8f378d90ad
[BUGFIX] fix bugs for running GCN on giant graphs. ( #561 )
...
* load mxnet csr.
* enable load large csr.
* fix
* fix.
* fix int overflow.
* fix test.
2019-05-25 11:09:33 -07:00
14af8402f0
[Perf] lazily create msg_index. ( #563 )
...
* lazily create msg_index.
* update test.
2019-05-24 13:41:03 -07:00
de54891fc8
[Test] Add gtest to project ( #547 )
...
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
2019-05-24 15:15:47 +08:00
86d60a1ff6
[DEMO] Add Pytorch demo for distributed sampler ( #562 )
...
* update
* update
* update
* add sender
* update
* remove duplicate cpde
2019-05-24 11:53:28 +08:00
ce27ebbbd0
[BUGIFX] fix a bug in edge_ids ( #560 )
...
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
2019-05-23 15:25:38 -07:00
924efc6520
[Perf] Improve performance of graph store. ( #554 )
...
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
2019-05-23 12:23:31 -07:00
49c4a9e4cc
[DEMO] Remove duplicate code for sampling ( #557 )
...
* update
* update
* re-use single-machine code
* update
* use relative path
* update
* update
* update
* add __init__.py
* add __init__.py
* import sys, os
* fix typo
* update
2019-05-23 16:57:35 +08:00
28379f927f
[DEMO] Reproduce numbers of distributed training in AMLC giant graph paper ( #556 )
...
* update
* update
* update
* update num_hops
* fix bug
* update
* report numbers of distributed training in AMLC giant graph paper
2019-05-23 14:23:14 +08:00
f99725adbc
all demo use python-3 ( #555 )
2019-05-23 10:38:05 +08:00
605b51857e
[Refactor] Immutable graph index ( #543 )
...
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
2019-05-21 14:28:06 -04:00
b2b8be25bb
[API] update graph store API. ( #549 )
...
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
2019-05-21 03:27:05 -07:00
cdfca992e1
[BUGFix] Improve multi-processing training ( #526 )
...
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
2019-05-20 14:23:19 -07:00
4b761571b2
[Feature] Add batch and unbatch for immutable graph ( #539 )
...
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
2019-05-20 11:08:43 -04:00
aa129522e0
[Bug Fix] Fix inplace op at backend ( #546 )
...
* Fix inplace operation
* fix line seprator
2019-05-19 23:43:54 -04:00
d533e9ba62
[BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block ( #515 )
2019-05-16 18:08:43 -07:00
d8c69d53e2
Fixing typo in JTNN after interface change ( #536 )
2019-05-16 09:43:03 +08:00
acefb9a9f0
[BUGFIX] fix the problems in the sampling tutorial. ( #523 )
...
* add index.
* update.
* update tutorial.
2019-05-07 18:19:07 -07:00
edf3951c18
[Tutorial] add sampling tutorial. ( #522 )
...
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
2019-05-07 17:29:18 -07:00
3a1392e641
[Model] add multiprocessing training with sampling. ( #484 )
...
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
2019-05-06 23:51:56 -07:00
e89519150f
disable pytorch reddit ( #512 )
2019-04-28 11:24:14 +08:00
99bc3ab83b
Use private buffer instead of global buffer ( #511 )
2019-04-26 12:48:01 +08:00
8c79885d8b
[BUGFIX] fix the error message in the data loader. ( #509 )
...
* fix.
* Change to required=True
* Update description
* Change back
2019-04-26 09:52:45 +09:00
2ff8ecfbac
update test_batch_size ( #510 )
2019-04-25 13:28:36 +08:00
ca48787ae4
fix tutorial ( #506 )
...
* Update 9_gat.py
* Update 1_gcn.py
2019-04-23 15:30:59 +08:00
3b96299a45
sampling ( #505 )
2019-04-23 12:14:38 +08:00
ad9da36add
[Model] Add DGI Model ( #501 )
2019-04-22 11:37:06 +08:00
fe7d5e9b8b
[Sampler] Change Distributed Sampler API ( #499 )
...
* Change Distributed Sampler API
* fix lint
* fix lint
* update demo
* update
* update
* update
* update demo
* update demo
2019-04-22 09:41:47 +08:00
3f4645918d
[Model] Add edge dropout to APPNP ( #493 )
...
* [Model] Add edge dropout to APPNP
[Model] Add edge dropout to APPNP
* [Model] Refactor the sampling examples (#498 )
* reorganize sampling code.
* speedup gcn_ns.
* speed up gcn_cv
* fix graphsage_cv.
* undo the modification.
* accel training.
* update readme.
* [Model] Add edge dropout to APPNP
[Model] Add edge dropout to APPNP
update
2019-04-18 20:58:28 +08:00
6124667f42
[Model] Refactor the sampling examples ( #498 )
...
* reorganize sampling code.
* speedup gcn_ns.
* speed up gcn_cv
* fix graphsage_cv.
* undo the modification.
* accel training.
* update readme.
2019-04-17 01:31:14 -07:00
a3febc061b
[Model] Add GIN Model ( #471 )
...
* add gin model
* convert dataset.py to data_ont_the_fly way and put it into dgl.data module
* convert dataset.py to data_ont_the_fly way and put it into dgl.data module
python code checked
* modified document and reference TUDataset; checked python part and bypass cpp part due to error
* change tensor to numpy in dataset and transform in collate@Dataloader
* Change minor format issue
Change minor format issue
* moved logging; adjusted tqdm etc
2019-04-17 14:57:07 +08:00
fb6af16f83
[BUG] Fix crash during Cython build ( #500 )
2019-04-17 14:18:22 +08:00
06e22f1ee4
[Test] Disable unittest for shared memory temporarily ( #497 )
...
* disable tests.
* import unittest.
2019-04-16 16:27:23 -07:00
b89dcce16c
[RPC] Refactoring networking APIs ( #496 )
...
* Refactoring network API
* update demo
* update
* update demo
* update demo
* add num_sender
* update
* fix lint
* fix lint
* fix lint
* update
2019-04-16 16:33:29 +08:00
688a9228a8
fix. ( #491 )
2019-04-11 15:38:35 -07:00
e2e04329a6
[BUG] Update URL for JTNN dataset
2019-04-11 22:23:34 +08:00
c69dbbb736
[Bug Fix] Fix TUDataset Processing bug ( #489 )
...
* add graph classification dataset
* add node label
* add TUDataset
* Modify to consistent with Qi Huang's implementation
* add docs
* Add docs
* Fix change of environment variable
* Update tu.py
* Update tu.py
* Fix error when add node with np.int64
* Fix processing bug
2019-04-11 19:30:50 +08:00
00fc680aae
[Dataset] Add TUDataset ( #473 )
...
* add graph classification dataset
* add node label
* add TUDataset
* Modify to consistent with Qi Huang's implementation
* add docs
* Add docs
* Fix change of environment variable
* Update tu.py
* Update tu.py
* Fix error when add node with np.int64
2019-04-10 09:37:53 -04:00
039a711da1
[Model] fix self-edge bug in GCN and GAT. ( #482 )
...
* Update gcn_mp.py
* Update train.py
* Update train.py
* Update train.py
* Update gat_batch.py
* Update gat_batch.py
* Update gcn_mp.py
2019-04-10 09:04:19 -04:00
fa887f69f4
[Model] Add APPNP model ( #480 )
...
* [Model] Add APPNP model
* update
* Revert "update"
This reverts commit a8e42d19c8b421fb382305a8f04d7426e78ea53c.
* update
* Update appnp.py
2019-04-09 19:54:09 +08:00
bfdd1eaa44
[Feature] Create shared memory graph store. ( #468 )
...
* accelerate gcn_ns.
* add timing.
* run infer with whole graph.
* distributed gcn_ns.
* reconstruct gcn_ns.
* minor fix.
* change graphsage_cv for numa.
* fix #OMP threads.
* accelerate graphsage_cv.
* fix a weird bug.
* add profiler in graphsage_cv.
* accelerate graphsage_cv.
manually aggregate neighbors' embeddings with pull.
* load csr directly in gcn_ns_sc.
* parallel sort for graph index.
* Revert "parallel sort for graph index."
This reverts commit 86fe2c7117fe5e56b0d481b39849c258b166945b.
* run gcn_ns_sc on GPUs.
* acc gcn_cv_sc.
* change gcn_cv for numa.
* fix gcn_cv to use numa and gpu.
* improve graphsage_cv to use numa and gpu.
* improve gcn_ns.
* improve graphsage_cv.
* init shared memory graph store.
* fix.
* enable init ndata.
* improve tests.
* add bidirectional communication.
* link to rt.
* fix compilation error.
* fix shared memory init.
* use MessageQueue for inter-process communication.
* reconstruct immutable graph csr.
* fix gcn.
* load csr to shared memory.
* fix minor bugs.
* add comments.
* refactor SharedMemory.
* fix bugs in ImmutableGraph.
* create CSR graph from shared memory.
* add more test for loading a csr graph.
* terminate graph store properly.
* allow initializing ndata in the graph store server.
* use RPC for inter-process communication.
* a script for loading a graph.
* allow customizing port.
* list all ndata and edata.
* support dtype.
* reorganize SharedMemoryGraphStore.
* fix ndata shape.
* reconstruct gcn_ns.
* print info.
* set omp in gcn_ns.
* reset sampling examples.
* fix lint.
* fix lint.
* reset gcn.
* disable shared memory in windows.
* fix.
* fix.
* reset changes.
* revert nodeflow changes.
* fix cmake.
* fix test.
* fix test.
* fix test.
* fix test.
* add comments.
* fix test.
* move vector out.
* fix lint.
* fix lint.
* move SharedMemory.
* update cmake.
* update comment.
* fix comments.
* Revert "update cmake."
This reverts commit 592445e37077f70a6e3f2e5245f9a3d086b04f3b.
* update cmake.
* add comments.
* rename.
* change the comment.
* fix a bug.
* rename.
* add comments.
* add comments.
* add init_edata.
* rewrite memory alloc.
* move vector to CSR.
* fix.
* init data.
* Revert "init data."
This reverts commit 2b217b9553911b7dd84a9f1d9b68430b5aa18e23.
* init data.
* init new columns correctly.
2019-04-08 12:09:02 -07:00
9565db6f9e
fix warning. ( #479 )
2019-04-06 09:55:58 +08:00
4ea42e3e08
fix gcn. ( #469 )
2019-04-05 10:56:09 -07:00
8058f1c579
[Fix] Update inner API of distributed sampler ( #478 )
...
* update inner API of distributed sampler
* update
2019-04-05 10:00:36 +08:00
da3ab84c3c
Add demo for distributed sampler ( #474 )
...
* add C++ rpc infrastructure and distributed sampler
* update
* update lint
* update lint
* update lint
* update
* update
* update
* updare
* update
* update
* update
* update serialize and unittest
* update serialize
* lint
* update
* update
* update
* update
* update
* update
* update unittest
* put Finalize() to __del__
* update unittest
* update
* delete buffer in Finalize
* update unittest
* update unittest
* update unittest
* update unittest
* update
* update
* fix small bug
* windows socket impl
* update API
* fix bug in serialize
* fix bug in serialzie
* set parent graph
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* fix
* fix windows compilation error
* fix windows error
* change API to lower-case
* update test
* fix typo
* update
* add SamplerPool
* add SamplerPool
* update
* update test
* update
* update
* update
* update
* add example
* update
* update
* add distributed sampler demo
* add index
* update demo of distributed sampler
* fix lower-case
* print subg index
* update README.md
* update
* remove --gpu args
2019-04-03 16:09:59 -07:00
9b4fb2fb32
[Doc]Fix 'git remote' command in contribution doc ( #470 )
...
* fix docs remote instead of remove
* Update CONTRIBUTORS.md
2019-04-01 09:05:17 +08:00
5d494c6207
[RPC] add C++ RPC infrastructure and distributed sampler ( #465 )
...
* add C++ rpc infrastructure and distributed sampler
* update
* update lint
* update lint
* update lint
* update
* update
* update
* updare
* update
* update
* update
* update serialize and unittest
* update serialize
* lint
* update
* update
* update
* update
* update
* update
* update unittest
* put Finalize() to __del__
* update unittest
* update
* delete buffer in Finalize
* update unittest
* update unittest
* update unittest
* update unittest
* update
* update
* fix small bug
* windows socket impl
* update API
* fix bug in serialize
* fix bug in serialzie
* set parent graph
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* fix
* fix windows compilation error
* fix windows error
* change API to lower-case
* update test
* fix typo
* update
* add SamplerPool
* add SamplerPool
* update
* update test
* update
* update
* update
* update
* add example
* update
* update
2019-03-29 16:48:45 +08:00
6066fee935
[Model][Feature] PinSage & Random Walk with Restart ( #453 )
...
* random walk traces generation
* remove outdated comments
* oops put in the wrong place
* explicit inline
* moving rand_r to util
* pinsage-like model on movielens
* the code runs now
* support cuda
* using readonly graph
* moving random walk to public function
* per-thread seed and openmp support
* pinsage-like model on movielens
* the code runs now
* support cuda
* using readonly graph
* using C random walk
* removing profile decorators
* param initialization
* no grad
* leaky relu fixes everything
* train and save
* WIP
* WIP
* WIP
* seems to work
* evaluation output
* swapping order of val/test and train
* debug
* hyperparam tuning
* prior/training dataset split changes
* random walk reorg
* random walk with restart
* signed comparison fix
* migrating random walk to nodeflow
* Revert "migrating random walk to nodeflow"
This reverts commit f2565347cced7c912a58a529b257c033d9f375b7.
* add README and remove dataset
* new endpoint
* lint
* lint x2
* oops forgot test
* including bpr - better for baseline
* addressing fixes
* throwing random walks out from SamplerOp class
* forgot to move RandomWalk; why did this even work?
* removing legacy garbage
* add todo
* address comments
* stupid bug fix
* call ndarrayvector converter to handle traces
2019-03-29 13:01:18 +08:00
2dff1aba83
[Bugfix] Fix shape of test_relation tensor in RGCN ( #464 )
2019-03-28 14:34:07 -04:00
5033d94530
[News] Add GTC 2019 Tutorial as news in readme ( #462 )
2019-03-23 17:39:09 -04:00
341dbea0a1
[Bug] Fix unused prefetch switch for NeighborSampler and LayerSampler ( #457 )
...
* forgot prefetch switch
* yet another stupid bug
2019-03-19 14:57:10 +08:00
Da Zheng