ca302a13ac
[Feature] Add dgl.nn.*.Sequential for usability ( #1166 )
...
* upd
* upd
* upd
* upd
* lint
* upd
* upd
* upd
* upd
Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com >
2020-01-09 12:39:22 +08:00
4f02bb751b
[KVStore] Made kvstore can support multiple NICs ( #1150 )
...
* API change of kvstore
* add demo for kvstore
* update
* remove duplicated log
* change queue size
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* change name
* update
* fix lint
* update
* update
* update
* update
* change message queue size to a python argument
* change default queue size to 2GB
* OMP_NUM_THREADS=1
* add multiple NICs support for kvstore
* test
* fix lint
* update
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* update
* update
* update
* update
2019-12-30 15:17:31 +08:00
9a0511c8e9
[NN] nn modules & examples update ( #890 )
...
* upd
* damn it
* fuck
* fuck pylint
* fudge
* remove some comments about MXNet
* upd
* upd
* damn it
* damn it
* fuck
* fuck
* upd
* upd
* pylint bastard
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
2019-11-03 17:29:42 -05:00
6a4b5ae98a
[Feature] Edge softmax on a subset of edges in the graph. ( #842 )
...
* upd
* add test
* fix
* upd
* merge
* hotfix
* upd
* fix
2019-09-09 13:44:56 +08:00
e17add5602
[NN] Add MXNet impl for TAGCN module. ( #799 )
...
* upd
* fig edgebatch edges
* add test
* trigger
* Update README.md for pytorch PinSage example.
Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.
* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.
1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
test/pytorch/test_nn.py work on both CPU and GPU
* Fix style
* Delete unused code
* Make agnostic test only related to tests/backend
1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu
* Fix code style
* fix
* doc
* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.
* Fix syntex
* Remove rand
* Add TAGCN nn.module and example
* Now tagcn can run on CPU.
* Add unitest for TGConv
* Fix style
* For pubmed dataset, using --lr=0.005 can achieve better acc
* Fix style
* Fix some descriptions
* trigger
* Fix doc
* Add nn.TGConv and example
* Fix bug
* Update data in mxnet.tagcn test acc.
* Fix some comments and code
* delete useless code
* Fix namming
* Fix bug
* Fix bug
* Add test code for mxnet TAGCov
* Update some docs
* Fix some code
* Update docs dgl.nn.mxnet
* Update weight init
* Fix
2019-08-28 13:19:17 +08:00
9314aabd1f
[Refactor] Interface of nn modules ( #798 )
...
* refactor
* upd mpnn
2019-08-27 22:29:25 +08:00
708765f0a1
[NN] RGCN modules ( #744 )
...
* rgcn module
* support id input
* WIP: model codes
* use faster index select
* dropout
* self loop
* WIP: link prediction
* fix lint
* WIP: docs
* docstring
* docstring
* merge two child classes
* mxnet rgcn module
* fix lint
* fix lint
* fix rename bug
* add uniform edge sampler
* fix fn name
* docstring
* fix mxnet rgcn module
* fix mx rgcn
* enable test on cuda
2019-08-23 16:38:48 -04:00
2bff8339dd
[Test] Provid a frame agnostic API to test nn modules on both CPU and CUDA side. ( #775 )
...
* upd
* fig edgebatch edges
* add test
* trigger
* Update README.md for pytorch PinSage example.
Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.
* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.
1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
test/pytorch/test_nn.py work on both CPU and GPU
* Fix style
* Delete unused code
* Make agnostic test only related to tests/backend
1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu
* Fix code style
* fix
* doc
* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.
* Fix syntex
* Remove rand
2019-08-21 16:40:41 +08:00
5d3f470b72
[Feature] DGL Pooling modules ( #669 )
...
* removal doc
* glob
* upd
* rm knn
* add softmax
* upd
* upd
* add broadcast and s2s
* optimize max_on
* forsaken changes to heterograph
* upd
* upd
* upd
* upd
* upd
* bugfix
* upd
* upd
* upd
* upd
* format upd
* upd format
* upd doc
* upd
* import order
* upd
* rm warnings
* fix
* upd test
* upd
* upd
* fix device
* upd
* upd
* upd
* upd
* remove 1.1
* upd
* trigger
* trigger
* add more tests
* fix device
* upd
* upd
* refactor
* fix?
* fix
* upd docstring
* refactor
* upd
* fix
* upd
* upd
* upd
* fix
* upd docs
* add shape
* refactor & upd doc
* upd doc
* upd
2019-08-06 01:43:25 +08:00
fc9d30fae4
[Graph] add local scope function ( #735 )
...
* add local scope function
* fix lint
* fix docstring
* change local_scope to local_var; add context manager
* address comments
2019-08-02 00:11:48 -04:00
2e9949d228
[BUGFIX] Fix bugs in shared mem graph store. ( #630 )
...
* fix graph store for Pytorch.
* add test.
* fix dtype error in test
* disable test on GPU.
* test avoid windows.
* fix shared-memory test.
* use script to control testing environment.
* update test.
* enable all tests.
* fix test script.
2019-06-09 13:08:47 -07:00
74e13eea61
[Model] Update GAT model code ( #622 )
...
* fix gat code to use latest edge softmax module
* avoid transpose
* update README
* use edge_softmax op
* mxnet edge softmax op
* mxnet gat
* update README
* fix unittest
* fix ci
* fix mxnet nn test; relax criteria for prod reducer
2019-06-08 21:41:38 -04:00
653428bdc7
[Feature][Kernel] DGL kernel support ( #596 )
...
* [Kernel] Minigun integration and fused kernel support (#519 )
* kernel interface
* add minigun
* Add cuda build
* functors
* working on binary elewise
* binary reduce
* change kernel interface
* WIP
* wip
* fix minigun
* compile
* binary reduce kernels
* compile
* simple test passed
* more reducers
* fix thrust problem
* fix cmake
* fix cmake; add proper guard for atomic
* WIP: bcast
* WIP
* bcast kernels
* update to new minigun pass-by-value practice
* broadcasting dim
* add copy src and copy edge
* fix linking
* fix none array problem
* fix copy edge
* add device_type and device_id to backend operator
* cache csr adj, remove cache for adjmat and incmat
* custom ops in backend and pytorch impl
* change dgl-mg kernel python interface
* add id_mapping var
* clean up plus v2e spmv schedule
* spmv schedule & clean up fall back
* symbolic message and reduce func, remove bundle func
* new executors
* new backend interface for dgl kernels and pytorch impl
* minor fix
* fix
* fix docstring, comments, func names
* nodeflow
* fix message id mapping and bugs...
* pytorch test case & fix
* backward binary reduce
* fix bug
* WIP: cusparse
* change to int32 csr for cusparse workaround
* disable cusparse
* change back to int64
* broadcasting backward
* cusparse; WIP: add rev_csr
* unit test for kernels
* pytorch backward with dgl kernel
* edge softmax
* fix backward
* improve softmax
* cache edge on device
* cache mappings on device
* fix partial forward code
* cusparse done
* copy_src_sum with cusparse
* rm id getter
* reduce grad for broadcast
* copy edge reduce backward
* kernel unit test for broadcasting
* full kernel unit test
* add cpu kernels
* edge softmax unit test
* missing ref
* fix compile and small bugs
* fix bug in bcast
* Add backward both
* fix torch utests
* expose infershape
* create out tensor in python
* fix c++ lint
* [Kernel] Add GPU utest and kernel utest (#524 )
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* [Kernel] Update kernel branch (#550 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* Fixing typo in JTNN after interface change (#536 )
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* [Kernel] Update kernel branch (#576 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* Fixing typo in JTNN after interface change (#536 )
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* all demo use python-3 (#555 )
* [DEMO] Reproduce numbers of distributed training in AMLC giant graph paper (#556 )
* update
* update
* update
* update num_hops
* fix bug
* update
* report numbers of distributed training in AMLC giant graph paper
* [DEMO] Remove duplicate code for sampling (#557 )
* update
* update
* re-use single-machine code
* update
* use relative path
* update
* update
* update
* add __init__.py
* add __init__.py
* import sys, os
* fix typo
* update
* [Perf] Improve performance of graph store. (#554 )
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
* [BUGIFX] fix a bug in edge_ids (#560 )
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
* [DEMO] Add Pytorch demo for distributed sampler (#562 )
* update
* update
* update
* add sender
* update
* remove duplicate cpde
* [Test] Add gtest to project (#547 )
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
* [Perf] lazily create msg_index. (#563 )
* lazily create msg_index.
* update test.
* [BUGFIX] fix bugs for running GCN on giant graphs. (#561 )
* load mxnet csr.
* enable load large csr.
* fix
* fix.
* fix int overflow.
* fix test.
* [BugFix] Fix error when bfs_level = 0 in Entity Classification with RGCN (#559 )
* [DEMO] Update demo of distributed sampler (#564 )
* update
* update
* update demo
* add network cpp test (#565 )
* Add unittest for C++ RPC (#566 )
* [CI] Fix CI for cpp test (#570 )
* fix CI for cpp test
* update port number
* [Docker] update docker image (#575 )
* update docker image
* specify lint version
* rm torch import from unified tests
* [Kernel][Scheduler][MXNet] Scheduler for DGL kernels and MXNet backend support (#541 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* edge softmax module
* WIP
* Fixing typo in JTNN after interface change (#536 )
* mxnet backend support
* improve reduce grad
* add max to unittest backend
* fix kernel unittest
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* lint
* lint
* win build
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* try
* fix
* fix
* fix
* fix
* fix
* try
* test
* test
* test
* try
* try
* try
* test
* fix
* try gen_target
* fix gen_target
* fix msvc var_args expand issue
* fix
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* WIP
* WIP
* all demo use python-3 (#555 )
* ToImmutable and CopyTo
* [DEMO] Reproduce numbers of distributed training in AMLC giant graph paper (#556 )
* update
* update
* update
* update num_hops
* fix bug
* update
* report numbers of distributed training in AMLC giant graph paper
* [DEMO] Remove duplicate code for sampling (#557 )
* update
* update
* re-use single-machine code
* update
* use relative path
* update
* update
* update
* add __init__.py
* add __init__.py
* import sys, os
* fix typo
* update
* [Perf] Improve performance of graph store. (#554 )
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
* [BUGIFX] fix a bug in edge_ids (#560 )
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
* DGLRetValue DGLContext conversion
* [DEMO] Add Pytorch demo for distributed sampler (#562 )
* update
* update
* update
* add sender
* update
* remove duplicate cpde
* [Test] Add gtest to project (#547 )
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
* Add support to convert immutable graph to 32 bits
* [Perf] lazily create msg_index. (#563 )
* lazily create msg_index.
* update test.
* fix binary reduce following new minigun template
* enable both int64 and int32 kernels
* [BUGFIX] fix bugs for running GCN on giant graphs. (#561 )
* load mxnet csr.
* enable load large csr.
* fix
* fix.
* fix int overflow.
* fix test.
* new kernel interface done for CPU
* docstring
* rename & docstring
* copy reduce and backward
* [BugFix] Fix error when bfs_level = 0 in Entity Classification with RGCN (#559 )
* [DEMO] Update demo of distributed sampler (#564 )
* update
* update
* update demo
* adapt cuda kernels to the new interface
* add network cpp test (#565 )
* fix bug
* Add unittest for C++ RPC (#566 )
* [CI] Fix CI for cpp test (#570 )
* fix CI for cpp test
* update port number
* [Docker] update docker image (#575 )
* update docker image
* specify lint version
* rm torch import from unified tests
* remove pytorch-specific test_function
* fix unittest
* fix
* fix unittest backend bug in converting tensor to numpy array
* fix
* mxnet version
* [BUGFIX] fix for MXNet 1.5. (#552 )
* remove clone.
* turn on numpy compatible.
* Revert "remove clone."
This reverts commit 17bbf76ed72ff178df6b3f35addc428048672457.
* revert format changes
* fix mxnet api name
* revert mistakes in previous revert
* roll back CI to 20190523 build
* fix unittest
* disable test_shared_mem_store.py for now
* remove mxnet/test_specialization.py
* sync win64 test script
* fix lowercase
* missing backend in gpu unit test
* transpose to get forward graph
* pass update all
* add sanity check
* passing test_specialization.py
* fix and pass test_function
* fix check
* fix pytorch softmax
* mxnet kernels
* c++ lint
* pylint
* try
* win build
* fix
* win
* ci enable gpu build
* init submodule recursively
* backend docstring
* try
* test win dev
* doc string
* disable pytorch test_nn
* try to fix windows issue
* bug fixed, revert changes
* [Test] fix CI. (#586 )
* disable unit test in mxnet tutorial.
* retry socket connection.
* roll back to set_np_compat
* try to fix multi-processing test hangs when it fails.
* fix test.
* fix.
* doc string
* doc string and clean up
* missing field in ctypes
* fix node flow schedule and unit test
* rename
* pylint
* copy from parent default context
* fix unit test script
* fix
* demo bug in nodeflow gpu test
* [Kernel][Bugfix] fix nodeflow bug (#604 )
* fix nodeflow bug
* remove debug code
* add build gtest option
* fix cmake; fix graph index bug in spmv.py
* remove clone
* fix div rhs grad bug
* [Kernel] Support full builtin method, edge softmax and unit tests (#605 )
* add full builtin support
* unit test
* unit test backend
* edge softmax
* apply edge with builtin
* fix kernel unit test
* disable mxnet test_shared_mem_store
* gen builtin reduce
* enable mxnet gpu unittest
* revert some changes
* docstring
* add note for the hack
* [Kernel][Unittest][CI] Fix MXNet GPU CI (#607 )
* update docker image for MXNet GPU CI
* force all dgl graph input and output on CPU
* fix gpu unittest
* speedup compilation
* add some comments
* lint
* add more comments
* fix as requested
* add some comments
* comment
* lint
* lint
* update pylint
* fix as requested
* lint
* lint
* lint
* docstrings of python DGL kernel entries
* disable lint warnings on arguments in kernel.py
* fix docstring in scheduler
* fix some bug in unittest; try again
* Revert "Merge branch 'kernel' of github.com:zzhang-cn/dgl into kernel"
This reverts commit 1d2299e68b004182ea6130b088de1f1122b18a49, reversing
changes made to ddc97fbf1bec2b7815c0da7c74f7ecb2f428889b.
* Revert "fix some bug in unittest; try again"
This reverts commit ddc97fbf1bec2b7815c0da7c74f7ecb2f428889b.
* more comprehensive kernel test
* remove shape check in test_specialization
2019-06-06 15:47:55 -04:00
372203f04a
[Test] Fix tests in test_shared_mem_store. ( #588 )
...
* fix test.
* better assert.
* more asserts.
* print to stderr.
* destroy g.
* fix tests.
* add timeout in sync_barrier.
* test _sync_barrier.
* fix.
* avoid printing messages.
* fix test.
* fix test.
* fix.
2019-06-01 21:18:01 -04:00
40dc18595b
[Test] fix CI. ( #586 )
...
* disable unit test in mxnet tutorial.
* retry socket connection.
* roll back to set_np_compat
* try to fix multi-processing test hangs when it fails.
* fix test.
* fix.
2019-05-29 21:40:16 -04:00
ce27ebbbd0
[BUGIFX] fix a bug in edge_ids ( #560 )
...
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
2019-05-23 15:25:38 -07:00
924efc6520
[Perf] Improve performance of graph store. ( #554 )
...
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
2019-05-23 12:23:31 -07:00
b2b8be25bb
[API] update graph store API. ( #549 )
...
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
2019-05-21 03:27:05 -07:00
cdfca992e1
[BUGFix] Improve multi-processing training ( #526 )
...
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
2019-05-20 14:23:19 -07:00
3a1392e641
[Model] add multiprocessing training with sampling. ( #484 )
...
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
2019-05-06 23:51:56 -07:00
06e22f1ee4
[Test] Disable unittest for shared memory temporarily ( #497 )
...
* disable tests.
* import unittest.
2019-04-16 16:27:23 -07:00
bfdd1eaa44
[Feature] Create shared memory graph store. ( #468 )
...
* accelerate gcn_ns.
* add timing.
* run infer with whole graph.
* distributed gcn_ns.
* reconstruct gcn_ns.
* minor fix.
* change graphsage_cv for numa.
* fix #OMP threads.
* accelerate graphsage_cv.
* fix a weird bug.
* add profiler in graphsage_cv.
* accelerate graphsage_cv.
manually aggregate neighbors' embeddings with pull.
* load csr directly in gcn_ns_sc.
* parallel sort for graph index.
* Revert "parallel sort for graph index."
This reverts commit 86fe2c7117fe5e56b0d481b39849c258b166945b.
* run gcn_ns_sc on GPUs.
* acc gcn_cv_sc.
* change gcn_cv for numa.
* fix gcn_cv to use numa and gpu.
* improve graphsage_cv to use numa and gpu.
* improve gcn_ns.
* improve graphsage_cv.
* init shared memory graph store.
* fix.
* enable init ndata.
* improve tests.
* add bidirectional communication.
* link to rt.
* fix compilation error.
* fix shared memory init.
* use MessageQueue for inter-process communication.
* reconstruct immutable graph csr.
* fix gcn.
* load csr to shared memory.
* fix minor bugs.
* add comments.
* refactor SharedMemory.
* fix bugs in ImmutableGraph.
* create CSR graph from shared memory.
* add more test for loading a csr graph.
* terminate graph store properly.
* allow initializing ndata in the graph store server.
* use RPC for inter-process communication.
* a script for loading a graph.
* allow customizing port.
* list all ndata and edata.
* support dtype.
* reorganize SharedMemoryGraphStore.
* fix ndata shape.
* reconstruct gcn_ns.
* print info.
* set omp in gcn_ns.
* reset sampling examples.
* fix lint.
* fix lint.
* reset gcn.
* disable shared memory in windows.
* fix.
* fix.
* reset changes.
* revert nodeflow changes.
* fix cmake.
* fix test.
* fix test.
* fix test.
* fix test.
* add comments.
* fix test.
* move vector out.
* fix lint.
* fix lint.
* move SharedMemory.
* update cmake.
* update comment.
* fix comments.
* Revert "update cmake."
This reverts commit 592445e37077f70a6e3f2e5245f9a3d086b04f3b.
* update cmake.
* add comments.
* rename.
* change the comment.
* fix a bug.
* rename.
* add comments.
* add comments.
* add init_edata.
* rewrite memory alloc.
* move vector to CSR.
* fix.
* init data.
* Revert "init data."
This reverts commit 2b217b9553911b7dd84a9f1d9b68430b5aa18e23.
* init data.
* init new columns correctly.
2019-04-08 12:09:02 -07:00
565f0c88fc
[WIP] [NN] Refactor NN package ( #406 )
...
* refactor graph conv
* docs & tests
* fix lint
* fix lint
* fix lint
* fix lint script
* fix lint
* Update
* Style fix
* Fix style
* Fix style
* Fix gpu case
* Fix for gpu case
* Hotfix edgesoftmax docs
* Handle repeated features
* Add docstring
* Set default arguments
* Remove dropout from nn.conv
* Fix
* add util fn for renaming
* revert gcn_spmv.py
* mx folder
* fix wierd bug
* fix mx
* fix lint
2019-02-25 18:41:21 -05:00
929742b588
[Feature] Reimplement Immutable graph index in DGL ( #342 )
...
* init.
* it's compiled.
* add immutable graph constructor.
* add immutable graph API.
* fix.
* impl get adjacency matrix.
* fix.
* fix graph_index from scipy matrix.
* add neighbor sampling.
* remap vertex ids.
* fix.
* move sampler test.
* fix tests.
* add comments
* remove mxnet-specific immutable graph.
* fix.
* fix lint.
* fix.
* try to fix windows compile error.
* fix.
* fix.
* add test.
* unify Graph and ImmutableGraph.
* fix bugs.
* fix compile.
* move immutable graph.
* fix.
* remove print.
* fix lint.
* fix
* fix lint.
* fix lint.
* fix test.
* fix comments.
* merge GraphIndex and ImmutableGraphIndex.
* temp fix.
* impl GetAdj.
* fix lint
* fix.
* fix.
* fix.
* fix.
* fix.
* use csr only for readonly graph.
* Revert "use csr only for readonly graph."
This reverts commit 8e24bb033af8504531b22849de5b7567b168e0d5.
* remove code.
* fix.
* fix.
* fix.
* fix.
* fix.
* fix.
* address comments.
* fix for comments.
* fix comments.
* revert.
* move test_graph_index to compute.
* fix.
* fix.
* impl GetAdj for coo.
* fix.
* fix tests.
* address comments.
* address comments.
* fix comment.
* address comments.
* use lambda.
* other comments.
* address comments.
* modify the semantics of edges.
* fix order.
* use DGLIdIter
* fix.
* remove NotImplemented.
* revert some code.
2019-01-17 09:59:53 +08:00
e19cd62ecd
[Test] Unify tests for different backends ( #333 )
...
* test basics
* batched graph & filter, mxnet filter fix
* frame and function; bugfix
* test graph adj and inc matrices
* fixing start = 0 for mxnet
* test index
* inplace update & line graph
* multi send recv
* more tests
* oops
* more tests
* removing old test files; readonly graphs for mxnet still kept
* modifying test scripts
* adding a placeholder for pytorch to reserve directory
* torch 0.4.1 compat fixes
* moving backend out of compute to avoid nose detection
* tests guide
* mx sparse-to-dense/sparse-to-numpy is buggy
* oops
* contribution guide for unit tests
* printing incmat
* printing dlpack
* small push
* typo
* fixing duplicate entries that causes undefined behavior
* move equal comparison to backend
2019-01-05 23:32:33 -05:00
4bd4d6e348
[Lint] Pylint ( #330 )
...
* fix lint for graph_index.py
* pylint for base.py
* pylint for batched_graph.py
* pylint for frame.py; simplify and fix bugs in frame when index is slice type
* pylint for graph.py
* pylint for immutable_graph_index.py
* pylint for init.py
* pylint for rest files in root package
* pylint for _ffi package
* pylint for function package
* pylint for runtime package
* pylint for runtime.ir package
* add pylint to ci
* fix mx tests
* fix lint errors
* fix ci
* fix as requested
* fix lint
2019-01-01 15:15:41 -05:00
0d0f44363c
[Bugfix] MX utest traversal memory corruption ( #312 )
...
* WIP
* temp fix mx traversal memory crash bug
2018-12-16 21:07:59 -08:00
f35ac5443e
[Graph][Model] Cache adj & inc; MX GCN ( #307 )
...
* add cache to adj and incmat
* Fix bug in cached adj/inc
* mx gcn spmv runnable; acc debugging...
* fix bug in mx gcn that loss is not correctly calculated
* fix mx utest
* fix as requested
* use raw parameter tensors rather than dense layer
* fix dropout
* Add numbers in readme
2018-12-16 22:51:01 -05:00
e2926544b6
[Graph] Better string repr ( #311 )
2018-12-16 12:23:49 -05:00
30107407fd
add prefetcher for neighbor sampler ( #298 )
2018-12-15 19:14:17 -08:00
3564fdc5f1
[Bugfix][Model] fix treelstm model ( #274 )
...
* fix bug after moving batcher out of dgl.data
* disable mx utest
2018-12-06 23:52:50 -05:00
7aa494b3bc
[Bugfix][Doc] explain the batch dimension in the doc and many fix ( #266 )
...
* add more unit tests for mxnet.
* fix.
* explain the batch dimension.
* update doc.
* disable unit tests on DFS.
* fix graph traversal.
2018-12-06 00:48:57 -05:00
063ed69cdf
[BUGFIX] fix a bug in creating immutable graph index. ( #251 )
...
* fix a bug in creating immutable graph index.
* fix for new changes in the backend API.
* fix for creating immutable graph index from coo matrix.
* retrigger
2018-12-05 17:51:50 -05:00
899d125bb1
Add tests for MXNet and throw NotImplementedError if not implemented ( #258 )
...
* add more unit tests for mxnet.
* fix.
2018-12-05 10:41:33 -08:00
7c7cc7e0c2
[sampler] Adjust the sampler API for the future extension. ( #243 )
...
* return seed ids.
* fix tests.
* implement.
2018-12-05 10:05:10 -08:00
21255b6528
[Bugfix] tolist and dependencies in dgl.data ( #239 )
...
* change Index.tolist -> Index.tonumpy; fix bug in traversal; remove dependencies in data
* fix import
* fix __all__ and some docstring
2018-12-05 12:20:59 -05:00
eafcb7e7f5
[Bugfix][MXNet] Fix edge order and builtin max bug in mx ( #247 )
...
* Fix edge order and builtin max bug in mx
* fix as requested
2018-12-05 01:32:54 -08:00
2c170a8c17
[Graph][Bugfix] Fix the API of map_to_subgraph_nid ( #226 )
...
* correct vid mapping API.
* fix sse.
2018-12-03 17:16:35 -05:00
8918cce0b5
[MXNet] add unit tests for mxnet ( #227 )
...
[MXNet] add unit tests for mxnet
2018-12-03 11:04:11 -05:00
b1eeb93449
[Bugfix][Runtime] Zero degree behaviors ( #177 )
...
* fix recv nodes are all 0deg; fix hybriddict does not through keyerror properly
* fallback to apply_nodes when all nodes are 0deg; WIP on pull spmv 0deg
* new 0deg behavior
* new 0deg behavior
* update mx utest for pull-0deg
* fix mx
* fix mx
* get rid of unnecessary sort-n-unique
2018-11-25 22:33:08 -05:00
deb653f8dc
[Runtime] Scheduler and Executor ( #140 )
...
* executor api
* draft executor interface
* WIP
* revert changes to avoid conflict with api change
* core scheduling logic
* WIP: build graph adj
* incidence matrix for in edges
* support incidence matrix for partial recv nodes
* improve
* build adjmat in scheduler
* graph store
* get degree bucketing schedule
* connect to c++ degree bucketing
* conceptual executor creation code
* executor comments
* fix
* more executor comments
* WIP: full send_and_recv schedule
* most schedulers
* simplify scheduler
* executors
* runtime
* builtin function base class
* adj indices and shape
* completely refactor scheduler
* rename and move bundled out to function.py
* use_edge_feature in msg func
* rewrite scheduler
* node edge executor
* connect with graph api
* handle zero degree
* misc
* fix test cases
* fix a good many bugs...
* remove old scheduler
* push and pull
* fix send recv
* c++ lint
* fix batched send recv
* hot fix for mxnet
* typo
* write back executor
* apply node edge
* clean up, doc string
* fix as requested
* refactor
* fix
* WIP
* WIP
* ir draft
* more on ir
* WIP: spmv schedule
* WIP
* recv schedule
* refactor
* WIP
* snr degree bucketing
* snr scheduler
* move prog to graph.py; rename
* unittest for send/recv
* remove some legacy codes
* WIP: update_all
* pass test_basics
* passed all current utests
* more utests; fix mx utest
* WIP: fixing zero deg initial value
* some tests
* fix 0deg problem
* fix mx
* fix mx
* some notes
* fix as requested
2018-11-22 12:15:50 -05:00
440aecee8f
[Test] add tests for built-in functions ( #159 )
...
* add spmv tests on mxnet.
* test backward.
* add tests for pull.
* use mxnet backend.
* address comment.
* add comments.
2018-11-20 23:39:54 -05:00
2389df8155
[API] Subgraph sampling API. ( #138 )
...
* add neighbor sampler.
* fix sampler.
* handle non-existing nodes differently in MapParentIdToSubgraphId.
* fix sampling tests.
* parallelize sampling.
* add more tests.
* fix a bug in subgraph loader.
* get number of sampled nodes directly.
* load mxnet CSR to DGLGraph directly.
* create graph from mxnet csr array directly.
* use subgraph loader in SSE.
* fix a bug in sampling.
* output framework tensor.
* fix.
* remove unnecessary code.
* fix tests.
* add SSE README.
* move to contrib.
* address comments.
* add tests.
2018-11-14 13:51:27 -08:00
048f6d7a30
[GraphIndex] refactor graph caching ( #150 )
...
* refactor graph caching
* fix mx test
* fix typo
2018-11-13 22:12:38 -05:00
bd0e4fa0b3
[MXNet][API] move to the new API ( #123 )
...
* move gat to the new api.
* fix gcn.
* update sse.
* fix dgl core.
* update sse.
* fix small bugs in dgl core.
* fix mxnet tests.
* retrigger
* address comments and fix more bugs.
* fix
* fix tests.
2018-11-08 10:00:47 -08:00
7241a9c008
[Backend] backend interface ( #109 )
...
* backend interface
* small fix
* more comments to the data type dict
* WIP
* convert_to and narrow
* WIP
* pytorch and numpy backend; WIP on mxnet backend
* mxnet backend
* narrow
* Fix all usages
* fix for mx
* fix for mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* revert jenkins
* add sparse_matrix api
* sparse matrix api
* some fixme
* Fix as requested
2018-11-04 21:42:39 -05:00
68ec624782
[API][Doc] API change & basic tutorials ( #113 )
...
* Add SH tutorials
* setup sphinx-gallery; work on graph tutorial
* draft dglgraph tutorial
* update readme to include document url
* rm obsolete file
* Draft the message passing tutorial
* Capsule code (#102 )
* add capsule example
* clean code
* better naming
* better naming
* [GCN]tutorial scaffold
* fix capsule example code
* remove previous capsule example code
* graph struc edit
* modified: 2_graph.py
* update doc of capsule
* update capsule docs
* update capsule docs
* add msg passing prime
* GCN-GAT tutorial Section 1 and 2
* comment for API improvement
* section 3
* Tutorial API change (#115 )
* change the API as discusses; toy example
* enable the new set/get syntax
* fixed pytorch utest
* fixed gcn example
* fixed gat example
* fixed mx utests
* fix mx utest
* delete apply edges; add utest for update_edges
* small change on toy example
* fix utest
* fix out in degrees bug
* update pagerank example and add it to CI
* add delitem for dataview
* make edges() return form that is compatible with send/update_edges etc
* fix index bug when the given data is one-int-tensor
* fix doc
2018-11-02 17:19:14 -04:00
9e9a9488f1
[GraphIndex] Immutable graph index using MXNet CSRArray ( #86 )
...
* add immutable graph index.
* update ImmutableGraphIndex.
* add benchmarks of subgraph generation.
* fix node_subgraphs.
* use scipy synthetic data in immutable graph.
* create immutable_graph.
* use spmv in gcn.
* fix immutable graph index.
* update graph index.
* create ImmutableSubgraphIndex.
* return subgraph node and edge mapping.
* fix benchmark.
* add mapping to subgraph nid.
* test in_edges and out_edges.
* update benchmark.
* Fix immutable subgraph.
* complete in_edges and out_edges.
* fix benchmarks.
* add test on node_subgraphs
* add sse.
* construct subgraphs in parallel.
* add in/out degree(s).
* make immutable graph index backend-specific.
* implement has_edge(s) and edge_id(s).
* Revert "use spmv in gcn."
This reverts commit 9cfed5f5fb3dd2ed9b98745348b1c0e9731ed7f7.
* implement node_subgraphs in GraphIndex.
* fix sse.
* address comments.
* address comments.
* Update mxnet docker.
2018-10-26 11:28:25 -07:00
0d6cd30be3
[CI] Enable mxnet CI. ( #105 )
...
* fix tests.
* enable mxnet CI.
* add mxnet tests.
* update jenkins.
* update mxnet docker image.
2018-10-26 12:04:41 -04:00
4af3f8bcc8
update the MXNet backend. ( #89 )
...
* update mxnet.
* add get_tvmtype.
* remove undefined test.
2018-10-18 12:49:13 -07:00
Zihao Ye