cadcc1c2a6
[Feature] add sparse embedding. ( #1497 )
...
* add sparse embedding.
* fix
* add test.
* man fixes.
* many fixes
* fix sparse emb.
* fix.
* fix lint.
* fix lint.
* fix kvstore.
* expose DistTensor.
* test sparse embeddings.
* add attach_grad to the backends.
* remove part_id
* fix.
* move backward computation.
* move more computation to backend.
* fix a bug when applying learning rate.
* fix a few things.
* fix a few things.
* add docstring
* fix.
* apply no_grad.
* fix tests.
* fix for other frameworks.
* add examples in docstring.
2020-07-03 10:42:09 -07:00
177011744d
[Test] Fix test of distributed graph ( #1748 )
...
* Revert "Serialize server and client start for dist_graph_store test (#1736 )"
This reverts commit da16ebf03d
2020-07-03 10:10:54 -07:00
168a88e570
[Sampling] NodeDataLoader for node classification ( #1635 )
...
* neighbor sampler data loader first commit
* more commit
* nodedataloader
* fix
* update RGCN example
* update OGB
* fixes
* fix minibatch RGCN crashing with self loop
* reverting gatconv test code
* fix
* change to new solution that doesn't require tf dataloader
* fix
* lint
* fix
* fixes
* change doc
* fix docstring
* docstring fixes
* return seeds and input nodes from data loader
* fixes
* fix test
* fix windows build problem
* add pytorch wrapper
* fixes
* add pytorch wrapper
* add unit test
* add -1 support to sample_neighbors & fix docstrings
* docstring fix
* lint
* add minibatch rgcn evaluations
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Tong He <hetong007@gmail.com >
2020-07-02 16:31:05 +08:00
da16ebf03d
Serialize server and client start for dist_graph_store test ( #1736 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-07-02 15:28:26 +08:00
087992f14a
update ( #1737 )
2020-07-02 13:35:57 +08:00
9eb0efcf2a
[KVStore] Test multi-server and multi-client ( #1732 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
2020-07-01 20:26:40 -07:00
2c88f7c0b0
[Distributed] Split data to ensure data locality for multiple clients ( #1710 )
...
* fix.
* fix tests.
* fix
* add tests.
* fix.
* have default rank.
* add comment.
* fix test.
* remove check
* simplify code.
* add test.
* split data evenly.
* simplify the distributed training code.
* add comments.
* add comments.
2020-07-01 13:03:40 -07:00
18a26fcfb1
[Data] Support HeteroGraph save/load ( #1526 )
...
* 111
* add history version test
* fix
* 111
* save
* ``
* fix1
* 111
* add save heterograph
* lint
* lint
* add tests
* minor fix
* fix
* docs
* add format tets
* use unique_ptr
* fix
* fix interface
* 111
* 111
* fix
* lint
* fix
* add support to s3
* fix
* fix
* fix leak
* fix
* fix docs
* fix
* linlt
* fix
* fix
* fix
* address comment
* address comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-30 13:34:50 +08:00
dd8d5289ef
[rpc] Add num_clients and sort client ID by machine-id ( #1713 )
...
* add num_clients to kvstore
* update
* update
* update
* update
* update
* update
* update
* fix lint
* update
* update
* update
* add test
* update
* update
* update
* update
* fix test
* update
* update
* update
* update
* update
* update
* update
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-28 18:40:38 -07:00
0015eff787
[DGLHeteroGraph] Add unittest for srcdst Order for DGLHeteroGraph.all_edges ( #1677 )
...
* Fix (#1 )
* Update
* Fix
* Fix
* Update
* Fix
* Fix
* Fix
* Fix
* Fixg
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Add test case
* Trigger CI (#2 )
* Update
* Fix
* Fix
* Update
* Fix
* Fix
* Fix
* Fix
* Fixg
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Add test case
* test
* Fix (#3 )
* Update
* Fix
* Fix
* Update
* Fix
* Fix
* Fix
* Fix
* Fixg
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Add test case
* test
* Fix
* Fix
* Roll back
* Roll back
* CI
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-06-29 01:08:03 +08:00
5747542f35
[Kernel] Migrate batching/unbatching on adjlist to CSR/COO ( #1687 )
...
* start
* coo csr union partition
* lint
* lint
* lint
* Add matrix->data transform
* update
* Fix window compile
* Add CSR support for DisjointPartition
* lint
* Fix
* Use IdArray Op
* Concat ready
* Fix and all pass
* resolve comments
* Add union COO C++ test
* Add C++ test for csr
* lint
* triger
* Update include
* Fix merge
* test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-28 23:33:55 +08:00
55072b4e83
[Feature] dgl.subgraph support Bool mask tensor as index. ( #1709 )
...
* dgl.subgraph support using Bool mask tensor
* support numpy boo tensor and add some test
* Update
* Fix docstring
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-28 22:08:04 +08:00
870da747ea
[CUDA][Kernel] More CUDA kernels; Standardize the behavior for sorted COO/CSR ( #1704 )
...
* add cub; array cumsum
* CSRSliceRows
* fix warning
* operator << for ndarray; CSRSliceRows
* add CSRIsSorted
* add csr_sort
* inplace coosort and outplace csrsort
* WIP: coo is sorted
* mv cuda_utils
* add AllTrue utility
* csr sort
* coo sort
* coo2csr for sorted coo arrays
* CSRToCOO from sorted
* pass tests for the new kernel changes
* cannot use inplace sort
* lint
* try fix msvc error
* Fix g.copy_to and g.asnumbits; ToBlock no longer uses CSC
* stash
* revert some hack
* revert some changes
* address comments
* fix
* fix to_block unittest
* add todo note
2020-06-28 18:37:28 +08:00
da8632cafe
fix parallel msbuild ( #1717 )
2020-06-28 17:50:37 +08:00
2f983abe64
fix knn graph bugs ( #1716 )
2020-06-28 17:31:38 +08:00
04d4680d5b
[Distributed] add constraints for metis partitioning. ( #1708 )
...
* add constraint.
* fix bugs.
* temp fix.
* fix lint
* add balance_ntypes and balance_edges
* add comments.
* fix.
* fix
2020-06-27 21:27:18 -07:00
45e1333e13
[Distributed] Optimize distributed sampling ( #1700 )
...
* optimize sampling neighbors from the local partition.
* fix minior bugs.
* fix lint
* overlap local sampling and remote sampling.
* fix.
* fix lint.
* fix a potential bug.
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-06-26 17:05:29 -07:00
4b96addca0
[BUGFIX] Fix bugs in range partitioning. ( #1703 )
...
* Fix bugs in partitioning.
* add comment.
* fix lint.
* fix test.
* add more tests.
* simplify the fix
* fix.
2020-06-26 15:59:45 -07:00
b742c559fe
[Feature]heterograph.edata and heterograph.ndata support multiple ntype and etype ( #1673 )
...
* g.edata and g.ndata support multiple ntype and etype
* Add test case
* support g.srcdata and g.dstdata
* Fix test
* lint
* Fix
* Fix some comments
* lint
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2020-06-26 21:11:22 +08:00
fa1478a4bd
[BUGFIX] fix a bug in partition book. ( #1698 )
...
* fix a bug.
* add msg for assert
2020-06-24 12:38:58 -07:00
b6c4deb4e9
[NN] ChebConv update ( #1460 )
...
* New version for the convolutional layer
* Minor changes
* Minor changes
* Update python/dgl/nn/pytorch/conv/chebconv.py
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com >
* Resolved variable miss-naming, import simplifying and raising warning
* Added dg_warnings instead of warnings.warn
* add doc
* upd
* upd
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com >
Co-authored-by: yzh119 <expye@outlook.com >
2020-06-24 15:59:44 +08:00
86c243b45f
Fix ( #1684 )
2020-06-23 16:15:53 +08:00
421763fb67
[Bugfix] Fix #1641 ( #1678 )
...
* fix #1641
* lint
2020-06-23 11:45:55 +08:00
071cba1f2b
[kernel] New SpMM & SDDMM kernel on CPU and CUDA ( #1644 )
...
* udp
* simplify
* sddmm dot cpu
* upd
* format
* upd
* compatible with MJ's PR
* lint
* upd
* upd
* upd
* python end
* upd
* upd
* lint
* lint
* upd
* upd
* upd
* upd
* upd
* lint
* fix mxnet
* upd
* lint
* use minjie's ptr
* macro
* upd
* reorg
* lint
* fix corner cases
* upd
* enrich cpu docs
* upd
* upd
* lint
* lint
* pylint
* sx review
* improve docstring
* python doc
* upd
* restructure
* lint
* upd test
* upd
* pylint
* fix corner cases and test
2020-06-22 23:18:32 +08:00
3d47693b1f
[Op] Farthest Point Sampler in Cpp and CUDA ( #1630 )
...
* working framework without actual algorithm logic
* rename
* fix
* fps passes compilation
* correct algorithm
* add cuda implementation
* update random start
* before refactor
* pass compilation but cuda not working
* working
* code working, will add docstring
* add mxnet support
* update docstring
* update doc and test
* cpplint
* cpcplint
* pylint
* temporary fix
* fix for win64
* fix unitetest
* fix
* fix
* remove comment
* move to geometry package
* remove redundant include
* add docstrings and comments
* add proof
* add validity check
2020-06-22 00:52:20 +08:00
c8b18b793c
[RPC] Support null NDArray in RPC communication ( #1653 )
2020-06-21 17:20:58 +08:00
f1b19a6b67
[CUDA] Many CUDA operators; Prepare for DGLGraph on CUDA ( #1660 )
...
* add cuda utils; change g.to; add g.device
* split array.h into several headers
* cuda index select
* file
* three cuda kernels
* add cuda elementwise arith and several others
* cuda CSRIsNonZero
* fix lint
* lint
* lint
* fix bug in changing ctx to property
* address comments
* remove unused codes
* address comments
2020-06-19 15:44:42 +08:00
42b0c38f81
[Kernel] Migrate traversal on adjlist to CSR ( #1650 )
...
* traversal to new framework
* add new
* Fix compile
* Pass test
* keep old version
* lint
* lint
* Fix
* Fix
* Fix compatability with new master
* Fix test and tutorials
* Update according to comments
* Fix test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-19 13:25:18 +08:00
fb248b678a
[RPC] Sampling over RPC ( #1616 )
...
* fix
* test1111
* 111
* 111
* fff
* lint
* 111
* lint
* lint
* 111
* fijx
* 111
* fix
* 111
* commit
* 111
* 111
* lint
* fix typo
* fix
* lint
* fix
* 111
* support mxnet
* support mxnet
* lint
* remove print
* fix
* fix test.
* fix test.
* fix test.
* try to fix an undetermistic error.
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-18 12:05:23 -07:00
ff5b5a4a10
[Distributed] support no shared memory. ( #1663 )
...
* support no shared memory.
* add test.
* add CAPI to check existence of shared memory.
* revert the change in src/runtime/ndarray.cc
* update docstring.
* fix compile.
2020-06-18 09:42:11 -07:00
0a4e8b32b6
[Optimization] Optimize pickling of heterograph ( #1570 )
...
* update
* update
* update
* update
* update
* update
* backward compatibility
* update
* update
* update
* update
* update
* test
* test
* update
* update
* update
* update
* update
* update
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-18 19:43:42 +08:00
b9ec3e36ec
[BUGFIX] reshuffle node features in range partitioning. ( #1659 )
...
* fix a bug.
* fix test.
* simplify.
* fix test.
* test edge features.
* fix.
2020-06-17 22:20:17 -07:00
b372b3c796
[KVStore] Add fast-pull for kvstore ( #1647 )
...
* add fast-pull
* update
* add fast-pull
* update
* update
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* add omp
* update
* update
2020-06-17 13:50:25 +08:00
e8a56dc1b6
[KVStore] make pull/push handler per tensor. ( #1646 )
...
* make pull/push handler per tensor.
* update.
2020-06-15 23:22:49 -07:00
41349dcef5
[Feature] Range partition ( #1522 )
...
* add reorder immutable graph.
* add python API.
* add reorder for csr.
* remove gk version.
* fix
* add cpp test.
* bug fixes
* fix tests.
* fix bugs and add check
* fix test.
* add omp.
* add comments.
* add coo reorder.
* fix a bug.
* handle reorder for different graph structues.
* fix lint.
* fix.
* add original ids.
* reshuffle nodes before metis partition.
* inner nodes are in contiguous Id range.
* reshuffle nodes/edges when partitioning.
* load partition return graph partition book.
* use inner_node/inner_edges
* add and test range partition book.
* count inner_edge correctly.
* fix lint.
* fix lint.
* fix lint.
* fix errors.
* fix errors.
* fix for TF.
* fix.
* fix.
* change docstring.
* support logical and.
* add comments.
* avoid copy.
* fix
* update docstring.
* fix a bug.
* add range search.
* fix
* fix a bug.
* add more tests.
* load graph partition book.
* support shared memory for range partition book.
* fix a bug.
* fix.
* fix lint.
* remove check
* fix test.
* remove num_nodes and num_edges
* fix lint.
* fix graph partition book.
* address comments.
* use makedirs.
* fix compile
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-06-15 11:43:04 -07:00
3e72c53ae0
[Bug?] Fix #1563 ( #1642 )
2020-06-15 22:00:21 +08:00
d6d517bb02
[Kernel] CUDA CSR2COO COOSort COO2CSR ( #1620 )
...
* add cuda source
* moving codes from kernel2 branch
* operator overloading
* Better error message for unsupported device
* fix c tests
* coo sort using cusparse
* move test_rpc to distributed
* lint
* address comments and add utests
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-15 14:35:12 +08:00
61f007c43b
[NN] Add low memory support for nn.RelGraphConv ( #1631 )
...
* Add low memory support for relgraphconv
y
* lint
* Add tf support
* Fix lint
* Fix mxnet
* Fix mxnet
* Fix
* minor fix
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-15 13:49:11 +08:00
fe207b4564
[HeteroGraph] Improve Error Messages for find_edges ( #1626 )
...
* Improve error message
* Fix
* Fix
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-11 21:29:47 +08:00
77006db209
[CI] Use spawn in unittest instead of fork ( #1627 )
...
* update
* update
* update
* update
* update
* update
* fix ci using spawn
Co-authored-by: aksnzhy <mctt90@gmail.com >
2020-06-11 18:15:40 +08:00
0c313e51e6
[rpc] Move test_rpc.py to distributed and use dynamic port binding ( #1623 )
...
* update
* update
* update
* update
* update
* update
* update
* update
2020-06-11 16:01:05 +08:00
a67ba946bc
[RPC] Add send_request_to_machine() and remote_call_to_machine() ( #1619 )
...
* add send_request_to_machine()
* update
* update
* update
* update
* update
* update
* fix lint
* update
2020-06-11 13:04:06 +08:00
50962c433f
Refactor distributed graph store ( #1597 )
...
* refactor graph store for new kvstore.
* fix kvstore.
* fix.
* fix lint complains.
* add docstring.
* small fix in graph partition book.
* fix.
* remove tests
* disable sampling.
* Revert "disable sampling."
This reverts commit 70451008f61ff1481d5dadbf8bd199470aee559d.
* Revert "remove tests"
This reverts commit 1394364243bdd73b669abde6193a74e2cda5521d.
2020-06-10 18:51:32 -07:00
48cc8fa803
[KVStore] Update kvstore test ( #1607 )
...
* remove freeze
* update
* update
* fix lint
* update test
* update
2020-06-09 16:25:35 +08:00
64f49703af
[KVStore] Re-write kvstore using DGL RPC infrastructure ( #1569 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update init_data
* update server_state
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* debug init_data
* update
* update
* update
* update
* update
* update
* test get_meta_data
* update
* update
* update
* update
* update
* debug push
* update
* update
* update
* update
* update
* update
* update
* update
* update
* use F.reverse_data_type_dict
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* update
* fix lint
* update
* fix lint
* update
* update
* update
* update
* fix test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* set random seed
* update
2020-06-04 16:48:53 +08:00
a936f9d9a6
[Hetero] Allow Batching Graphs with Zero Nodes/Edges ( #1575 )
...
* Allow batching graphs with zero nodes and edges
* Fix the case without node/edge features
* Fix lint
2020-06-02 13:08:16 +08:00
b9e4a5b742
[Test] add regression tests for graph partitioning. ( #1561 )
...
* add tests.
* 111
* fix
* Update asv.conf.json
* fix.
* benchmark partition with livejournal.
* fix benchmark
* fix.
* fix.
* remove ogb
* Revert "Update asv.conf.json"
This reverts commit dd327a5564f4ef01795e444e79b17265b9c8b391.
* change branch
* depend pandas
* Revert "change branch"
This reverts commit 1d4f93756492a93f2e3cde07229a59a31a8c380b.
* Update README.md
Co-authored-by: VoVAllen <jz1749@nyu.edu >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-05-28 22:28:57 +08:00
7639b5e70c
[Fix] New StreamWithBuffer interface ( #1557 )
...
* WIP: rpc components
* client & server
* move network package to rpc
* fix include
* fix compile
* c api
* wip: test
* add basic tests
* missing file
* [RPC] Zero copy serializer (#1517 )
* zerocopy serialization
* add test for HeteroGraph
* fix lint
* remove unnecessary codes
* add comment
* lint
* lint
* disable pylint for now
* add include for win
* windows guard
* lint
* lint
* skip test on windows
* refactor
* add comment
* fix
* comment
* 1111
* fix
* Update Jenkinsfile
* [RPC] Implementation of RPC infra (#1544 )
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* remove client.cc and server.cc
* fix lint
* update
* update
* fix linr
* update
* fix lint
* update
* update
* update
* update
* update
* update
* update test
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update comment
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Refactor StreamWithBuffer (#1550 )
* refactor
* fix with new interface
* remove copy
* fix
* remove comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-05-27 01:54:25 +08:00
3e69692233
[RPC] New RPC infrastructure. ( #1549 )
...
* WIP: rpc components
* client & server
* move network package to rpc
* fix include
* fix compile
* c api
* wip: test
* add basic tests
* missing file
* [RPC] Zero copy serializer (#1517 )
* zerocopy serialization
* add test for HeteroGraph
* fix lint
* remove unnecessary codes
* add comment
* lint
* lint
* disable pylint for now
* add include for win
* windows guard
* lint
* lint
* skip test on windows
* refactor
* add comment
* fix
* comment
* 1111
* fix
* Update Jenkinsfile
* [RPC] Implementation of RPC infra (#1544 )
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* remove client.cc and server.cc
* fix lint
* update
* update
* fix linr
* update
* fix lint
* update
* update
* update
* update
* update
* update
* update test
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update comment
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-05-22 18:48:37 +08:00
70dc2ee9c6
[Feature] Add a request_format api and enrich related docstring. ( #1528 )
...
* upd
* upd
* lint
* fix
* bloody lint
2020-05-16 01:17:06 +08:00
Da Zheng