2c88f7c0b0
[Distributed] Split data to ensure data locality for multiple clients ( #1710 )
...
* fix.
* fix tests.
* fix
* add tests.
* fix.
* have default rank.
* add comment.
* fix test.
* remove check
* simplify code.
* add test.
* split data evenly.
* simplify the distributed training code.
* add comments.
* add comments.
2020-07-01 13:03:40 -07:00
69e6afdf82
Fix crashing on SIOCGIFADDR ( #1733 )
...
During scanning the list of interfaces,
some of them can be configured invalidly:
lack of ip, link down etc.
This patch handles the error
and inform about problem with concrete interface.
2020-07-01 22:47:24 +08:00
18a26fcfb1
[Data] Support HeteroGraph save/load ( #1526 )
...
* 111
* add history version test
* fix
* 111
* save
* ``
* fix1
* 111
* add save heterograph
* lint
* lint
* add tests
* minor fix
* fix
* docs
* add format tets
* use unique_ptr
* fix
* fix interface
* 111
* 111
* fix
* lint
* fix
* add support to s3
* fix
* fix
* fix leak
* fix
* fix docs
* fix
* linlt
* fix
* fix
* fix
* address comment
* address comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-30 13:34:50 +08:00
dd8d5289ef
[rpc] Add num_clients and sort client ID by machine-id ( #1713 )
...
* add num_clients to kvstore
* update
* update
* update
* update
* update
* update
* update
* fix lint
* update
* update
* update
* add test
* update
* update
* update
* update
* fix test
* update
* update
* update
* update
* update
* update
* update
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-28 18:40:38 -07:00
5747542f35
[Kernel] Migrate batching/unbatching on adjlist to CSR/COO ( #1687 )
...
* start
* coo csr union partition
* lint
* lint
* lint
* Add matrix->data transform
* update
* Fix window compile
* Add CSR support for DisjointPartition
* lint
* Fix
* Use IdArray Op
* Concat ready
* Fix and all pass
* resolve comments
* Add union COO C++ test
* Add C++ test for csr
* lint
* triger
* Update include
* Fix merge
* test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-28 23:33:55 +08:00
55072b4e83
[Feature] dgl.subgraph support Bool mask tensor as index. ( #1709 )
...
* dgl.subgraph support using Bool mask tensor
* support numpy boo tensor and add some test
* Update
* Fix docstring
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-28 22:08:04 +08:00
2f983abe64
fix knn graph bugs ( #1716 )
2020-06-28 17:31:38 +08:00
02d3197407
[Distributed] Pytorch example of distributed GraphSage. ( #1495 )
...
* add train_dist.
* Fix sampling example.
* use distributed sampler.
* fix a bug in DistTensor.
* fix distributed training example.
* add graph partition.
* add command
* disable pytorch parallel.
* shutdown correctly.
* load diff graphs.
* add ip_config.txt.
* record timing for each step.
* use ogb
* add profiler.
* fix a bug.
* add train_dist.
* Fix sampling example.
* use distributed sampler.
* fix a bug in DistTensor.
* fix distributed training example.
* add graph partition.
* add command
* disable pytorch parallel.
* shutdown correctly.
* load diff graphs.
* add ip_config.txt.
* record timing for each step.
* use ogb
* add profiler.
* add Ips of the cluster.
* fix exit.
* support multiple clients.
* balance node types and edges.
* move code.
* remove run.sh
* Revert "support multiple clients."
* fix.
* update train_sampling.
* fix.
* fix
* remove run.sh
* update readme.
* update readme.
* use pytorch distributed.
* ensure all trainers run the same number of steps.
* Update README.md
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal >
2020-06-28 13:09:40 +08:00
04d4680d5b
[Distributed] add constraints for metis partitioning. ( #1708 )
...
* add constraint.
* fix bugs.
* temp fix.
* fix lint
* add balance_ntypes and balance_edges
* add comments.
* fix.
* fix
2020-06-27 21:27:18 -07:00
45e1333e13
[Distributed] Optimize distributed sampling ( #1700 )
...
* optimize sampling neighbors from the local partition.
* fix minior bugs.
* fix lint
* overlap local sampling and remote sampling.
* fix.
* fix lint.
* fix a potential bug.
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-06-26 17:05:29 -07:00
4b96addca0
[BUGFIX] Fix bugs in range partitioning. ( #1703 )
...
* Fix bugs in partitioning.
* add comment.
* fix lint.
* fix test.
* add more tests.
* simplify the fix
* fix.
2020-06-26 15:59:45 -07:00
b742c559fe
[Feature]heterograph.edata and heterograph.ndata support multiple ntype and etype ( #1673 )
...
* g.edata and g.ndata support multiple ntype and etype
* Add test case
* support g.srcdata and g.dstdata
* Fix test
* lint
* Fix
* Fix some comments
* lint
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2020-06-26 21:11:22 +08:00
fa1478a4bd
[BUGFIX] fix a bug in partition book. ( #1698 )
...
* fix a bug.
* add msg for assert
2020-06-24 12:38:58 -07:00
b6c4deb4e9
[NN] ChebConv update ( #1460 )
...
* New version for the convolutional layer
* Minor changes
* Minor changes
* Update python/dgl/nn/pytorch/conv/chebconv.py
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com >
* Resolved variable miss-naming, import simplifying and raising warning
* Added dg_warnings instead of warnings.warn
* add doc
* upd
* upd
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com >
Co-authored-by: yzh119 <expye@outlook.com >
2020-06-24 15:59:44 +08:00
d4daceb81d
remove message ( #1685 )
2020-06-23 22:44:49 +08:00
7afd1d0d79
[KVStore] Fix a small bug ( #1686 )
...
* update
* update
* update
* update
* update
2020-06-23 19:20:42 +08:00
86c243b45f
Fix ( #1684 )
2020-06-23 16:15:53 +08:00
85e660cb6b
[hotfix] Fix #1682 ( #1683 )
...
* upd
* upd to_hetero
2020-06-23 15:59:02 +08:00
fc7cd27598
[RPC] accelerate sampling RPC ( #1669 )
...
* support null ndarray
* speed up
* placeholder
* fix
* lint
* lint
* fix
* done
* comment
* Update sampling.py
* Update sampling.py
* Update sampling.py
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-22 11:05:06 -07:00
071cba1f2b
[kernel] New SpMM & SDDMM kernel on CPU and CUDA ( #1644 )
...
* udp
* simplify
* sddmm dot cpu
* upd
* format
* upd
* compatible with MJ's PR
* lint
* upd
* upd
* upd
* python end
* upd
* upd
* lint
* lint
* upd
* upd
* upd
* upd
* upd
* lint
* fix mxnet
* upd
* lint
* use minjie's ptr
* macro
* upd
* reorg
* lint
* fix corner cases
* upd
* enrich cpu docs
* upd
* upd
* lint
* lint
* pylint
* sx review
* improve docstring
* python doc
* upd
* restructure
* lint
* upd test
* upd
* pylint
* fix corner cases and test
2020-06-22 23:18:32 +08:00
558673e139
[Model] PointNet and PointNet++ for point cloud ( #1510 )
...
* commit patch
* commit patch
* pointnet basic
* fix data
* reorg
* reorg
* temp status
* remove validate set
* add partseg data and model
* partseg miou
* clean up
* fix loss
* network definition match paper
* fix
* fix miou
* update data format
* fix
* fix
* working pointnet ssg cls
* avoid some pytorch bug
* fix script
* update hyperparams
* add msg module
* try different dataset
* update new dataset info
* quick fix to subgraph
* fix speed
* update training
* update
* fix bs
* update docstring
* update
* update
* remove parallel reduction in fps
* switch to kernel fps, training is 30% faster
Co-authored-by: Ubuntu <ubuntu@ip-172-31-20-181.us-west-2.compute.internal >
2020-06-22 19:50:40 +08:00
3d47693b1f
[Op] Farthest Point Sampler in Cpp and CUDA ( #1630 )
...
* working framework without actual algorithm logic
* rename
* fix
* fps passes compilation
* correct algorithm
* add cuda implementation
* update random start
* before refactor
* pass compilation but cuda not working
* working
* code working, will add docstring
* add mxnet support
* update docstring
* update doc and test
* cpplint
* cpcplint
* pylint
* temporary fix
* fix for win64
* fix unitetest
* fix
* fix
* remove comment
* move to geometry package
* remove redundant include
* add docstrings and comments
* add proof
* add validity check
2020-06-22 00:52:20 +08:00
613df6f96b
[Bugfix]fix the bug for knngraph ( #1667 ) ( #1668 )
...
* fix the bug for knngraph (#1667 )
* fix the bug for knngraph (#1667 )
* fix the bug for knngraph (#1667 )
* fix the bug for knngraph (#1667 )
* fix the bug for knngraph (#1667 )
* fix the bug for knngraph (#1667 )
Co-authored-by: jiruyi <18810981229@gmail.com >
2020-06-20 16:07:28 +08:00
f1b19a6b67
[CUDA] Many CUDA operators; Prepare for DGLGraph on CUDA ( #1660 )
...
* add cuda utils; change g.to; add g.device
* split array.h into several headers
* cuda index select
* file
* three cuda kernels
* add cuda elementwise arith and several others
* cuda CSRIsNonZero
* fix lint
* lint
* lint
* fix bug in changing ctx to property
* address comments
* remove unused codes
* address comments
2020-06-19 15:44:42 +08:00
42b0c38f81
[Kernel] Migrate traversal on adjlist to CSR ( #1650 )
...
* traversal to new framework
* add new
* Fix compile
* Pass test
* keep old version
* lint
* lint
* Fix
* Fix
* Fix compatability with new master
* Fix test and tutorials
* Update according to comments
* Fix test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-19 13:25:18 +08:00
fb248b678a
[RPC] Sampling over RPC ( #1616 )
...
* fix
* test1111
* 111
* 111
* fff
* lint
* 111
* lint
* lint
* 111
* fijx
* 111
* fix
* 111
* commit
* 111
* 111
* lint
* fix typo
* fix
* lint
* fix
* 111
* support mxnet
* support mxnet
* lint
* remove print
* fix
* fix test.
* fix test.
* fix test.
* try to fix an undetermistic error.
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-18 12:05:23 -07:00
ff5b5a4a10
[Distributed] support no shared memory. ( #1663 )
...
* support no shared memory.
* add test.
* add CAPI to check existence of shared memory.
* revert the change in src/runtime/ndarray.cc
* update docstring.
* fix compile.
2020-06-18 09:42:11 -07:00
0a4e8b32b6
[Optimization] Optimize pickling of heterograph ( #1570 )
...
* update
* update
* update
* update
* update
* update
* backward compatibility
* update
* update
* update
* update
* update
* test
* test
* update
* update
* update
* update
* update
* update
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-18 19:43:42 +08:00
dc86bd421e
[Doc] fix a doc for partition_graph. ( #1657 )
...
* fix a doc for partition_graph.
* Update partition.py
* fix.
2020-06-17 22:45:05 -07:00
b9ec3e36ec
[BUGFIX] reshuffle node features in range partitioning. ( #1659 )
...
* fix a bug.
* fix test.
* simplify.
* fix test.
* test edge features.
* fix.
2020-06-17 22:20:17 -07:00
8fccdaf60d
[RPC] HandleCtrlC Register for handling Ctrl+C event. ( #1652 )
...
* update
* update
* update
* update
* fix lint
2020-06-17 14:54:13 +08:00
b372b3c796
[KVStore] Add fast-pull for kvstore ( #1647 )
...
* add fast-pull
* update
* add fast-pull
* update
* update
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* add omp
* update
* update
2020-06-17 13:50:25 +08:00
e8a56dc1b6
[KVStore] make pull/push handler per tensor. ( #1646 )
...
* make pull/push handler per tensor.
* update.
2020-06-15 23:22:49 -07:00
41349dcef5
[Feature] Range partition ( #1522 )
...
* add reorder immutable graph.
* add python API.
* add reorder for csr.
* remove gk version.
* fix
* add cpp test.
* bug fixes
* fix tests.
* fix bugs and add check
* fix test.
* add omp.
* add comments.
* add coo reorder.
* fix a bug.
* handle reorder for different graph structues.
* fix lint.
* fix.
* add original ids.
* reshuffle nodes before metis partition.
* inner nodes are in contiguous Id range.
* reshuffle nodes/edges when partitioning.
* load partition return graph partition book.
* use inner_node/inner_edges
* add and test range partition book.
* count inner_edge correctly.
* fix lint.
* fix lint.
* fix lint.
* fix errors.
* fix errors.
* fix for TF.
* fix.
* fix.
* change docstring.
* support logical and.
* add comments.
* avoid copy.
* fix
* update docstring.
* fix a bug.
* add range search.
* fix
* fix a bug.
* add more tests.
* load graph partition book.
* support shared memory for range partition book.
* fix a bug.
* fix.
* fix lint.
* remove check
* fix test.
* remove num_nodes and num_edges
* fix lint.
* fix graph partition book.
* address comments.
* use makedirs.
* fix compile
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-06-15 11:43:04 -07:00
3e72c53ae0
[Bug?] Fix #1563 ( #1642 )
2020-06-15 22:00:21 +08:00
ba2ee7bdcd
[Bug] fixing from_edge_list ( #1615 )
...
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-15 18:12:55 +08:00
d6d517bb02
[Kernel] CUDA CSR2COO COOSort COO2CSR ( #1620 )
...
* add cuda source
* moving codes from kernel2 branch
* operator overloading
* Better error message for unsupported device
* fix c tests
* coo sort using cusparse
* move test_rpc to distributed
* lint
* address comments and add utests
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-15 14:35:12 +08:00
61f007c43b
[NN] Add low memory support for nn.RelGraphConv ( #1631 )
...
* Add low memory support for relgraphconv
y
* lint
* Add tf support
* Fix lint
* Fix mxnet
* Fix mxnet
* Fix
* minor fix
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-06-15 13:49:11 +08:00
04522a76c6
[bugfix] Quick fix of #1547 ( #1600 )
...
* upd
* upd
2020-06-15 09:27:37 +08:00
57d111f9e5
[Bug] Fix broken initialization in Reddit ( #1634 )
2020-06-14 03:08:40 +08:00
02dd6d9021
[Feature] Add dot product attention ( #1604 )
...
* Add dotproduct attention
* [Feature] Add dotproduct attention
* [Feature] Add dotproduct attention
* [Feature] Add dotproduct attention
* [New] Update landing page
* [New] Update landing page
* [New] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
* [Doc] Update landing page
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-12 09:29:39 +08:00
fe207b4564
[HeteroGraph] Improve Error Messages for find_edges ( #1626 )
...
* Improve error message
* Fix
* Fix
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-06-11 21:29:47 +08:00
a67ba946bc
[RPC] Add send_request_to_machine() and remote_call_to_machine() ( #1619 )
...
* add send_request_to_machine()
* update
* update
* update
* update
* update
* update
* fix lint
* update
2020-06-11 13:04:06 +08:00
50962c433f
Refactor distributed graph store ( #1597 )
...
* refactor graph store for new kvstore.
* fix kvstore.
* fix.
* fix lint complains.
* add docstring.
* small fix in graph partition book.
* fix.
* remove tests
* disable sampling.
* Revert "disable sampling."
This reverts commit 70451008f61ff1481d5dadbf8bd199470aee559d.
* Revert "remove tests"
This reverts commit 1394364243bdd73b669abde6193a74e2cda5521d.
2020-06-10 18:51:32 -07:00
484bbcc8f1
[Test] Fix partition ci fail ( #1614 )
2020-06-10 15:44:38 +08:00
8eab08d021
[KVStore] Remove Freeze flag ( #1605 )
...
* remove freeze
* update
* update
* fix lint
2020-06-09 15:32:02 +08:00
36c7b77167
[LifeSci] Move to Independent Repo ( #1592 )
...
* Move LifeSci
* Remove doc
2020-06-05 21:28:24 +08:00
64f49703af
[KVStore] Re-write kvstore using DGL RPC infrastructure ( #1569 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update init_data
* update server_state
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* debug init_data
* update
* update
* update
* update
* update
* update
* test get_meta_data
* update
* update
* update
* update
* update
* debug push
* update
* update
* update
* update
* update
* update
* update
* update
* update
* use F.reverse_data_type_dict
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* update
* fix lint
* update
* fix lint
* update
* update
* update
* update
* fix test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* set random seed
* update
2020-06-04 16:48:53 +08:00
2b9d06b6e2
[Data] Retry downloading once if dataset initialization fails ( #1520 )
...
* dataloader guard
* more fixes
* sets overwrite to true by default
* Update utils.py
* fix
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2020-06-03 17:21:05 +08:00
90d2118da5
[Dataset] Change Cora Split ( #1583 )
...
* Use new version of Cora
* Fix import
2020-06-03 15:45:38 +08:00
Da Zheng