864fd4f7da
[DistDGL] fix docstring typo
2023-10-30 08:44:25 +08:00
5dfaf99e3c
[Performance] Add NCCL support ( #5929 )
...
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
2023-10-12 11:17:15 -04:00
8b23ec6082
Fix a bug in standalone mode ( #6179 )
2023-08-21 02:03:02 -07:00
6519ec2722
[BugFix] Fix API convert_dgl_partition_to_csc_sampling_graph ( #5992 )
2023-07-13 12:32:57 -04:00
0d9b6bfd5a
[TensorpipeDeprecation] remove long live server support from DistDGL ( #5931 )
2023-07-03 09:26:27 +08:00
d22049e879
[TensorpipeDeprecation] remove unused arguments from wait_for_senders() ( #5917 )
2023-06-28 16:32:29 +08:00
1e886cb9cf
[DistDGL] sort node/edge_map to obtain expected id ranges ( #5872 )
2023-06-15 13:53:53 +08:00
894ae31ffc
[DistDGL] keep net_type in public API even it is deprecated ( #5875 )
2023-06-14 21:35:41 +08:00
72ec1c95c6
[DistDGL] remove tensorpipe cpp code ( #5850 )
...
Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com >
2023-06-12 10:50:58 +08:00
0f3223310d
[DistDGL] remove tensorpipe from python APIs ( #5848 )
2023-06-09 16:35:51 +08:00
d0fa2909ef
[GraphBolt] import when necessary for DistDGL ( #5755 )
2023-06-01 11:22:12 +08:00
00972deecf
[GraphBolt] convert dgl partition to CSCSamplingGraph ( #5736 )
2023-05-29 16:49:39 +08:00
fbb26ee5f6
[Dist] fix crash issue if empty test set ( #5653 )
2023-05-11 17:21:31 +08:00
1ab0170a10
[Distributed] Ensure round-robin edge file downloads, reduce logging, other improvements. ( #5578 )
...
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
2023-04-27 11:17:55 -07:00
7f5da69709
[Misc] Rename number_of_edges and number_of_nodes to num_edges and num_nodes in dist related python files. ( #5489 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal >
2023-03-27 15:55:37 +08:00
d2ed57e94b
[DistDGL] add log when load graph and data ( #5350 )
...
* [DistDGL] add log when load graph and data
* fix lint
* refine log
---------
Co-authored-by: Israt Nisa <neesha295@gmail.com >
2023-03-10 14:21:32 +08:00
0b3a447ba4
auto format distributed ( #5317 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal >
2023-02-19 08:33:09 +08:00
3a2a5031e9
[API Deprecation]Deprecate candidates in dgl.distributed ( #5116 )
2023-01-06 16:17:44 +08:00
ab0c0ec68f
[API Deprecation] Deprecate candidates in convert module ( #4988 ) ( #5115 )
2023-01-06 14:45:52 +08:00
cd3fa03035
[Dist] init node/edge data store for Node/EdgeDataView in appropriate place ( #4906 )
...
* [Dist] instantiate NodeDataView in lazy mode
* fix test failure
* init node/edge data store at the very beginning
* fix test failures
* refine comment
* add more tests
2022-11-21 09:52:36 +08:00
6e1cc7dac2
[Dist] fix is_node check according to policy ( #4911 )
...
* [Dist] fix is_node check according to policy
* add more tests
2022-11-17 09:27:50 +08:00
3132da2826
Deprecate ( #4864 )
...
rename DGLHeteroGraph to DGLGraph
2022-11-15 10:01:01 +08:00
5f0b61b858
[Dist] enable access DistGraph.edges via canonical etype ( #4814 )
...
* [Dist] enable access DistGraph.edges via canonical etype
* refine code
* refine test
* refine code
2022-11-07 21:35:41 +08:00
1e8c645d8a
[Dist] fix max length to 80 introduced in deprecate etype PR ( #4809 )
2022-11-04 11:48:36 +08:00
ed8e9c4440
[Dist] deprecate etype and always use canonical etype for partition and load ( #4777 )
...
* [Dist] deprecate etype and always use canonical etype for partition and load
* enable canonical etypes in dist part pipeline
* resolve rebase conflicts
* fix lint
* fix test failure
* throw exception if outdated part config is loaded
* refine
* refine
* revert unnecessary change
* fix typo
2022-11-04 10:08:18 +08:00
2caa6bd02d
[Dist] Enable save and load for Distributed Optimizer ( #4752 )
...
* add save/load for distributed optimizer
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-19.ap-northeast-1.compute.internal >
2022-11-01 14:30:16 +08:00
2bca4759c1
[Sampling] Enable sampling with edge masks in sample_etype_neighbors ( #4749 )
...
* sample neighbors with masks
* oops
* refactor again
* remove
* remove debug code
* rename macro
* address comments
* more stuff
* remove
* fix
* try fix unit test
* oops
* fix test
* oops
* change name
* rename a lot of stuff
* oops
* ugh
* misc fixes
* lint
* address a lot of comments
* lint
* lint
* fix
* that was silly
* fix
* fix
* fix
* oops
2022-10-29 15:49:44 +08:00
1990e797e1
[Dist] Reduce startup overhead: sort etypes and save in specified formats ( #4735 )
...
* [Dist] reduce startup overhead: enable to save in specified formats
* [Dist] reduce startup overhead: sort partitions when generating
* sort csc/csr only whenmultiple etypes
* refine
2022-10-26 19:07:10 +08:00
83ada2964d
[Doc] Fix the doc for the partition_graph function ( #4708 )
...
* Fix the doc for the partition_graph func
* Lint for a long line
* Fix a lint
Co-authored-by: Triston Cao <tristonc@nvidia.com >
2022-10-18 12:06:05 +08:00
b13092172d
[Dist] Reduce peak memory in DistDGL ( #4687 )
...
* [Dist] Reduce peak memory in DistDGL: avoid validation, release memory once loaded
* remove orig_id from ndata/edata for partition_graph()
* delete orig_id from ndata/edata in dist part pipeline
* reduce dtype size and format before saving graphs
* fix lint
* ETYPE requires to be int32/64 for CSRSortByTag
* fix test failure
* refine
2022-10-17 13:35:55 +08:00
408eba2479
Multiple dist dl sampler ( #4704 )
...
* [Dist] enable iterate multiple dist dataloaders simultaneously
* format file
* add support for any number of dataloaders
* fix lint
* refine code
2022-10-12 15:51:45 +08:00
a208e8868b
[Misc] Black auto fix. ( #4680 )
...
* [Misc] Black auto fix.
* fix pylint disable
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
2022-10-08 14:15:56 +08:00
5bf17547f7
[DistTensor] add bitwise or function ( #4638 )
...
* first commit
* add test
* fixes
* ah this is how you skip setup
* fix
* ugh
* address comments
* i like black
2022-09-30 19:38:37 +08:00
3bcb268a36
[Doc] Update distributed chapter according to new pipeline ( #4275 )
...
* dist index chapter
* preproc chapter
* rst
* tools page
* partition chapter
* rst
* hetero chapter
* 7.1 step1
* add parmetis back
* changed based on feedback
* address comments
2022-08-17 13:30:04 +08:00
39987bc599
[Feature] enable graph partition book support canonical etypes ( #4343 )
...
* [Feature] enable graph partition book support canonical etypes
* fix lint
* fix lint
* add todo
* refine according to review comments
* fix lint
* refine naming
* revert PartitionPolicy __init__
* refine docstring
* fix doc string
2022-08-16 11:52:02 +08:00
43ba94ee49
[BugFix] fix etype check in DistGraph.edge_subgraph ( #4322 )
2022-08-03 11:56:00 +08:00
4dd16f5dd8
[BugFix] enable DistGraph.find_edge() works with str or tuple of str ( #4319 )
2022-08-01 20:47:15 +08:00
7cd531c4d6
[Dist][Optim] Change op order in SparseAdagrad to be numerically closer to PyTorch ( #4253 )
...
Signed-off-by: Serge Panev <spanev@nvidia.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-07-25 14:53:53 +08:00
8292bf329f
[Dist] remove deprecated arguments from initialize() ( #4284 )
2022-07-25 10:45:19 +08:00
740cd70610
[Doc]Fix doc typo in DistEmbedding ( #4258 )
...
* [Doc] fix docstring typo
* Update sparse_emb.py
* Update sparse_emb.py
* update link
2022-07-19 10:37:14 +08:00
14e4e1b0f7
[Dist] enable to specify sort_etype for sample_etype_neighbours ( #4212 )
...
* [Dist] enable to specify sort_etype for sample_etype_neighbours
* fix lint
* pass argument instead of env
* fix lint and doc string
* refine args
* remove unnecessary lines
* debug only
* debug add sort time log
* change interface
* fix typo
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-11 18:37:25 +08:00
c65d6fa55f
[Dist] format dtypes when loading graph in server ( #4228 )
...
* [Dist] format dtypes when loading graph in server
* add test
* refine
* add comments
2022-07-11 14:08:17 +08:00
ab1b28111d
[Dist] etype is not guaranteed to be sorted ( #4156 )
2022-06-23 09:23:08 +08:00
69226588a5
[Dist] defer to load node/edge feats ( #4143 )
...
* [Dist] defer to load node/edge feats
* fix lint
* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
* fix lint
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
2022-06-20 19:44:37 +08:00
b258729b3f
[Dist] set socket as default backend for RPC ( #4120 )
...
* [Dist] set socket as default backend for RPC
* add tests both for socket and tensorpipe
2022-06-16 13:08:19 +08:00
abcc9cce83
disable multiple groups tests due to random failure in CI ( #4101 )
2022-06-09 17:40:53 +08:00
cac3720b48
[Dist] enable time out when fetching msg ( #4043 )
...
* [ist] enable time out when fetching msg
* fix lint error
* minor refinements
* improve minor log
* fix dist test
* fix timeout issue in tensorpipe
2022-06-08 20:20:03 +08:00
53835bdb75
[Dist][BugFix] enable sampling on bipartite ( #4014 )
...
* [Dist][BugFix] enable sampling on bipartite
* add comments for tests
2022-05-18 10:14:54 +08:00
22e218d335
[Dist] Enable maximum try times for socket backend via DGL_DIST_MAX_T… ( #3977 )
...
* [Dist] Enable maximum try times for socket backend via DGL_DIST_MAX_TRY_TIMES
* reset env before/after test
* print log for info when trying to connect
* fix
* print log in python instead of cpp
2022-05-11 13:03:06 +08:00
edf2d52666
[Dist][Optim] Fixed race conditions in distributed SparseAdam and SparseAdagrad ( #3971 )
...
* * Fixed race condition bug in distributed/optim/pytorch/sparse_optim.py's SparseAdam::update, corresponding with the bug fixed in the non-distributed version in https://github.com/dmlc/dgl/pull/3013 , though using the newer Event-based approach from that corresponding function. The race condition would often result in NaNs, like the previously fixed bug. https://github.com/dmlc/dgl/issues/2760
* * Fixed race condition bug in SparseAdagrad::update corresponding with the one fixed in SparseAdam::update in the previous commit. Same info applies.
* * Fixed typo in all copies of a repeatedly-copied comment near bug fixed 3 commits ago, checking all implementations nearby for a corresponding bug. (All of them appear to have been fixed as of 2 commits ago.)
* * Removed trailing whitespace
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
2022-05-09 09:05:31 -07:00
Rhett Ying