 Wenxuan Cao
|
17017c2899
|
[distGB] graphbolt graph edge's mask will be filled with 0 if these edges have no mask initial (#7846)
|
2025-01-09 19:52:10 +08:00 |
|
|
Rhett Ying
|
8772c02db7
|
[DistGB] fix prob attr not exist (#7783)
|
2024-09-06 13:56:20 +08:00 |
|
|
Rhett Ying
|
12841c675b
|
[Dist] backward compatible with dgl.dataloading.DistDataLoader (#7782)
|
2024-09-06 11:21:52 +08:00 |
|
|
Rhett Ying
|
9782c021de
|
[Dist] move DistNode/EdgeDataLoader to distributed (#7755)
|
2024-08-29 16:10:05 +08:00 |
|
|
Hongzhi (Steve), Chen
|
a208e8868b
|
[Misc] Black auto fix. (#4680)
* [Misc] Black auto fix.
* fix pylint disable
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
|
2022-10-08 14:15:56 +08:00 |
|
|
Rhett Ying
|
6255c95aae
|
[Fix] too many open files (#3694)
* [Fix] too many open files
|
2022-01-28 13:24:20 +08:00 |
|
|
Jinjing Zhou
|
8bc91414e9
|
[Distributed] Fix distributed training hang with multiple samplers (#3169)
Rewrite the multiprocessing worker pool
|
2021-08-01 21:31:04 +08:00 |
|
|
Da Zheng
|
34426a98e9
|
[Distributed] Distributed heterograph training (#3069)
* support hetero RGCN.
* fix.
* simplify code.
* sample_neighbors return heterograph directly.
* avoid using to_heterogeneous.
* compute canonical etypes in advance.
* fix tests.
* fix.
* fix distributed data loader for heterograph.
* use NodeDataLoader.
* fix bugs in partitioning on heterogeneous graphs.
* fix lint.
* fix tests.
* fix.
* fix.
* fix bugs.
* fix tests.
* fix.
* enable coo for distributed.
* fix.
* fix.
* fix.
* fix.
* fix.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
|
2021-07-17 10:57:00 +08:00 |
|
|
Da Zheng
|
bfbbefa793
|
[Distributed] Fix a bug in multiprocessing sampling. (#2826)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-73-81.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
|
2021-04-08 01:29:33 -07:00 |
|
|
Da Zheng
|
97863ab85b
|
[Distributed] Cleanup the process pool correctly when the process exits. (#2781)
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
|
2021-03-29 14:53:51 -07:00 |
|
|
Da Zheng
|
25ac334403
|
[Distributed] Heterogeneous graph support (#2457)
* Distributed heterograph (#3)
* heterogeneous graph partition.
* fix graph partition book for heterograph.
* load heterograph partitions.
* update DistGraphServer to support heterograph.
* make DistGraph runnable for heterograph.
* partition a graph and store parts with homogeneous graph structure.
* update DistGraph server&client to use homogeneous graph.
* shuffle node Ids based on node types.
* load mag in heterograph.
* fix per-node-type mapping.
* balance node types.
* fix for homogeneous graph
* store etype for now.
* fix data name.
* fix a bug in example.
* add profiler in rgcn.
* heterogeneous RGCN.
* map homogeneous node ids to hetero node ids.
* fix graph partition book.
* fix DistGraph.
* shuffle eids.
* verify eids and their mappings when loading a partition.
* Id map from homogneous Ids to per-type Ids.
* verify partitioned results.
* add test for distributed sampler.
* add mapping from per-type Ids to homogeneous Ids.
* update example.
* fix DistGraph.
* Revert "add profiler in rgcn."
This reverts commit 36daaed8b660933dac8f61a39faec3da2467d676.
* add tests for homogeneous graphs.
* fix a bug.
* fix test.
* fix for one partition.
* fix for standalone training and evaluation.
* small fix.
* fix two bugs.
* initialize projection matrix.
* small fix on RGCN.
* Fix rgcn performance (#17)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal>
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix.
* fix test.
* fix lint.
* test partitions.
* remove redundant test for partitioning.
* remove commented code.
* fix partition.
* fix tests.
* fix RGCN.
* fix test.
* fix test.
* fix test.
* fix.
* fix a bug.
* update dmlc-core.
* fix.
* fix rgcn.
* update readme.
* add comments.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal>
* fix.
* fix.
* add div_int.
* fix.
* fix.
* fix lint.
* fix.
* fix.
* fix.
* adjust.
* move code.
* handle heterograph.
* return pytorch tensor in GPB.
* remove some tests in example.
* add to_block for distributed training.
* use distributed to_block.
* remove unnecessary function in DistGraph.
* remove distributed to_block.
* use pytorch tensor.
* fix a bug in ntypes and etypes.
* enable norm.
* make the data loader compatible with the old format.
* fix.
* add comments.
* fix a bug.
* add test for heterograph.
* support partition without reshuffle.
* add test.
* support partition without reshuffle.
* fix.
* add test.
* fix bugs.
* fix lint.
* fix dataset.
* fix for mxnet.
* update docstring.
* rename to floor_div
* avoid exposing NodePartitionPolicy and EdgePartitionPolicy.
* fix docstring.
* fix error.
* fixes.
* fix comments.
* rename.
* rename.
* explain IdMap.
* fix docstring.
* fix docstring.
* update docstring.
* remove the code of returning heterograph.
* remove argument.
* fix example.
* make GraphPartitionBook an abstract class.
* fix.
* fix.
* fix a bug.
* fix a bug in example
* fix a bug
* reverse heterograph sampling.
* temp fix.
* fix lint.
* Revert "temp fix."
This reverts commit c450717b9f578b8c48769c675f2a19d6c1e64381.
* compute norm.
* Revert "reverse heterograph sampling."
This reverts commit bd6deb7f52998de76508f800441ff518e2fadcb9.
* fix.
* move id_map.py
* remove check
* add more comments.
* update docstring.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal>
|
2021-01-24 19:18:00 -08:00 |
|
|
Jinjing Zhou
|
cd204a4ac9
|
[Distributed] Use barrier instead of sleep in DistDataloader (#2086)
* use barrier instead of sleep
* lint
|
2020-08-21 18:01:09 +08:00 |
|
|
Da Zheng
|
75ffc31f76
|
[Doc] Update the docstring of distributed APIs. (#2025)
* add doc.
* update DistGraph.
* add DistTensor.
* update DistEmbedding.
* add partition.py
* add sampling.
* fix.
* add graph partition book and create a base class.
* fix test.
* add rst.
* update doc rst.
* update.
* fix.
* fix docs
* update distributed tensor and embeddings.
* add checks.
* update DistGraph.
* update initialization.
* fix graph partition book.
* update graph partition book.
* update partition.
* update partition.
* fix.
* add example code.
* update DistGraph
* Update python/dgl/distributed/dist_context.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Update python/dgl/distributed/dist_context.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Update python/dgl/distributed/dist_dataloader.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Update python/dgl/distributed/dist_dataloader.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Update python/dgl/distributed/dist_dataloader.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* update initialize.
* update dataloader.
* update distgraph.
* update DistGraph.
* update DistTensor.
* update.
* more updates.
* fix lint.
* add num_nodes and num_edges
Co-authored-by: Chao Ma <mctt90@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
|
2020-08-18 11:53:45 -07:00 |
|
|
Da Zheng
|
ee30b2aadd
|
[Distributed] Fix standalone (#1974)
* fix tests.
* fix.
* remove a test.
* make code work in the standalone mode.
* fix example.
* more fix.
* make DistDataloader work with num_workers=0
* fix DistDataloader tests.
* fix.
* fix lint.
* fix cleanup.
* fix test
* remove unnecessary code.
* remove tests.
* fix.
* fix.
* fix.
* fix example
* fix.
* fix.
* fix launch script.
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>
|
2020-08-10 15:49:28 -07:00 |
|
|
Da Zheng
|
bab32d5ba1
|
[Distributed] Fix dataloader (#1970)
* fix dataloader.
* initialize iterator of DistDataloader correctly.
* update test.
|
2020-08-07 18:09:21 -07:00 |
|
|
Jinjing Zhou
|
4f499c7ffb
|
[Distributed] DistDataloader (#1901)
* 111
* 111
* fix
* 111
* fix
* 11
* fix
* lint
* Update __init__.py
* lint
* fix
* lint
* fix
* fix
* fix
* fix
* fix
* try fix
* try fix
* fix
* Revert "fix"
This reverts commit a0b954fd4e99b7df92b53db8334dcb583d6e1551.
* fixes.
* fix.
* fix test.
* fix exit.
* fix.
* fix
* fix
* lint
* lint
* lint
* fix
* Update .gitignore
* 111
* fix
* 111
* 111
* fff
* 1111
* 111
* 1325315
* ffff
* f???
* fff
* 1111
* 111
* fix
* 111
* asda
* 1111
* 11
* 123
* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊
* spawn
* 1231231
* up
* 111
* fix
* fix
* Revert "fix"
This reverts commit 7373f95312fdcaa36d2fc330bf242339e89c045d.
* fix
* fix
* 1111
* fix
* fix tests
* start kvclient as early as possible.
* lint
* fix test
* lint
* 1111
* fix
* fix
* 111
* fix
* fix
* 1
* fix
* fix
* lint
* fix
* lint
* lint
* remove quit
* fix
* lint
* fix
* fix several
* lint
* fix minor
* fix
* lint
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
|
2020-08-05 17:43:17 +08:00 |
|
|
Jinjing Zhou
|
71cf78652d
|
Revert "[Distributed] DistDataloader (#1870)" (#1876)
This reverts commit 6557291fdc.
|
2020-07-28 17:57:57 +08:00 |
|
|
Jinjing Zhou
|
6557291fdc
|
[Distributed] DistDataloader (#1870)
* 111
* 111
* fix
* 111
* fix
* 11
* fix
* lint
* Update __init__.py
* lint
* fix
* lint
* fix
* fix
* fix
* fix
* fix
* try fix
* try fix
* fix
* Revert "fix"
This reverts commit a0b954fd4e99b7df92b53db8334dcb583d6e1551.
* fixes.
* fix.
* fix test.
* fix exit.
* fix.
* fix
* fix
* lint
* lint
* lint
* fix
* Update .gitignore
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
|
2020-07-28 16:36:37 +08:00 |
|