文件历史

3 次代码提交

作者 SHA1 备注 提交日期
Rhett Ying 9731e02325 [Dist] enable to chunk node/edge data into arbitrary number of chunks (#4930)
* [Dist] enable to chunk node/edge data into arbitrary number of chunks

* [Dist] enable to split node/edge data into arbitrary parts

* refine code

* Format boolean to uint8 forcely to avoid dist.scatter failure

* convert boolean to int8 before scatter and revert it after scatter

* refine code

* fix test

* refine code

* move test utilities into utils.py

* update comment

* fix empty data

* update

* update

* fix empty data issue

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* remove unnecessary shuffle data

* separate array_split into standalone utility

* add example

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2022-12-15 10:01:39 +08:00
Rhett Ying 32dc1af67e [Dist] generate partition meta for ParMETIS pipeline (#5020)
* [Dist] generate partition meta for ParMETIS
2022-12-14 21:15:54 +08:00
kylasa eae6ce2a4f ParMETIS wrapper script to enable ParMETIS to process chunked dataset format (#4605)
* Creating ParMETIS wrapper script to run parmetis using one script from user perspective

* Addressed all the CI comments from PR https://github.com/dmlc/dgl/pull/4529

* Addressing CI comments.

* Isort, and black changes.

* Replaced python with python3

* Replaced single quote with double quotes per suggestion.

* Removed print statement

* Addressing CI Commets.

* Addressing CI review comments.

* Addressing CI comments as per chime discussion with Rui

* CI Comments, Black and isort changes

* Align with code refactoring, black, isort and code review comments.

* Addressing CI review comments, and fixing merge issues with the master branch

* Updated with proper unit test skip decorator
2022-10-03 12:23:57 -07:00