mirrors / dmlc--dgl

提交

3,326 提交 34 分支 42 Git 标签
peizhou/negative__patch
提交图

3326 次代码提交

此分支
此分支
所有分支
作者 SHA1 备注 提交日期
Hongzhi (Steve), Chen 529b26627b autofix (#5337) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 20:12:43 +08:00
Hongzhi (Steve), Chen 5bfa813761 [Misc] All overrun on master CI. (#5360) 
* exclude_master

* fix

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 20:08:01 +08:00
Hongzhi (Steve), Chen e41ce0c66a [Misc] Update Jenkins status. (#5356) 
* test

* blabla

* add

* reformat

* balbla

* rollback

* remove

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 18:45:00 +08:00
Rhett Ying 197f1d25ed [CI] use more light-weight node for lint check (#5358) 2023-02-21 18:44:30 +08:00
peizhou001 ed2e540911 [Enhancement] Change id hash map (#5304) 
* change concurrent id hash map
 2023-02-21 16:47:03 +08:00
Hongzhi (Steve), Chen 4706b6bf72 [Misc] Autoformat python dgl. (#5338) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 15:24:06 +08:00
Hongzhi (Steve), Chen 71f7e96aeb [Misc] Abort previous CI. (#5346) 
* Abort previous CI

* script

* newtry

* more

* fix

* comment

* add notes

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 15:09:30 +08:00
Hongzhi (Steve), Chen 27b3cd87e2 fix (#5349) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 14:02:25 +08:00
Hongzhi (Steve), Chen b8ed4d9ba3 [Misc] Revert status.py (#5348) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 14:00:08 +08:00
Hongzhi (Steve), Chen e4e0132972 [Misc] Add debug output to avoid silent failure. (#5347) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 13:42:00 +08:00
Hongzhi (Steve), Chen 0127a2ea4d [Misc] Update Jenkins status. (#5345) 
* update

* polish

* polish

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 12:29:00 +08:00
Hongzhi (Steve), Chen e330954ca6 [Misc] Print jenkins response for test. (#5344) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 11:32:29 +08:00
Hongzhi (Steve), Chen deed945625 [Misc] Revert jenkins update change and add build url in log. (#5343) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 11:18:05 +08:00
Hongzhi (Steve), Chen f87e98e4b5 Update Jenkins response status (#5342) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 11:06:56 +08:00
Hongzhi (Steve), Chen 36065bfd4e autofix (#5336) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 09:39:35 +08:00
Hongzhi (Steve), Chen a566b60be4 auto-fix (#5331) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-21 01:25:16 +08:00
Hongzhi (Steve), Chen d18274885d autofix2 (#5333) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-20 23:27:27 +08:00
Hongzhi (Steve), Chen 3e5137feaf [Misc] Autoformat python dgl (Part 2). (#5332) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-20 20:56:25 +08:00
Hongzhi (Steve), Chen 6e58f5f1a5 [Misc] Auto format with manual fix of 3 files and add pylint: disable= too-many-lines for functional.py. (#5330) 
* blabal

* 2more

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-20 17:14:10 +08:00
Rhett Ying f17c0d6224 [CI] add boto3 to interactive with AWS services like batch (#5329) 
* [CI] add boto3 to interactive with AWS services like batch

* Update dgl_dev.yml.template
 2023-02-20 11:37:33 +08:00
Hongzhi (Steve), Chen dce899190e [Misc] Auto-reformat multiple python folders. (#5325) 
* auto-reformat

* lintrunner

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-20 10:09:22 +08:00
Hongzhi (Steve), Chen ab812179a3 [Misc] Auto reformat data/. (#5318) 
* data

* lintrunner

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 22:50:55 +08:00
Hongzhi (Steve), Chen f0759a96ee [Misc] Auto-reformat test/. (#5324) 
* auto-format-test

* more

* remove

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 22:50:37 +08:00
Hongzhi (Steve), Chen 9fe5092c17 lintrunner (#5326) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 22:50:07 +08:00
Hongzhi (Steve), Chen 9836f78e77 autoformat (#5322) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 11:24:16 +08:00
Hongzhi (Steve), Chen 704bcaf6dd examples (#5323) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 08:35:15 +08:00
Hongzhi (Steve), Chen 6bc8216118 [Misc] auto-format tools. (#5321) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 08:34:23 +08:00
Hongzhi (Steve), Chen 63ac788ff4 auto-reformat-nn (#5319) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 08:33:30 +08:00
Hongzhi (Steve), Chen 0b3a447ba4 auto format distributed (#5317) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-19 08:33:09 +08:00
Hongzhi (Steve), Chen 74c9d27d16 [Misc] Auto-format tests. (#5313) 
* [Misc] Auto-format tests.

* more

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-17 23:33:44 +08:00
Hongzhi (Steve), Chen 86193c26f7 [Misc] Reorg python test (with backend) according to the guideline. (#5312) 
* rename

* remove test transpose

* udpate

* rename

* cleanup

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-17 16:11:39 +08:00
Hongzhi (Steve), Chen d9e550369f [Misc] Reorg python test according to the guideline. (#5307) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-17 16:08:20 +08:00
Hongzhi (Steve), Chen 2bf2566fed remove (#5309) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-17 12:14:02 +08:00
Hongzhi (Steve), Chen 03dbe75808 blabal (#5306) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-17 11:17:08 +08:00
kylasa 9ce800d2f3 [DistDGL][Optimizations]Rehash code to optimize for loop (#5224) 
* Rehash code to optimize for loop

Reduced number of instructions in for loop, which exchanging edge features. This will reduce the number of times numpy's intersect1d is invoked (saving the runtime and memory overhead needs of numpy).

* Applying lintrunner patch to data_shuffle.py
 2023-02-16 09:34:07 -08:00
kylasa e25f47dea2 [DistDGL][Mem_Optimizations]Edge Ownership processes are computed on the fly when required. (#5225) 
* Edge Ownership processes are computed on the fly when required.

Earlier we were storing Edge ownership processes after the dataset was retrieved from the disk. For massively large datasets, each node can handle upto 5 Billion edges, this means storing owner process-ids will consume 5 * 8 = 40GB. This memory will be hanging around until the edges are exchanged.

To reduce the memory footprint of the pipeline, we no longer store the ownership process-ids in the 'edge_data' dictionary after reading the dataset from the disk. Instead, we compute them on the fly at the time of exchanging edges.

Another optimization is not to send/receive all the messages in a one single large message. Instead we now split the total number edges into chunks, limited by 8 GB per node. And we iterate until all the chunks are exchanged.

Once all the edges are exchanged, as a sanity check, we compute the total number of edges in the system and compare it with the original value before edge shuffling, in a final assert statement before return the result to the caller.

* Applying lintrunner patch.
 2023-02-16 09:33:40 -08:00
Songqing Zhang 1329be9606 [Misc] Fix build warnings (#5303) 
Co-authored-by: songqing.zhang <songqing.zhang@shopee.com>
 2023-02-16 16:44:50 +08:00
Hongzhi (Steve), Chen 85dfcea0bc [Dev] Create a script for kick off python test. (#5298) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-16 14:57:46 +08:00
Hongzhi (Steve), Chen ccbc30fdd2 [Dev] Create a script for building dgl from source. (#5294) 
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>
 2023-02-16 13:46:05 +08:00
nv-dlasalle 2bbca12ad5 [bugfix] Fix assertions in /src/runtime/workspace.h and expand unit tests for sparse optimizer (#5299) 
* Fix assertions for size 0 workspaces

* Expand unit test to cover case of communication

* Fixes

* Format

* Fix c++ formatting
 2023-02-16 13:43:03 +08:00
Hongzhi (Steve), Chen a5e3139155 Set USE_LIBXSMM default to OFF. (#5287) 
* Set USE_LIBXSMM default to OFF.

* update

---------

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-16 13:08:11 +08:00
Hongzhi (Steve), Chen 0552466c9c sort (#5281) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-16 13:06:46 +08:00
Hongzhi (Steve), Chen f7a0c4bbf9 remove (#5280) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-16 12:57:49 +08:00
Tingyu Wang 19b3cea166 [Model] Update CuGraphRelGraphConv to use pylibcugraphops=23.02 (#5217) 
* update cugraph_relgraphconv

* update equality test

* update cugraph rgcn example

* update RelGraphConvAgg based on latest API changes

* enable fallback option to fg when fanout is large

---------

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
 2023-02-15 20:11:20 +08:00
Quan (Andy) Gan 5f1babf0bf fix build for multiple pytorch versions (#5289) 2023-02-15 15:22:50 +08:00
Hongzhi (Steve), Chen 40c968b78b fix (#5293) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-14 17:18:45 +08:00
Quan (Andy) Gan fb8677b419 [Test] Move example test from unit test to examples (#5290) 
* move example test from unit test to examples

* oops

* oops

* ok
 2023-02-14 16:25:05 +08:00
Hongzhi (Steve), Chen 267d285597 [Dev] Create canonical dev conda environment. (#5278) 
* create_dev_conda_env

* fix

* polish

* nlabla

* cleanup

* more

* add_more

* polish

---------

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2023-02-14 16:07:07 +08:00
kylasa 432c71ef25 Code changes to fix order sensitivity of the pipeline (#5288) 
Following changes are made in this PR.
1. In dataset_utils.py, when reading edges from disk we follow the order defined by the STR_EDGE_TYPE key in the metadata.json file. This order is implicitly used to assign edgeid to edge types. This same order is used to read edges from the disk as well.
2. Now the unit test framework will also randomize the order of edges read from the disk. This is done for the edges when reading from the disk for the unit tests.

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2023-02-13 10:56:26 -08:00
Krzysztof Sadowski f5afc6eaa2 fix rounding issue (#5159) 
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
 2023-02-13 18:02:55 +08:00