mirrors / dmlc--dgl

提交

2,937 提交 34 分支 42 Git 标签
test_sparse_compile
提交图

2937 次代码提交

此分支
此分支
所有分支
作者 SHA1 备注 提交日期
czkkkkkk 08c50eb7c1 [Subgraph] State in the docstring that nodes cannot have any duplicate value when inducing a subgraph (#4637) 2022-09-26 16:25:49 +08:00
czkkkkkk 544a2cd192 [Sparse] Add reduction operators (#4600) 
* [Sparse] Add reduction operators

* Fix linting problem

* Fix linting problem

* Fix linting problem

* Fix docstring

* Fix test cases

* Fix linting problem

* Skip reduction unittests
 2022-09-26 16:06:05 +08:00
Hongzhi (Steve), Chen 80b22ad881 [Readability] Polish udf.py (#4534) 
* Polish udf.py

* Update udf.py

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-26 14:50:34 +08:00
Israt Nisa 7dba19913e Sparse ops: Support elementwise ops between sparse and diagonal matrix (#4617) 
* initial commit

* address comments

* lint check

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
 2022-09-23 22:22:51 -04:00
kylasa ace76327dd Garbage Collection and memory snapshot code for debugging partitioning pipeline (target as master branch) (#4598) 
* Squashed commit of the following:

commit e605a550b3783dd5f24eb39b6873a2e0e79be9c7
Author: kylasa <kylasa@gmail.com>
Date:   Thu Sep 15 14:45:39 2022 -0700

    Delete pyproject.toml

commit f2db9e700d817212b67b5227f6472d218f0c74f2
Author: kylasa <kylasa@gmail.com>
Date:   Thu Sep 15 14:44:40 2022 -0700

    Changes suggested by isort program to sort imports.

commit 5a6078beac6218a4f1fb378c169f04dda7396425
Author: kylasa <kylasa@gmail.com>
Date:   Thu Sep 15 14:39:50 2022 -0700

    addressing code review comments from the CI process.

commit c8e92decb7aebeb32c7467108e16f058491443ab
Author: kylasa <kylasa@gmail.com>
Date:   Wed Sep 14 18:23:59 2022 -0700

    Corrected a typo in the import statement

commit 14ddb0e9b553d5be3ed2c50d82dee671e84ad8c9
Author: kylasa <kylasa@gmail.com>
Date:   Tue Sep 13 18:47:34 2022 -0700

    Memory snapshot code for debugging memory footprint of the graph partitioning pipeline

Squashed commit done

* Addressing code review comments.

* Update utils.py

* dummy change to trigger CI tests

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
 2022-09-23 15:37:29 -07:00
Chang Liu 533afa851e [Example][Refactor] Graphsage link prediction example refactor (#4526) 
* Refactor link pred example for graphsage

* Use ogb evaluator + README update

* Update

* Add comments
 2022-09-23 16:45:32 +08:00
czkkkkkk db64cc3760 [Sparse] Add Transposition for SparseMatrix and DiagMatrix (#4597) 
* [Sparse] Add Transposition

* Fix docstring

* Fix linting problem

* Minor fix

* Minor fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-23 15:17:55 +08:00
Vibhu Jawa d04534c940 [Bugfix][Hotfix] Fix cugraph Pytorch CI failures (#4622) 
* get_more_logs

* Update cugraph_unit_test.sh

* Update test_basics.py

* Update cugraph_unit_test.sh

* Update tests/cugraph/test_basics.py

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-23 11:49:03 +08:00
Rhett Ying be11409312 [CI] Enable DGL cugaph nightly CI (#4573) 
* [CI] Enable DGL cugaph nightly CI

* fix build issue

* remove runtime
 2022-09-22 15:26:29 +08:00
Rhett Ying dc258603df update dev list 2022-09-22 14:58:50 +08:00
Tong He 33c8abb9f4 [Doc] Update Readme for example Hilander (#4587) 
The example was updated in https://github.com/dmlc/dgl/pull/3972 but the README file was not. This PR updates it to reflect the correct environment requirement.

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-22 13:54:13 +08:00
Ereboas 92bff4d2bb [Example] SEAL+NGNN for ogbl (#4550) 
* Use black for formatting

* limit line width to 80 characters.

* Use a backslash instead of directly concatenating

* file structure adjustment.

* file structure adjustment(2)

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
 2022-09-22 13:09:27 +08:00
YanJun-Zhao ec4271bf87 [Example] Refactor GNNExplainer Example (#4560) 
* debug

* debug

* readme

* fix readme

* fix readme

* Update

* Update

* update

* fix bug of syn2

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
 2022-09-21 13:34:39 +08:00
Xin Yao 880b3b1f97 [Fix] Enable lint check for cuh files and fix compiler warnings (#4585) 
* disable warning for tensorpipe

* fix warning

* enable lint check for cuh files

* resolve comments
 2022-09-21 10:09:14 +08:00
peizhou001 166b273bec add ssh port config for dispatchdata (#4557) 2022-09-20 20:41:41 +08:00
Mufei Li 73b9f3a1da [Sparse] Migrate DiagMatrix class (#4567) 
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
 2022-09-20 17:17:43 +08:00
Quan (Andy) Gan 8366774122 [Feature] Support canonical edge types in HeteroGraphConv (#4440) 
* fix

* Update hetero.py

* why did i remove this

Co-authored-by: Xin Yao <xiny@nvidia.com>
 2022-09-20 14:15:26 +08:00
Quan (Andy) Gan 5ebd3bf05a [DataLoader] Disable the usage of shared memory when persistent_workers=False on single GPU (#4497) 
* toggle shared memory usage

* Update dataloader.py

Co-authored-by: Xin Yao <xiny@nvidia.com>
 2022-09-20 11:40:33 +08:00
nv-dlasalle 2630d2eb02 [Performance][bugfix] Implement is_unibipartite in C++ with caching. (#4556) 
* updates

* Enable caching C++ result

* Add missing docstring

* Remove unused function

* Add unit test

* Address comments
 2022-09-19 16:46:27 -07:00
Xin Yao cded5b80fe [Feature] Bump DLPack to v0.7 and decouple DLPack from the core library (#4454) 
* rename `DLContext` to `DGLContext`

* rename `kDLGPU` to `kDLCUDA`

* replace DLTensor with DGLArray

* fix linting

* Unify DGLType and DLDataType to DGLDataType

* Fix FFI

* rename DLDeviceType to DGLDeviceType

* decouple dlpack from the core library

* fix bug

* fix lint

* fix merge

* fix build

* address comments

* rename dl_converter to dlpack_convert

* remove redundant comments
 2022-09-19 16:02:43 +08:00
Hongzhi (Steve), Chen f1689ad0e1 [Misc]Add pyproject.toml for autopep8 & black. (#4543) 
* Add pyproject.toml for autopep8.

* Add pyproject.toml for autopep8.

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-19 11:38:58 +08:00
Hongzhi (Steve), Chen 36dbccee12 Delete pylint.yml 2022-09-16 18:35:41 +08:00
Hongzhi (Steve), Chen d0b04f7eec Update pylint.yml 2022-09-16 18:29:58 +08:00
Hongzhi (Steve), Chen e069549a74 Update pylint.yml 2022-09-16 18:17:39 +08:00
Hongzhi (Steve), Chen 07a465b8be Update pylint. (#4576) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 17:37:09 +08:00
Hongzhi (Steve), Chen 11a679c085 [Misc] Use another workflow. (#4575) 
* Update pylint.

* Use another workflow.

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 17:33:17 +08:00
Hongzhi (Steve), Chen d08b28dec1 Update pylint. (#4574) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 17:11:12 +08:00
Hongzhi (Steve), Chen 5f5739192a [Misc] Update the python version in Pylint workflow for flake8. (#4572) 
* remove

* Add pylint.

* Change the python version for pylint.

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 17:01:46 +08:00
Hongzhi (Steve), Chen 97b3c5c8a7 Add Pylint workflow for flake8. (#4571) 
* remove

* Add pylint.

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 16:50:19 +08:00
Hongzhi (Steve), Chen 565bbdbb66 remove (#4570) 
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 16:34:29 +08:00
Rhett Ying a3e00c4453 [CI] refine stage logic (#4565) 
* [CI] refine stage logic

* refine

* refine
 2022-09-16 16:28:04 +08:00
Hongzhi (Steve), Chen 795371ad0e [Misc] Try use official pylint workflow. (#4568) 
* polish update_version

* update pylint workflow.

* add

* revert.

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 16:10:04 +08:00
Hongzhi (Steve), Chen ec96407c3c [Misc] Add flake8 lint workflow. (#4566) 
* Add pyproject.toml for autopep8.

* Add pyproject.toml for autopep8.

* Add flake8 annotation in workflow.

* remove

* add

* clean up

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
 2022-09-16 15:47:41 +08:00
Rhett Ying 7f984b64b2 Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563) 
This reverts commit ec171c648a.
 2022-09-16 13:57:50 +08:00
Vibhu Jawa ec171c648a [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525) 
* Added cugraph nightly scripts

* Removed nvcr.io//nvidia/pytorch:22.04-py3 reference

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
 2022-09-16 11:28:30 +08:00
Rhett Ying 1bc77c5e61 [examples]educe memory consumption (#4558) 
* [examples]educe memory consumption

* reffine help message

* refine
 2022-09-15 19:27:20 +08:00
Xin Yao 9a00cf194f [Feature] Import PyTorch's CUDA stream management (#4503) 
* add set_stream

* add .record_stream for NDArray and HeteroGraph

* refactor dgl stream Python APIs

* test record_stream

* add unit test for record stream

* use pytorch's stream

* fix lint

* fix cpu build

* address comments

* address comments

* add record stream tests for dgl.graph

* record frames and update dataloder

* add docstring

* update frame

* add backend check for record_stream

* remove CUDAThreadEntry::stream

* record stream for newly created formats

* fix bug

* fix cpp test

* fix None c_void_p to c_handle
 2022-09-15 16:54:52 +08:00
Rhett Ying 099b173f6f [DistPart] expose timeout config for process group (#4532) 
* [DistPart] expose timeout config for process group

* refine code

* Update tools/distpartitioning/data_proc_pipeline.py

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-15 15:28:36 +08:00
Israt Nisa cf03592743 Move mock version of dgl_sparse library to DGL main repo (#4524) 
* init

* Add api doc for sparse library

* support op btwn matrices with differnt sparsity

* Fixed docstring

* addresses comments

* lint check

* change keyword format to fmt

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
 2022-09-14 12:08:20 -04:00
Hongzhi (Steve), Chen 701c6fe754 [Readability] Auto fix setup.py and update-version.py (#4446) 
* Auto fix update-version

* Auto fix setup.py

* Auto fix update-version

* Auto fix setup.py

* [Doc] Change random.py to random_partition.py in guide on distributed partition pipeline (#4438)

* Update distributed-preprocessing.rst

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>

* fix unpinning when tensoradaptor is not available (#4450)

* [Doc] fix print issue in tutorial (#4459)

* [Example][Refactor] Refactor RGCN example (#4327)

* Refactor full graph entity classification

* Refactor rgcn with sampling

* README update

* Update

* Results update

* Respect default setting of self_loop=false in entity.py

* Update

* Update README

* Update for multi-gpu

* Update

* [doc] fix invalid link in user guide (#4468)

* [Example] directional_GSN for ogbg-molpcba (#4405)

* version-1

* version-2

* version-3

* update examples/README

* Update .gitignore

* update performance in README, delete scripts

* 1st approving review

* 2nd approving review

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Clarify the message name, which is 'm'. (#4462)

Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>

* [Refactor] Auto fix view.py. (#4461)

Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* [Example] SEAL for OGBL (#4291)

* [Example] SEAL for OGBL

* update index

* update

* fix readme typo

* add seal sampler

* modify set ops

* prefetch

* efficiency test

* update

* optimize

* fix ScatterAdd dtype issue

* update sampler style

* update

Co-authored-by: Quan Gan <coin2028@hotmail.com>

* [CI] use https instead of http (#4488)

* [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487)

* [BugFix] fix crash due to incorrect dtype in dgl.to_block()

* fix test failure in TF

* [Feature] Make TensorAdapter Stream Aware (#4472)

* Allocate tensors in DGL's current stream

* make tensoradaptor stream-aware

* replace TAemtpy with cpu allocator

* fix typo

* try fix cpu allocation

* clean header

* redirect AllocDataSpace as well

* resolve comments

* [Build][Doc] Specify the sphinx version (#4465)

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* reformat

* reformat

* Auto fix update-version

* Auto fix setup.py

* reformat

* reformat

Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Chang Liu <chang.liu@utexas.edu>
Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
 2022-09-14 13:36:18 +08:00
Rhett Ying 11532198b6 [CI] only known devs are authorized to trigger CI (#4518) 
* [CI] only known devs are authorized to trigger CI

* fix if author is null

* add comments
 2022-09-08 15:37:01 +08:00
ndickson-nvidia 684f66b7aa [Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323 #4316) (#4389) 
* * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph
* Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information

* * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing

* * Added test to ensure dgl.remove_self_loop function correctly updates batch information

* * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph

* * Formatting

* * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph

* * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass
* Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed

* * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments

* * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test

* * Added doc comments for new parameters

* * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph
* Added test cases for more than k coincident points

* * Updated doc comments for output_batch parameters for clarity

* * Linter formatting fixes

* * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu

* * Rewording in doc comments

* * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-07 08:35:26 +08:00
Chang Liu 1c9d2a0302 [Feature] Unify the cuda stream used in core library (#4480) 
* Use an internal cuda stream for CopyDataFromTo

* small fix white space

* Fix to compile

* Make stream optional in copydata for compile

* fix lint issue

* Update cub functions to use internal stream

* Lint check

* Update CopyTo/CopyFrom/CopyFromTo to use internal stream

* Address comments

* Fix backward CUDA stream

* Avoid overloading CopyFromTo()

* Minor comment update

* Overload copydatafromto in cuda device api

Co-authored-by: xiny <xiny@nvidia.com>
 2022-09-06 10:14:55 +08:00
peizhou001 62af41c245 [Bug] Enable turn on/off libxsmm at runtime (#4455) 
* enable turn on/off libxsmm at runtime by adding a global config and related API


Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal>
 2022-09-05 17:38:34 +08:00
Israt Nisa 06608f847c [Cleanup] Remove duplicate entries of CUB submodule (issue# 4395) (#4499) 
* remove third_part/cub

* remove from third_party

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
 2022-09-05 13:23:08 +08:00
nv-dlasalle 806def67a1 [Cleanup] Remove async_transferer (#4505) 
* Remove async_transferer

* remove test

* Remove AsyncTransferer

Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Xin Yao <yaox12@outlook.com>
 2022-09-05 11:46:06 +08:00
Chang Liu fce9614089 [Example][Refactor] Refactor graphsage multigpu and full-graph example (#4430) 
* Add refactors for multi-gpu and full-graph example

* Fix format

* Update

* Update

* Update
 2022-09-05 09:05:56 +08:00
Quan (Andy) Gan 2dd06b2152 [Build][Doc] Specify the sphinx version (#4465) 
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2022-09-01 13:52:08 +08:00
Xin Yao 2b76674002 [Feature] Make TensorAdapter Stream Aware (#4472) 
* Allocate tensors in DGL's current stream

* make tensoradaptor stream-aware

* replace TAemtpy with cpu allocator

* fix typo

* try fix cpu allocation

* clean header

* redirect AllocDataSpace as well

* resolve comments
 2022-08-31 20:09:40 +08:00
Rhett Ying 468c0ca4b4 [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487) 
* [BugFix] fix crash due to incorrect dtype in dgl.to_block()

* fix test failure in TF
 2022-08-31 16:09:46 +08:00