文件历史

提交图

1069 次代码提交

作者 SHA1 备注 提交日期
Minjie Wang 49a4436ac9 [MPOps] Add edge-wise message passing operators u_op_v (#4801)
* edgewise mpops

* formatting

* black formatting

* isort

* fix lint

* remove typing module; address comments

* fix gpu ut; rename test file
2022-11-04 09:23:09 +08:00
ZhenyuLU_Heliodore 9d9280cb0e [NN] Add DegreeEncoder for graph transformer (#4742)
* Add files via upload

This file will contain several utility modules for Graph Transformer. DegreeEncoder has been implemented in this file now.

* Update graph_transformer.py

* Update nn-pytorch.rst

Add graph_transformer.DegreeEncoder in Utility Modules

* Update test_nn.py

* Update __init__.py

* Update graph_transformer.py

* Update test_nn.py

* Update graph_transformer.py

fix unused import

* Update graph_transformer.py

add module doc-string

* Update graph_transformer.py

* Update graph_transformer.py

Co-authored-by: rudongyu <ru_dongyu@outlook.com>
2022-11-02 13:29:41 +08:00
Serge Panev c5e83757de [Dist][Test] Add tests for multi-node DistGraph (#4396)
Signed-off-by: Serge Panev <spanev@nvidia.com>

Signed-off-by: Serge Panev <spanev@nvidia.com>
2022-11-01 17:41:33 +08:00
peizhou001 2caa6bd02d [Dist] Enable save and load for Distributed Optimizer (#4752)
* add save/load for distributed  optimizer

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-19.ap-northeast-1.compute.internal>
2022-11-01 14:30:16 +08:00
Zhiteng Li ed66a20970 [NN] Add a learned laplacian positional encoder (#4750)
* add a learned laplacian positional encoder

* leverage black to beautify the python code

* refine according to dongyu's comments

Co-authored-by: rudongyu <ru_dongyu@outlook.com>
2022-10-31 11:11:37 +08:00
Quan (Andy) Gan 2bca4759c1 [Sampling] Enable sampling with edge masks in sample_etype_neighbors (#4749)
* sample neighbors with masks

* oops

* refactor again

* remove

* remove debug code

* rename macro

* address comments

* more stuff

* remove

* fix

* try fix unit test

* oops

* fix test

* oops

* change name

* rename a lot of stuff

* oops

* ugh

* misc fixes

* lint

* address a lot of comments

* lint

* lint

* fix

* that was silly

* fix

* fix

* fix

* oops
2022-10-29 15:49:44 +08:00
Quan (Andy) Gan 72781efbcf [Sampling] Enable sampling with edge masks on homogeneous graph (#4748)
* sample neighbors with masks

* oops

* refactor again

* remove

* remove debug code

* rename macro

* address comments

* address comment

* address comments

* rename a lot of stuff

* oops
2022-10-28 09:04:33 +08:00
Infinity_X 4b1fb6816a [Model] Heterogeneous graph support for GNNExplainer (#4401)
* [Model] Heterogeneous graph support for GNNExplainer (#1)

* add HeteroGNNExplainer

* GNNExplainer for heterogeenous graph

* fix typo

* variable name cleanup

* added HeteroGNNExplainer test

* added doc indexing for HeteroGNNExplainer

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/nn/pytorch/explain/gnnexplainer.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update gnnexplainer.py

Change DGLHeteroGraph to DGLGraph, and specified parameter inputs

* Added ntype parameter to the explainer_node call

* responding to @mufeili's comment regarding restoring empty lines at appriopiate places to be consistent with existing practices

* responding to @mufeili's comment regarding restoring empty lines at appriopiate places that were missed in the last commit

* docstring comments added based on @mufeili suggestions

* indorporated @mufeili requested changes related to docstring model declaration.

* example model and test_nn.py added for explain_graphs

* explain_nodes fixed and fixed the way hetero num nodes and edges are handled

* white spaces removed

* lint issues fixed

* explain_graph model updated

* explain nodes model updated

* minor fixes related to gpu compatability

* cuda support added

* simplify WIP

* _init_masks for ennexplainer updated to match heterographs

* Update

* model simplified and docstring comments updated

* nits: docstring udpated

* lint check issues updated

* lint check updated

* soem formatting updated

* disabling int32 testing for GNNExplainer

* Update

Co-authored-by: Kangkook Jee <kangkook.jee@gmail.com>
Co-authored-by: ahadjawaid <94938815+ahadjawaid@users.noreply.github.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: kxm180046 <kxm180046@utdallas.edu>
Co-authored-by: Kunal Mukherjee <kunmukh@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>
2022-10-27 21:01:36 +08:00
Mufei Li 98adca6805 [Transform] Fix graph structure corruption with transform (#4753)
* Update

* Update

* Update

* Update

* Update test_transform.py

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
2022-10-27 15:34:24 +08:00
Rhett Ying ea4d9e832d [Dist] fix etype issue in dist part pipeline (#4754)
* [Dist] fix etype issue in dist part pipeline

* add comments
2022-10-27 09:44:04 +08:00
Rhett Ying 1990e797e1 [Dist] Reduce startup overhead: sort etypes and save in specified formats (#4735)
* [Dist] reduce startup overhead: enable to save in specified formats

* [Dist] reduce startup overhead: sort partitions when generating

* sort csc/csr only whenmultiple etypes

* refine
2022-10-26 19:07:10 +08:00
peizhou001 743516f37b add standalone tools for generating canonical etypes (#4626)
* add a standalone tool for change etypes to canonical etypes in part config
2022-10-19 23:34:39 +08:00
Rhett Ying c78ddee265 [Feature] enable to save graphs in multiple formats (#4266)
* [Feature] enable to save graphs in multiple formats

* use StreamWithCount to accomodate formats

* fix lint

* fix dynamic cast issue

* fix lint

* refine docstring and code naming

* update docstring
2022-10-19 14:53:44 +08:00
Rhett Ying 6a46072593 [Dist] decouple num_chunks and num_parts for graphs with edge feature (#4729)
* [Dist] decouple num_chunks and num_parts for graphs with edge feature

* fix test failure
2022-10-19 13:47:54 +08:00
Rhett Ying b63fc41a38 [BugFix] cugraph must be imported before torch (#4730) 2022-10-19 10:10:01 +08:00
Rhett Ying b13092172d [Dist] Reduce peak memory in DistDGL (#4687)
* [Dist] Reduce peak memory in DistDGL: avoid validation, release memory once loaded

* remove orig_id from ndata/edata for partition_graph()

* delete orig_id from ndata/edata in dist part pipeline

* reduce dtype size and format before saving graphs

* fix lint

* ETYPE requires to be int32/64 for CSRSortByTag

* fix test failure

* refine
2022-10-17 13:35:55 +08:00
Rhett Ying a5d21c2bfc [Sampling] handle fanout=-1 differently from fanout>0 in sample_etype_neighbors() (#4716) 2022-10-13 15:39:06 +08:00
LuckyLiuM f846d902cf Add Metapath2vec module (#4660)
* metapath2vec package

* fix bugs --metapath2vec package

* add unittest and fix bugs

* fix pyling messages

* del init.py

* fix bugs

* modify metapath2vec and add deepwalk

* metapath2vec module

* Update

* Update

* rollback to initial metapath2vec

* Update

* Update

* Update

* Update

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
2022-10-13 01:01:07 +08:00
Rhett Ying 408eba2479 Multiple dist dl sampler (#4704)
* [Dist] enable iterate multiple dist dataloaders simultaneously

* format file

* add support for any number of dataloaders

* fix lint

* refine code
2022-10-12 15:51:45 +08:00
Hongzhi (Steve), Chen ea48ce7a37 [Misc] Black auto fix. (#4697)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
2022-10-11 13:31:13 +08:00
Hongzhi (Steve), Chen 89a4cc4db2 [Misc] Black auto fix. (#4694)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
2022-10-11 00:12:30 +08:00
Hongzhi (Steve), Chen 98325b1097 [Misc] Black auto fix. (#4691)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
2022-10-10 10:53:51 +08:00
Zhiteng Li 29434e6501 [Refactor] Refactor Laplacian positional encoding transform (#4628)
* refactor the function and module of laplacian_pe

* add blank lines between design doc

* 1st approving review

* update the unittest

* fix lint issues

* fix trailing space

* update by mufei's comments and fix backend bugs

* del test file

Co-authored-by: rudongyu <ru_dongyu@outlook.com>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-10-08 13:03:03 +08:00
kylasa eae6ce2a4f ParMETIS wrapper script to enable ParMETIS to process chunked dataset format (#4605)
* Creating ParMETIS wrapper script to run parmetis using one script from user perspective

* Addressed all the CI comments from PR https://github.com/dmlc/dgl/pull/4529

* Addressing CI comments.

* Isort, and black changes.

* Replaced python with python3

* Replaced single quote with double quotes per suggestion.

* Removed print statement

* Addressing CI Commets.

* Addressing CI review comments.

* Addressing CI comments as per chime discussion with Rui

* CI Comments, Black and isort changes

* Align with code refactoring, black, isort and code review comments.

* Addressing CI review comments, and fixing merge issues with the master branch

* Updated with proper unit test skip decorator
2022-10-03 12:23:57 -07:00
kylasa 1f471396d8 Edge Feature support for input graph datasets for dist. graph partitioning pipeline (#4623)
* Added support for edge features.

* Added comments and removing unnecessary print statements.

* updated data_shuffle.py to remove compile error.

* Repaled python3 with python to match CI test framework.

* Removed unrelated files from the pull request.

* Isort changes.

* black changes on this file.

* Addressing CI review comments.

* Addressing CI comments.

* Removed duplicated and resolved merge conflict code.

* Addressing CI Comments from Rui.

* Addressing CI comments, and fixing merge issues.

* Addressing CI comments, code refactoring, isort and black
2022-10-03 01:02:15 -07:00
Quan (Andy) Gan 5bf17547f7 [DistTensor] add bitwise or function (#4638)
* first commit

* add test

* fixes

* ah this is how you skip setup

* fix

* ugh

* address comments

* i like black
2022-09-30 19:38:37 +08:00
czkkkkkk bb7b3c6f25 [Sparse] Add SparseMatrix unittest and fix docstring problem (#4627)
* [Sparse] Add SparseMatrix unittest and fix docstring problem

* Minor fix

* Update

* check permission

* rm future annonations

* Skip create_from_csr and create_from_csc tests because Pytorch 1.9.0 does not have torch.sparse_csr_tensor

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-09-30 11:53:33 +08:00
Israt Nisa 3d01340d23 [Sparse] Add elementwise operators for diagonal matrix (#4644)
* Add elementwise operators for diagonal matrix

* minor

* resolve ambiguity of ops

* lint check

* lint check

* lint check

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-09-29 13:08:50 +08:00
LuckyLiuM eb729c5401 Add Deepwalk module (#4562)
* metapath2vec package

* fix bugs --metapath2vec package

* add unittest and fix bugs

* fix pyling messages

* del init.py

* fix bugs

* modify metapath2vec and add deepwalk

* deepwalk module

* del metapath2vec files

* Update

* Update

* Update

* Update

* Update

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-09-29 09:30:56 +08:00
Israt Nisa fa05ccb961 Add sddmm operator in mock_sparse library (#4579)
* sddmm init

* SDDMM with N-D nonzero values

* drop support for vector shaped non zero elements

* address comments

* skip cpu test

* skip GPU test too

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-09-28 11:50:30 -04:00
Rhett Ying cf19254a19 [Dist] enable to partition many chunks into less partitions via pipeline (#4620)
* [Dist] enable to partition many chunks into less partitions via pipeline

* refine

* add meta file for num_parts, add more tests, refine docstring

* remove args.num_parts

* create pydantic class for partition metadata

* refine

* rename json file
2022-09-28 16:52:39 +08:00
Rhett Ying 6c1500d49f [Dist] save original node/edge IDs into separate files (#4649)
* [Dist] save original node/edge IDs into separate files

* separate nids and eids
2022-09-28 14:30:24 +08:00
Mufei Li 977b1ba448 [Sparse] Migration of Unary, Softmax, Matmul Ops (#4616)
* Update

* lint

* Fix

* Update

* Update

* Update

* update

* Fix

* Update

* Fix

* Fix

* Fix

* CI

* Update

* Update

* Update

* update test

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
2022-09-27 19:34:46 +08:00
Xin Yao 188bc2bf71 [Fix][Unittest] Fix test stream (#4635)
* fix test stream

* init cusparse handle
2022-09-27 12:41:26 +08:00
czkkkkkk 544a2cd192 [Sparse] Add reduction operators (#4600)
* [Sparse] Add reduction operators

* Fix linting problem

* Fix linting problem

* Fix linting problem

* Fix docstring

* Fix test cases

* Fix linting problem

* Skip reduction unittests
2022-09-26 16:06:05 +08:00
Israt Nisa 7dba19913e Sparse ops: Support elementwise ops between sparse and diagonal matrix (#4617)
* initial commit

* address comments

* lint check

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-09-23 22:22:51 -04:00
czkkkkkk db64cc3760 [Sparse] Add Transposition for SparseMatrix and DiagMatrix (#4597)
* [Sparse] Add Transposition

* Fix docstring

* Fix linting problem

* Minor fix

* Minor fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-09-23 15:17:55 +08:00
Vibhu Jawa d04534c940 [Bugfix][Hotfix] Fix cugraph Pytorch CI failures (#4622)
* get_more_logs

* Update cugraph_unit_test.sh

* Update test_basics.py

* Update cugraph_unit_test.sh

* Update tests/cugraph/test_basics.py

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-09-23 11:49:03 +08:00
Rhett Ying be11409312 [CI] Enable DGL cugaph nightly CI (#4573)
* [CI] Enable DGL cugaph nightly CI

* fix build issue

* remove runtime
2022-09-22 15:26:29 +08:00
Xin Yao 880b3b1f97 [Fix] Enable lint check for cuh files and fix compiler warnings (#4585)
* disable warning for tensorpipe

* fix warning

* enable lint check for cuh files

* resolve comments
2022-09-21 10:09:14 +08:00
peizhou001 166b273bec add ssh port config for dispatchdata (#4557) 2022-09-20 20:41:41 +08:00
Mufei Li 73b9f3a1da [Sparse] Migrate DiagMatrix class (#4567)
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
2022-09-20 17:17:43 +08:00
Quan (Andy) Gan 8366774122 [Feature] Support canonical edge types in HeteroGraphConv (#4440)
* fix

* Update hetero.py

* why did i remove this

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-09-20 14:15:26 +08:00
nv-dlasalle 2630d2eb02 [Performance][bugfix] Implement is_unibipartite in C++ with caching. (#4556)
* updates

* Enable caching C++ result

* Add missing docstring

* Remove unused function

* Add unit test

* Address comments
2022-09-19 16:46:27 -07:00
Xin Yao cded5b80fe [Feature] Bump DLPack to v0.7 and decouple DLPack from the core library (#4454)
* rename `DLContext` to `DGLContext`

* rename `kDLGPU` to `kDLCUDA`

* replace DLTensor with DGLArray

* fix linting

* Unify DGLType and DLDataType to DGLDataType

* Fix FFI

* rename DLDeviceType to DGLDeviceType

* decouple dlpack from the core library

* fix bug

* fix lint

* fix merge

* fix build

* address comments

* rename dl_converter to dlpack_convert

* remove redundant comments
2022-09-19 16:02:43 +08:00
Rhett Ying 7f984b64b2 Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563)
This reverts commit ec171c648a.
2022-09-16 13:57:50 +08:00
Vibhu Jawa ec171c648a [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)
* Added cugraph nightly scripts

* Removed nvcr.io//nvidia/pytorch:22.04-py3 reference

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-09-16 11:28:30 +08:00
Xin Yao 9a00cf194f [Feature] Import PyTorch's CUDA stream management (#4503)
* add set_stream

* add .record_stream for NDArray and HeteroGraph

* refactor dgl stream Python APIs

* test record_stream

* add unit test for record stream

* use pytorch's stream

* fix lint

* fix cpu build

* address comments

* address comments

* add record stream tests for dgl.graph

* record frames and update dataloder

* add docstring

* update frame

* add backend check for record_stream

* remove CUDAThreadEntry::stream

* record stream for newly created formats

* fix bug

* fix cpp test

* fix None c_void_p to c_handle
2022-09-15 16:54:52 +08:00
Rhett Ying 099b173f6f [DistPart] expose timeout config for process group (#4532)
* [DistPart] expose timeout config for process group

* refine code

* Update tools/distpartitioning/data_proc_pipeline.py

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-09-15 15:28:36 +08:00
Israt Nisa cf03592743 Move mock version of dgl_sparse library to DGL main repo (#4524)
* init

* Add api doc for sparse library

* support op btwn matrices with differnt sparsity

* Fixed docstring

* addresses comments

* lint check

* change keyword format to fmt

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-09-14 12:08:20 -04:00