3d01340d23
[Sparse] Add elementwise operators for diagonal matrix ( #4644 )
...
* Add elementwise operators for diagonal matrix
* minor
* resolve ambiguity of ops
* lint check
* lint check
* lint check
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
2022-09-29 13:08:50 +08:00
eb729c5401
Add Deepwalk module ( #4562 )
...
* metapath2vec package
* fix bugs --metapath2vec package
* add unittest and fix bugs
* fix pyling messages
* del init.py
* fix bugs
* modify metapath2vec and add deepwalk
* deepwalk module
* del metapath2vec files
* Update
* Update
* Update
* Update
* Update
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-09-29 09:30:56 +08:00
fa05ccb961
Add sddmm operator in mock_sparse library ( #4579 )
...
* sddmm init
* SDDMM with N-D nonzero values
* drop support for vector shaped non zero elements
* address comments
* skip cpu test
* skip GPU test too
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-09-28 11:50:30 -04:00
977b1ba448
[Sparse] Migration of Unary, Softmax, Matmul Ops ( #4616 )
...
* Update
* lint
* Fix
* Update
* Update
* Update
* update
* Fix
* Update
* Fix
* Fix
* Fix
* CI
* Update
* Update
* Update
* update test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
2022-09-27 19:34:46 +08:00
08c50eb7c1
[Subgraph] State in the docstring that nodes cannot have any duplicate value when inducing a subgraph ( #4637 )
2022-09-26 16:25:49 +08:00
544a2cd192
[Sparse] Add reduction operators ( #4600 )
...
* [Sparse] Add reduction operators
* Fix linting problem
* Fix linting problem
* Fix linting problem
* Fix docstring
* Fix test cases
* Fix linting problem
* Skip reduction unittests
2022-09-26 16:06:05 +08:00
80b22ad881
[Readability] Polish udf.py ( #4534 )
...
* Polish udf.py
* Update udf.py
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
2022-09-26 14:50:34 +08:00
7dba19913e
Sparse ops: Support elementwise ops between sparse and diagonal matrix ( #4617 )
...
* initial commit
* address comments
* lint check
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
2022-09-23 22:22:51 -04:00
db64cc3760
[Sparse] Add Transposition for SparseMatrix and DiagMatrix ( #4597 )
...
* [Sparse] Add Transposition
* Fix docstring
* Fix linting problem
* Minor fix
* Minor fix
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-09-23 15:17:55 +08:00
ec4271bf87
[Example] Refactor GNNExplainer Example ( #4560 )
...
* debug
* debug
* readme
* fix readme
* fix readme
* Update
* Update
* update
* fix bug of syn2
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-09-21 13:34:39 +08:00
73b9f3a1da
[Sparse] Migrate DiagMatrix class ( #4567 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
2022-09-20 17:17:43 +08:00
8366774122
[Feature] Support canonical edge types in HeteroGraphConv ( #4440 )
...
* fix
* Update hetero.py
* why did i remove this
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-09-20 14:15:26 +08:00
5ebd3bf05a
[DataLoader] Disable the usage of shared memory when persistent_workers=False on single GPU ( #4497 )
...
* toggle shared memory usage
* Update dataloader.py
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-09-20 11:40:33 +08:00
2630d2eb02
[Performance][bugfix] Implement is_unibipartite in C++ with caching. ( #4556 )
...
* updates
* Enable caching C++ result
* Add missing docstring
* Remove unused function
* Add unit test
* Address comments
2022-09-19 16:46:27 -07:00
cded5b80fe
[Feature] Bump DLPack to v0.7 and decouple DLPack from the core library ( #4454 )
...
* rename `DLContext` to `DGLContext`
* rename `kDLGPU` to `kDLCUDA`
* replace DLTensor with DGLArray
* fix linting
* Unify DGLType and DLDataType to DGLDataType
* Fix FFI
* rename DLDeviceType to DGLDeviceType
* decouple dlpack from the core library
* fix bug
* fix lint
* fix merge
* fix build
* address comments
* rename dl_converter to dlpack_convert
* remove redundant comments
2022-09-19 16:02:43 +08:00
9a00cf194f
[Feature] Import PyTorch's CUDA stream management ( #4503 )
...
* add set_stream
* add .record_stream for NDArray and HeteroGraph
* refactor dgl stream Python APIs
* test record_stream
* add unit test for record stream
* use pytorch's stream
* fix lint
* fix cpu build
* address comments
* address comments
* add record stream tests for dgl.graph
* record frames and update dataloder
* add docstring
* update frame
* add backend check for record_stream
* remove CUDAThreadEntry::stream
* record stream for newly created formats
* fix bug
* fix cpp test
* fix None c_void_p to c_handle
2022-09-15 16:54:52 +08:00
cf03592743
Move mock version of dgl_sparse library to DGL main repo ( #4524 )
...
* init
* Add api doc for sparse library
* support op btwn matrices with differnt sparsity
* Fixed docstring
* addresses comments
* lint check
* change keyword format to fmt
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
2022-09-14 12:08:20 -04:00
701c6fe754
[Readability] Auto fix setup.py and update-version.py ( #4446 )
...
* Auto fix update-version
* Auto fix setup.py
* Auto fix update-version
* Auto fix setup.py
* [Doc] Change random.py to random_partition.py in guide on distributed partition pipeline (#4438 )
* Update distributed-preprocessing.rst
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
* fix unpinning when tensoradaptor is not available (#4450 )
* [Doc] fix print issue in tutorial (#4459 )
* [Example][Refactor] Refactor RGCN example (#4327 )
* Refactor full graph entity classification
* Refactor rgcn with sampling
* README update
* Update
* Results update
* Respect default setting of self_loop=false in entity.py
* Update
* Update README
* Update for multi-gpu
* Update
* [doc] fix invalid link in user guide (#4468 )
* [Example] directional_GSN for ogbg-molpcba (#4405 )
* version-1
* version-2
* version-3
* update examples/README
* Update .gitignore
* update performance in README, delete scripts
* 1st approving review
* 2nd approving review
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Clarify the message name, which is 'm'. (#4462 )
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
* [Refactor] Auto fix view.py. (#4461 )
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
* [Example] SEAL for OGBL (#4291 )
* [Example] SEAL for OGBL
* update index
* update
* fix readme typo
* add seal sampler
* modify set ops
* prefetch
* efficiency test
* update
* optimize
* fix ScatterAdd dtype issue
* update sampler style
* update
Co-authored-by: Quan Gan <coin2028@hotmail.com >
* [CI] use https instead of http (#4488 )
* [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487 )
* [BugFix] fix crash due to incorrect dtype in dgl.to_block()
* fix test failure in TF
* [Feature] Make TensorAdapter Stream Aware (#4472 )
* Allocate tensors in DGL's current stream
* make tensoradaptor stream-aware
* replace TAemtpy with cpu allocator
* fix typo
* try fix cpu allocation
* clean header
* redirect AllocDataSpace as well
* resolve comments
* [Build][Doc] Specify the sphinx version (#4465 )
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
* reformat
* reformat
* Auto fix update-version
* Auto fix setup.py
* reformat
* reformat
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
Co-authored-by: Xin Yao <xiny@nvidia.com >
Co-authored-by: Chang Liu <chang.liu@utexas.edu >
Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: rudongyu <ru_dongyu@outlook.com >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2022-09-14 13:36:18 +08:00
684f66b7aa
[Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323 #4316 ) ( #4389 )
...
* * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph
* Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information
* * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing
* * Added test to ensure dgl.remove_self_loop function correctly updates batch information
* * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph
* * Formatting
* * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph
* * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass
* Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed
* * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments
* * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test
* * Added doc comments for new parameters
* * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph
* Added test cases for more than k coincident points
* * Updated doc comments for output_batch parameters for clarity
* * Linter formatting fixes
* * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu
* * Rewording in doc comments
* * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-09-07 08:35:26 +08:00
1c9d2a0302
[Feature] Unify the cuda stream used in core library ( #4480 )
...
* Use an internal cuda stream for CopyDataFromTo
* small fix white space
* Fix to compile
* Make stream optional in copydata for compile
* fix lint issue
* Update cub functions to use internal stream
* Lint check
* Update CopyTo/CopyFrom/CopyFromTo to use internal stream
* Address comments
* Fix backward CUDA stream
* Avoid overloading CopyFromTo()
* Minor comment update
* Overload copydatafromto in cuda device api
Co-authored-by: xiny <xiny@nvidia.com >
2022-09-06 10:14:55 +08:00
62af41c245
[Bug] Enable turn on/off libxsmm at runtime ( #4455 )
...
* enable turn on/off libxsmm at runtime by adding a global config and related API
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal >
2022-09-05 17:38:34 +08:00
806def67a1
[Cleanup] Remove async_transferer ( #4505 )
...
* Remove async_transferer
* remove test
* Remove AsyncTransferer
Co-authored-by: Xin Yao <xiny@nvidia.com >
Co-authored-by: Xin Yao <yaox12@outlook.com >
2022-09-05 11:46:06 +08:00
468c0ca4b4
[BugFix] fix crash due to incorrect dtype in dgl.to_block() ( #4487 )
...
* [BugFix] fix crash due to incorrect dtype in dgl.to_block()
* fix test failure in TF
2022-08-31 16:09:46 +08:00
ef93518da3
[Refactor] Auto fix view.py. ( #4461 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-08-30 18:31:03 +08:00
47993776df
[Feature] Rework Dataloader cpu affinitization as helper method ( #4126 )
...
* Add helper method for temporary affinitization of compute threads
* Rework DL affinitization as single helper
* Add example usage in benchmarks
* Fix python linter warnings
* Fix affinity helper params
* Use NUMA node 0 cores only by default
* Fix benchmarks
* Fix lint errors
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-08-18 22:34:12 +08:00
3bcb268a36
[Doc] Update distributed chapter according to new pipeline ( #4275 )
...
* dist index chapter
* preproc chapter
* rst
* tools page
* partition chapter
* rst
* hetero chapter
* 7.1 step1
* add parmetis back
* changed based on feedback
* address comments
2022-08-17 13:30:04 +08:00
39987bc599
[Feature] enable graph partition book support canonical etypes ( #4343 )
...
* [Feature] enable graph partition book support canonical etypes
* fix lint
* fix lint
* add todo
* refine according to review comments
* fix lint
* refine naming
* revert PartitionPolicy __init__
* refine docstring
* fix doc string
2022-08-16 11:52:02 +08:00
a88e7f7e28
[Example]rgcn-ogbn-mag ( #4331 )
...
* rgcn-ogbn-mag
* Add link in README.md
* correct code-format,add the reset_parameters function to the HeteroEmbedding module
* add the annotation in hetero.py
* add a unit test
* modify format
* Update
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-50-143.us-west-2.compute.internal >
2022-08-10 20:12:31 +08:00
919b783875
[Example][Bugfix] Fix infograph example ( #4298 )
...
* Fix infograph example
* Update
* Revert the changes and update Doc
* Update
* Split lines to pass CI-lint
* Update
* Update
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-08-10 17:10:25 +08:00
62c827c81c
[Bug] A bunch of fixes in edge_softmax_hetero ( #4336 )
...
* bunch of fixes
* Update test_edge_softmax_hetero.py
* Update test_edge_softmax_hetero.py
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-08-09 15:16:48 +08:00
5ba5106aca
[Bugfix] Fix the default value of num_bases in RelGraphConv module ( #4321 )
...
* Fix doc and default settings for RelGraphConv
* Add unit test
* Split msg in two lines to pass CI-lint
2022-08-07 16:33:16 +08:00
43ba94ee49
[BugFix] fix etype check in DistGraph.edge_subgraph ( #4322 )
2022-08-03 11:56:00 +08:00
4dd16f5dd8
[BugFix] enable DistGraph.find_edge() works with str or tuple of str ( #4319 )
2022-08-01 20:47:15 +08:00
44b6864114
[Feature] Enable UVA for Weighted Samplers ( #4314 )
...
* enable use for weighted neighbor sampler and biased random walk
* add unit tests
* fix for mxnet/tf
* fix typo
2022-08-01 17:56:51 +08:00
d6957c28ff
[BugFix] fix incorrect _bias and bias usage ( #4310 )
2022-07-30 17:40:54 +08:00
86c81b4e92
[Feature] Add CUDA Weighted Neighborhood Sampling ( #4064 )
...
* add weighted sampling without replacement (A-Chao)
* improve Algorithm A-Chao with block-wise prefix sum
* correctly fill out_idxs
* implement weighted sampling with replacement
* small fix
* merge host-side code of weighted/uniform sampling
* enable unit tests for cuda weighted sampling
* move thrust/cub wrapper to the cmake file
* update docs accordingly
* fix linting
* fix linting
* fix unit test
* Bump external CUB/Thrust versions
* Fix code style and update description of algorithm design
* [Feature] GPU support weighted graph neighbor sampling
commit by pengqirong(OPPO)
* merge pengqirong's implementation
* revert the change to cub and thrust
* fix linting
* use DeviceSegmentedSort for better performance
* add more comments
* add necessary notes
* add necessary notes
* resolve some comments
* define THRUST_CUB_WRAPPED_NAMESPACE
* fix doc
Co-authored-by: 彭齐荣 <657017034@qq.com >
2022-07-29 11:08:48 +08:00
2cf05c5342
[Transform] Allow add data to self loop created by AddSelfLoop or add_self_loop ( #4261 )
...
* Update
* Update functional.py
* Update
* Update test_transform.py
* Update
* Update functional.py
* Update functional.py
* Update functional.py
* Update functional.py
* Update
* Update
* Update functional.py
* Update functional.py
* Update functional.py
* Update functional.py
* Update module.py
* Update test_transform.py
* Update test_transform.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-07-27 15:14:03 +08:00
4f7972957b
[Bugfix] Fix dataloader pytorch cuda indexing ( #4297 )
...
* Modify to repro crash
* Revert to orig. scenario and add fix
* Update
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-26 16:59:58 +08:00
7e6a6b4ac0
[Feature] Add CUDA Weighted Randomwalk Sampling ( #4243 )
...
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* fix empty prob array && enable non-uniform for restart && enable unit tests
* update doc and guide for randomwalk and pinsage
* update comments
Co-authored-by: zhenliangqiu <ubuntu@ip-172-31-24-245.ap-southeast-1.compute.internal >
Co-authored-by: xiny <xiny@nvidia.com >
2022-07-26 13:42:39 +08:00
7cd531c4d6
[Dist][Optim] Change op order in SparseAdagrad to be numerically closer to PyTorch ( #4253 )
...
Signed-off-by: Serge Panev <spanev@nvidia.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-07-25 14:53:53 +08:00
8292bf329f
[Dist] remove deprecated arguments from initialize() ( #4284 )
2022-07-25 10:45:19 +08:00
740cd70610
[Doc]Fix doc typo in DistEmbedding ( #4258 )
...
* [Doc] fix docstring typo
* Update sparse_emb.py
* Update sparse_emb.py
* update link
2022-07-19 10:37:14 +08:00
fdbf5a0fda
[DGL-Go][Doc] Update DGL-Go version to 0.0.2 and misc fix from bug bash ( #4236 )
...
* Update
* Update
* Update
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-142.us-west-2.compute.internal >
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-14 21:30:38 +08:00
2efdaa5df3
[Bug] Revert clearing backward cache for retain_graph flag ( #4249 )
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-07-14 16:32:01 +08:00
82ca781911
[Bug] Record stream when using another CUDA stream for data transfer ( #4250 )
...
* record stream when using another cuda stream for data transfer
* fix linting
* fix None stream
2022-07-14 13:17:37 +08:00
2f322a94b9
[BugFix] fetch device from graph ( #4245 )
...
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-14 10:25:07 +08:00
c56e27a807
[Bugfix] Accessing data from the indexes stored in same device ( #4242 )
...
* First update to fix two examples
* Update to fix RGCN/graphsage example and dataloader
* Update
2022-07-13 09:02:07 +08:00
14e4e1b0f7
[Dist] enable to specify sort_etype for sample_etype_neighbours ( #4212 )
...
* [Dist] enable to specify sort_etype for sample_etype_neighbours
* fix lint
* pass argument instead of env
* fix lint and doc string
* refine args
* remove unnecessary lines
* debug only
* debug add sort time log
* change interface
* fix typo
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-11 18:37:25 +08:00
c65d6fa55f
[Dist] format dtypes when loading graph in server ( #4228 )
...
* [Dist] format dtypes when loading graph in server
* add test
* refine
* add comments
2022-07-11 14:08:17 +08:00
9ae117d3cd
[Feature] Add to_cugraph and from_cugraph to DGL ( #4166 )
...
* Added to_cugraph and from_cugraph functionality
* fix from_cugraph example
* Addressed Reviews
* Fix linting
* Apply suggestions to docstrings from code review
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
* Add API docs and remove `from_cugraph` alias
* move cugraph tests to tests/cugraph/test_basics.py
* remove ununsed imports from test_basics.py
* remove pytest.importorskip as no longer needed
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-07-07 11:52:17 +08:00
Israt Nisa