* Update from master (#4584)
* [Example][Refactor] Refactor graphsage multigpu and full-graph example (#4430)
* Add refactors for multi-gpu and full-graph example
* Fix format
* Update
* Update
* Update
* [Cleanup] Remove async_transferer (#4505)
* Remove async_transferer
* remove test
* Remove AsyncTransferer
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Xin Yao <yaox12@outlook.com>
* [Cleanup] Remove duplicate entries of CUB submodule (issue# 4395) (#4499)
* remove third_part/cub
* remove from third_party
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
* [Bug] Enable turn on/off libxsmm at runtime (#4455)
* enable turn on/off libxsmm at runtime by adding a global config and related API
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal>
* [Feature] Unify the cuda stream used in core library (#4480)
* Use an internal cuda stream for CopyDataFromTo
* small fix white space
* Fix to compile
* Make stream optional in copydata for compile
* fix lint issue
* Update cub functions to use internal stream
* Lint check
* Update CopyTo/CopyFrom/CopyFromTo to use internal stream
* Address comments
* Fix backward CUDA stream
* Avoid overloading CopyFromTo()
* Minor comment update
* Overload copydatafromto in cuda device api
Co-authored-by: xiny <xiny@nvidia.com>
* [Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323#4316) (#4389)
* * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph
* Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information
* * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing
* * Added test to ensure dgl.remove_self_loop function correctly updates batch information
* * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph
* * Formatting
* * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph
* * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass
* Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed
* * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments
* * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test
* * Added doc comments for new parameters
* * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph
* Added test cases for more than k coincident points
* * Updated doc comments for output_batch parameters for clarity
* * Linter formatting fixes
* * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu
* * Rewording in doc comments
* * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* [CI] only known devs are authorized to trigger CI (#4518)
* [CI] only known devs are authorized to trigger CI
* fix if author is null
* add comments
* [Readability] Auto fix setup.py and update-version.py (#4446)
* Auto fix update-version
* Auto fix setup.py
* Auto fix update-version
* Auto fix setup.py
* [Doc] Change random.py to random_partition.py in guide on distributed partition pipeline (#4438)
* Update distributed-preprocessing.rst
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
* fix unpinning when tensoradaptor is not available (#4450)
* [Doc] fix print issue in tutorial (#4459)
* [Example][Refactor] Refactor RGCN example (#4327)
* Refactor full graph entity classification
* Refactor rgcn with sampling
* README update
* Update
* Results update
* Respect default setting of self_loop=false in entity.py
* Update
* Update README
* Update for multi-gpu
* Update
* [doc] fix invalid link in user guide (#4468)
* [Example] directional_GSN for ogbg-molpcba (#4405)
* version-1
* version-2
* version-3
* update examples/README
* Update .gitignore
* update performance in README, delete scripts
* 1st approving review
* 2nd approving review
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* Clarify the message name, which is 'm'. (#4462)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
* [Refactor] Auto fix view.py. (#4461)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* [Example] SEAL for OGBL (#4291)
* [Example] SEAL for OGBL
* update index
* update
* fix readme typo
* add seal sampler
* modify set ops
* prefetch
* efficiency test
* update
* optimize
* fix ScatterAdd dtype issue
* update sampler style
* update
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* [CI] use https instead of http (#4488)
* [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487)
* [BugFix] fix crash due to incorrect dtype in dgl.to_block()
* fix test failure in TF
* [Feature] Make TensorAdapter Stream Aware (#4472)
* Allocate tensors in DGL's current stream
* make tensoradaptor stream-aware
* replace TAemtpy with cpu allocator
* fix typo
* try fix cpu allocation
* clean header
* redirect AllocDataSpace as well
* resolve comments
* [Build][Doc] Specify the sphinx version (#4465)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* reformat
* reformat
* Auto fix update-version
* Auto fix setup.py
* reformat
* reformat
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Chang Liu <chang.liu@utexas.edu>
Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* Move mock version of dgl_sparse library to DGL main repo (#4524)
* init
* Add api doc for sparse library
* support op btwn matrices with differnt sparsity
* Fixed docstring
* addresses comments
* lint check
* change keyword format to fmt
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
* [DistPart] expose timeout config for process group (#4532)
* [DistPart] expose timeout config for process group
* refine code
* Update tools/distpartitioning/data_proc_pipeline.py
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* [Feature] Import PyTorch's CUDA stream management (#4503)
* add set_stream
* add .record_stream for NDArray and HeteroGraph
* refactor dgl stream Python APIs
* test record_stream
* add unit test for record stream
* use pytorch's stream
* fix lint
* fix cpu build
* address comments
* address comments
* add record stream tests for dgl.graph
* record frames and update dataloder
* add docstring
* update frame
* add backend check for record_stream
* remove CUDAThreadEntry::stream
* record stream for newly created formats
* fix bug
* fix cpp test
* fix None c_void_p to c_handle
* [examples]educe memory consumption (#4558)
* [examples]educe memory consumption
* reffine help message
* refine
* [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)
* Added cugraph nightly scripts
* Removed nvcr.io//nvidia/pytorch:22.04-py3 reference
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
* Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563)
This reverts commit ec171c648a.
* [Misc] Add flake8 lint workflow. (#4566)
* Add pyproject.toml for autopep8.
* Add pyproject.toml for autopep8.
* Add flake8 annotation in workflow.
* remove
* add
* clean up
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* [Misc] Try use official pylint workflow. (#4568)
* polish update_version
* update pylint workflow.
* add
* revert.
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* [CI] refine stage logic (#4565)
* [CI] refine stage logic
* refine
* refine
* remove (#4570)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* Add Pylint workflow for flake8. (#4571)
* remove
* Add pylint.
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* [Misc] Update the python version in Pylint workflow for flake8. (#4572)
* remove
* Add pylint.
* Change the python version for pylint.
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* Update pylint. (#4574)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* [Misc] Use another workflow. (#4575)
* Update pylint.
* Use another workflow.
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* Update pylint. (#4576)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* Update pylint.yml
* Update pylint.yml
* Delete pylint.yml
* [Misc]Add pyproject.toml for autopep8 & black. (#4543)
* Add pyproject.toml for autopep8.
* Add pyproject.toml for autopep8.
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
* [Feature] Bump DLPack to v0.7 and decouple DLPack from the core library (#4454)
* rename `DLContext` to `DGLContext`
* rename `kDLGPU` to `kDLCUDA`
* replace DLTensor with DGLArray
* fix linting
* Unify DGLType and DLDataType to DGLDataType
* Fix FFI
* rename DLDeviceType to DGLDeviceType
* decouple dlpack from the core library
* fix bug
* fix lint
* fix merge
* fix build
* address comments
* rename dl_converter to dlpack_convert
* remove redundant comments
Co-authored-by: Chang Liu <chang.liu@utexas.edu>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Xin Yao <yaox12@outlook.com>
Co-authored-by: Israt Nisa <neesha295@gmail.com>
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal>
Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com>
* [Deprecation] Dataset Attributes (#4546)
* Update
* CI
* CI
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
* [Example] Bug Fix (#4665)
* Update
* CI
* CI
* Update
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
* Update
Co-authored-by: Chang Liu <chang.liu@utexas.edu>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Xin Yao <yaox12@outlook.com>
Co-authored-by: Israt Nisa <neesha295@gmail.com>
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal>
Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com>
* add argument reorder=False for citation_graph
* add description of the argument reorder
* add reordered/un_reordered save_path
* add version number postfix
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* [Fix] be able to parse ids if numeric and non-numeric values are used together
* add required package info and cache note into docstring
* duplicate node id is not allowed
* [Feature] support non-numeric node_id/src_id/dst_id/graph_id and rename CSVDataset
* change return value when iterate dataset
* refine data_parser
* force reload
* add ut
* add doc link
* install dep
* fix ci
* fix ut; more comments
* remove deprecated attributes in rdf datasets; fix label feature name
* address comments
* fix ut for other frameworks
* [Feature] add CSVDataset to load data from csv files
* add CSVDataset class file
* install pyyaml when running unit tests
* install pandas for unit tests
* utilize pydantic for YAML config check
* generate yaml and csv files during test
* make more keys as optional
* remove/rename several keys in yaml config and more tets though looks a bit clumsy
* fix test failure on mxnet
* pass /path/to/dataset instead of yaml path
* code refinement
* code refine
* change several yaml field such as feat and graph_id
* merge graph generation logic
* refine code
* Refactored_first_version
* DGLCSVDataset works for single heterograph
* add more tests
* fix test failure in mxnet
* add docstring
* use list comprehension for dict
* fix version in YAML
* refine data length assert
* use dict.pop directly
* remove ambiguous variable names
* refine graph id missing logic
* refine graph create call
* separate node/edge/graph data parser
* remove separator in DefaultDataParser
* refine validation error log for yaml field
* minor check
* refine code via dict.get()
* move load_from_csv into Node/Edge/GraphData
* move _parse_node/edge/graph_data into Node/Edge/GraphData
* refine id-related fields check
* check duplicate ntypes/etypes when load yaml
* refine docstring
* squeeze node labels in FraudDataset
* fix RLModule
* update results in README.md
* fix KeyError in full graph training
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* [Model] add model example CARE-GNN
* update README
* improvements based on the review feedback
* fix missing item()
Co-authored-by: zhjwy9343 <6593865@qq.com>
* PPIDataset
* Revert "PPIDataset"
This reverts commit 264bd0c960cfa698a7bb946dad132bf52c2d0c8a.
* Hide implementations that may cause confusions to the user
* fix number of classes in Amazon Cobuy datasets
* add four papers
* Fix doc string
* fix readme
* fix readme
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* fix breakline in fakenews.py
* fix inconsistent argument name
* modify incorrect example and deprecated graph type
* modify docstring and example in knn_graph
* fix incorrect node type
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [BugFix] fix problems in data split
* fix format problems in docstring
* modify statistics to fit in dgl nature
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: zhjwy9343 <6593865@qq.com>
* csr and csc creation
* fix
* fix
* fixes to adj transpose
* fine
* raise error if indptr did not match number of nodes
* fix
* huh?
* oh
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* [Feature] add positive negative statistics
* [Feature] add permission information and fix import problem
* fix backend incompatible problem
* modify random split to remove sklearn usage
* modify file read to remove pandas usage
* add datasets into doc
* add random seed in data splitting
* add dataset unit test
* usage permission information update
Co-authored-by: zhjwy9343 <6593865@qq.com>
Chang Liu