* add set_stream
* add .record_stream for NDArray and HeteroGraph
* refactor dgl stream Python APIs
* test record_stream
* add unit test for record stream
* use pytorch's stream
* fix lint
* fix cpu build
* address comments
* address comments
* add record stream tests for dgl.graph
* record frames and update dataloder
* add docstring
* update frame
* add backend check for record_stream
* remove CUDAThreadEntry::stream
* record stream for newly created formats
* fix bug
* fix cpp test
* fix None c_void_p to c_handle
* * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph
* Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information
* * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing
* * Added test to ensure dgl.remove_self_loop function correctly updates batch information
* * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph
* * Formatting
* * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph
* * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass
* Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed
* * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments
* * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test
* * Added doc comments for new parameters
* * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph
* Added test cases for more than k coincident points
* * Updated doc comments for output_batch parameters for clarity
* * Linter formatting fixes
* * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu
* * Rewording in doc comments
* * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* Use an internal cuda stream for CopyDataFromTo
* small fix white space
* Fix to compile
* Make stream optional in copydata for compile
* fix lint issue
* Update cub functions to use internal stream
* Lint check
* Update CopyTo/CopyFrom/CopyFromTo to use internal stream
* Address comments
* Fix backward CUDA stream
* Avoid overloading CopyFromTo()
* Minor comment update
* Overload copydatafromto in cuda device api
Co-authored-by: xiny <xiny@nvidia.com>
* rgcn-ogbn-mag
* Add link in README.md
* correct code-format,add the reset_parameters function to the HeteroEmbedding module
* add the annotation in hetero.py
* add a unit test
* modify format
* Update
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-50-143.us-west-2.compute.internal>
* Added to_cugraph and from_cugraph functionality
* fix from_cugraph example
* Addressed Reviews
* Fix linting
* Apply suggestions to docstrings from code review
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
* Add API docs and remove `from_cugraph` alias
* move cugraph tests to tests/cugraph/test_basics.py
* remove ununsed imports from test_basics.py
* remove pytest.importorskip as no longer needed
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* [CI] add new stage specific forcuda related features based on nvidia+pytorch
* build and test for gpu_nv
* fix build failure
* fix unit tests
* make -j
* install cython beforehand
* copy cython lib
* test cugraph tests only
* fix typo
* separate test script for cugraph
* refactor build dgl shell
* [Feature] extend sort_csr/csc_by_tag to edge
* fix test ffailure in tensorflow
* refine sorting by edges
* fix docstring
* remove unnecessary mem
Co-authored-by: Xin Yao <xiny@nvidia.com>
* * Workaround for graph data saving/loading compatibility problem in Column class. There may be more places in DGL with the same issue, due to using Python serialization, instead of a more cohesive, comprehensive strategy. This is just a local fix.
* Add checking for non-empty states
* Add unit test
* Handle the case of columns without storage
Co-authored-by: ndickson <ndickson@nvidia.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
* Update nccl communicator for when NCCL is missing
* Use static_cast
* Add doc string
* Fix whitespace
* Resrtict unit test to GPU runs
Co-authored-by: Xin Yao <xiny@nvidia.com>
* Add uva by default to embedding
* More updates
* Update optimizer
* Add new uva functions
* Expose new pinned memory function
* Add unit tests
* Update formatting
* Fix unit test
* Handle auto UVA case when training is on CPU
* Allow per-embedding decisions for whether to use UVA
* Address spares_optim.py comments
* Remove unused templates
* Update unit test
* Use dgl allocate memory for pinning
* allow automatically unpin
* workaround for d2h copy with a different dtype
* fix linting
* update error message
* update copyright
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* Explicitly unpin tensoradapter allocated arrays
* Undo unrelated change
* Add unit test
* update unit test
* add pinned_by_dgl flag to NDArray::Container
* use dgl.ndarray for holding the pinning status
* update multi-gpu uva inference
* reinterpret cast NDArray::Container* to DLTensor* in MoveAsDLTensor
* update unpin column and examples
* add unit test for unpin column
Co-authored-by: Dominique LaSalle <dlasalle@nvidia.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
Xin Yao