* Split from NCCL PR
* Fix type in comment
* Expand documentation for sparse_all_to_all_push
* Restore previous behavior in example
* Re-work optimizer to use NCCL based on gradient location
* Allow for running with embedding on CPU but using NCCL for gradient exchange
* Optimize single partition case
* Fix pylint errors
* Add missing include
* fix gradient indexing
* Fix line continuation
* Migrate 'first_step'
* Skip tests without enough GPUs to run NCCL
* Improve empty tensor handling for pytorch 1.5
* Fix indentation
* Allow multiple NCCL communicator to coexist
* Improve handling of empty message
* Update python/dgl/nn/pytorch/sparse_emb.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* Update python/dgl/nn/pytorch/sparse_emb.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* Keepy empty tensor dimensionaless
* th.empty -> th.tensor
* Preserve shape for empty non-zero dimension tensors
* Use shared state, when embedding is shared
* Add support for gathering an embedding
* Fix typo
* Fix more typos
* Fix backend call
* Use NodeDataLoader to take advantage of ddp
* Update training script to share memory
* Only squeeze last dimension
* Better handle empty message
* Keep embedding on the target device GPU if dgl_sparse if false in RGCN example
* Fix typo in comment
* Add asserts
* Improve documentation in example
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* Make multigpu graphsage use whole datset
* Specify queeze dimension
* Remove squeeze dimension
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Added a special implementation for MetapathBasedRandomWalkStep for the Uniform randomwalk case
* Fixed all linting issues
* add comment
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* [CPU Parallel] implementation of parallel_for with grain size
* Create parallel_for.h
* Update parallel_for.h
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [Doc] modify the dimension of weight to Numpy broadcasting rule
* [Doc] modify the user defined reduce function
Co-authored-by: zhjwy9343 <6593865@qq.com>
* SpMM for heterograph
* C APIs SDDMM heterograph
* passes initial result
* renamed eid with nid
* aggregation on same ntype for multiple etypes
* fix link check failure
* lint check part 2
* lint check part 3
* Fixed SpMMCmpCsr Min op
* added mem references
* fixed fill(Max/Min), added const
* removed newline
* brought back docstring
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* add two fraud dataset
* add comments and modify to heterogenous graph
* modify the node name based on datasets
* add assertion message
Co-authored-by: zhjwy9343 <6593865@qq.com>
* Add NCCL as a submodule
* Allow using third_party/nccl or system nccl
* Add nccl_external as a dependency
* Fix conditional
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* PPIDataset
* Revert "PPIDataset"
This reverts commit 264bd0c960cfa698a7bb946dad132bf52c2d0c8a.
* data pipeline user guide
* remove chapter numbers
* Update data.rst
* image in dataset userguide
* update links using ref
* modify the link of save_graphs and load_graphs in dataset user guide
* move image to s3 server.
* fix links and ref
* Hide implementations that may cause confusions to the user
* add papers
* fix number of classes in Amazon Cobuy datasets
* add two papers to Awesome paper list
* Update README.md
* add four papers
* add two papers
* Add awesome papers
* Fix index in paper list
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* Split NCCL wrapper from sparse optimizer and sparse embedding
* Add more unit tests for single node nccl
* Fix unit test for tf
* Switch to device histogram
* Fix histgram issues
* Finish migration to histogram
* Handle cases with zero send/recieve data
* Start on partition object
* Get compiling
* Updates
* Add unit tests
* Switch to partition object
* Fix linting issues
* Rename partition file
* Add python doc
* Fix python assert and finish doxygen comments
* Remove stubs for range based partition to satisfy pylint
* Wrap unit test in GPU only
* Wrap explicit cuda call in ifdef
* Merge with partition.py
* update docstrings
* Cleanup partition_op
* Add Workspace object
* Switch to using workspace object
* Move last remainder based function out of nccl_api
* Add error messages
* Update docs with examples
* Fix linting erros
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* add bruteforce impl
* add support for bruteforce-sharemem
* modify python API
* add tests
* change file path
* change python API
* fix lint
* fix test
* also check worst_dist in the last few dim
* use heap and early-stop on CPU
* fix lint
* fix lint
* add device check
* use cuda function to determine max shared mem
* use cuda to determine block info
* add memory free for tmp var
* update doc-string and add dist option
* fix lint
* add more tests
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Tomasz Patejko