5be937a7fb
[Kernel] Slicing Batched Graphs ( #2965 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update
* Update
* Add files via upload
* Add files via upload
* Update
* Lint
* Add files via upload
* Lint
* Update
* Update
* Update
* Update
* Update
* Lint Fix
* Lint
Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-161.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-06-10 12:07:44 +08:00
2df4a95f3e
[Feature] Make to_heterogeneous(to_homogeneous(hg)) return hg ( #2958 )
...
* make to_heterogeneous and to_homogeneous invertible
* docstring
* oops
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-06-03 18:50:18 +08:00
b8fe2b4837
[Feature][Sampler] Sort CSR by tag ( #1664 )
...
* update
* update
* update
* update
* lint
* lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* lint
* update
* clone
* update
* update
* update
* update
* replace idarray with ndarray
* refactor cpp part
* refactor python part
* debug
* refactor interface
* test and doc
* lint and test
* lint
* fix
* fix
* fix
* const
* doc
* fix
* fix
* fix
* fix
* fix & doc
* fix
* fix
* fix
* fix
* fix
* fix
* update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-01 23:33:56 +08:00
2ad7a9e9dd
[Fix] Fix #2808 ( #2917 )
...
* Fix #2808
* fix
* fix
* fix
* fix
* fix
* fix
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-05-31 19:05:58 +08:00
ae8dbe6d3c
[Feature][Performance] Implement NCCL wrapper for communicating NodeEmbeddings and sparse gradients. ( #2825 )
...
* Split NCCL wrapper from sparse optimizer and sparse embedding
* Add more unit tests for single node nccl
* Fix unit test for tf
* Switch to device histogram
* Fix histgram issues
* Finish migration to histogram
* Handle cases with zero send/recieve data
* Start on partition object
* Get compiling
* Updates
* Add unit tests
* Switch to partition object
* Fix linting issues
* Rename partition file
* Add python doc
* Fix python assert and finish doxygen comments
* Remove stubs for range based partition to satisfy pylint
* Wrap unit test in GPU only
* Wrap explicit cuda call in ifdef
* Merge with partition.py
* update docstrings
* Cleanup partition_op
* Add Workspace object
* Switch to using workspace object
* Move last remainder based function out of nccl_api
* Add error messages
* Update docs with examples
* Fix linting erros
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-05-20 10:58:17 -07:00
6e7f19f27b
[Distributed] add distributed in-degree and out-degree. ( #2918 )
...
* add distributed in-degree and out-degree.
* update comments.
* fix a bug.
* add tests.
* add tests.
* fix a bug.
* fix docstring.
* update doc.
* fix
* fix.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-05-18 19:46:09 +08:00
657c220de7
[Feature] Python interface for adjacency matrix summation and multiplication ( #2893 )
...
* test commit
* fixes
* oops
* add docs
* lint
* why does it say I have a trailing whitespace
* oh ok
* fixes
* why there's an invalid argument error
* address comments
* fix
* address comments
2021-05-17 19:47:02 +08:00
103444c58f
Remove __len__ method from DGLGraph ( #2902 )
...
* Update heterograph.py
* remove unit tests
* replace tutorial
2021-05-11 14:06:18 +08:00
ab2bd1f13d
[Feature] Add cuda support for Sparse Matrix multiplication, summation and masking ( #2782 )
...
* init cuda support
* cuSPARSE err
* passed unittest for csr_mm/SpGEMM. int64 not supported
* Debugging cuSPARSE error 3
* csrgeam only supports int32?
* disabling int64 for cuda
* refactor and add CSRMask
* lint
* oops
* remove todo
* rewrite CSRMask with CSRGetData
* lint
* fix test
* address comments
* lint
* fix
* addresses comments and rename BUG_ON
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-71.ec2.internal >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-04-27 16:06:35 +08:00
d76af4d4c1
[Distributed] Return the ID mapping in graph partitioning. ( #2857 )
...
* return mapping.
* support heterogeneous graph.
* more test.
* fix lint.
* fix for diff backends.
* fix.
* fix.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-04-22 17:51:42 +08:00
e70138bbe4
[Performance][GPU] Enable GPU uniform edge sampling ( #2716 )
...
* Start on uniform GPU sampling
* Save more work
* Get cu file compiling
* Update sampling
* More changes
* Get GPU sampling for uniform probabilities solved
* Fix batch tensor migration
* Fix
* update kernels
* expand blocking
* Undo testing change
* Cut down on sampling overhead
* Fix replacement
* Update unit tests
* Add option to gpu sample in graphsage
* Copy only csc to gpu
* Add ogbn support
* Fix linting
* Remove nvtx from sample
* Improve documentation and error checking
* Expand documentation
* Update assert checking
* delete extra space
* Use standard dataloader when dataset is a dictionary
* ogb -> ogbn
* Fix edge selection determinism
* Fix typos
* Remove nvtx
* Add comment for self.fanout_arrays and assert
* Fix linting
* Migrate to scalarbatcher
* Fix indentation
* Fix batcher
* Fix indexing
* Only use databatcher for GPU
* Convert to DGL NDArray to PyTorch Tensor
* Add optimization for PyTorch's F.tensor() for list of GPU tensors
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-04-14 18:55:09 -07:00
e6bf54cdcb
[Feature] Support aggregate multiple edge features in to_simple. ( #2623 )
...
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* fix
* refactor
* upd test
* large feat_len or n in segment reduce
* lint
2021-02-21 10:05:42 +08:00
bc3a532f5e
[Sampling] Implement dgl.to_block() for the GPU ( #2339 )
...
* Add start of to_block gpu implementation
* Pull in more changes from 0.4.2 cuda_to_block
* Move more code to IdArray
* Refactor DeviceNodeMapMaker
* Updates
* get compiling
* Integrate to_block
* Fix ID allocation
* Minor fixes
* Cleanup cuda calls to use cuda_common
* Reduce kernel calls
* Lint cleanup
* Expand documentation
* Remove unused function
* Rename variables for consistency
* Add doxygen comments
* Fix file extension
* Remove raw asynccopy for deviceapi
* Remove unused function
* Fix block/tile configuration
* Add cuda_device_common.cuh
* Add basic hashtable
* Migrate part of hashtable
* Refactor to use external hashtable
* Make functions members
* Format hash table functions
* Migrate duplicate filling
* Move last function over
* Refactor with cu file
* lint c++ code
* Move context check to C++ code
* Use macro switch
* Add missing files
* Update docstring
* update docs
* Move atomic functions
* Refactor hashtable
* Fix linting
* Expand docs
* Fix mismatched argument names
* Switch doxygen comments from using @param to \param
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-02-08 13:00:43 +08:00
0346b0aa80
[Test] Add test for Cython and force rebuild when run setup.py ( #2635 )
2021-02-05 22:38:46 +08:00
fb4a0508c6
[bugfix] Solve the boundary issue in backward function of segment sum ( #2610 )
...
* upd
* trigger
* upd
2021-02-03 18:50:07 +08:00
0f9056edb9
[feature] Implement missing CUDA operators for COO format (part 1). ( #2565 )
...
* upd
* upd
* upd
* upd
* fix
* upd
* upd
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-01-25 16:15:22 +08:00
2288098ba6
[Feature] Converting a block to a graph and allow block construction ( #2555 )
...
* first commit
* more commit
* fix and docstring
* add to api reference
* address comments
* Update convert.py
2021-01-24 12:44:54 +08:00
8c8c5b1dc1
[Bug fix] Stop erasing schemas of frames with no data rows ( #2529 )
...
* added option for batching empty data, fixes #2526
* added option for batching empty data, fixes #2526
* decreased line lengths
* removed trailing whitespace
* fixed wrong feature name
* [Bugfix] stop erasing schema of frames w/ no cols (fixes #2528 )
* removed arg doc
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-01-17 10:32:47 +08:00
0778766411
[Bug fix] [Feature] added option for batching empty data ( #2527 )
...
* added option for batching empty data, fixes #2526
* added option for batching empty data, fixes #2526
* decreased line lengths
* removed trailing whitespace
* fixed wrong feature name
* now default behavior when all graphs are empty
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-01-14 15:25:13 +08:00
2caac086c2
[Bug] send_and_recv and pull may write to wrong places ( #2497 )
...
* fix
* fix test
2021-01-06 17:18:37 +08:00
0c2a2ea1e2
[Bugfix] Handle a Corner Case of Batching after Removing Nodes/Edges ( #2465 )
...
* Update
* Update
* Update
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-01-05 22:22:03 +08:00
f960468f50
[feature] APIs for manually set batch_num_nodes and batch_num_edges ( #2430 )
...
* wip
* udp
* upd
* fix typo
* lint
* lint
* upd
* upd
* lint
* lint
* upd
* upd
* tftest
* fix
* fallback numpy
* fix tf
* docstring
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2021-01-04 23:43:58 +08:00
d3c1645550
[Performance][Test] New low memory implementation for RGCN and related regression tests ( #2468 )
...
* WIP
* finish lowmem impl; benchmarking
* wip
* wip
* fix benchmarks
* fix bug in searchsorted
* update readme
* update
* lint and ogb dependency
* add flags to to_homogeneous
* fix docstring
* address comments
2021-01-04 22:23:59 +08:00
3d1f2e870a
delete shared memory when receiving signals. ( #2419 )
...
* delete shared memory when receive signal.
* rename.
* fix lint.
* fix lint.
* fix compile.
* Fix.
* we need to report error if the shared memory exist.
* disable tensorflow test for shared memory.
* revert.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-12-26 10:41:10 -08:00
3adbfa1812
[Performance] Use segment operators for graph readout. ( #2361 )
...
* upd
* upd
* update
* upd
* upd
* upd
* fix
* lint
* lint
* pylint
* doc
2020-11-23 01:45:33 +08:00
6eda605ccd
[BugFix][Dataloading] Disable AsyncTransferer copying to anywhere but the GPU ( #2323 )
...
* Disable copying for anywhere but the GPU
* Remove unused import and remove references to transferring from the GPU from the docs
* Skip gpu test in cpu mode
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-11-07 21:34:54 +08:00
5f19876358
[Bugfix] GINDataset check whether labels are all valid ( #2319 )
...
* PPIDataset
* Revert "PPIDataset"
This reverts commit 264bd0c960cfa698a7bb946dad132bf52c2d0c8a.
* check whether the original labels are valid
* add unit test for gin dataset
* bug of asscalar in mxnet<=1.6
* mxnet<=1.6 asscalar requires ndarray to be shaped (1,)
* skip gpu while testing datasets
* Update test_data.py
* test of gin dataset takes too much time
2020-11-05 09:35:10 +08:00
f673fc2553
[Dataloading] Add class for copying tensors to/from the GPU on a non-default stream ( #2284 )
...
* Add async transferer class
* Add async ndarray copy interface
* Add python bindings
* Fix comment
* Add python class
* Fix linting issues
* Add python unit test
* Update python interface
* move async_transferer to cuda only directory
* Fix linting issue
* Move out of contrib
* Add doc strings
* Move test compute from backend
* Update comment
* Fix test naming
* Fix argument usage
* Wrap/unwrap backend parameters
* Move to dataloading
* Move to 'dataloading'
* Make GPU/CPU compatible
* Fix unit tests
* Add docs
* Use only backend interface for datamovement in unit test
2020-10-30 07:29:06 -07:00
7b97298150
fix bug in flatten and is_unibipartite ( #2279 )
2020-10-17 12:11:24 +08:00
c341520d5b
[bugfix] Fix the behavior of min/max reducer for 1-dim dta. ( #2250 )
...
* udp
* add test
* udp
* fix mxnet
2020-09-29 17:27:16 +08:00
40caf1ab50
[HeteroGraph] Fix the failure of apply_nodes when the input function changes feature size for all nodes ( #2223 )
...
* Fix
* Fix
2020-09-24 08:54:28 +08:00
7d8522a29f
[Bug] fix cumsum on an empty array with prepend_zero returning an empty array ( #2179 )
...
* fix cumsum
* udp
Co-authored-by: Zihao <expye@outlook.com >
2020-09-11 18:23:44 +08:00
8f4594074b
[Hotfix] Fix degree bucket edge ordering ( #2176 )
...
* fix degree bucket edge ordering
* unit test
* fix
2020-09-11 14:16:52 +08:00
cbd55eb12f
[performance] Batch DGLGraph in C++ end. ( #2155 )
...
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* fix
* upd
Co-authored-by: VoVAllen <jz1749@nyu.edu >
2020-09-11 02:54:27 +08:00
ac570c1dae
[Bugfix] Fix flatten not wrapping unit graph ( #2170 )
...
* fix flatten not wrapping unit graph
* fix doc
2020-09-10 17:41:52 +08:00
c9c6171b9f
Revert "[Feature] Basic utils to handle raw data features ( #2102 )" ( #2147 )
...
This reverts commit 33a8bb93bfwmjlyjemaine@gmail.com >
2020-09-08 10:33:59 +08:00
ac5c79cdb8
[Bug] Fixes issues in feature pickling and transmission for subgraphs and blocks ( #2139 )
...
* bug fixes
* remove __deepcopy__; not sure what the behavior should be
* lint
* skip gpu test
* fix
* fix dist dataloader
* add comment
* remove assert
2020-09-03 12:48:59 +08:00
9eaace9216
rename create_format_ to create_formats_ ( #2126 )
2020-08-29 22:49:27 +08:00
642bdbaf04
[Traversal] Fall back to CPU for Graph Traversal as a Workaround ( #2115 )
...
* workaround
* Allow Test for Traversal on GPU Graphs
Co-authored-by: Ubuntu <ubuntu@ip-172-31-1-5.us-west-2.compute.internal >
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2020-08-29 16:34:31 +08:00
3808dc950b
[hotfix] Disable hypersparse memory optimization. ( #2121 )
...
* upd
* upd
2020-08-28 15:07:53 +08:00
33a8bb93bf
[Feature] Basic utils to handle raw data features ( #2102 )
...
* add feature utils and add test for feature norm
* Add docstring and test
* upd
* dis able some test
* Update
* update doc string
* update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-68-185.ec2.internal >
2020-08-27 11:36:39 +08:00
2ce426d956
[Bug] add_edges() crashes if the input tensor is empty ( #2100 )
...
* [Bug] add_edges() crashes if the input tensor is empty
* lint
* fix
2020-08-24 16:04:41 +08:00
3af0e91c0e
[Frame] Fix for Column Index Device ( #2055 )
...
* Update frame.py
* add unit test
* fix test
* fix
* fix
* fix
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-08-19 16:10:52 +08:00
dacc7afeb2
[Convert] Reserve node/edge IDs in to_heterogeneous ( #2045 )
...
* Fix
* Fix
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-08-18 23:16:33 +08:00
9088c63592
[Feature] Change the Default Value of transpose in DGLGraph.adj ( #2046 )
...
* Update adj
* Fix
2020-08-18 22:45:46 +08:00
47c96dd114
[Test] Remove Test for metapath_random_walk ( #2039 )
...
* Update test_randomwalk.py
* Update test_randomwalk.py
2020-08-18 11:19:31 +08:00
aa19df1bbb
[Bugfix] Fix batching isolated node types ( #2035 )
...
* [Bugfix] Fix batching isolated ntypes
* fix test
* remove test.py
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-08-18 09:29:05 +08:00
be444e52d9
[Doc/Feature] Refactor, doc update and behavior fix for graphs ( #1983 )
...
* Update graph
* Fix for dgl.graph
* from_scipy
* Replace canonical_etypes with relations
* from_networkx
* Update for hetero_from_relations
* Roll back the change of canonical_etypes to relations
* heterograph
* bipartite
* Update doc
* Fix lint
* Fix lint
* Fix test cases
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Update
* Fix test
* Fix
* Update
* Use DGLError
* Update
* Update
* Update
* Update
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Fix
* Update
* Update
* Update
* Update
* Update
* Update
* rewrite sanity checks
* delete unnecessary checks
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Update
* Update
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Fix
* Update
* Fix
* Update
* Fix
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-08-18 04:26:29 +08:00
cd48435206
[Transform] Docstring and subframes ( #1962 )
...
* update knn graph docs
* more docs
* [Doc] transform module docstrings
* remove copy_ndata and copy_edata
* fix
* lint
* fix
* fix
* fix
* clean up docstrings
* fix docstring
* dtype specifications
* addresses comments
* fix
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-08-13 10:41:43 +08:00
6294677f8a
[hotfix] Set reduce results to all zero for nodes with zero in-degrees. ( #2011 )
2020-08-13 01:20:56 +08:00
Mufei Li