05c53ca38e
[Performance] Prefer parallelized conversion to CSC from COO instead of transposing CSR ( #2793 )
...
* fix coo2csr speed
* add comments
2021-04-05 16:38:36 +08:00
b2e35e6aa9
[Performance] Linear UniformChoice optimization ( #2710 )
...
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Zihao Ye <expye@outlook.com >
2021-04-01 16:29:49 +08:00
0b57ce1867
[Bug] Disable cpu fp16 ( #2783 )
...
* disable cpu fp16
* spell mistakes
2021-03-25 18:04:39 +08:00
929d863447
[Feature] Sparse-sparse matrix multiplication, addition, and masking ( #2753 )
...
* test
* more stuff
* add test
* fixes
* optimize algo
* replace unordered_map with arrays
* lint
* lint x2
* oops
* disable gpu csrmm tests
* remove gpu invocation
* optimize with openmp
* remove python functions
* add back with docstrings
* lint
* lint
* update python interface
* functionize
* functionize
* lint
* lint
2021-03-24 17:24:41 +08:00
0ff7127a0f
[Bugfix] Wrap cub with CUB_NS_PREFIX and remove dependency on Thrust to linking issues with Torch 1.8 ( #2758 )
...
* Wrap cub with prefixes and remove thrust
* Using counting iterator
Co-authored-by: Zihao Ye <expye@outlook.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-03-22 20:06:57 +08:00
74c38a1f58
Print error messages when using TCP socket ( #2763 )
...
* print error messages.
* fix.
2021-03-21 21:00:22 -07:00
c88fca5055
[Feature] Add edge coarsening for homogeneous undirected graphs ( #2691 )
...
* finish graph matching gpu version
* use C++ shuffle
* finish graph matching
* fix bug
* fix bug
* change name and use swap
* upt
* fix format problem
* fix format problem
* stronger test
* upt
* upt
* change python api
* upt
* upt
* format check
* upt
* upt
* fix bug
Co-authored-by: Tong He <hetong007@gmail.com >
2021-03-09 16:35:27 +08:00
62dd1c8689
fix doc typo ( #2721 )
2021-03-05 23:30:21 +08:00
e6bf54cdcb
[Feature] Support aggregate multiple edge features in to_simple. ( #2623 )
...
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* fix
* refactor
* upd test
* large feat_len or n in segment reduce
* lint
2021-02-21 10:05:42 +08:00
bc3a532f5e
[Sampling] Implement dgl.to_block() for the GPU ( #2339 )
...
* Add start of to_block gpu implementation
* Pull in more changes from 0.4.2 cuda_to_block
* Move more code to IdArray
* Refactor DeviceNodeMapMaker
* Updates
* get compiling
* Integrate to_block
* Fix ID allocation
* Minor fixes
* Cleanup cuda calls to use cuda_common
* Reduce kernel calls
* Lint cleanup
* Expand documentation
* Remove unused function
* Rename variables for consistency
* Add doxygen comments
* Fix file extension
* Remove raw asynccopy for deviceapi
* Remove unused function
* Fix block/tile configuration
* Add cuda_device_common.cuh
* Add basic hashtable
* Migrate part of hashtable
* Refactor to use external hashtable
* Make functions members
* Format hash table functions
* Migrate duplicate filling
* Move last function over
* Refactor with cu file
* lint c++ code
* Move context check to C++ code
* Use macro switch
* Add missing files
* Update docstring
* update docs
* Move atomic functions
* Refactor hashtable
* Fix linting
* Expand docs
* Fix mismatched argument names
* Switch doxygen comments from using @param to \param
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-02-08 13:00:43 +08:00
460bb42df7
fix build problems ( #2594 )
2021-01-29 17:46:31 +08:00
7bab1365e2
[feature] Supporting half precision floating data type (fp16). ( #2552 )
...
* add tvm as submodule
* compilation is ok but calling fails
* can call now
* pack multiple modules, change names
* upd
* upd
* upd
* fix cmake
* upd
* upd
* upd
* upd
* fix
* relative path
* upd
* upd
* upd
* singleton
* upd
* trigger
* fix
* upd
* count reducible
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* only keep related files
* upd
* upd
* upd
* upd
* lint
* lint
* lint
* lint
* pylint
* upd
* upd
* compilation
* fix
* upd
* upd
* upd
* upd
* upd
* upd
* upd doc
* refactor
* fix
* upd number
Co-authored-by: Zhi Lin <linzhilynn@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-78.us-east-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-21-156.us-east-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-01-28 11:21:58 +08:00
a7e941c379
[Feature] Add support for sparse embedding ( #2451 )
...
* Add sparse embedding for dgl and update rgcn example
* upd
* Fix
* Revert "Fix"
This reverts commit 4da87cdfb8b8c3506b7fc7376cd2385ba8045c2a.
* Fix
* upd
* upd
* Fix
* Add unitest and update impl
* fix
* Clean up rgcn example code
* upd
* upd
* update
* Fix
* update score
* sparse for sage
* remove model sparse
* upd
* upd
* remove global norm
* revert delete model_sparse.py
* update according to comments
* Fix doc
* upd
* Fix test
* upd
* lint
* lint
* lint
* upd
* upd
* clean up
Co-authored-by: Ubuntu <ubuntu@ip-172-31-56-220.ec2.internal >
2021-01-28 00:26:49 +08:00
2576647c1f
[Performance] Improve COO to CSR, and sort columns of CSR only when necessary. ( #2391 )
...
* Remove double-checking sorted
* Remove sorting of CSR by default
* Update unit test to use unsorted matix
* delete whitespace
* Expand unit tests
* Replace cusparse sort
* Fix row column sorting
* Explicitly don't sort columns
* Fix linting errors
* Fix bit-width calculation
* Fix sorting assertion and unit test
* Fix linting
* Improve CPU COO2CSR
* Remove references
* Rename and add documentation to edge encoding/decoding funcionts
* Fix sorting keys as 64 bit
* Revert cosmetic changes to unit tests
* Update documentation
* Update complexity documentation for coo to csr conversion
* Remove COOIsSorted check in CPU implementation too
2021-01-27 15:33:34 +08:00
0f9056edb9
[feature] Implement missing CUDA operators for COO format (part 1). ( #2565 )
...
* upd
* upd
* upd
* upd
* fix
* upd
* upd
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-01-25 16:15:22 +08:00
25ac334403
[Distributed] Heterogeneous graph support ( #2457 )
...
* Distributed heterograph (#3 )
* heterogeneous graph partition.
* fix graph partition book for heterograph.
* load heterograph partitions.
* update DistGraphServer to support heterograph.
* make DistGraph runnable for heterograph.
* partition a graph and store parts with homogeneous graph structure.
* update DistGraph server&client to use homogeneous graph.
* shuffle node Ids based on node types.
* load mag in heterograph.
* fix per-node-type mapping.
* balance node types.
* fix for homogeneous graph
* store etype for now.
* fix data name.
* fix a bug in example.
* add profiler in rgcn.
* heterogeneous RGCN.
* map homogeneous node ids to hetero node ids.
* fix graph partition book.
* fix DistGraph.
* shuffle eids.
* verify eids and their mappings when loading a partition.
* Id map from homogneous Ids to per-type Ids.
* verify partitioned results.
* add test for distributed sampler.
* add mapping from per-type Ids to homogeneous Ids.
* update example.
* fix DistGraph.
* Revert "add profiler in rgcn."
This reverts commit 36daaed8b660933dac8f61a39faec3da2467d676.
* add tests for homogeneous graphs.
* fix a bug.
* fix test.
* fix for one partition.
* fix for standalone training and evaluation.
* small fix.
* fix two bugs.
* initialize projection matrix.
* small fix on RGCN.
* Fix rgcn performance (#17 )
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal >
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix lint.
* fix.
* fix test.
* fix lint.
* test partitions.
* remove redundant test for partitioning.
* remove commented code.
* fix partition.
* fix tests.
* fix RGCN.
* fix test.
* fix test.
* fix test.
* fix.
* fix a bug.
* update dmlc-core.
* fix.
* fix rgcn.
* update readme.
* add comments.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal >
* fix.
* fix.
* add div_int.
* fix.
* fix.
* fix lint.
* fix.
* fix.
* fix.
* adjust.
* move code.
* handle heterograph.
* return pytorch tensor in GPB.
* remove some tests in example.
* add to_block for distributed training.
* use distributed to_block.
* remove unnecessary function in DistGraph.
* remove distributed to_block.
* use pytorch tensor.
* fix a bug in ntypes and etypes.
* enable norm.
* make the data loader compatible with the old format.
* fix.
* add comments.
* fix a bug.
* add test for heterograph.
* support partition without reshuffle.
* add test.
* support partition without reshuffle.
* fix.
* add test.
* fix bugs.
* fix lint.
* fix dataset.
* fix for mxnet.
* update docstring.
* rename to floor_div
* avoid exposing NodePartitionPolicy and EdgePartitionPolicy.
* fix docstring.
* fix error.
* fixes.
* fix comments.
* rename.
* rename.
* explain IdMap.
* fix docstring.
* fix docstring.
* update docstring.
* remove the code of returning heterograph.
* remove argument.
* fix example.
* make GraphPartitionBook an abstract class.
* fix.
* fix.
* fix a bug.
* fix a bug in example
* fix a bug
* reverse heterograph sampling.
* temp fix.
* fix lint.
* Revert "temp fix."
This reverts commit c450717b9f578b8c48769c675f2a19d6c1e64381.
* compute norm.
* Revert "reverse heterograph sampling."
This reverts commit bd6deb7f52998de76508f800441ff518e2fadcb9.
* fix.
* move id_map.py
* remove check
* add more comments.
* update docstring.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-171.ec2.internal >
2021-01-24 19:18:00 -08:00
9d90faf0b9
Use ALG2 for SpMM in cuSparse ( #2550 )
2021-01-21 11:12:30 +08:00
4d89b54efa
[Bug] Fix munmap() using wrong parameter ( #2519 )
...
* fix munmap() using wrong parameter
* rename variables
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-01-14 15:00:49 +08:00
4208ce2b9e
[Feature] Tvm integration ( #2367 )
...
Co-authored-by: Zihao Ye <expye@outlook.com >
2020-12-31 17:40:25 +08:00
3d1f2e870a
delete shared memory when receiving signals. ( #2419 )
...
* delete shared memory when receive signal.
* rename.
* fix lint.
* fix lint.
* fix compile.
* Fix.
* we need to report error if the shared memory exist.
* disable tensorflow test for shared memory.
* revert.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-12-26 10:41:10 -08:00
e805470101
addressing post-merge comments ( #2455 )
2020-12-26 17:58:15 +08:00
9a7235faf2
[Performance] Use allocator from PyTorch if possible ( #2328 )
...
* first commit
* some thoughts
* move around
* more commit
* more fixes
* now it uses torch allocator
* fix symbol export error
* fix
* fixes
* test fix
* add script
* building separate library per version
* fix for vs2019
* more fixes
* fix on windows build
* update jenkinsfile
* auto copy built dlls for windows
* lint and installation guide update
* fix
* specify conda environment
* set environment for ci
* fix
* fix
* fix
* fix again
* revert
* fix cmake
* fix
* switch to using python interpreter path
* remove scripts
* debug
* oops sorry
* Update index.rst
* Update index.rst
* copies automatically, no need for this
* do not print message if library not found
* tiny fixes
* debug on nightly
* replace add_compile_definitions to make CMake 3.5 happy
* fix linking to wrong lib for multiple pytorch envs
* changed building strategy
* fix nightly
* fix windows
* fix windows again
* setup bugfix
* address comments
* change README
2020-12-25 13:57:51 +08:00
e379e52585
[hotfix] Make USE_AVX a flag in cmake to avoid compilation error for arm user ( #2428 )
...
* upd cmake
* upd
* format
2020-12-17 17:29:15 +08:00
8ff4798075
[RPC] Make the port reusable after the socket is closed. ( #2418 )
...
* reuse.
* fix compile.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
2020-12-15 13:43:52 +08:00
9dff5419ea
[Performance][Hotfix] Disable openmp in arithmetic operation ( #2412 )
...
* disable openmp in arithmetic operation
* lint
* Update array_op_impl.cc
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-12-10 13:31:56 +08:00
27b0558465
change msg_seq from int to int_64 ( #2410 )
2020-12-09 13:55:30 +08:00
6b02babbad
[doc] Add docstring for segment reduce. ( #2375 )
2020-11-27 18:04:13 +08:00
5ac74f86a0
disable gespmm ( #2371 )
2020-11-26 15:59:46 +08:00
6897f55a3a
[Performance]: Remove indptr correction for CooToCSR ( #2356 )
2020-11-26 11:28:24 +08:00
3adbfa1812
[Performance] Use segment operators for graph readout. ( #2361 )
...
* upd
* upd
* update
* upd
* upd
* upd
* fix
* lint
* lint
* pylint
* doc
2020-11-23 01:45:33 +08:00
061c2a364e
upd ( #2352 )
2020-11-17 17:20:55 +08:00
f8ebcd7f90
[Performance] Dynamic cpu kernel V3 for SpMMSumCsr all Ops ( #2309 )
...
* support AVX512
* env DGL_CPU_INTEL_KERNEL_ENABLED=1
* env DGL_CPU_INTEL_KERNEL_LOG=1
* Add unittest test_spmm.cc
Co-authored-by: Izabela Mazur <izabela.mazur@intel.com >
Co-authored-by: Michal Szarmach <michal.szarmach@intel.com >
Review patch
2020-11-17 15:12:23 +08:00
501b2b75a5
[Bug] Multiple fixes for CUDA 11 support ( #2333 )
...
* multiple fixes
* fix CI
* fiddle
* revert stubs
* remove stubs
* poke
* remove linking of driver library
* minor
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-11-13 15:37:29 +08:00
d89f825d8c
upd ( #2336 )
2020-11-12 18:45:15 +08:00
92a3d07df2
[Kernel] Use tree reduction for SDDMM-dot ( #2335 )
...
* multiple fixes
* fix CI
* fiddle
* revert stubs
* upd
* upd
* unmerge
* unmerge
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-11-12 17:32:16 +08:00
272cb9e29a
[kernel] Select GE-SpMM when feature size is large. ( #2306 )
...
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
2020-11-06 13:29:38 +08:00
d453d72dd1
[Doc][Dataloading] Expand documentation of AsyncTransferer ( #2313 )
...
* Update docs
* Make non-default streams non-blocking
2020-11-02 12:28:30 -08:00
f673fc2553
[Dataloading] Add class for copying tensors to/from the GPU on a non-default stream ( #2284 )
...
* Add async transferer class
* Add async ndarray copy interface
* Add python bindings
* Fix comment
* Add python class
* Fix linting issues
* Add python unit test
* Update python interface
* move async_transferer to cuda only directory
* Fix linting issue
* Move out of contrib
* Add doc strings
* Move test compute from backend
* Update comment
* Fix test naming
* Fix argument usage
* Wrap/unwrap backend parameters
* Move to dataloading
* Move to 'dataloading'
* Make GPU/CPU compatible
* Fix unit tests
* Add docs
* Use only backend interface for datamovement in unit test
2020-10-30 07:29:06 -07:00
7b97298150
fix bug in flatten and is_unibipartite ( #2279 )
2020-10-17 12:11:24 +08:00
7d19b33c24
OMP pragma was added to support hetero_graph loop. ( #2148 )
...
* OMP pragma was added to support hetero_graph loop, as follows:
pragma omp parallel for
* OMP pragma will be enabled iff the gpu is used for
Co-authored-by: Zihao Ye <expye@outlook.com >
2020-10-10 01:43:12 +08:00
36daf66f62
update ( #2236 )
2020-09-25 14:48:46 +08:00
b25bbe642e
Loop reorder ( #2201 )
2020-09-15 20:10:43 +08:00
7d8522a29f
[Bug] fix cumsum on an empty array with prepend_zero returning an empty array ( #2179 )
...
* fix cumsum
* udp
Co-authored-by: Zihao <expye@outlook.com >
2020-09-11 18:23:44 +08:00
cbd55eb12f
[performance] Batch DGLGraph in C++ end. ( #2155 )
...
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* fix
* upd
Co-authored-by: VoVAllen <jz1749@nyu.edu >
2020-09-11 02:54:27 +08:00
ac570c1dae
[Bugfix] Fix flatten not wrapping unit graph ( #2170 )
...
* fix flatten not wrapping unit graph
* fix doc
2020-09-10 17:41:52 +08:00
2c04ecb55a
[hotfix] Skip CUDA kernel launch when number of blocks/threads is zero. ( #2144 )
...
* upd
* upd
* upd
* upd
* lint
* upd
* upd
* fmt
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2020-09-10 16:05:43 +08:00
567c5acf4e
[Feature?] BUG_ON macro and sanity check of FindEdges ( #2168 )
...
* Sanity check for FindEdges
* [Feature?] BUG_ON() macro for sanity check
2020-09-10 15:28:12 +08:00
7816c5a249
[hotfix] IsContiguous failed when tensor's size on first dimension is 1. ( #2127 )
...
* udp
* fix
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-08-30 09:26:03 +08:00
3808dc950b
[hotfix] Disable hypersparse memory optimization. ( #2121 )
...
* upd
* upd
2020-08-28 15:07:53 +08:00
715b3b167d
upd ( #2117 )
2020-08-27 21:15:53 +08:00
Quan (Andy) Gan