文件历史

提交图

951 次代码提交

作者 SHA1 备注 提交日期
Xin Yao d3ae7544cd [Feature] aten::Relabel_() for the GPU (#3445)
* relabel gpu

* unittest for ralebl_ on the GPU

* finish Relabel_ for the GPU

* copyright

* re-enable the unittest for edge_subgrah on the GPU

* fix unittest for tensorflow

* use a fixed number of threads

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-11-04 08:58:31 -07:00
Mufei Li f46080a4d1 [Feature] k-hop Subgraph Extraction (#3458)
* Update

* Fix

* Fix

* Update

* Update

* Update

* Fix CI

* Fix

* Fix

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Fix for TF
2021-11-04 15:47:35 +08:00
Shaked Brody e2f33fd5cc [NN][Model] GATv2 (#3473)
* [Model][Core] GATv2

* lint

* gatv2conv.py

* lint

* lint

* style and docs

* lint

* gatv2conv fix

Co-authored-by: Shaked Brody shakedbr@campus.technion.ac.il <shakedbr@tangerine.cslcs.technion.ac.il>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2021-11-03 21:55:56 +08:00
Kamil Kamiński 51c6509704 [NN] Add EGATConv nn.module (#3425)
* added nn pytorch egatconv

* aligned with test build

* aligned with test build

* fixed wihite spaces

* fixed wihite spaces

* fixed wihite spaces

* added missing egatconv in imports

* added indentation in forward

* GATConv based implementation

* removed **kw_args

* added dgl relative imports

* PR corrections

* added DGL Error to EGATConv imports

* Update test_nn.py

Co-authored-by: Argusmocny <k.kaminski@cent.uw.edu.pl>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2021-10-28 00:38:08 +08:00
Xin Yao a8c81018c5 [Sampling] Implement dgl.compact_graphs() for the GPU (#3423)
* gpu compact graph template

* cuda compact graph draft

* fix typo

* compact graphs

* pass unit test but fail in training

* example using EdgeDataLoader on the GPU

* refactor cuda_compact_graph and cuda_to_block

* update training scripts

* fix linting

* fix linting

* fix exclude_edges for the GPU

* add --data-cpu & fix copyright
2021-10-20 22:07:35 -07:00
Rhett Ying 8798872f54 [Bug] Do not skip graphconv even no edge exists (#3416) 2021-10-14 14:34:42 +08:00
Rhett Ying 7c7b60be18 [BugFix] add count_nonzero() into SA_Client (#3417) 2021-10-12 06:17:30 -07:00
Rhett Ying 2d88db5a3c [Bug] check dtype before convert to gk (#3414) 2021-10-12 14:34:40 +08:00
Rhett Ying f9fd7fd7f7 [BugFix] extract gz into target dir (#3389) 2021-09-30 11:35:40 +08:00
Rhett Ying e234fcfa8f [Feature] enable create/set/free cuda stream for internal use (#3334)
* [Feature] enable create/set/free cuda stream for internal use

* add unit test

* fix unit test failure on mxnet and tf

* refactor stream wrapper

* fix lint error

* fix lint error
2021-09-29 15:35:02 +08:00
Jingcheng Yu 5cf48fc69c [Feature] Implement one thread multiple socket (#3200)
Co-authored-by: JingchengYu94 <jingchengyu94@gmail.com>
2021-09-27 21:45:52 -07:00
Quan (Andy) Gan a04a8d066e [Feature] Graceful handling of exceptions thrown within OpenMP blocks (#3353)
* graceful c++ exception in OpenMP

* credits

* add test

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-09-22 13:44:46 +08:00
mszarma bc14829fb3 [Feature] Exclude edges in sample_neighbors (#2971)
* [Feature] Exclude edges in sample_neighbors

Extending sample_neighbors and sample_frontier
API to support exclude_edges parameter.

exclude_edges support tensor and dict data
Feature enable excluding certain edges
during neighborhood sampling
Exclude_edges contains EID's of edges
which will be excluded
during neighbor picking for seed nodes.

Added test case for heterograph and homograph
RFC issue id: 2944

* compatibility

* fix

* fix

Co-authored-by: Quan Gan <coin2028@hotmail.com>
2021-09-22 02:15:56 +08:00
Rhett Ying bacc9047f2 [BugFix] initialize data if null when converting from row sorted coo to csr (#3360) 2021-09-17 17:07:05 +08:00
nv-dlasalle 2647afc9b3 [Performance][Feature] Add src_nodes paramter to to_block() to avoid cost running unique() when available. (#2973)
* Add lhs_nodes are paremeter to to_block

* Update unit test

* Switch to simplified node conversion

* Switch lhs_nodes to be in/out parameter

* Update docs

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2021-09-16 09:05:16 +08:00
Rhett Ying f4c79f7f6d [Performance] improve coo2csr space complexity when row is not sorted (#3326)
* [Performance] improve coo2csr space complexity when row is not sorted

* [Perf] replace std::vector<> by NDArray

* keep both impl of unsorted coo to csr and choose according to graph density dynamically

* refine criteria to choose btw Unsorted algos

Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-27.us-west-2.compute.internal>
2021-09-14 10:41:55 +08:00
esang 6454c791b8 [Bugfix] Fix bugs of farthest_point_sampler (#3327)
* fix start_idx

* fix the bug when cuda > 0

Co-authored-by: Tong He <hetong007@gmail.com>
2021-09-10 17:50:06 +08:00
Israt Nisa d17b299a13 [Feature] Update edge features (apply_edge) for heterogeneous graph (#3250)
* rgcn with new heterograph API

* apply_edge() forward for multi relation

* undoing changes from rgcn-hetero

* backward apply_edge(copy_u) added

* unittest for apply_edge(copy_e)

* Compatible with new PRs

* resolving conflict with master

* Bringing back change after resolving conflict

* minor

* minor

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2021-09-09 22:39:08 +08:00
Israt Nisa 298e4fa627 [Feature] Support builtin binary message function for heterogenenous graph (#3273)
* Added binary builtinMsgFunc forward() for heterograph

* Added backward for u_op_v

* Supports all binary builtin forward

* Supports binary message funcs with reduce func sum

* lint check

* removed import torch from unittest

* enabled GPU test

* lint check

* Fixed docstrings

* rename func get_hs_id

* edited comment

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2021-09-07 10:01:33 -07:00
Jinjing Zhou c81efdf215 Remove deprecated kernels (#3316)
* remove

* remove

* fix

* remove

* remove
2021-09-06 13:59:41 +08:00
Rhett Ying 5a2451047f [Feature] enable to specify stream in UnitGraph::CopyTo() which could lead to async copy (#3297)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2021-09-01 17:57:20 +08:00
xiang song(charlie.song) f4fe518ff7 [Feature] Add a HINT for the per edge type sampler of heterogeneous DistGraph that highlighting the etypes are sorted already. (#3260)
* pass cpp test

* distgraph use sorted edge flag.

* lint

* triger

* update test

Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
2021-09-01 15:36:01 +08:00
nv-dlasalle a53783c81d [Performance] Only copy and slice index's in subcolumn when data is used. (#2930)
* Make column double indices lazy

* Copy indices to proper contexts

* Fix initialization

* Add unit test

* Fix unit test for tensorflow

* Remove unused member

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-08-30 19:17:25 -07:00
Da Zheng e5ed7adc12 [Distributed] Fix a bug in sampling an empty frontier (#3298)
* handle empty frontiers.

* fix lint.

* fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2021-08-29 10:54:46 -07:00
nv-dlasalle 7f9279397a [Feature][DistDGL] Add NCCL support for range based partitions (#3213)
* Implement range based NDArrayPartition

* Finish implement range based partition support

* Add unit test

* Fix whitepace

* Add Kernel suffix

* Fix argument passing

* Add doxygen docs and improve variable naming

* Add unit test

* Add function for converting a partition book

* Add example to partition_op docs

* Fix dtype conversion for mxnet and tensorflow
2021-08-19 21:17:55 -07:00
nv-dlasalle f634950825 [Performance][Feature] Implement edge excluding in EdgeDataLoader on GPU (#3226)
* Update filter code

* Add unit tests

* Fixes

* Switch to indices

* Rename functions

* Fix linting

* Fix whitespace

* Add doc

* Fix heterograph

* Change workspace allocation

* Fix linting

* Fix docs in filter.py

* Add todo

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-08-19 12:24:15 -07:00
Quan (Andy) Gan a536772ed2 fix cuda 11.1 crashing bug (#3265) 2021-08-18 21:16:49 +08:00
Eric Kim ac01e88082 [Tools] In tools/launch.py, correctly pass all DGL client/server env vars if udf is a multi-command (#3245)
* Correctly pass all DGL client/server env vars if udf is a multi-command

* Refactor to use wrap_cmd_with_local_envvars() helper fn
2021-08-16 18:54:48 -07:00
xiang song(charlie.song) bcf92f6c21 [Bugfix] Distributed training can not work with dgl.dataloading.negative_sampler (#3215)
* Fix dist negative data loader bug

* upd

* Fix

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2021-08-06 20:41:20 +08:00
Eric Kim c40bbf4f33 [Tools] Refactor tools/launch.py to handle more python binary names (#3205)
* Refactors torch dist launcher udf-wrap code to handle more python versions

* minor changes
2021-08-02 17:31:02 +08:00
David Min c793593577 [Feature] Add multi-GPU UnifiedTensor unit test (#3184)
* Add pytorch-direct version

* remove

* Add multi-gpu unified tensor test for pytorch

* relocate verification step to each process

* reduce number of workers

* add parameter

Co-authored-by: shhssdm <shhssdm@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-08-02 12:41:03 +08:00
Rhett Ying c4791fd46e [bugfix] fix default ntypes/etypes consistency between dgl.DGLGraph and dgl.graph (#3198) 2021-07-29 14:05:28 +08:00
Da Zheng cb4a83a233 [Distributed] Enable distributed EdgeDataLoader (#3192)
* make heterogeneous find_edges

* add distributed EdgeDataLoader.

* fix.

* fix a bug.

* fix bugs.

* add tests on distributed heterogeneous graph sampling.

* fix.

Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
2021-07-28 15:28:27 +08:00
xiang song(charlie.song) ba7e7cf931 [New Feature] Per edge type sampler for to_homogeneous graphs. (#3131)
* fix.

* fix.

* fix.

* fix.

* Fix test

* Deprecate old DistEmbedding impl, use synchronized embedding impl

* Basic imple of heterogeneous on homogenenous sampling

* make pass

* Pass C++ test

* Add python test code

* lint

* lint

* Add MultiLayerEtypeNeighborSampler

* Add unitest for single machine dataloader

* Add dist dataloader test for edge type sampler

* Fix lint

* fix

* support for per etype sample

* Fix some bug and enable distributed training with per edge sample

* fix

* Now distributed training works

* turn off some mxnet

* turn off mxnet for some dist test

* fix

* upd

* upd according to the comments

* Fix

* Fix test and now distributed works.

* upd

* upd

* Fix

* Fix bug

* remove dead code.

* upd

* Fix

* upd

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2021-07-28 11:51:32 +08:00
Da Zheng 34426a98e9 [Distributed] Distributed heterograph training (#3069)
* support hetero RGCN.

* fix.

* simplify code.

* sample_neighbors return heterograph directly.

* avoid using to_heterogeneous.

* compute canonical etypes in advance.

* fix tests.

* fix.

* fix distributed data loader for heterograph.

* use NodeDataLoader.

* fix bugs in partitioning on heterogeneous graphs.

* fix lint.

* fix tests.

* fix.

* fix.

* fix bugs.

* fix tests.

* fix.

* enable coo for distributed.

* fix.

* fix.

* fix.

* fix.

* fix.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
2021-07-17 10:57:00 +08:00
David Min 905c0aa578 [Feature][Performance][GPU] Introducing UnifiedTensor for efficient zero-copy host memory access from GPU (#3086)
* Add pytorch-direct version

* Initial commit of unified tensor

* Merge branch 'master' of https://github.com/davidmin7/dgl

* Remove unnecessary things

* Fix error message

* Fix/Add descriptions

* whitespace fix

* add unpin

* disable IndexSelectCPUFromGPU with no CUDA

* add a newline for unified_tensor.py

* Apply changes based on feedback

* add 'os' module

* skip unified tensor unit test for cpu only

* Update tests/pytorch/test_unified_tensor.py

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* reflect feedback

Co-authored-by: shhssdm <shhssdm@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2021-07-17 00:06:20 +08:00
xiang song(charlie.song) d7390763f0 [Distributed] Deprecate old DistEmbedding impl, use synchronized embedding impl (#3111)
* fix.

* fix.

* fix.

* fix.

* Fix test

* Deprecate old DistEmbedding impl, use synchronized embedding impl

* update doc

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-07-14 00:15:53 +08:00
Mufei Li f0fafa2062 [Bug fix] Fix batch information with remove_nodes/edges applied (#3119)
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix lint

* Fix

* Update

* Fix test cases

* Fix

* add docstrings

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-07-13 16:27:31 +08:00
Quan (Andy) Gan 2e19ba8025 add exclude self option for EdgeDataLoader (#3122)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2021-07-13 13:30:16 +08:00
Quan (Andy) Gan b576e617ad [Feature] Add left normalizer for GCN (#3114)
* add left normalizer for gcn

* fix

* fixes and some bug stuff
2021-07-13 12:49:35 +08:00
Rhett Ying 186ef59283 [Feature] apply dgl.reorder() onto several node classification datase… (#3102)
* [Feature] apply dgl.reorder() onto several node classification datasets in DGL

* rebase on latest dgl.reorder_graph()
2021-07-13 08:52:17 +08:00
Rhett-Ying 175f53decf [Feature] enable edge reorder in dgl.reorder_graph() (#3113)
* [Feature] enable edge reorder in dgl.reorder_graph()

* refine doc string

* refine doc string for dgl.reorder_graph

* refine doc string further
2021-07-12 14:11:46 +08:00
Rhett-Ying 32d1f3ac97 [BugFix] skip frames whose num_rows is zero in dgl.heterograph.combine_frames() (#3110)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2021-07-07 14:15:06 +08:00
Israt Nisa 188152b853 [Feature] Add Heterograph support on Python for builtin unary msg functions (copy_u, copy_e) (#2989)
* heterograph for binary func

* Added SDDMM support

* Added unittest

* added binary test cases

* unary mfuncs works

* Fixed lint err

* lint check and others

* link check

* fixed import *_hetero issue

* lint check

* replace torch with dgl backend

* lint cehck

* removed torch from test

* skip mxnet unittest

* skip gpu test

* Remove unused/duplicated code

* minor

* changed data structure of ndata and edata

* link check

* reorganized

* minor lint

* minor lint

* raise error for udf func

* lint check

* fix for CUDA 10.1

* add a note for future cross-type max/min reducing

* Add support CUDA < 11

* lint check

* tidied C code

* remove dummy GSDDMM_hetero backward implementation

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
2021-07-06 20:41:58 +08:00
Minjie Wang d3e4460b98 [Doc] Add docstring for missing APIs (#3088)
* add docstring for missing API; fix some docstring

* rename apis; address comments
2021-07-05 18:40:07 +08:00
Da Zheng 485c04cff6 [Distributed] Fix a few bugs in distributed API (#3094)
* fix.

* fix.

* fix.

* fix.

* Fix test

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
2021-07-05 15:12:02 +08:00
Da Zheng 0884d02465 [Distributed] Fix bugs in partitioning on heterogeneous graphs. (#3085)
* fix bugs in partitioning on heterogeneous graphs.

* fix.

* fix.

* fix example.

* fix.

* fix test.

* fix.

* fix.

* fix.

* fix tests.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
2021-07-02 21:16:14 +08:00
Jinjing Zhou 0b3a6216f5 [Test] Enable kvstore test (#3079)
* try enable kvstore test

* fix

* fix

* seperate out kvstore test

* add comment
2021-07-02 14:54:38 +08:00
nv-dlasalle a0390dde93 [Feature] Add dgl.utils.is_sorted_srcdst() (#2685)
* Add dgl.utils.is_sorted_srcdst

* Fix linting issues

* delete blank line

* Specify datatype to index tensor in test

* Force integer conversion

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-07-02 11:29:24 +08:00
Rhett-Ying 4307fe8895 [Feature] Add dgl.reorder() to re-order graph according to specified … (#3063)
* [Feature] Add dgl.reorder() to re-order graph according to specified strategy

* fix unit test failure for metis reorder

* fix unit test failure on mxnet_cpu

* refine unit test for dgl.reorder

* fix unit test failure on mxnet

* fix array_equal error for mxnet unit test

* fix unit test failure for mxnet

* convert metis output to numpy array explicitly

Co-authored-by: Tong He <hetong007@gmail.com>
2021-06-30 13:29:08 +08:00