mirrors / dmlc--dgl

文件历史

4,090 提交 34 分支 42 Git 标签
2.2.x
提交图

1938 次代码提交

作者 SHA1 备注 提交日期
Da Zheng 34426a98e9 [Distributed] Distributed heterograph training (#3069) 
* support hetero RGCN.

* fix.

* simplify code.

* sample_neighbors return heterograph directly.

* avoid using to_heterogeneous.

* compute canonical etypes in advance.

* fix tests.

* fix.

* fix distributed data loader for heterograph.

* use NodeDataLoader.

* fix bugs in partitioning on heterogeneous graphs.

* fix lint.

* fix tests.

* fix.

* fix.

* fix bugs.

* fix tests.

* fix.

* enable coo for distributed.

* fix.

* fix.

* fix.

* fix.

* fix.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
 2021-07-17 10:57:00 +08:00
David Min 905c0aa578 [Feature][Performance][GPU] Introducing UnifiedTensor for efficient zero-copy host memory access from GPU (#3086) 
* Add pytorch-direct version

* Initial commit of unified tensor

* Merge branch 'master' of https://github.com/davidmin7/dgl

* Remove unnecessary things

* Fix error message

* Fix/Add descriptions

* whitespace fix

* add unpin

* disable IndexSelectCPUFromGPU with no CUDA

* add a newline for unified_tensor.py

* Apply changes based on feedback

* add 'os' module

* skip unified tensor unit test for cpu only

* Update tests/pytorch/test_unified_tensor.py

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* reflect feedback

Co-authored-by: shhssdm <shhssdm@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
 2021-07-17 00:06:20 +08:00
Jingcheng Yu b379dbd6c8 Optimize dist_graph/_split_even_to_part memory usage (#3132) 
Co-authored-by: yujingcheng02 <yujingcheng02@meituan.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2021-07-15 11:51:56 +08:00
Rhett Ying 5cdf7cfa98 [Doc] refine metis part of dgl.reorder_graph() (#3136) 
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
 2021-07-15 10:14:39 +08:00
Mufei Li 3f6f694159 [Bug fix] Various fix from bug bash (#3133) 
* Update

* Update

* Update dependencies

* Update

* Update

* Fix ogbn-products gat

* Update

* Update

* Reformat

* Fix typo in node2vec_random_walk

* Specify file encoding

* Working for 6.7

* Update

* Fix subgraph

* Fix doc for sample_neighbors_biased

* Fix hyperlink

* Add example for udf cross reducer

* Fix

* Add example for slice_batch

* Replace dgl.bipartite

* Fix GATConv

* Fix math rendering

* Fix doc

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-156.us-west-2.compute.internal>
 2021-07-15 09:17:15 +08:00
Quan (Andy) Gan 066bef21f0 [Doc] Fix CoraFull number of edges inconsistency in doc (#3139) 2021-07-14 15:06:57 +08:00
xiang song(charlie.song) d7390763f0 [Distributed] Deprecate old DistEmbedding impl, use synchronized embedding impl (#3111) 
* fix.

* fix.

* fix.

* fix.

* Fix test

* Deprecate old DistEmbedding impl, use synchronized embedding impl

* update doc

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-07-14 00:15:53 +08:00
Mufei Li f0fafa2062 [Bug fix] Fix batch information with remove_nodes/edges applied (#3119) 
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix lint

* Fix

* Update

* Fix test cases

* Fix

* add docstrings

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-07-13 16:27:31 +08:00
Quan (Andy) Gan 2e19ba8025 add exclude self option for EdgeDataLoader (#3122) 
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-07-13 13:30:16 +08:00
Quan (Andy) Gan b576e617ad [Feature] Add left normalizer for GCN (#3114) 
* add left normalizer for gcn

* fix

* fixes and some bug stuff
 2021-07-13 12:49:35 +08:00
Rhett Ying 186ef59283 [Feature] apply dgl.reorder() onto several node classification datase… (#3102) 
* [Feature] apply dgl.reorder() onto several node classification datasets in DGL

* rebase on latest dgl.reorder_graph()
 2021-07-13 08:52:17 +08:00
Rhett-Ying 175f53decf [Feature] enable edge reorder in dgl.reorder_graph() (#3113) 
* [Feature] enable edge reorder in dgl.reorder_graph()

* refine doc string

* refine doc string for dgl.reorder_graph

* refine doc string further
 2021-07-12 14:11:46 +08:00
Kay Liu 0ce92a86ed [BugFix] fix problems found in bug bash (#3116) 
* fix breakline in fakenews.py

* fix inconsistent argument name

* modify incorrect example and deprecated graph type

* modify docstring and example in knn_graph

* fix incorrect node type

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2021-07-12 13:34:50 +08:00
Wilfried L. Bounsi c7449818a1 Update documentation for remove_edges function (#3123) 
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2021-07-12 12:54:21 +08:00
Quan (Andy) Gan 5798ee8d98 [Performance] Add a warning for ChebConv (#3099) 
* add a warning for chebconv

* fix and docstrings

* update bgnn

* fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-07-09 14:49:15 +08:00
Quan (Andy) Gan 2f41fcd986 [Build] fix various build problems (#3117) 2021-07-08 11:39:33 +08:00
Kay Liu bb89dee7ff [BugFix] fix problems in data split (#3082) 
* [BugFix] fix problems in data split

* fix format problems in docstring

* modify statistics to fit in dgl nature

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: zhjwy9343 <6593865@qq.com>
 2021-07-07 16:07:23 +08:00
Rhett-Ying 32d1f3ac97 [BugFix] skip frames whose num_rows is zero in dgl.heterograph.combine_frames() (#3110) 
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-07-07 14:15:06 +08:00
Quan (Andy) Gan 751685a979 [Doc] Add documentation for DGLGraph.adj_sparse (#3096) 
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-07-07 11:24:47 +08:00
Israt Nisa 188152b853 [Feature] Add Heterograph support on Python for builtin unary msg functions (copy_u, copy_e) (#2989) 
* heterograph for binary func

* Added SDDMM support

* Added unittest

* added binary test cases

* unary mfuncs works

* Fixed lint err

* lint check and others

* link check

* fixed import *_hetero issue

* lint check

* replace torch with dgl backend

* lint cehck

* removed torch from test

* skip mxnet unittest

* skip gpu test

* Remove unused/duplicated code

* minor

* changed data structure of ndata and edata

* link check

* reorganized

* minor lint

* minor lint

* raise error for udf func

* lint check

* fix for CUDA 10.1

* add a note for future cross-type max/min reducing

* Add support CUDA < 11

* lint check

* tidied C code

* remove dummy GSDDMM_hetero backward implementation

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
 2021-07-06 20:41:58 +08:00
Jinjing Zhou 4556fab8e6 [Fix] Use logging to print warning message in socket creation (#3098) 
* fix

* fix

* lint

* fix

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2021-07-06 15:33:43 +08:00
Minjie Wang d3e4460b98 [Doc] Add docstring for missing APIs (#3088) 
* add docstring for missing API; fix some docstring

* rename apis; address comments
 2021-07-05 18:40:07 +08:00
Da Zheng 485c04cff6 [Distributed] Fix a few bugs in distributed API (#3094) 
* fix.

* fix.

* fix.

* fix.

* Fix test

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
 2021-07-05 15:12:02 +08:00
WangYQ 0851b4df5d Sageconv doc (#3093) 
* revise the doc of forward function in SAGEConv

* remove trailing space
 2021-07-05 09:25:56 +08:00
Da Zheng 0884d02465 [Distributed] Fix bugs in partitioning on heterogeneous graphs. (#3085) 
* fix bugs in partitioning on heterogeneous graphs.

* fix.

* fix.

* fix example.

* fix.

* fix test.

* fix.

* fix.

* fix.

* fix tests.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
 2021-07-02 21:16:14 +08:00
Rhett-Ying 04dce1e630 [Doc] refine doc and optimize metis part for dgl.reorder() (#3090) 
Co-authored-by: Tong He <hetong007@gmail.com>
 2021-07-02 20:14:47 +08:00
nv-dlasalle a0390dde93 [Feature] Add dgl.utils.is_sorted_srcdst() (#2685) 
* Add dgl.utils.is_sorted_srcdst

* Fix linting issues

* delete blank line

* Specify datatype to index tensor in test

* Force integer conversion

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
 2021-07-02 11:29:24 +08:00
Rhett-Ying 95f8ec83b5 [Doc] fix docstring in to_block() (#3081) 2021-07-02 09:45:45 +08:00
Quan (Andy) Gan a97aaa4903 fix (#3076) 2021-06-30 18:27:05 +08:00
Quan (Andy) Gan 7823ae2dbb add TUDataset notice (#3075) 
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-06-30 14:22:45 +08:00
Rhett-Ying 4307fe8895 [Feature] Add dgl.reorder() to re-order graph according to specified … (#3063) 
* [Feature] Add dgl.reorder() to re-order graph according to specified strategy

* fix unit test failure for metis reorder

* fix unit test failure on mxnet_cpu

* refine unit test for dgl.reorder

* fix unit test failure on mxnet

* fix array_equal error for mxnet unit test

* fix unit test failure for mxnet

* convert metis output to numpy array explicitly

Co-authored-by: Tong He <hetong007@gmail.com>
 2021-06-30 13:29:08 +08:00
Tong He 789b03f433 [Doc] Add KNN Benchmark Results (#3055) 
* add knn benchmark result

* update
 2021-06-28 16:13:56 +08:00
Quan (Andy) Gan 427a5a96e0 [Doc] Update NodeDataLoader and EdgeDataLoader for GPU-based neighbor sampling (#3046) 
* update docstrings and tidy code

* add docs

* address comments

* Update __init__.py

* address comments
 2021-06-25 10:46:21 +08:00
Quan (Andy) Gan acd21a6d60 [Feature] Support direct creation from CSR and CSC (#3045) 
* csr and csc creation

* fix

* fix

* fixes to adj transpose

* fine

* raise error if indptr did not match number of nodes

* fix

* huh?

* oh

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-06-25 10:05:47 +08:00
xiang song(charlie.song) 2f7ca41459 [Bug fix] Use shared memory for grad sync when NCCL is not avaliable as PyTorch distributed backend. (#3034) 
* Use shared memory for grad sync when NCCL is not avaliable as PyTorch distributed backend.

Fix small bugs and update unitests

* Fix bug

* update test

* update test

* Fix unitest

* Fix unitest

* Fix test

* Fix

* simple update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-24-212.ec2.internal>
 2021-06-24 12:06:54 +08:00
Qidong Su e56bbafd25 [Feature] Biased Neighbor Sampling (#2987) 
* update

* update

* update

* update

* lint

* lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* lint

* update

* clone

* update

* update

* update

* update

* replace idarray with ndarray

* refactor cpp part

* refactor python part

* debug

* refactor interface

* test and doc

* lint and test

* lint

* fix

* fix

* fix

* const

* doc

* fix

* fix

* fix

* fix

* fix & doc

* fix

* fix

* update

* update

* update

* merge

* doc

* doc

* lint

* fix

* more tests

* doc

* fix

* fix

* update

* update

* update

* fix

* fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-06-23 17:44:51 +08:00
Tomohiro Endo 5961c14026 [Doc] EdgeWeightNorm Fix doc (#3051) 
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-06-23 15:49:31 +08:00
Quan (Andy) Gan e667545da5 [Feature] Node2vec (#2992) 
* add seal example

* 1. add paper infomation in examples/README
2. adjust codes
3. option test

* use latest `to_simple` to replace coalesce graph function

* remove outdated codes

* remove useless comment

* Node2vec
1.implement node2vec random walk c++ op
2.implement node2vec model
3.implement node2vec example

* add CMakeLists file modify

* refine c++ codes

* refine c++ codes

* add missing whitespace

* refine python codes

* add codes

* add node2vec_impl.h

* fix codes

* fix code style problem

* fixes

* remove

* lots of changes

* add benchmark

* fixes

Co-authored-by: smilexuhc <smile.xuhc@gmail.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-06-23 14:47:56 +08:00
nv-dlasalle 7359481497 [Feature][GPU] Add function for setting weights of a sparse embedding on multiple GPUs. (#3047) 
* add unit test

* Extend NDArrayPartition object

* Add method for setting embedding, and improve documentation

* Sync before returning

* Use name unique to sparse embedding class to avoid delete

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
 2021-06-22 08:48:12 -07:00
Mufei Li ff519f98c3 [API] Standardize Subgraph APIs (#2929) 
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix

* Update

* Fix subgraph tests

* Capture stdout for distributed test

* Capture stdout for distributed test

* Update

* Update

* Update

* Update subgraph.cc

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-06-21 19:53:37 +08:00
Kay Liu 9706eaa895 [Feature] add permission information and fix import problems (#3036) 
* [Feature] add positive negative statistics

* [Feature] add permission information and fix import problem

* fix backend incompatible problem

* modify random split to remove sklearn usage

* modify file read to remove pandas usage

* add datasets into doc

* add random seed in data splitting

* add dataset unit test

* usage permission information update

Co-authored-by: zhjwy9343 <6593865@qq.com>
 2021-06-21 11:55:15 +08:00
zhjwy9343 55e7796ac5 Revert "[Feature] add permission information and fix import problems (#2998)" (#3029) 
This reverts commit cba5b188b5.

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
 2021-06-17 13:58:38 +08:00
Kay Liu cba5b188b5 [Feature] add permission information and fix import problems (#2998) 
* [Feature] add positive negative statistics

* [Feature] add permission information and fix import problem

* fix backend incompatible problem

* modify random split to remove sklearn usage

* modify file read to remove pandas usage

* add datasets into doc

Co-authored-by: zhjwy9343 <6593865@qq.com>
 2021-06-16 18:24:17 +08:00
Da Zheng aaec3d8a0b [Distributed] Support hierarchical partitioning (#3000) 
* add.

* fix.

* fix.

* fix.

* fix.

* add tests.

* support node split and edge split.

* support 1 partition.

* add tests.

* fix.

* fix test.

* use hierarchical partition.

* add check.

Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-57.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
 2021-06-16 16:58:23 +08:00
nv-dlasalle 8932285652 [bugfix] Fix sparse_optim when the state is stored on the CPU (fixes #2760) (#3013) 
* Fix sparse optimizer to wait on copies to the CPU

* Fix linting

* Fix typo

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
 2021-06-15 11:37:32 -07:00
Leonardo Castorina 63aed58204 [Doc] Fix typo in DGL base dataset class (#2977) #3017 
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
 2021-06-15 16:15:52 +08:00
Tianqi Zhang (张天启) 64d0f3f355 [Feature] Add NN-descent support for the KNN graph function in dgl (#2941) 
* add bruteforce impl

* add nn descent implementation

* change doc-string

* remove redundant func

* use local rng for cuda

* fix lint

* fix lint

* fix bug

* fix bug

* wrap nndescent_knn_graph into knn

* fix lint

* change function names

* add comment for dist funcs

* let the compiler do the unrolling

* use better blocksize setting

* remove redundant line

* check the return of the cub calls

Co-authored-by: Tong He <hetong007@gmail.com>
 2021-06-15 14:25:02 +08:00
Jinjing Zhou a303f07819 fix #2952 (#3010) 2021-06-14 15:09:14 +08:00
Jinjing Zhou 17141dd372 [Fix] Fix to_homo for graph with zero nodes ntype (#3011) 
* fix #2870

* lint

* fix
 2021-06-14 14:34:13 +08:00
nv-dlasalle 8b64ae59b8 [Performance] Perform to_block on the GPU when the dataloader is created with a GPU device. (#3016) 
* add output device for dataloading

* Update dataloader

* Get sampler device from dataloader

* Fix line length

* Update examples

* Fix to_block GPU for empty relation types

* Handle the case where the DistGraph has None for the underlying graph

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
 2021-06-13 18:51:00 +08:00