794ec4a440
[BugFix] fix unstable sort when using dataloader with HeteroGraph ( #4147 )
...
* fix unstable sort
* add torch version check
* reformat
* split too long comments
* Update dataloader.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-06-22 18:05:42 +08:00
69226588a5
[Dist] defer to load node/edge feats ( #4143 )
...
* [Dist] defer to load node/edge feats
* fix lint
* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
* fix lint
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
2022-06-20 19:44:37 +08:00
b258729b3f
[Dist] set socket as default backend for RPC ( #4120 )
...
* [Dist] set socket as default backend for RPC
* add tests both for socket and tensorpipe
2022-06-16 13:08:19 +08:00
9a6f292493
[Bugfix] Fix fail to create_shared_mem_array in ddp spawn train #4110 ( #4111 )
...
* Fix fail to create_shared_mem_array in ddp spawn train #4110
Fix fail to create_shared_mem_array in ddp spawn train #4110
* [Bugfix] Fix fail to create_shared_mem_array in ddp spawn train #4110
[Bugfix] Fix fail to create_shared_mem_array in ddp spawn train #4110
Replace random.seed() to random_ = random.Random()
* Update pytorch.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-06-14 23:39:36 +08:00
defa292bc0
[Dataset] Add Flickr and Yelp dataset ( #4099 )
...
* Add Flickr and Yelp dataset
* Update flickr.py
* update
* Update yelp.py
* Update yelp.py
* update
* Update yelp.py
* Update test_data.py
* Update yelp.py
* update
* Update test_data.py
* Update yelp.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-14 16:14:12 +08:00
92e7733065
[dataset] Add a reorder flag to builtin datasets ( #4104 )
...
* add argument reorder=False for citation_graph
* add description of the argument reorder
* add reordered/un_reordered save_path
* add version number postfix
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-12 12:35:30 +08:00
abcc9cce83
disable multiple groups tests due to random failure in CI ( #4101 )
2022-06-09 17:40:53 +08:00
cac3720b48
[Dist] enable time out when fetching msg ( #4043 )
...
* [ist] enable time out when fetching msg
* fix lint error
* minor refinements
* improve minor log
* fix dist test
* fix timeout issue in tensorpipe
2022-06-08 20:20:03 +08:00
d92a4e8b64
[Bug] Fixed missing dgl.transforms.functional __all__ entries ( #4089 )
2022-06-08 09:31:44 +08:00
c9c79b4401
typo fix in TUDataset docs ( #4056 )
...
Co-authored-by: decoherencer <decoherencer@users.noreply.github.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-03 13:29:32 +08:00
efd909e62e
[NN] Enhance EGATConv branch ( #4062 )
...
* enhance EGATConv| nfeats as tuples
* egatconv modified for bipartite graphs
* modified docstrings
* added/modified unittests for EGATConv
* Update egatconv.py
* rectified lint errors
Co-authored-by: rijulizer <riju.mukherjee@gmail.com >
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-03 11:53:10 +08:00
89655cfda2
[Dataset] Add WikiCS Dataset ( #4035 )
...
* Fix bugs & Update dataset
* Update
* Update wikics.py
* Update wikics.py
* Update test_data.py
* Update wikics.py
* Update wikics.py
* Update wikics.py
* update
* Update module.py
* Update dgl.data.rst
* Update wikics.py
* Update wikics.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-02 23:03:17 +08:00
d9c25521bc
[Data] AsGraphPredDataset ( #4073 )
...
* Update
* CI
* Update
* Update
* Fix
* Fix
2022-06-02 16:18:48 +08:00
7ec165c2c5
default value typo fix in GlobalUniform negative sampler ( #4060 )
...
Co-authored-by: decoherencer <decoherencer@users.noreply.github.com >
2022-05-28 13:22:41 +08:00
3c129ad71f
[Bugfix] Cython CAPI holding GIL causes deadlock when Python callback is asynchronous ( #4036 )
...
* cython nogil
* move APIs to internal and add unit test
* fix lint
* disable callback array test
2022-05-25 10:02:44 +08:00
230b886ec5
[Bug fix] Misc Fix for Transforms and NN Modules ( #4038 )
...
* Update module.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update module.py
* Update
* Update
* Update
2022-05-24 18:08:08 +08:00
d70a362dba
Update shadow.py ( #4034 )
...
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-05-23 19:06:18 +08:00
3fe5eea791
[NN] Label Propagation & Directional Graph Networks ( #4017 )
...
* add label propagation module
* fix prev bug in example
* add dgn
* fix linting and doc issues
* update label propagation & dgn
* update label propagation & dgn
* update example
* fix unit test
* fix agg heritage issue
* fix agg issue
* fix lint
* fix idx
* fix lp gpu issue
* Update
* Update
Co-authored-by: mufeili <mufeili1996@gmail.com >
2022-05-20 15:23:19 +08:00
53835bdb75
[Dist][BugFix] enable sampling on bipartite ( #4014 )
...
* [Dist][BugFix] enable sampling on bipartite
* add comments for tests
2022-05-18 10:14:54 +08:00
de5543e3db
Update functional.py ( #4015 )
2022-05-17 14:14:04 +08:00
e0e8736fa0
[Feature] Added floating-point conversion functions to dgl.transforms.functional ( #3890 )
...
* * Added half_(), float_(), and double_() functions to DGLHeteroGraph, HeteroNodeDataView, and HeteroEdgeDataView, for converting floating-point tensor data to float16, float32, or float64 precision
* * Extracted out private functions for floating-point type conversion, to reduce code duplication
* * Added test for floating-point data conversion functions, half_(), float_(), and double_()
* * Moved half_(), float_(), and double_() functions from HeteroNodeDataView and HeteroEdgeDataView to Frame class
* * Updated test_float_cast() to use dgl.heterograph instead of dgl.graph
* Added to CONTRIBUTORS.md
* * Changed data type conversion to be deferred until the data is accessed, to avoid redundant conversions of data that isn't used.
* * Addressed issues flagged by linter
* * Worked around a bug in the old version of mxnet that's currently used for DGL testing
* * Only defer Column data type conversion if there is a pending device transfer or index sampling to be done. This is expected to be the desired behaviour based on discussions of a few use cases, as described in the comments.
* * Moved floating-point feature data conversion functions to dgl.transforms.functional
* Changed them from in-place behaviour to shallow copy (out-of-place) behaviour
* * Fixed linter issues
* * Removed lines that unintentionally added to_half, to_float, and to_double to DGLHeteroGraph
* Moved _init_api line to the end of the file again
* * Removed one of the two leading underscores from Frame.__astype_float, making it not fully private
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com >
2022-05-16 18:17:15 -07:00
1f2e696080
Prevent users from attempting to pin PyTorch non-contiguous tensors or views only encompassing part of tensor. ( #3992 )
...
* Disable pinning non-contiguous memory
* Prevent views from being converted for write
* Fix linting
* Add unit tests
* Improve error message for users
* Switch to pytest function
* exclude mxnet and tensorflow from inplace pinning
* Add skip
* Restrict to pytorch backend
* Use backend to retrieve device
* Fix capitalization in decorator
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-05-16 14:49:36 -07:00
65b0b9e8c3
[Dataset & Transform] Synthetic Datasets for Explainability and SIGNDiffusion Transform ( #3982 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Update
* Update
2022-05-16 22:32:09 +08:00
4c14781427
[Optimization] Memory consumption optimization on index shuffling in dataloader ( #3980 )
...
* fix
* revert
* Update dataloader.py
2022-05-14 02:11:01 +08:00
22e218d335
[Dist] Enable maximum try times for socket backend via DGL_DIST_MAX_T… ( #3977 )
...
* [Dist] Enable maximum try times for socket backend via DGL_DIST_MAX_TRY_TIMES
* reset env before/after test
* print log for info when trying to connect
* fix
* print log in python instead of cpp
2022-05-11 13:03:06 +08:00
edf2d52666
[Dist][Optim] Fixed race conditions in distributed SparseAdam and SparseAdagrad ( #3971 )
...
* * Fixed race condition bug in distributed/optim/pytorch/sparse_optim.py's SparseAdam::update, corresponding with the bug fixed in the non-distributed version in https://github.com/dmlc/dgl/pull/3013 , though using the newer Event-based approach from that corresponding function. The race condition would often result in NaNs, like the previously fixed bug. https://github.com/dmlc/dgl/issues/2760
* * Fixed race condition bug in SparseAdagrad::update corresponding with the one fixed in SparseAdam::update in the previous commit. Same info applies.
* * Fixed typo in all copies of a repeatedly-copied comment near bug fixed 3 commits ago, checking all implementations nearby for a corresponding bug. (All of them appear to have been fixed as of 2 commits ago.)
* * Removed trailing whitespace
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
2022-05-09 09:05:31 -07:00
34981bd676
[Doc] Fix compact_graphs example docstring ( #3953 )
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-05-08 15:58:00 +08:00
b2ec5a2ea6
[Transform] Add FeatMask and RowFeatNormalizer transforms ( #3968 )
...
* [Model]P-GNN
* updata
* [Example]P-GNN
* Update README.md
* Add NodeFeatureMasking and NormalizeFeatures
* Update
* Update transforms.rst
* Update
* Update
* Update
* Update test_transform.py
* Update
* Update
* Update test_transform.py
* Update module.py
* Update module.py
* Update module.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-05-07 11:59:39 +08:00
5b97a1a26e
add bias argument ( #3970 )
2022-05-05 10:17:24 +08:00
5561af4517
[Feature] Dataloader worker affinitization ( #3723 )
...
* PR3355 + CSR conversion workaround
* Remove debug code
* Fix convention errors
* Remove wrongly added code section during merge
* Update to reflect dataloading changes
* Fix missing changes
* Remove comment
* Fix linter errors
* Fix trailing whitespace
* Add wrapper around worker init function
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-04-28 13:00:04 +08:00
37be02a486
[Feature] enable socket net_type for rpc ( #3951 )
...
* [Feature] enable socket net_type for rpc
* fix lint
* fix lint
* fix build issue on windows
* fix test failure on windows
* fix test failure
* fix cpp unit test failure
* net_type blocking max_try_times
* fix other comments
* fix lint
* fix comment
* fix lint
* fix cpp
2022-04-28 00:10:06 +08:00
c3baf4332b
[Bug] Fix multi-GPU edge classification crashing with pure GPU sampling ( #3946 )
...
* fix
* fix
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-04-27 22:59:21 +08:00
248bece633
[NN] GINEConv ( #3934 )
...
* Update
* Update
* Update
* Update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-04-25 14:41:05 +08:00
4375c2d7e4
[Doc] Fix documentation in dgl.multiprocessing namespace ( #3929 )
...
* fix docs
* remove
* oh
* fix
2022-04-16 00:35:06 +08:00
e06e63d5d5
[Feature] Enable UVA sampling with CPU indices ( #3892 )
...
* enable UVA sampling with CPU indices
* add docs
* add more docs
* lint
* fix
* fix
* better error message
* use mp.Barrier instead of queues
* revert
* revert
* oops
* revert dgl.multiprocessing.spawn
* Update pytorch.py
2022-04-12 20:56:51 +08:00
0d878ff8ef
[Example] Cleaned GraphSAGE node classification example with PyTorch Lightning ( #3863 )
...
* cleaned pl node classification example
* conform to PL's method of updating the dataloader
* update
* lint
* fix test
* fix
2022-04-12 17:26:23 +08:00
baa9292899
[bugfix] Fix EdgePredictionSampler for UVA sampling ( #3904 )
...
* Add failing unit test
* Fix negative sampler edge types
* fix test
* oops
* revert
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2022-04-12 15:25:21 +08:00
5fcd7f290f
[Feature] Enable UVA for GPU PinSAGE and RandomWalk ( #3857 )
...
* enable uva for pinsage sampler
* unit test
* modify some checks on the python side
* remove legacy random walk code
* update unit test
* update unit test
* fix unit test
* adjust checks
* move some checks to c++
* move max_nodes check to cuda kernel
* fix ci for tf
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com >
2022-04-11 20:01:23 +08:00
23a5e674e7
[fix] device cannot be Nonetype object. ( #3822 )
...
* fix device = none to cpu.
* fix singleton-comparison
* modify device is none
* fix use uva
* add docs
* add edge dataloader
* add doc
* fix trailing-whitespace
* modify default
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-04-11 14:58:27 +08:00
43b9d40e5e
[Doc] Fix Doc for EGNNConv ( #3919 )
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-04-11 11:01:15 +08:00
adb3a7c11e
[BugFix] record/restore pin status when pickle/unpickle ( #3914 )
...
* [BugFix] record/restore pin status when pickle/unpickle
* disable test on TF
* set version as expected
* unpin memory in test
2022-04-09 11:42:23 +08:00
dc5035b1f3
[NN] Add EGNN & PNA ( #3901 )
...
* add EGNN & PNA
* fix egnn issues
* fix pna issues
* update pna conv
* add doc strings
* update pnaconv
* fix unused args issue
* fix moment aggregation issue
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-04-07 17:44:57 +08:00
11e910b6f8
[Doc] Hotfix for GraphSAINT Doc ( #3898 )
...
* Update graphsaint.py
* Update graphsaint.py
2022-03-31 19:22:21 +08:00
35e66f426a
[Bugfix] Fix UVA sampling with partially specified node types ( #3897 )
...
* fix uva with partial node types
* lint
* skip tensorflow unit test
2022-03-31 17:07:02 +08:00
4c7bd314b4
[Feature] Radius Graph ( #3829 )
...
* radius graph
* remove trailing whitespaces from docs
* disable invalid name for transform func
* disable radius graph module invalid name
* move pylint disable before init
* fix missing nodes from point set
* update docs indexing
* add compute mode as optional param
* radius graph test
* remove trailing whitespaces
* fix precision when comparing tensors
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-03-30 17:23:29 +08:00
290b7c25c2
[Transform] Random Walk Positional Encoding & Laplacian Positional Encoding ( #3869 )
...
* [Transform] Random Walk Positional Encoding & Laplacian Positional Encoding
* Update python/dgl/transforms/module.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
* Update python/dgl/transforms/functional.py & python/dgl/transforms/module.py
* update python/dgl/transforms/functional.py
* update doc indexing
* update unit test
* support weighted graphs for random_walk_pe
* change reference
* fix linter errors
* fix datatype compatibility with MXNet & TF
* fix device errors
* fix result precision errors
* change tensorflow abs api
* reboost CI
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-03-30 15:59:28 +08:00
9fee20b91d
[Sampler] [Example] SAINTSampler and Simplify GraphSAINT Example ( #3879 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-03-30 13:51:37 +08:00
d8375c10ec
[BugFix] involve indices_devices for other dataset object. ( #3810 )
...
* involve indices_devices for other dataset object.
* modify raise error when device not found.
* remove empty line
* fix line too long
2022-03-30 11:50:03 +08:00
fbbca994b7
[Feature] METIS Partition with Communication Volume Minimization ( #3821 )
...
* upd
* upd
* upd
* upd
* upd
* fix OpenMP compatibility issues
* typo
* partition
* misc
* fix typo
* num_parts=1
* import torch
* long
* print info
* print info
* print info
* upd
* remove debug code
* revert partition.py
* fix cut count
* fix cut count
* Revert "fix cut count"
This reverts commit 10926b4fd48f45c8f1ddb58be7db6c22e653effd.
* Revert "fix cut count"
This reverts commit 76465283bef093a2b4209ad70dd15d2437b2ec8a.
* type of deprecate
* typo in deprecate info
* fix typo
* use cv for partitioning
* CE
* no message
* revert
* typo
* add objtype
* no message
* fix bug
* fix bug
* fix bug
* ?
* semicolon
* drop tensors
* no message
* backward
* backward
* max op
* store X.shape
* th
* test
* Revert "test"
This reverts commit 92b3b2f64a3a1128590098fa03ce429c5466e6ce.
* test
* tolist
* debug
* to cuda
* tuple
* fix bug
* remove X
* no message
* fix bug
* workload balance
* Revert "workload balance"
This reverts commit d7f8e4a16ba2a7eabb4a9bb945523bfe6623e723.
* reverse
* Revert "reverse"
This reverts commit 8a71cf25685aa7d889b9b8881b46f7a16b7d6e6d.
* Revert "Revert "reverse""
This reverts commit 196b143932d5cf9813576ece7c990b63d322d063.
* Revert "Revert "Revert "reverse"""
This reverts commit cf9e89a07013582056e7cde235e51331aca7fa9c.
* no message
* Merge commit '5498cf051e9cf450dcb798ca09920bbde008a661'
# Conflicts:
# python/dgl/distributed/partition.py
* Revert "Merge commit '5498cf051e9cf450dcb798ca09920bbde008a661'"
This reverts commit f79be2ad777897c7025b28308454cad81ad6bb27.
* fix bug
* third party
* no message
* try to avoid memory leak
* try to avoid memory leak
* avoid memory leak with no hope
* Revert "avoid memory leak with no hope"
This reverts commit c77befe9479f46758e744642f66dd209b50eef7d.
* no message
* Revert "no message"
This reverts commit 478cb28fe25fb1002b2f1dc202bb9bdaad8b2a56.
* del
* Revert "del"
This reverts commit 1b468e45ce646b400ff3ffa61a0b2da058b3bdfd.
* no message
* no message
* Revert "no message"
This reverts commit 92e4f5561ed42da0606618b2fff9f1ad5ed439d9.
* third party
* document
* Update metis_partition.cc
* Update metis_partition_hetero.cc
* Update metis_partition_hetero.cc
* Update partition.py
* Update partition.py
* Update partition.py
Co-authored-by: yzh119 <expye@outlook.com >
Co-authored-by: chwan-rice <54331508+chwan-rice@users.noreply.github.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2022-03-27 14:22:01 -07:00
ae3316c846
[Doc] Update doc of dataloading package. ( #3886 )
...
* wip: dataloading doc
* update dataloading package doc and many others
* lint
2022-03-26 21:59:02 +08:00
maqy