b131920066
[Feature] Add removed edges in distributed graph partitioning to handle heterogeneous graph ( #3137 )
...
* Added code for Rectifying (TypeError: unhashable type: 'slice') when copying file
* 1) added distributed preprocessing code to create ParMetis Input from CSV files
2) add code to run pm_dglpart on multiple machines
3) added support for recreating heteregenous graph from homo geneous graph based on dropped edges, as ParMetis currently only supports homogeneous graphs
* move to pandas
* Added comments and remove drop_duplicates as it was redundant
* Addressed Pr Comments
* Rename variable
* Added comment
* Added comment
* updated ReadMe
Co-authored-by: Ankit Garg <gaank@amazon.com >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-08-02 11:41:13 -07:00
39764da491
[model] add model example GeniePath ( #3199 )
...
* [model] add model example GeniePath
* improvements based on feedback
* improvements based on feedback
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-07-30 18:09:42 +08:00
6f93c6aa74
fix ( #3167 )
...
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-07-30 13:07:22 +08:00
a107993f10
[Model] add model example CARE-GNN ( #3187 )
...
* [Model] add model example CARE-GNN
* update README
* improvements based on the review feedback
* fix missing item()
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-07-29 15:24:15 +08:00
2583ec5934
[Doc] Fix WeightBasis documentation ( #3189 )
...
* fix
* fix type
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-07-28 18:56:10 +08:00
ba7e7cf931
[New Feature] Per edge type sampler for to_homogeneous graphs. ( #3131 )
...
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* Basic imple of heterogeneous on homogenenous sampling
* make pass
* Pass C++ test
* Add python test code
* lint
* lint
* Add MultiLayerEtypeNeighborSampler
* Add unitest for single machine dataloader
* Add dist dataloader test for edge type sampler
* Fix lint
* fix
* support for per etype sample
* Fix some bug and enable distributed training with per edge sample
* fix
* Now distributed training works
* turn off some mxnet
* turn off mxnet for some dist test
* fix
* upd
* upd according to the comments
* Fix
* Fix test and now distributed works.
* upd
* upd
* Fix
* Fix bug
* remove dead code.
* upd
* Fix
* upd
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-07-28 11:51:32 +08:00
59a7d0d1c0
[Model] add model example GCN-based Anti-Spam ( #3145 )
...
* add model example GCN-based Anti-Spam
* update example index
* add usage info
* improvements as per comments
* fix image invisiable problem
* add image file
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-07-24 23:33:25 +08:00
c0719ec574
train test on face use concat bce ( #3180 )
2021-07-24 20:49:09 +08:00
65ecbb9492
change numbers to concat bce ( #3175 )
2021-07-23 12:54:28 +08:00
9d56d386c8
[Doc] Update the example folder README
2021-07-20 19:12:06 +08:00
34426a98e9
[Distributed] Distributed heterograph training ( #3069 )
...
* support hetero RGCN.
* fix.
* simplify code.
* sample_neighbors return heterograph directly.
* avoid using to_heterogeneous.
* compute canonical etypes in advance.
* fix tests.
* fix.
* fix distributed data loader for heterograph.
* use NodeDataLoader.
* fix bugs in partitioning on heterogeneous graphs.
* fix lint.
* fix tests.
* fix.
* fix.
* fix bugs.
* fix tests.
* fix.
* enable coo for distributed.
* fix.
* fix.
* fix.
* fix.
* fix.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
2021-07-17 10:57:00 +08:00
905c0aa578
[Feature][Performance][GPU] Introducing UnifiedTensor for efficient zero-copy host memory access from GPU ( #3086 )
...
* Add pytorch-direct version
* Initial commit of unified tensor
* Merge branch 'master' of https://github.com/davidmin7/dgl
* Remove unnecessary things
* Fix error message
* Fix/Add descriptions
* whitespace fix
* add unpin
* disable IndexSelectCPUFromGPU with no CUDA
* add a newline for unified_tensor.py
* Apply changes based on feedback
* add 'os' module
* skip unified tensor unit test for cpu only
* Update tests/pytorch/test_unified_tensor.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
* reflect feedback
Co-authored-by: shhssdm <shhssdm@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-07-17 00:06:20 +08:00
88f20eec4e
fix tags ( #3140 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-07-15 11:02:44 +08:00
3f6f694159
[Bug fix] Various fix from bug bash ( #3133 )
...
* Update
* Update
* Update dependencies
* Update
* Update
* Fix ogbn-products gat
* Update
* Update
* Reformat
* Fix typo in node2vec_random_walk
* Specify file encoding
* Working for 6.7
* Update
* Fix subgraph
* Fix doc for sample_neighbors_biased
* Fix hyperlink
* Add example for udf cross reducer
* Fix
* Add example for slice_batch
* Replace dgl.bipartite
* Fix GATConv
* Fix math rendering
* Fix doc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-156.us-west-2.compute.internal >
2021-07-15 09:17:15 +08:00
d7390763f0
[Distributed] Deprecate old DistEmbedding impl, use synchronized embedding impl ( #3111 )
...
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* update doc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-07-14 00:15:53 +08:00
ee6bc95175
Update a comment in pytorch HGT example ( #3101 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-07-13 19:09:17 +08:00
b576e617ad
[Feature] Add left normalizer for GCN ( #3114 )
...
* add left normalizer for gcn
* fix
* fixes and some bug stuff
2021-07-13 12:49:35 +08:00
0ce92a86ed
[BugFix] fix problems found in bug bash ( #3116 )
...
* fix breakline in fakenews.py
* fix inconsistent argument name
* modify incorrect example and deprecated graph type
* modify docstring and example in knn_graph
* fix incorrect node type
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-07-12 13:34:50 +08:00
3641829211
fix docs ( #3126 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-07-12 11:43:15 +08:00
fc55225f08
Fix module name ( #3128 )
2021-07-12 11:15:02 +08:00
5798ee8d98
[Performance] Add a warning for ChebConv ( #3099 )
...
* add a warning for chebconv
* fix and docstrings
* update bgnn
* fix
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-07-09 14:49:15 +08:00
0d1dcdcd2e
[bugfix] fix a bug found in v0.7 bug bash in the 'grace' model ( #3109 )
...
* Update aug.py
* Update aug.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2021-07-07 14:50:02 +08:00
22e8c120be
[Example ] fix faiss issue on k 3 for hilander model ( #3105 )
...
* fix faiss issue on k 3
* Add Hilander paper link
Co-authored-by: Tong He <hetong007@gmail.com >
2021-07-06 13:22:56 +08:00
9c41c22d7b
[Model] Official implementation for HiLANDER model. ( #3087 )
...
* add hilander model implementation draft
* use focal loss
* fix
* change data root
* add necessary scripts
* update download links
* update
* update example table
* fix
* update readme with numbers
* add empty folder
* only eval at the end
* set up hilander
* inform results may fluctuate
* address comments
Co-authored-by: sneakerkg <xiaotj1990327@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-212.us-east-2.compute.internal >
2021-07-04 20:08:58 +08:00
0884d02465
[Distributed] Fix bugs in partitioning on heterogeneous graphs. ( #3085 )
...
* fix bugs in partitioning on heterogeneous graphs.
* fix.
* fix.
* fix example.
* fix.
* fix test.
* fix.
* fix.
* fix.
* fix tests.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
2021-07-02 21:16:14 +08:00
02e5d47d2a
[Example][Bugfix] Fix RGCN example datasplitting ( #3037 )
...
* Fix example datasplitting
* Remove left-over split parameter
* Remove unused parameter
* Set epoch when using multiple GPUs
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-06-29 10:16:28 +08:00
7f4086da7a
cuda() to to(device) ( #3064 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-06-28 11:56:24 +08:00
2f43cdb3c8
deal with situation where num_layers equals 1 ( #3066 )
2021-06-28 10:17:16 +08:00
427a5a96e0
[Doc] Update NodeDataLoader and EdgeDataLoader for GPU-based neighbor sampling ( #3046 )
...
* update docstrings and tidy code
* add docs
* address comments
* Update __init__.py
* address comments
2021-06-25 10:46:21 +08:00
acd21a6d60
[Feature] Support direct creation from CSR and CSC ( #3045 )
...
* csr and csc creation
* fix
* fix
* fixes to adj transpose
* fine
* raise error if indptr did not match number of nodes
* fix
* huh?
* oh
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-25 10:05:47 +08:00
31f4483af7
add HAN sampling demo ( #3005 )
...
Co-authored-by: maqy1995 <maqingyun0718>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-06-23 20:27:18 +08:00
cb2327d4a9
[Example] Fix error in gcn train due to missing argument ( #3040 )
...
* Fix example
* Remove register_data_args and revert to dataset
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-23 15:50:07 +08:00
e667545da5
[Feature] Node2vec ( #2992 )
...
* add seal example
* 1. add paper infomation in examples/README
2. adjust codes
3. option test
* use latest `to_simple` to replace coalesce graph function
* remove outdated codes
* remove useless comment
* Node2vec
1.implement node2vec random walk c++ op
2.implement node2vec model
3.implement node2vec example
* add CMakeLists file modify
* refine c++ codes
* refine c++ codes
* add missing whitespace
* refine python codes
* add codes
* add node2vec_impl.h
* fix codes
* fix code style problem
* fixes
* remove
* lots of changes
* add benchmark
* fixes
Co-authored-by: smilexuhc <smile.xuhc@gmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-23 14:47:56 +08:00
ff519f98c3
[API] Standardize Subgraph APIs ( #2929 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Fix subgraph tests
* Capture stdout for distributed test
* Capture stdout for distributed test
* Update
* Update
* Update
* Update subgraph.cc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-06-21 19:53:37 +08:00
aaec3d8a0b
[Distributed] Support hierarchical partitioning ( #3000 )
...
* add.
* fix.
* fix.
* fix.
* fix.
* add tests.
* support node split and edge split.
* support 1 partition.
* add tests.
* fix.
* fix test.
* use hierarchical partition.
* add check.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-57.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
2021-06-16 16:58:23 +08:00
18eaad17cc
fix bugs ( #3008 )
...
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-06-16 15:27:47 +08:00
36c6c64957
Replace QM9_v2 with built-in QM9Edge Dataset ( #3026 )
...
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2021-06-16 14:35:40 +08:00
305d5c16df
[Example] update ogbn-arxiv and ogbn-proteins results ( #3018 )
...
* [Example] GCN on ogbn-arxiv dataset
* Add README.md
* Update GCN implementation on ogbn-arxiv
* Update GCN on ogbn-arxiv
* Fix typo
* Use evaluator to get results
* Fix duplicated
* Fix duplicated
* Update GCN on ogbn-arxiv
* Add GAT for ogbn-arxiv
* Update README.md
* Update GAT on ogbn-arxiv
* Update README.md
* Update README
* Update README.md
* Update README.md
* Add GAT implementation for ogbn-proteins
* Add GAT implementation for ogbn-proteins
* Add GAT implementation for ogbn-proteins
* Add GAT implementation for ogbn-proteins
* Add GAT implementation for ogbn-proteins
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Update examples for ogbn-arxiv
* Add examples for ogbn-products.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-proteins.
* Update examples for ogbn-products.
* Update examples for ogbn-products.
* Update examples for ogbn-arxiv.
* Update examples for ogbn-arxiv.
* Update examples for ogbn-arxiv.
* Update examples for ogbn-proteins.
* Update examples for ogbn-products.
* Update examples for ogbn-proteins.
* Update examples for ogbn-arxiv.
* Update examples for ogbn-proteins.
* Update README.md
* [Example] ogbn-arxiv & ogbn-proteins
* [Example] ogbn-arxiv & ogbn-proteins
* [Example] ogbn-arxiv & ogbn-proteins
* [Example] ogbn-arxiv & ogbn-proteins
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
Co-authored-by: Zihao Ye <expye@outlook.com >
2021-06-16 12:33:43 +08:00
b4ad59d77f
[Model] reformat and update performance of grace ( #3009 )
...
* reformat and update performance
* Update README.md
* Fix typos
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2021-06-15 17:02:37 +08:00
8b64ae59b8
[Performance] Perform to_block on the GPU when the dataloader is created with a GPU device. ( #3016 )
...
* add output device for dataloading
* Update dataloader
* Get sampler device from dataloader
* Fix line length
* Update examples
* Fix to_block GPU for empty relation types
* Handle the case where the DistGraph has None for the underlying graph
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-06-13 18:51:00 +08:00
79b057f0cf
Fix example to test_acc is always computed at last epoch ( #3014 )
2021-06-12 18:02:00 -07:00
17d604b5c7
[Feature] Allow using NCCL for communication in dgl.NodeEmbedding and dgl.SparseOptimizer ( #2824 )
...
* Split from NCCL PR
* Fix type in comment
* Expand documentation for sparse_all_to_all_push
* Restore previous behavior in example
* Re-work optimizer to use NCCL based on gradient location
* Allow for running with embedding on CPU but using NCCL for gradient exchange
* Optimize single partition case
* Fix pylint errors
* Add missing include
* fix gradient indexing
* Fix line continuation
* Migrate 'first_step'
* Skip tests without enough GPUs to run NCCL
* Improve empty tensor handling for pytorch 1.5
* Fix indentation
* Allow multiple NCCL communicator to coexist
* Improve handling of empty message
* Update python/dgl/nn/pytorch/sparse_emb.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
* Update python/dgl/nn/pytorch/sparse_emb.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
* Keepy empty tensor dimensionaless
* th.empty -> th.tensor
* Preserve shape for empty non-zero dimension tensors
* Use shared state, when embedding is shared
* Add support for gathering an embedding
* Fix typo
* Fix more typos
* Fix backend call
* Use NodeDataLoader to take advantage of ddp
* Update training script to share memory
* Only squeeze last dimension
* Better handle empty message
* Keep embedding on the target device GPU if dgl_sparse if false in RGCN example
* Fix typo in comment
* Add asserts
* Improve documentation in example
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-06-10 21:19:00 -07:00
9497a9be85
[Bugfix][Examples] Fix graphsage multigpu training example training set size ( #3002 )
...
* Make multigpu graphsage use whole datset
* Specify queeze dimension
* Remove squeeze dimension
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-06-10 16:35:08 -07:00
6c59fee99b
[Model] HetGNN/GTN/HAN/NSHE/MAGNN examples ( #2993 )
...
* commit hetgnn
* Update README.md
* Delete examples/pytorch/hetgnn directory
* Update README.md
* Update README.md
* Update README.md
* Revert "Update README.md"
This reverts commit 14cdb3da76c58665ff879eae17ecf9eb28b66cce.
* Update README.md
* Add files via upload
* Update README.md
* Update README.md
2021-06-08 08:54:27 +08:00
972a9f1323
[Doc] Re-organize the code for dgl.geometry, and expose it in the doc ( #2982 )
...
* reorg and expose dgl.geometry
* fix lint
* fix test
* fix
2021-06-07 11:19:39 +08:00
6e9d16459e
[Fix] Fixed AttributeError with new version of torchtext (0.9+). ( #2985 )
2021-06-05 02:01:20 +08:00
60426278bf
[Example] TAHIN ( #2864 )
...
* tahin
* readme
* readme
* readme
* readme
* readme
* readme
* main
* main
* new_line
* update
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-06-03 16:03:55 +08:00
22d0df8b6a
[Doc] Update link_predict.py docstring ( #2960 )
...
corrected docstring, the model uses two R-GCN layers
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-02 12:08:28 +08:00
18dbaebebc
[Distributed] Specify the graph format for distributed training ( #2948 )
...
* explicitly set the graph format.
* fix.
* fix.
* fix launch script.
* fix readme.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
2021-05-26 18:09:30 +08:00
c018436588
[Example] add latent dirichlet allocation ( #2883 )
...
* add lda model
* tweak latent dirichlet allocation
* Update README.md
* Update README.md
* update example index
* update header
* minor tweak
* add example test
* update doc
* Update README.md
* Update README.md
* add partial_fit for free
* Update examples/pytorch/lda/lda_model.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
* Update examples/pytorch/lda/example_20newsgroups.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
* Update lda_model.py
* bugfix torch Gamma uses rate parameter
Co-authored-by: Yifei Ma <yifeim@amazon.com >
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-05-17 20:25:36 +08:00
Ankit Garg