文件历史

594 次代码提交

作者 SHA1 备注 提交日期
WangYQ f5bba284f8 [Example] Add EGES example (#3756)
* add eges example

* remove csv files and add data link

* Update README.md

* Update main.py

* Update model.py

* Update sampler.py

* Update utils.py

* Update model.py

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-04-14 16:23:08 +08:00
Tong He a0bf5daa65 fix (#3903)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-04-13 12:28:19 +08:00
Mufei Li c380f0b5fa Update gatv2.py (#3921)
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-04-13 09:47:52 +08:00
Quan (Andy) Gan e06e63d5d5 [Feature] Enable UVA sampling with CPU indices (#3892)
* enable UVA sampling with CPU indices

* add docs

* add more docs

* lint

* fix

* fix

* better error message

* use mp.Barrier instead of queues

* revert

* revert

* oops

* revert dgl.multiprocessing.spawn

* Update pytorch.py
2022-04-12 20:56:51 +08:00
Quan (Andy) Gan 0d878ff8ef [Example] Cleaned GraphSAGE node classification example with PyTorch Lightning (#3863)
* cleaned pl node classification example

* conform to PL's method of updating the dataloader

* update

* lint

* fix test

* fix
2022-04-12 17:26:23 +08:00
Serge Panev f931c6ba54 [Examples] Add pure gpu mode in the GraphSAGE node classification and link prediction (#3856)
Signed-off-by: Serge Panev <spanev@nvidia.com>

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-04-12 13:03:03 +08:00
Mufei Li 01e50626d3 [Example] Add TorchMetrics in README (#3913)
* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-04-09 23:25:51 +08:00
nv-dlasalle 27a6eb564c [Examples] Update graphsage multi-gpu example to use mutliple GPUs for validation and testing. (#3827)
* Update graphsage multi-gpu example to use mutliple GPUs for validation and
testing.

* Remove argmax

* Fix rebase error

* Add more documentation to example and simplify

* Switch to name shared memory

* Add comment about how training is distributed

* Restore iteration count

* fix munmap error reporting for better error messages

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-04-06 00:32:37 +08:00
Mufei Li 9fee20b91d [Sampler] [Example] SAINTSampler and Simplify GraphSAINT Example (#3879)
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-03-30 13:51:37 +08:00
Liang Wang 0dd500e386 [BugFix] Remove redundant reference of networkx package in pagerank.py (#3888)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-03-30 11:54:03 +08:00
Sai Kandregula 6752bd4515 [Example] layers number fix in GAT example (#3872)
* gat layers fix

* minor bug

Co-authored-by: decoherencer <decoherencer@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-03-26 15:02:16 +08:00
Quan (Andy) Gan 7d4160869c [Bug] Fix multiple issues in distributed multi-GPU GraphSAGE example (#3870)
* fix distributed multi-GPU example device

* try Join

* update version requirement in README

* use model.join

* fix docs

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-03-25 22:04:02 +08:00
Xin Yao 172949d429 [Example] Fix multi-GPU RGCN example (#3871)
* fix multi-gpu rgcn example

* remove dgl.multiprocessing in turorials

* add a comment

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-03-23 09:28:36 -07:00
nv-dlasalle 57d2f31f20 [Examples] Add pure gpu example of graphsage (#3796)
* Add pure_gpu example of graphsage

* move to advanced directory

Co-authored-by: Quan Gan <coin2028@hotmail.com>
2022-03-10 17:33:23 -08:00
RecLusIve-F f908f35c38 [Example]Add P-GNN example (#3823)
* [Model]P-GNN

* updata

* [Example]P-GNN

* Update README.md

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-03-10 17:39:32 +08:00
Quan (Andy) Gan 44638b9336 fix ddp dataloader in heterogeneous cases (#3801) 2022-03-07 16:56:43 +08:00
Rhett Ying 0528e90d7d [Examples] fix path for load_graph (#3797) 2022-03-03 15:36:48 +08:00
Rhett Ying 0ec439247a [Examples] re-locate load_graph for share (#3784) 2022-03-01 09:32:53 +08:00
Xin Yao 861666fa55 [Bugfix] Fix RGCN Example (#3785) 2022-02-28 18:55:06 +08:00
Quan (Andy) Gan d41d07d0f6 [Doc and bugfix] Add docs and user guide and update tutorial for sampling pipeline (#3774)
* huuuuge update

* remove

* lint

* lint

* fix

* what happened to nccl

* update multi-gpu unsupervised graphsage example

* replace most of the dgl.mp.process with torch.mp.spawn

* update if condition for use_uva case

* update user guide

* address comments

* incorporating suggestions from @jermainewang

* oops

* fix tutorial to pass CI

* oops

* fix again

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-02-28 02:21:51 +08:00
Mufei Li 6d9433b04f [Transform] [Doc] Rename transform to transforms and update doc (#3765)
* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Fix
2022-02-24 21:28:15 +08:00
Minjie Wang 0227ddfb66 [NN] Rework RelGraphConv and HGTConv (#3742)
* WIP: TypedLinear and new RelGraphConv

* wip

* further simplify RGCN

* a bunch of tweak for performance; add basic cpu support

* update on segmm

* wip: segment.cu

* new backward kernel works

* fix a bunch of bugs in kernel; leave idx_a for future

* add nn test for typed_linear

* rgcn nn test

* bugfix in corner case; update RGCN README

* doc

* fix cpp lint

* fix lint

* fix ut

* wip: hgtconv; presorted flag for rgcn

* hgt code and ut; WIP: some fix on reorder graph

* better typed linear init

* fix ut

* fix lint; add docstring
2022-02-23 12:10:20 +08:00
Quan (Andy) Gan 4f00d5ac6f [Bugfix] Fix graph being duplicated in multi-GPU and CPU dataloader workers (#3760)
* fix shared memory issue

* oops

* add explanation

* add explanation
2022-02-22 18:00:47 +08:00
Quan (Andy) Gan 3f138ebaaf [Bugfix] Bug fixes in new dataloader (#3727)
* fixes

* fix

* more fixes

* update

* oops

* lint?

* temporarily revert - will fix in another PR

* more fixes

* skipping mxnet test

* address comments

* fix DDP

* fix edge dataloader exclusion problems

* stupid bug

* fix

* use_uvm option

* fix

* fixes

* fixes

* fixes

* fixes

* add evaluation for cluster gcn and ddp

* stupid bug again

* fixes

* move sanity checks to only support DGLGraphs

* pytorch lightning compatibility fixes

* remove

* poke

* more fixes

* fix

* fix

* disable test

* docstrings

* why is it getting a memory leak?

* fix

* update

* updates and temporarily disable forkingpickler

* update

* fix?

* fix?

* oops

* oops

* fix

* lint

* huh

* uh

* update

* fix

* made it memory efficient

* refine exclude interface

* fix tutorial

* fix tutorial

* fix graph duplication in CPU dataloader workers

* lint

* lint

* Revert "lint"

This reverts commit 805484dd553695111b5fb37f2125214a6b7276e9.

* Revert "lint"

This reverts commit 0bce411b2b415c2ab770343949404498436dc8b2.

* Revert "fix graph duplication in CPU dataloader workers"

This reverts commit 9e3a8cf34c175d3093c773f6bb023b155f2bd27f.

Co-authored-by: xiny <xiny@nvidia.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-22 01:30:31 +08:00
Mufei Li e9c3c0e8ad [Model] Simplify RGCN
* Update (#5)

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* FIx

* Try

* Update

* Update

* Update

* Fix

* Update

* Fix

* Fix

* Fix

* Fix

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Update

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update README.md

* Update

* Fix

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>

* Update

* Update

* Fix

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-57-123.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-02-17 15:33:27 +08:00
Jonathan Broadbent 3b34a5a7ec Add links to RNAGlib in the DGL Readme (#3728)
* Add RNAGlib to examples and DGL-powered-projects

* Add RNAGlib to examples and DGL-powered-projects

* Add RNAGlib to examples and DGL-powered-projects
2022-02-11 15:00:18 -08:00
Xin Yao 738e8318fd [Feature] CUDA UVA sampling for MultiLayerNeighborSampler (#3674)
* implement pin_memory/unpin_memory/is_pinned for dgl.graph

* update python docstring

* update c++ docstring

* add test

* fix the broken UnifiedTensor

* XPU_SWITCH for kDLCPUPinned

* a rough version ready for testing

* eliminate extra context parameter for pin/unpin

* update train_sampling

* fix linting

* fix typo

* multi-gpu uva sampling case

* disable new format materialization for pinned graphs

* update python doc for pin_memory_

* fix unit test

* UVA sampling for link prediction

* dispatch most csr ops

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update doc

* update examples

* change unitgraph and heterograph's PinMemory to in-place

* update examples for multi-gpu uva sampling

* update doc

* fix linting

* fix cpu build

* fix is_pinned for DistGraph

* fix is_pinned for DistGraph

* update graphsage unsupervised example

* update doc for gpu sampling

* update some check for sampling device switching

* fix linting

* adapt for new dataloader

* fix linting

* fix

* fix some name issue

* adjust device check

* add unit test for uva sampling & fix some zero_copy bug

* fix linting

* update num_threads in graphsage examples

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-09 14:06:17 +08:00
Jinjing Zhou 0767c5fcc9 Fix dist example padding problem (#3687) 2022-02-07 17:44:44 +08:00
Quan (Andy) Gan 701b4fccc2 [Sampling] New sampling pipeline plus asynchronous prefetching (#3665)
* initial update

* more

* more

* multi-gpu example

* cluster gcn, finalize homogeneous

* more explanation

* fix

* bunch of fixes

* fix

* RGAT example and more fixes

* shadow-gnn sampler and some changes in unit test

* fix

* wth

* more fixes

* remove shadow+node/edge dataloader tests for possible ux changes

* lints

* add legacy dataloading import just in case

* fix

* update pylint for f-strings

* fix

* lint

* lint

* lint again

* cherry-picking commit fa9f494

* oops

* fix

* add sample_neighbors in dist_graph

* fix

* lint

* fix

* fix

* fix

* fix tutorial

* fix

* fix

* fix

* fix warning

* remove debug

* add get_foo_storage apis

* lint
2022-01-30 16:13:00 +08:00
PengZhang 8f99b13193 feature: add a parse parameter degree_as_nlabel for pytorch-gin demo (#3676)
* feature: add a parse parameter degree_as_nlabel for pytorch-gin demo

* fix some typo

* [fix]: allow to benchmark all of the 9 dataset.

* [Feature] add epoch number to log

* [Feature]:simply list the command lines for all datasets (https://github.com/dmlc/dgl/pull/3676#discussion_r790270705) and run a test.

* Update README.md

Co-authored-by: Ubuntu <ubuntu@ip-172-31-10-175.ap-northeast-1.compute.internal>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-01-25 19:24:41 +08:00
nxznm 9a6b81ef7d [Bugfix] Improve CompGCN (#3663)
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-01-23 22:44:29 +08:00
Zekuan (Kay) Liu ed4134ed74 [Example] fix auc in caregnn example (#3647)
Co-authored-by: zhjwy9343 <6593865@qq.com>
2022-01-21 18:33:29 +08:00
Da Zheng 14ab462fd8 Update examples/pytorch/graphsage/experimental/README.md
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2022-01-20 09:19:47 -08:00
Da Zheng 3b1978a379 fix. 2022-01-19 12:49:55 -08:00
Da Zheng 8d14a739bc fix. (#3652)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-164.us-west-2.compute.internal>
2022-01-15 09:12:13 +08:00
xcwan 4889c57822 Add 'nccl' backend in train_dist.py and fix pad_data function cuda bug (#3607)
* Add nccl backend  and fix pad_data function cuda bug

* Update train_dist.py

* Update train_dist.py

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-12-24 18:45:01 +08:00
Zak Jost 42897c3652 [Example] An example that re-creates the PyG OGB performance on ogbnmag (#3563)
* Adding initial files of example

* Removing old timing code

* Improving doc strings and fixing some minor bugs

* Merging from upstream and addressing PR comments

Co-authored-by: zakjost <jostza@amazon.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2021-12-20 22:22:25 +08:00
Jinjing Zhou 421c362266 Update train_dist.py (#3594)
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-12-20 20:24:45 +08:00
Vasimuddin Md 78e0dae693 [DistGNN, Graph partitioning] Libra partition (#3376)
* added distgnn plus libra codebase

* Dist application codes

* added comments in partition code. changed the interface of partitioning call.

* updated readme

* create libra partitioning branch for the PR

* removed disgnn files for first PR

* updated kernel.cc

* added libra_partition.cc and moved libra code from kernel.cc to libra_partition.cc

* fixed lint error; merged libra2dgl.py and main_Libra.py to libra_partition.py; added graphsage/distgnn folder and partition script.

* removed libra2dgl.py

* fixed the lint error and cleaned the code.

* revisions due to PR comments. added distgnn/tools contains partitions routines

* update 2 PR revision I

* fixed errors; also improved the runtime by 10x.

* fixed minor lint error

* fixed some more lints

* PR revision II changed the interface of libra partition function

* rewrite docstring

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-12-15 10:02:31 +08:00
Jinjing Zhou 987db374fd Fix for distributed training (#3542)
* tmp fix

* add description
2021-12-06 18:35:29 +08:00
Israt Nisa 490c5a8d72 [Model] RGCN with new heterograph API (#3025)
* rgcn with new heterograph API

* added new apply_edge()

* optimized forward pass

* renaming from *hetero to *heteroAPI

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2021-11-30 12:59:35 -08:00
Jinjing Zhou da53275a17 Fix tgn example (#3543) 2021-11-29 14:58:38 +08:00
Rhett Ying cd6d113874 [BugFix] fix dimension unmatch issue and legacy issue of torchtext (#3539) 2021-11-24 15:08:06 +08:00
Harsh Sinha 4bf70f0941 [Bugfix] issue #3527 (#3528)
* Fix issue 3527

* Changed default device

* Added g to device
2021-11-23 08:52:26 +08:00
Mufei Li 9e7fbf9567 [NN] JumpingKnowledge (#3512)
* Update

* Fix
2021-11-19 16:37:25 +08:00
Krzysztof Sadowski 81915f5591 [Examples] RGCN Heterogeneous on ogbn-mag (#3371)
* upload

* cleanup of unused code

* default gpu training/inference

* layer norm instead of batch norm

* fix for default inference mode

* simplified embedding forward method

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2021-11-18 00:05:24 +08:00
Quan (Andy) Gan b8ce0f41a3 [Sampling] Cluster-GCN and ShaDow-GNN DataLoader (#3487)
* first commit

* next commit

* third commit

* add ShaDow-GNN sampler and unit tests

* fixes

* lint

* cr*p

* lint

* fix lint

* fixes and more unit tests

* more tests

* fix docs

* lint

* fix

* fix

* fix

* fixes

* fix doc
2021-11-16 13:16:25 +08:00
Yuchen 7c771d0d6c [BugFix] fix #3429 and update results of caregnn (#3441)
* squeeze node labels in FraudDataset

* fix RLModule

* update results in README.md

* fix KeyError in full graph training

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-11-10 16:00:12 +08:00
yifeim fe6e01ad10 [Model] Lda subgraph (#3206)
* add word_ids and simplify

* simplify

* add word_ids to be removed later

* remove word_ids

* seems to work

* tweak

* transpose word_z

* add word_ids example

* check api compatibility

* improve compatibility

* update doc

* tweak verbose

* restore word_z layout; tweak

* tweak

* tweak doc

* word_cT

* use log_weight and some other tweaks

* rewrite README

* update equations

* rewrite for clarity and pass tests

* tweak

* bugfix import

* fix unit test

* fix mult to be the same as old versions

* tweak

* could be a bugfix

* 0/0=nan

* add doc_subgraph utility function

* minor cache optimization

* minor cache tweak

* add environmental variable to trade cache speed for memory

* update README

* tweak

* add sparse update pass unit test

* simplify sparse update

* improve low-memory efficiency

* tweak

* add sample expectation scores to allow resampling

* simplify

* update comment

* avoid edge cases

* bugfix pred scores

* simplify

* add save function

Co-authored-by: Yifei Ma <yifeim@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-11-08 17:44:04 +08:00
Rhett Ying 2a757d4aab Remove self-loops and duplicate edges before ParMETIS and restore when converting to DGLGraph (#3472)
* save self-loops and duplicated edges separately.

* [BugFix] sort graph by dgl.ETYPE

* fix bugs in verify script

* fix verify logic

* refine README

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2021-11-08 11:17:19 +08:00