文件历史

提交图

1382 次代码提交

作者 SHA1 备注 提交日期
Rhett Ying ccaa0bf282 [Doc] refine docstring and examples for reorder_graph (#3770)
* [Doc] refine docstring and examples for reorder_graph

* refine docstring
2022-02-24 17:31:00 +08:00
Rhett Ying 609a49a9ba [Doc] fix example in dataset adapter docstring (#3772) 2022-02-24 16:57:25 +08:00
Rhett Ying 2f4146a4c2 [Dataset] add train_idx/val_idx/test_idx as dataset attributes (#3769)
* add train_idx/val_idx/test_idx as dataset attributes

* refine docstring
2022-02-24 15:17:40 +08:00
Rhett Ying 8db2dd3355 [Dataset][Fix] Allow non-numeric values and some fix on doc (#3757)
* [Fix] be able to parse ids if numeric and non-numeric values are used together

* add required package info and cache note into docstring

* duplicate node id is not allowed
2022-02-23 17:26:12 +08:00
Minjie Wang 0227ddfb66 [NN] Rework RelGraphConv and HGTConv (#3742)
* WIP: TypedLinear and new RelGraphConv

* wip

* further simplify RGCN

* a bunch of tweak for performance; add basic cpu support

* update on segmm

* wip: segment.cu

* new backward kernel works

* fix a bunch of bugs in kernel; leave idx_a for future

* add nn test for typed_linear

* rgcn nn test

* bugfix in corner case; update RGCN README

* doc

* fix cpp lint

* fix lint

* fix ut

* wip: hgtconv; presorted flag for rgcn

* hgt code and ut; WIP: some fix on reorder graph

* better typed linear init

* fix ut

* fix lint; add docstring
2022-02-23 12:10:20 +08:00
Quan (Andy) Gan 4f00d5ac6f [Bugfix] Fix graph being duplicated in multi-GPU and CPU dataloader workers (#3760)
* fix shared memory issue

* oops

* add explanation

* add explanation
2022-02-22 18:00:47 +08:00
Quan (Andy) Gan 3f138ebaaf [Bugfix] Bug fixes in new dataloader (#3727)
* fixes

* fix

* more fixes

* update

* oops

* lint?

* temporarily revert - will fix in another PR

* more fixes

* skipping mxnet test

* address comments

* fix DDP

* fix edge dataloader exclusion problems

* stupid bug

* fix

* use_uvm option

* fix

* fixes

* fixes

* fixes

* fixes

* add evaluation for cluster gcn and ddp

* stupid bug again

* fixes

* move sanity checks to only support DGLGraphs

* pytorch lightning compatibility fixes

* remove

* poke

* more fixes

* fix

* fix

* disable test

* docstrings

* why is it getting a memory leak?

* fix

* update

* updates and temporarily disable forkingpickler

* update

* fix?

* fix?

* oops

* oops

* fix

* lint

* huh

* uh

* update

* fix

* made it memory efficient

* refine exclude interface

* fix tutorial

* fix tutorial

* fix graph duplication in CPU dataloader workers

* lint

* lint

* Revert "lint"

This reverts commit 805484dd553695111b5fb37f2125214a6b7276e9.

* Revert "lint"

This reverts commit 0bce411b2b415c2ab770343949404498436dc8b2.

* Revert "fix graph duplication in CPU dataloader workers"

This reverts commit 9e3a8cf34c175d3093c773f6bb023b155f2bd27f.

Co-authored-by: xiny <xiny@nvidia.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-22 01:30:31 +08:00
Jinjing Zhou f247d29f6c Minor fix to DGL Enter (#3753)
* [Fix] Convert float64 to float32 when creating tensor

* fix

Co-authored-by: RhettYing <rhett_ying@qq.com>
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-02-18 18:04:07 +08:00
Rhett Ying 5558ce2911 [Fix] Convert float64 to float32 when creating tensor (#3751)
* [Fix] Convert float64 to float32 when creating tensor

* refine docstring
2022-02-18 17:34:07 +08:00
Jinjing Zhou 539335cece DGL Enter (#3690)
* add

* fix

* fix

* fix

* fix

* add

* add

* fix

* fix

* fix

* new loader

* fix

* fix

* fix for 3.6

* fix

* add

* add receipes and also some bug fixes

* fix

* fix

* fix

* fix receipies

* allow AsNodeDataset to work on ogb

* add ut

* many fixes for nodepred-ns pipeline

* receipe for nodepred-ns

* Update enter/README.md

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>

* fix layers

* fix

* fix

* fix

* fix

* fix multiple issues

* fix for citation2

* fix comment

* fix

* fix

* clean up

* fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
2022-02-18 15:55:41 +08:00
Rhett Ying 80fb4dbe26 [Doc] fix examples (#3747) 2022-02-17 18:46:13 +08:00
dddg617 9e358dfe66 [NN] HeteroLinear and HeteroEmbedding (#3678)
* modify hetero

* modify rst document

* update hetero

* update hetero

* update hetero

* update hetero

* Update

* Update

* Update

* Update

* 20220216

* Update

* Update

* Fix

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: ShelkerX <925089962@qq.com>
2022-02-17 16:16:51 +08:00
Mufei Li e9c3c0e8ad [Model] Simplify RGCN
* Update (#5)

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* FIx

* Try

* Update

* Update

* Update

* Fix

* Update

* Fix

* Fix

* Fix

* Fix

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Update

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update README.md

* Update

* Fix

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>

* Update

* Update

* Fix

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-57-123.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-02-17 15:33:27 +08:00
Rhett Ying 22272de652 [Fix] normalize by dst if edge_weight is None (#3744)
* [Fix] normalize by dst if edge_weight is None

* [Doc] fix math formula display issue
2022-02-17 13:51:37 +08:00
RhettYing 6106a99d0a refine 2022-02-17 02:00:10 +00:00
RhettYing e0f054fb27 [Doc] add doc for DGLCSVDataset 2022-02-17 00:52:16 +00:00
Rhett Ying 39121dfdb8 [Feature] support non-numeric node_id/src_id/dst_id/graph_id and rena… (#3740)
* [Feature] support non-numeric node_id/src_id/dst_id/graph_id and rename CSVDataset

* change return value when iterate dataset

* refine data_parser

* force reload
2022-02-17 08:35:39 +08:00
Rhett Ying 42f8c8f335 [doc] fix module typo (#3734)
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-02-16 11:10:06 +08:00
Rhett Ying 97a6d740ad [doc] fix error in example results (#3735)
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-02-16 09:40:45 +08:00
Rhett Ying 4f7ad2582c [doc] fix error in example (#3736) 2022-02-15 18:44:15 +08:00
Mufei Li 8b8fd2c0be [Dataset] Add transform argument to built-in datasets (#3733)
* Update

* Fix

* Update
2022-02-15 16:45:47 +08:00
Israt Nisa b3d3a2c4b0 [Feature] Gather mm (#3641)
* init

* init

* working cublasGemm

* benchmark high-mem/low-mem, err gather_mm output

* cuda kernel for bmm like kernel

* removed cpu copy for E_per_Rel

* benchmark code from Minjie

* fixed cublas results in gathermm sorted

* use GPU shared mem in unsorted gather mm

* minor

* Added an optimal version of gather_mm_unsorted

* lint

* init gather_mm_scatter

* cublas transpose added

* fixed h_offset for multiple rel

* backward unittest

* cublas support to transpose W

* adding missed file

* forgot to add header file

* lint

* lint

* cleanup

* lint

* docstring

* lint

* added unittest

* lint

* lint

* unittest

* changed err type

* skip cpu test

* skip CPU code

* move in-len loop inside

* lint

* added check different dim length for B

* w_per_len is optional now

* moved gather_mm to pytorch/backend with backward support

* removed a_/b_trans support

* transpose op inside GEMM call

* removed out alloc from API, changed W 2D to 3D

* Added se_gather_mm, Separate API for sortedE

* Fixed gather_mm (unsorted) user interface

* unsorted gmm backward + separate CAPI for un/sorted A

* typecast to float to support atomicAdd

* lint typecast

* lint

* added gather_mm_scatter

* minor

* const

* design changes

* Added idx_a, idx_b support gmm_scatter

* dgl doc

* lint

* adding gather_mm in ops

* lint

* lint

* minor

* removed benchmark files

* minor

* empty commit

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-02-14 22:27:23 -05:00
Jinjing Zhou dc78e11c3f [Dataset] Adapter to convert a dataset for link prediction task (#3699)
* add ut

* add doc link

* install dep

* add

* Revert "install dep"

This reverts commit e574a8377144749056c6849b655004df2771e179.

* add

* merge fix

* rm files

* fix

* fix

* fix

* fix

* fix typo

* fix tf

* fix

* fix

* fix

* fix

* fix

* fix dependency

* fix test

* fix

* fix

* add doc

* fix

* fix

* fix test

* fix test

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-02-11 16:59:14 +08:00
ranzhejiang bc8f8b0b7a New fused edge_softmax op (#3650)
* [feature] edge softmax refact.

* delete file

* fix backward and cmake version

* fix backward

* format function

* fix setting

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* refix

* add cuda kernel for backward and rename some function

* add benchmark for edge_softmax

* fix format

* remove cuda_backwrd

* fix code format and add comment for op on CPU

* fix lint

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-11 13:34:24 +08:00
Rhett Ying 45ac57260a [data] refine AsNodePredDataset and add tests for DGLCSVDataset (#3722)
* [data] refine AsNodePredDataset and add tests for DGLCSVDataset

* fix

* remove add_self_loop

* refine
2022-02-10 17:46:39 +08:00
Rhett Ying fcd8ed9acc [Feature] Launch Long Live Servers and Multiple Client Groups (#3688)
* enable to launch multiple client groups sequentially

* launch simultaneously is enabled

* refine docstring

* revert unnecessary change

* [DOC] add doc for long live server

* refine

* refine doc

* refine doc
2022-02-09 15:56:38 +08:00
Xin Yao 738e8318fd [Feature] CUDA UVA sampling for MultiLayerNeighborSampler (#3674)
* implement pin_memory/unpin_memory/is_pinned for dgl.graph

* update python docstring

* update c++ docstring

* add test

* fix the broken UnifiedTensor

* XPU_SWITCH for kDLCPUPinned

* a rough version ready for testing

* eliminate extra context parameter for pin/unpin

* update train_sampling

* fix linting

* fix typo

* multi-gpu uva sampling case

* disable new format materialization for pinned graphs

* update python doc for pin_memory_

* fix unit test

* UVA sampling for link prediction

* dispatch most csr ops

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update doc

* update examples

* change unitgraph and heterograph's PinMemory to in-place

* update examples for multi-gpu uva sampling

* update doc

* fix linting

* fix cpu build

* fix is_pinned for DistGraph

* fix is_pinned for DistGraph

* update graphsage unsupervised example

* update doc for gpu sampling

* update some check for sampling device switching

* fix linting

* adapt for new dataloader

* fix linting

* fix

* fix some name issue

* adjust device check

* add unit test for uva sampling & fix some zero_copy bug

* fix linting

* update num_threads in graphsage examples

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-09 14:06:17 +08:00
KoyamaSohei fa343873a8 Fix docs on GraphSAGE normalization (#3711)
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-07 17:56:04 +08:00
nv-dlasalle f282ee30ad [bugfix] Fix set_default_backend() keyword (#3710)
* Add unit test

* Fix typo

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-07 16:15:02 +08:00
Jeremy Goh 25c9221b83 Add check for aggregator_type enum in SAGEConv init (#3691)
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-02-03 12:02:35 +08:00
Quan (Andy) Gan 701b4fccc2 [Sampling] New sampling pipeline plus asynchronous prefetching (#3665)
* initial update

* more

* more

* multi-gpu example

* cluster gcn, finalize homogeneous

* more explanation

* fix

* bunch of fixes

* fix

* RGAT example and more fixes

* shadow-gnn sampler and some changes in unit test

* fix

* wth

* more fixes

* remove shadow+node/edge dataloader tests for possible ux changes

* lints

* add legacy dataloading import just in case

* fix

* update pylint for f-strings

* fix

* lint

* lint

* lint again

* cherry-picking commit fa9f494

* oops

* fix

* add sample_neighbors in dist_graph

* fix

* lint

* fix

* fix

* fix

* fix tutorial

* fix

* fix

* fix

* fix warning

* remove debug

* add get_foo_storage apis

* lint
2022-01-30 16:13:00 +08:00
Minjie Wang 5152a87982 [Data] Utility function and class for converting a dataset for node prediction (#3695)
* add ut

* add doc link

* install dep

* fix  ci

* fix ut; more comments

* remove deprecated attributes in rdf datasets; fix label feature name

* address comments

* fix ut for other frameworks
2022-01-30 14:31:37 +08:00
Mufei Li 56b5d0e53a [Transform] Allow Disabling Adding Reverse Edges for Self Loops (#3701)
* Update

* Update

* Update
2022-01-29 18:38:09 +08:00
Quan (Andy) Gan 92db4bd5a0 fix (#3700) 2022-01-28 20:51:15 +08:00
Quan (Andy) Gan da877df238 [CI] Migrate to pylint 2.6.0 (#3698)
* migrate to pylint 2.6.0

* fix

* fix?

* ???

* oops
2022-01-28 18:32:18 +08:00
nv-dlasalle 268f617642 Fix race condition in creating directory (#3696)
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-01-28 15:58:03 +08:00
Rhett Ying 6255c95aae [Fix] too many open files (#3694)
* [Fix] too many open files
2022-01-28 13:24:20 +08:00
Mufei Li 05c6c3c5ec [NN] Fix GINConv (#3692)
* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix lint

* lint

* Update

* Update

* lint fix

* Fix CI

* Fix

* Fix CI

* Update

* Fix

* Update

* Update

* Update ginconv.py

* Update test_nn.py

Co-authored-by: Ubuntu <ubuntu@ip-172-31-31-136.us-west-2.compute.internal>
2022-01-27 12:46:29 +08:00
Jinjing Zhou c8fef62928 Fix wrong logging level (#3689) 2022-01-26 22:59:20 +08:00
Rhett Ying 02e4cd8b56 [Feature] long live server for multiple client groups (#3645)
* [Feature] long live server for multiple client groups

* generate globally unique name for DistTensor within DGL automatically
2022-01-26 15:21:14 +08:00
Mufei Li 2b98e764e8 [Transform] Modules for Augmentation (#3668)
* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix lint

* lint

* Update

* Update

* lint fix

* Fix CI

* Fix

* Fix CI

* Update

* Fix

* Update

* Update

* Augmentation (#10)

* Update

* PPR

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* CI

* lint

* lint

* Update

* Update

* Fix AddEdge

* try import

* Update

* Fix

* CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-31-136.us-west-2.compute.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-01-25 20:29:46 +08:00
Jinjing Zhou ba62b730b4 Disable tensoradapter and "using backend: xxx" log by default (#3684)
* fix

* lint

* lint

* address comment
2022-01-25 19:32:54 +08:00
Jeremy Goh 8f5baa15e4 [Doc] Fix spelling, references and update info on building docs (#3682)
* Fix ref to message-passing guide

* Fix pygments and spacing

* Update build documentation steps in README.md

* Use links

* Adjust parameters in SAGEConv docstring in same order as init

* Fix spelling error

* Change doc link
2022-01-25 17:22:18 +08:00
Xin Yao 40b44a43a9 [Feature] Pin dgl.graph to the page-locked memory (#3616)
* implement pin_memory/unpin_memory/is_pinned for dgl.graph

* update python docstring

* update c++ docstring

* add test

* fix the broken UnifiedTensor

* eliminate extra context parameter for pin/unpin

* fix linting

* fix typo

* disable new format materialization for pinned graphs

* update python doc for pin_memory_

* fix unit test

* update doc

* change unitgraph and heterograph's PinMemory to in-place

* update comments for NDArray's PinMemory_ and PinData

* update doc

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-01-21 16:13:12 +08:00
Rhett Ying e4cb4a3779 [Fix] reduce error msg, refine fetch logic of available ports (#3658)
* [Fix] reduce error msg, refine fetch logic of available ports

* un-initialize client before sending shutdown request

* fix import error

* print connect failure log only in debug mode

* enable DMLC_LOG_DEBUG=1 in CI
2022-01-19 09:41:58 +08:00
Rhett Ying 9ec9df57ca [Fix] check and load dependencies when needed (#3655)
* [Fix] check and load dependencies when needed

* refine rdflib import
2022-01-18 16:10:47 +08:00
Mufei Li 48cbea720e [Transform] Module Interface for Transform (#3636)
* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix lint

* lint

* Update

* Update

* lint fix

* Fix CI

* Fix

* Fix CI

* Update

* Fix

* Update

* Update

* resolve conflict

* Fix CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-31-136.us-west-2.compute.internal>
2022-01-17 17:45:07 +08:00
Da Zheng 2dd114ed24 [Distributed] Measure peak memory size in graph partitioning (#3633)
* measure peak memory size.

* fix lint.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-164.us-west-2.compute.internal>
2022-01-13 15:41:53 -08:00
Quan (Andy) Gan d03138e2cb [Bugfix?] Remove simple graph requirement for to_bidirected (#3630)
* remove simple graph requirement

* doc

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-01-13 18:46:06 +08:00
Andrew 7f83d7456a [Feature] method for merging graphs into graphs (#3522)
* Added graph updating method and tests. resolves #3488

* removed spaces around named args

* customizing indices for graph's idtype and ctx

* changing torch ops to generic backend ops

* changing tensors to np arrays

* created dgl merge function and tests

* Changed per-graph edge updates to single ag update

* removed update method and tests

* reformat newlines & spaces

* concatenating in one-shot instead of iteratively

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-01-12 00:20:22 +08:00