文件历史

361 次代码提交

作者 SHA1 备注 提交日期
Xin Yao 79b0a50afc [Unittest][Fix] Several unit tests fixes for Ampere+ and PyTorch 1.12+ (#4213)
* Fix test_csrmm for tensor core

* unset allow tf32 flag

* update test unified tensor

* skip fp16 for CPU
2022-07-14 19:59:57 +08:00
Rhett Ying dcf1699280 [BugFix] check whether etype sorted when sampling (#4198) 2022-07-01 15:52:14 +08:00
Rhett Ying 6a6597a02a [Feature] extend sort_csr/csc_by_tag to edge (#4164)
* [Feature] extend sort_csr/csc_by_tag to edge

* fix test ffailure in tensorflow

* refine sorting by edges

* fix docstring

* remove unnecessary mem

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-07-01 13:16:11 +08:00
Minjie Wang f7dae4533a [CI] Reduce CI workload (#4196)
* try optimize CI

* fix go test; adjust timing report

* disable certain tests for mx/tf backends

* fix ut

* add pydantic
2022-06-30 20:01:16 +08:00
nv-dlasalle d2a22984c1 [bugfix] Implement __setstate__ for Column (fixes #4107) (#4174)
* * Workaround for graph data saving/loading compatibility problem in Column class.  There may be more places in DGL with the same issue, due to using Python serialization, instead of a more cohesive, comprehensive strategy.  This is just a local fix.

* Add checking for non-empty states

* Add unit test

* Handle the case of columns without storage

Co-authored-by: ndickson <ndickson@nvidia.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-06-30 10:10:44 +08:00
Mufei Li 8b19c28788 Update test_transform.py (#4190) 2022-06-29 21:03:23 +08:00
nv-dlasalle 1dddaad4f0 [bugfix] Allow communicators of size one when NCCL is missing (#3713)
* Update nccl communicator for when NCCL is missing

* Use static_cast

* Add doc string

* Fix whitespace

* Resrtict unit test to GPU runs

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-06-29 11:48:16 +08:00
Mufei Li cb39dbfa13 Update (#4178) 2022-06-28 15:44:03 +08:00
Xin Yao 077e002fe5 [Bugfix][Rework] Automatically unpin tensors pinned by DGL (rework #3997) (#4135)
* Explicitly unpin tensoradapter allocated arrays

* Undo unrelated change

* Add unit test

* update unit test

* add pinned_by_dgl flag to NDArray::Container

* use dgl.ndarray for holding the pinning status

* update multi-gpu uva inference

* reinterpret cast NDArray::Container* to DLTensor* in MoveAsDLTensor

* update unpin column and examples

* add unit test for unpin column

Co-authored-by: Dominique LaSalle <dlasalle@nvidia.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
2022-06-23 13:56:54 +08:00
RecLusIve-F defa292bc0 [Dataset] Add Flickr and Yelp dataset (#4099)
* Add Flickr and Yelp dataset

* Update flickr.py

* update

* Update yelp.py

* Update yelp.py

* update

* Update yelp.py

* Update test_data.py

* Update yelp.py

* update

* Update test_data.py

* Update yelp.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-06-14 16:14:12 +08:00
Huarui HE 92e7733065 [dataset] Add a reorder flag to builtin datasets (#4104)
* add argument reorder=False for citation_graph

* add description of the argument reorder

* add reordered/un_reordered save_path

* add version number postfix

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-06-12 12:35:30 +08:00
ndickson-nvidia eabcc58e41 [Bug][Feature] Added cublasGemm<__half> specialization (#3988) (#4029)
* * Added specialization of cublasGemm function for `__half` type, to try to address https://github.com/dmlc/dgl/issues/3988

* * Added USE_FP16 guard

* * Added test cases to test_segment_mm, to test newly-added FP16 specialization of cublasGemm

* * Replaced for loop in test_segment_mm with pytest.mark.parametrize, as recommended

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-06-07 16:00:02 +08:00
RecLusIve-F 89655cfda2 [Dataset] Add WikiCS Dataset (#4035)
* Fix bugs & Update dataset

* Update

* Update wikics.py

* Update wikics.py

* Update test_data.py

* Update wikics.py

* Update wikics.py

* Update wikics.py

* update

* Update module.py

* Update dgl.data.rst

* Update wikics.py

* Update wikics.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-06-02 23:03:17 +08:00
Mufei Li d9c25521bc [Data] AsGraphPredDataset (#4073)
* Update

* CI

* Update

* Update

* Fix

* Fix
2022-06-02 16:18:48 +08:00
Quan (Andy) Gan c577dc9fc3 add sanity check (#4050) 2022-05-28 18:51:33 +08:00
Minjie Wang 3c129ad71f [Bugfix] Cython CAPI holding GIL causes deadlock when Python callback is asynchronous (#4036)
* cython nogil

* move APIs to internal and add unit test

* fix lint

* disable callback array test
2022-05-25 10:02:44 +08:00
Mufei Li 230b886ec5 [Bug fix] Misc Fix for Transforms and NN Modules (#4038)
* Update module.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update module.py

* Update

* Update

* Update
2022-05-24 18:08:08 +08:00
nv-dlasalle 1425150459 Fix test naming (#4023)
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-05-23 17:21:06 +08:00
ndickson-nvidia e0e8736fa0 [Feature] Added floating-point conversion functions to dgl.transforms.functional (#3890)
* * Added half_(), float_(), and double_() functions to DGLHeteroGraph, HeteroNodeDataView, and HeteroEdgeDataView, for converting floating-point tensor data to float16, float32, or float64 precision

* * Extracted out private functions for floating-point type conversion, to reduce code duplication

* * Added test for floating-point data conversion functions, half_(), float_(), and double_()

* * Moved half_(), float_(), and double_() functions from HeteroNodeDataView and HeteroEdgeDataView to Frame class

* * Updated test_float_cast() to use dgl.heterograph instead of dgl.graph

* Added to CONTRIBUTORS.md

* * Changed data type conversion to be deferred until the data is accessed, to avoid redundant conversions of data that isn't used.

* * Addressed issues flagged by linter

* * Worked around a bug in the old version of mxnet that's currently used for DGL testing

* * Only defer Column data type conversion if there is a pending device transfer or index sampling to be done.  This is expected to be the desired behaviour based on discussions of a few use cases, as described in the comments.

* * Moved floating-point feature data conversion functions to dgl.transforms.functional
* Changed them from in-place behaviour to shallow copy (out-of-place) behaviour

* * Fixed linter issues

* * Removed lines that unintentionally added to_half, to_float, and to_double to DGLHeteroGraph
* Moved _init_api line to the end of the file again

* * Removed one of the two leading underscores from Frame.__astype_float, making it not fully private

Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
2022-05-16 18:17:15 -07:00
nv-dlasalle 1f2e696080 Prevent users from attempting to pin PyTorch non-contiguous tensors or views only encompassing part of tensor. (#3992)
* Disable pinning non-contiguous memory

* Prevent views from being converted for write

* Fix linting

* Add unit tests

* Improve error message for users

* Switch to pytest function

* exclude mxnet and tensorflow from inplace pinning

* Add skip

* Restrict to pytorch backend

* Use backend to retrieve device

* Fix capitalization in decorator

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-05-16 14:49:36 -07:00
Mufei Li 65b0b9e8c3 [Dataset & Transform] Synthetic Datasets for Explainability and SIGNDiffusion Transform (#3982)
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update
2022-05-16 22:32:09 +08:00
Xin Yao 03024f9587 [Peformance] Remove unnecessary induced vertices in EdgeSubgraph (#3978)
* remove unnecessary induced vertices in EdgeSubgraph

* add unit test
2022-05-16 15:03:42 +08:00
RecLusIve-F b2ec5a2ea6 [Transform] Add FeatMask and RowFeatNormalizer transforms (#3968)
* [Model]P-GNN

* updata

* [Example]P-GNN

* Update README.md

* Add NodeFeatureMasking and NormalizeFeatures

* Update

* Update transforms.rst

* Update

* Update

* Update

* Update test_transform.py

* Update

* Update

* Update test_transform.py

* Update module.py

* Update module.py

* Update module.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-05-07 11:59:39 +08:00
Serge Panev 764b5d0524 Fix test_module_heat_kernel test (#3941)
Signed-off-by: Serge Panev <spanev@nvidia.com>

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-04-23 13:32:29 +08:00
Quan (Andy) Gan 0d878ff8ef [Example] Cleaned GraphSAGE node classification example with PyTorch Lightning (#3863)
* cleaned pl node classification example

* conform to PL's method of updating the dataloader

* update

* lint

* fix test

* fix
2022-04-12 17:26:23 +08:00
nv-dlasalle baa9292899 [bugfix] Fix EdgePredictionSampler for UVA sampling (#3904)
* Add failing unit test

* Fix negative sampler edge types

* fix test

* oops

* revert

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
2022-04-12 15:25:21 +08:00
Xin Yao 5fcd7f290f [Feature] Enable UVA for GPU PinSAGE and RandomWalk (#3857)
* enable uva for pinsage sampler

* unit test

* modify some checks on the python side

* remove legacy random walk code

* update unit test

* update unit test

* fix unit test

* adjust checks

* move some checks to c++

* move max_nodes check to cuda kernel

* fix ci for tf

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
2022-04-11 20:01:23 +08:00
Rhett Ying adb3a7c11e [BugFix] record/restore pin status when pickle/unpickle (#3914)
* [BugFix] record/restore pin status when pickle/unpickle

* disable test on TF

* set version as expected

* unpin memory in test
2022-04-09 11:42:23 +08:00
Quan (Andy) Gan 35e66f426a [Bugfix] Fix UVA sampling with partially specified node types (#3897)
* fix uva with partial node types

* lint

* skip tensorflow unit test
2022-03-31 17:07:02 +08:00
rudongyu 290b7c25c2 [Transform] Random Walk Positional Encoding & Laplacian Positional Encoding (#3869)
* [Transform] Random Walk Positional Encoding & Laplacian Positional Encoding

* Update python/dgl/transforms/module.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update python/dgl/transforms/functional.py & python/dgl/transforms/module.py

* update python/dgl/transforms/functional.py

* update doc indexing

* update unit test

* support weighted graphs for random_walk_pe

* change reference

* fix linter errors

* fix datatype compatibility with MXNet & TF

* fix device errors

* fix result precision errors

* change tensorflow abs api

* reboost CI

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-03-30 15:59:28 +08:00
Quan (Andy) Gan a0e8cf0d3e [Bugfix] Fix UVA sampling with lazy feature slicing (#3862)
* fix uva sampling with features

* fix

* add is_listlike function to distinguish strings from sequence

* fix

Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
2022-03-24 20:13:01 +08:00
Mufei Li 6d9433b04f [Transform] [Doc] Rename transform to transforms and update doc (#3765)
* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Fix
2022-02-24 21:28:15 +08:00
Rhett Ying 2f4146a4c2 [Dataset] add train_idx/val_idx/test_idx as dataset attributes (#3769)
* add train_idx/val_idx/test_idx as dataset attributes

* refine docstring
2022-02-24 15:17:40 +08:00
Rhett Ying 8db2dd3355 [Dataset][Fix] Allow non-numeric values and some fix on doc (#3757)
* [Fix] be able to parse ids if numeric and non-numeric values are used together

* add required package info and cache note into docstring

* duplicate node id is not allowed
2022-02-23 17:26:12 +08:00
Minjie Wang 0227ddfb66 [NN] Rework RelGraphConv and HGTConv (#3742)
* WIP: TypedLinear and new RelGraphConv

* wip

* further simplify RGCN

* a bunch of tweak for performance; add basic cpu support

* update on segmm

* wip: segment.cu

* new backward kernel works

* fix a bunch of bugs in kernel; leave idx_a for future

* add nn test for typed_linear

* rgcn nn test

* bugfix in corner case; update RGCN README

* doc

* fix cpp lint

* fix lint

* fix ut

* wip: hgtconv; presorted flag for rgcn

* hgt code and ut; WIP: some fix on reorder graph

* better typed linear init

* fix ut

* fix lint; add docstring
2022-02-23 12:10:20 +08:00
Quan (Andy) Gan 3f138ebaaf [Bugfix] Bug fixes in new dataloader (#3727)
* fixes

* fix

* more fixes

* update

* oops

* lint?

* temporarily revert - will fix in another PR

* more fixes

* skipping mxnet test

* address comments

* fix DDP

* fix edge dataloader exclusion problems

* stupid bug

* fix

* use_uvm option

* fix

* fixes

* fixes

* fixes

* fixes

* add evaluation for cluster gcn and ddp

* stupid bug again

* fixes

* move sanity checks to only support DGLGraphs

* pytorch lightning compatibility fixes

* remove

* poke

* more fixes

* fix

* fix

* disable test

* docstrings

* why is it getting a memory leak?

* fix

* update

* updates and temporarily disable forkingpickler

* update

* fix?

* fix?

* oops

* oops

* fix

* lint

* huh

* uh

* update

* fix

* made it memory efficient

* refine exclude interface

* fix tutorial

* fix tutorial

* fix graph duplication in CPU dataloader workers

* lint

* lint

* Revert "lint"

This reverts commit 805484dd553695111b5fb37f2125214a6b7276e9.

* Revert "lint"

This reverts commit 0bce411b2b415c2ab770343949404498436dc8b2.

* Revert "fix graph duplication in CPU dataloader workers"

This reverts commit 9e3a8cf34c175d3093c773f6bb023b155f2bd27f.

Co-authored-by: xiny <xiny@nvidia.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-22 01:30:31 +08:00
Rhett Ying 5558ce2911 [Fix] Convert float64 to float32 when creating tensor (#3751)
* [Fix] Convert float64 to float32 when creating tensor

* refine docstring
2022-02-18 17:34:07 +08:00
Jinjing Zhou 539335cece DGL Enter (#3690)
* add

* fix

* fix

* fix

* fix

* add

* add

* fix

* fix

* fix

* new loader

* fix

* fix

* fix for 3.6

* fix

* add

* add receipes and also some bug fixes

* fix

* fix

* fix

* fix receipies

* allow AsNodeDataset to work on ogb

* add ut

* many fixes for nodepred-ns pipeline

* receipe for nodepred-ns

* Update enter/README.md

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>

* fix layers

* fix

* fix

* fix

* fix

* fix multiple issues

* fix for citation2

* fix comment

* fix

* fix

* clean up

* fix

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
2022-02-18 15:55:41 +08:00
Mufei Li e9c3c0e8ad [Model] Simplify RGCN
* Update (#5)

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* FIx

* Try

* Update

* Update

* Update

* Fix

* Update

* Fix

* Fix

* Fix

* Fix

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Update

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update README.md

* Update

* Fix

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>

* Update

* Update

* Fix

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-240.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-57-123.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-02-17 15:33:27 +08:00
Rhett Ying 39121dfdb8 [Feature] support non-numeric node_id/src_id/dst_id/graph_id and rena… (#3740)
* [Feature] support non-numeric node_id/src_id/dst_id/graph_id and rename CSVDataset

* change return value when iterate dataset

* refine data_parser

* force reload
2022-02-17 08:35:39 +08:00
Mufei Li 8b8fd2c0be [Dataset] Add transform argument to built-in datasets (#3733)
* Update

* Fix

* Update
2022-02-15 16:45:47 +08:00
Israt Nisa b3d3a2c4b0 [Feature] Gather mm (#3641)
* init

* init

* working cublasGemm

* benchmark high-mem/low-mem, err gather_mm output

* cuda kernel for bmm like kernel

* removed cpu copy for E_per_Rel

* benchmark code from Minjie

* fixed cublas results in gathermm sorted

* use GPU shared mem in unsorted gather mm

* minor

* Added an optimal version of gather_mm_unsorted

* lint

* init gather_mm_scatter

* cublas transpose added

* fixed h_offset for multiple rel

* backward unittest

* cublas support to transpose W

* adding missed file

* forgot to add header file

* lint

* lint

* cleanup

* lint

* docstring

* lint

* added unittest

* lint

* lint

* unittest

* changed err type

* skip cpu test

* skip CPU code

* move in-len loop inside

* lint

* added check different dim length for B

* w_per_len is optional now

* moved gather_mm to pytorch/backend with backward support

* removed a_/b_trans support

* transpose op inside GEMM call

* removed out alloc from API, changed W 2D to 3D

* Added se_gather_mm, Separate API for sortedE

* Fixed gather_mm (unsorted) user interface

* unsorted gmm backward + separate CAPI for un/sorted A

* typecast to float to support atomicAdd

* lint typecast

* lint

* added gather_mm_scatter

* minor

* const

* design changes

* Added idx_a, idx_b support gmm_scatter

* dgl doc

* lint

* adding gather_mm in ops

* lint

* lint

* minor

* removed benchmark files

* minor

* empty commit

Co-authored-by: Israt Nisa <nisisrat@amazon.com>
2022-02-14 22:27:23 -05:00
Jinjing Zhou dc78e11c3f [Dataset] Adapter to convert a dataset for link prediction task (#3699)
* add ut

* add doc link

* install dep

* add

* Revert "install dep"

This reverts commit e574a8377144749056c6849b655004df2771e179.

* add

* merge fix

* rm files

* fix

* fix

* fix

* fix

* fix typo

* fix tf

* fix

* fix

* fix

* fix

* fix

* fix dependency

* fix test

* fix

* fix

* add doc

* fix

* fix

* fix test

* fix test

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-02-11 16:59:14 +08:00
Rhett Ying 45ac57260a [data] refine AsNodePredDataset and add tests for DGLCSVDataset (#3722)
* [data] refine AsNodePredDataset and add tests for DGLCSVDataset

* fix

* remove add_self_loop

* refine
2022-02-10 17:46:39 +08:00
nv-dlasalle f282ee30ad [bugfix] Fix set_default_backend() keyword (#3710)
* Add unit test

* Fix typo

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-02-07 16:15:02 +08:00
Quan (Andy) Gan 701b4fccc2 [Sampling] New sampling pipeline plus asynchronous prefetching (#3665)
* initial update

* more

* more

* multi-gpu example

* cluster gcn, finalize homogeneous

* more explanation

* fix

* bunch of fixes

* fix

* RGAT example and more fixes

* shadow-gnn sampler and some changes in unit test

* fix

* wth

* more fixes

* remove shadow+node/edge dataloader tests for possible ux changes

* lints

* add legacy dataloading import just in case

* fix

* update pylint for f-strings

* fix

* lint

* lint

* lint again

* cherry-picking commit fa9f494

* oops

* fix

* add sample_neighbors in dist_graph

* fix

* lint

* fix

* fix

* fix

* fix tutorial

* fix

* fix

* fix

* fix warning

* remove debug

* add get_foo_storage apis

* lint
2022-01-30 16:13:00 +08:00
Minjie Wang 5152a87982 [Data] Utility function and class for converting a dataset for node prediction (#3695)
* add ut

* add doc link

* install dep

* fix  ci

* fix ut; more comments

* remove deprecated attributes in rdf datasets; fix label feature name

* address comments

* fix ut for other frameworks
2022-01-30 14:31:37 +08:00
Mufei Li 56b5d0e53a [Transform] Allow Disabling Adding Reverse Edges for Self Loops (#3701)
* Update

* Update

* Update
2022-01-29 18:38:09 +08:00
Mufei Li 2b98e764e8 [Transform] Modules for Augmentation (#3668)
* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix lint

* lint

* Update

* Update

* lint fix

* Fix CI

* Fix

* Fix CI

* Update

* Fix

* Update

* Update

* Augmentation (#10)

* Update

* PPR

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* CI

* lint

* lint

* Update

* Update

* Fix AddEdge

* try import

* Update

* Fix

* CI

Co-authored-by: Ubuntu <ubuntu@ip-172-31-31-136.us-west-2.compute.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-01-25 20:29:46 +08:00
Xin Yao 40b44a43a9 [Feature] Pin dgl.graph to the page-locked memory (#3616)
* implement pin_memory/unpin_memory/is_pinned for dgl.graph

* update python docstring

* update c++ docstring

* add test

* fix the broken UnifiedTensor

* eliminate extra context parameter for pin/unpin

* fix linting

* fix typo

* disable new format materialization for pinned graphs

* update python doc for pin_memory_

* fix unit test

* update doc

* change unitgraph and heterograph's PinMemory to in-place

* update comments for NDArray's PinMemory_ and PinData

* update doc

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2022-01-21 16:13:12 +08:00