文件历史

87 次代码提交

作者 SHA1 备注 提交日期
Rhett Ying 0c5d6f1b00 [BUILD] enable dgl sparse build in default (#5175)
* enable dgl sparse build in default

* disable sparse build on cugraph

* update
2023-01-16 14:01:39 +08:00
peizhou001 56ffb650fe [API Deprecation]Deprecate contrib module (#5114) 2023-01-06 20:16:01 +08:00
Minjie Wang 025e43210f [Sparse] Add tests for sparse examples; Migrate from mock_sparse to mock_sparse2 (#5073)
* pass gcn

* example tests

* pass gcnii, hgnn, sgc, sign

* black

* test file

* add torchmetrics

* migrate appnp, c_and_s, twirls

* fix style

* minor fix

* fix hypergraphatt and twirls

* pass gat

* fix torchmetric

* lintrunner

Co-authored-by: Mufei Li <mufeili1996@gmail.com>
2022-12-29 08:55:25 +08:00
Rhett Ying 9731e02325 [Dist] enable to chunk node/edge data into arbitrary number of chunks (#4930)
* [Dist] enable to chunk node/edge data into arbitrary number of chunks

* [Dist] enable to split node/edge data into arbitrary parts

* refine code

* Format boolean to uint8 forcely to avoid dist.scatter failure

* convert boolean to int8 before scatter and revert it after scatter

* refine code

* fix test

* refine code

* move test utilities into utils.py

* update comment

* fix empty data

* update

* update

* fix empty data issue

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* remove unnecessary shuffle data

* separate array_split into standalone utility

* add example

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2022-12-15 10:01:39 +08:00
czkkkkkk 7c059e86c7 [Sparse] Add sparse matrix C++ implementation (#4773)
* [Sparse] Add sparse matrix C++ implementation

* Add documentation

* Update

* Minor fix

* Move Python code to dgl/mock_sparse2

* Move headers to include

* lint

* Update

* Add dgl_sparse directory

* Move src code to dgl_sparse

* Add __init__.py in tests to avoid naming conflict

* Add dgl sparse so in Jenkinsfile

* Complete docstring & SparseMatrix basic op

* lint

* Disable win tests
2022-11-07 18:01:19 +08:00
Xin Yao 96297fb8fd [Feature] Add bfloat16 (bf16) support (#4648)
* add bf16 specializations

* remove SWITCH_BITS

* enable amp for bf16

* remove SWITCH_BITS for cpu kernels

* enbale bf16 based on CUDART

* fix compiling for sm<80

* fix cpu build

* enable unit tests

* update doc

* disable test for CUDA < 11.0

* address comments

* address comments
2022-11-06 19:48:36 +08:00
Hongzhi (Steve), Chen ea48ce7a37 [Misc] Black auto fix. (#4697)
Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>
2022-10-11 13:31:13 +08:00
Rhett Ying be11409312 [CI] Enable DGL cugaph nightly CI (#4573)
* [CI] Enable DGL cugaph nightly CI

* fix build issue

* remove runtime
2022-09-22 15:26:29 +08:00
Xin Yao 880b3b1f97 [Fix] Enable lint check for cuh files and fix compiler warnings (#4585)
* disable warning for tensorpipe

* fix warning

* enable lint check for cuh files

* resolve comments
2022-09-21 10:09:14 +08:00
Rhett Ying 7f984b64b2 Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563)
This reverts commit ec171c648a.
2022-09-16 13:57:50 +08:00
Vibhu Jawa ec171c648a [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)
* Added cugraph nightly scripts

* Removed nvcr.io//nvidia/pytorch:22.04-py3 reference

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-09-16 11:28:30 +08:00
Rhett Ying 86656a6b34 [CI] use https instead of http (#4488) 2022-08-31 14:59:40 +08:00
Mufei Li 2e8ae9f980 [Dist][CI] Unit test for the new distributed partitioning pipeline (#4394)
* chunked graph data format

* Update

* Update

* Update task_distributed_test.sh

* Update

* Update

* Revert "Update"

This reverts commit 03c461870f19375fb03125b061fc853ab555577f.

* Update

* Update

* ssh-keygen

* CI

* install openssh

* openssh

* Update

* CI

* Update

* Update

Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-142.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-87.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-20-21.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>
2022-08-19 14:20:56 +08:00
Rhett Ying 8d3c58202a [CI] upgrade python version to 3.7.0 (#4406)
* [CI] upgrade python version to 3.7.0

* do not upgrade for mxnet cpu due to seg fault

* fix test failure for mxnet
2022-08-17 08:02:44 +00:00
Rhett Ying b3242e90b8 [CI] separate distributed tests from torch cpu tests (#4313)
* [CI] separate distributed tests from torch cpu tests

* remove TF related env
2022-07-30 16:32:35 +08:00
Rhett Ying 17f1432ab2 [DistTest] fix incorrect shell if statement (#4304)
* [DistTest] fix incorrect shell if statement

* fix incorrect use of dist.initialize()
2022-07-28 20:14:47 +08:00
Xin Yao 79b0a50afc [Unittest][Fix] Several unit tests fixes for Ampere+ and PyTorch 1.12+ (#4213)
* Fix test_csrmm for tensor core

* unset allow tf32 flag

* update test unified tensor

* skip fp16 for CPU
2022-07-14 19:59:57 +08:00
Chang Liu 885be1784b [Example][Refactor] Refactor GCN example (#4160)
* Refactor GCN example

* Refactor GCN based on graphsage

* Readme update

* Minor update

* update

* Remove user-defined GCN implementation

* README update

* Update

* Update CONTRIBUTORS.md

* update task_example_test

Co-authored-by: Xin Yao <xiny@nvidia.com>
2022-07-05 15:58:31 +08:00
Rhett Ying 85f281170f [CI] Add new CI stage for testing cugraph (#4171)
* [CI] add new stage specific forcuda related features based on nvidia+pytorch

* build and test for gpu_nv

* fix build failure

* fix unit tests

* make -j

* install cython beforehand

* copy cython lib

* test cugraph tests only

* fix typo

* separate test script for cugraph

* refactor build dgl shell
2022-07-05 10:57:07 +08:00
Minjie Wang f7dae4533a [CI] Reduce CI workload (#4196)
* try optimize CI

* fix go test; adjust timing report

* disable certain tests for mx/tf backends

* fix ut

* add pydantic
2022-06-30 20:01:16 +08:00
Serge Panev 652f4c0743 [Dist] Add env var for non-default SSH configs in tests (#4098)
Signed-off-by: Serge Panev <spanev@nvidia.com>

Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-06-15 12:38:50 +08:00
Rhett Ying 9501ed6a07 [Dist] master port should be fixed for all trainers (#4108)
* [Dist] master port should be fixed for all trainers

* add tests for tools/launch.py
2022-06-14 11:22:22 +08:00
Rhett Ying c1ff4c9b41 [DistTest] add basic pipeline for dist test across machines (#3984)
* [DistTest] add basic pipeline for dist test across machines

* move launch remote cmd to separate file

* add test for rpc

* fix function naming rule

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-06-08 12:37:02 +08:00
nv-dlasalle 7a065a9c56 [Build][Tests] Enable FP16 for GPU builds in CI (#4030)
* Enable FP16 for GPU builds in CI

* Limit default GPU archs to pascal and above

* Disable FP16 dispatching for cuda architectures less than 60

* Fix linting

* Fix typos
2022-05-26 09:48:28 -07:00
Rhett Ying 65e6b04d8b [Build] enable USE_AVX in CI and regression tests (#4004) 2022-05-12 21:29:11 -07:00
Vikram Sharma cb39eae1c4 Supporting DGL build with arch specific optimization (#3996)
Based on the pull request: https://github.com/dmlc/dgl/pull/3983
2022-05-11 12:38:42 -07:00
Rhett Ying 22d7f92407 [CI] print commit/build ID into comment (#3989) 2022-05-10 16:04:21 +08:00
Mufei Li 334e6434d2 [DGL-Go] CI for DGL-Go (#3959)
* Update

* Update

* Fix

* Update

* CI

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update
2022-05-06 12:26:50 +08:00
Rhett Ying 3f792071f6 [BugFix] fix CI status (#3963) 2022-04-29 14:47:19 +08:00
Rhett Ying 56614f320f [BugFix] fix job status in master CI (#3962)
* [BugFix] fix job status in master CI

* finalize
2022-04-29 12:54:13 +08:00
Daniil Sizov 5561af4517 [Feature] Dataloader worker affinitization (#3723)
* PR3355 + CSR conversion workaround

* Remove debug code

* Fix convention errors

* Remove wrongly added code section during merge

* Update to reflect dataloading changes

* Fix missing changes

* Remove comment

* Fix linter errors

* Fix trailing whitespace

* Add wrapper around worker init function

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2022-04-28 13:00:04 +08:00
Rhett Ying 2eaa58e047 [CI] refine status check (#3958) 2022-04-27 15:28:26 +08:00
Jinjing Zhou 338db32d21 [Feature] Try upload report to s3 (#3891)
refactor CI report and log
2022-04-22 16:02:12 +08:00
Jinjing Zhou ab50eb9b4e Add unbuffered to distributed test (#3732)
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com>
2022-02-15 10:23:45 +08:00
Jinjing Zhou dc78e11c3f [Dataset] Adapter to convert a dataset for link prediction task (#3699)
* add ut

* add doc link

* install dep

* add

* Revert "install dep"

This reverts commit e574a8377144749056c6849b655004df2771e179.

* add

* merge fix

* rm files

* fix

* fix

* fix

* fix

* fix typo

* fix tf

* fix

* fix

* fix

* fix

* fix

* fix dependency

* fix test

* fix

* fix

* add doc

* fix

* fix

* fix test

* fix test

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2022-02-11 16:59:14 +08:00
Minjie Wang 5152a87982 [Data] Utility function and class for converting a dataset for node prediction (#3695)
* add ut

* add doc link

* install dep

* fix  ci

* fix ut; more comments

* remove deprecated attributes in rdf datasets; fix label feature name

* address comments

* fix ut for other frameworks
2022-01-30 14:31:37 +08:00
Rhett Ying e4cb4a3779 [Fix] reduce error msg, refine fetch logic of available ports (#3658)
* [Fix] reduce error msg, refine fetch logic of available ports

* un-initialize client before sending shutdown request

* fix import error

* print connect failure log only in debug mode

* enable DMLC_LOG_DEBUG=1 in CI
2022-01-19 09:41:58 +08:00
Rhett Ying 95c0ff6302 [Feature] add CSVDataset to load data from csv files (#3547)
* [Feature] add CSVDataset to load data from csv files

* add CSVDataset class file

* install pyyaml when running unit tests

* install pandas for unit tests

* utilize pydantic for YAML config check

* generate yaml and csv files during test

* make more keys as optional

* remove/rename several keys in yaml config and more tets though looks a bit clumsy

* fix test failure on mxnet

* pass /path/to/dataset instead of yaml path

* code refinement

* code refine

* change several yaml field such as feat and graph_id

* merge graph generation logic

* refine code

* Refactored_first_version

* DGLCSVDataset works for single heterograph

* add more tests

* fix test failure in mxnet

* add docstring

* use list comprehension for dict

* fix version in YAML

* refine data length assert

* use dict.pop directly

* remove ambiguous variable names

* refine graph id missing logic

* refine graph create call

* separate node/edge/graph data parser

* remove separator in DefaultDataParser

* refine validation error log for yaml field

* minor check

* refine code via dict.get()

* move load_from_csv into Node/Edge/GraphData

* move _parse_node/edge/graph_data into Node/Edge/GraphData

* refine id-related fields check

* check duplicate ntypes/etypes when load yaml

* refine docstring
2022-01-10 17:21:57 +08:00
Jinjing Zhou 0b3a6216f5 [Test] Enable kvstore test (#3079)
* try enable kvstore test

* fix

* fix

* seperate out kvstore test

* add comment
2021-07-02 14:54:38 +08:00
Jinjing Zhou 9664cdffd3 [Build] Make nccl optional (#3056)
* fix

* remove nvidiasmi

* fix

* fix docs

* fix

* fix

* 1

* fix

* remove

* skip deprecated kernel

* fix

* Revert "skip deprecated kernel"

This reverts commit c5ceb7f60dbbaf065b81cc3680757fd611d90ad3.

* fix
2021-06-27 22:36:01 +08:00
Mufei Li ff519f98c3 [API] Standardize Subgraph APIs (#2929)
* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix

* Update

* Fix subgraph tests

* Capture stdout for distributed test

* Capture stdout for distributed test

* Update

* Update

* Update

* Update subgraph.cc

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2021-06-21 19:53:37 +08:00
Da Zheng b1628f2398 [Tutorial] Distributed node classification. (#2969)
* add init version.

* fix build.

* fix format.

* fix.

* fix.

* fix format.

* update README.

* avoid running CI on distributed training tutorials.

* Update tutorials/dist/1_node_classification.py

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* fix.

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2021-06-04 21:50:16 +08:00
Jinjing Zhou 6a56562a7c [CI] Use k8s cluster (#2957)
* add

* fix

* set default

* fix

* try master

* try fix

* try

* fix

* 111

* fix

* fix

* update

* ccc

* try

* fix

* fix

* try new machine

* fix

* fix

* fix

* Revert "fix"

This reverts commit e716d66b046f92fe7ae368947a51a036a7a3188a.

* try

* more parrallel

* use k8s for all

* fix name

* try not specify instance type

* ci

* use one yaml

* Revert "use one yaml"

This reverts commit 717d8d852be39fbf2e2e45f9f224aa97907c372c.

* add timeout

* fix permission

* mount efs

* print

* fix pvc

* fix

* restrict num of gpu instances

* check

* fix

* fix
2021-06-04 18:31:11 +08:00
yifeim c018436588 [Example] add latent dirichlet allocation (#2883)
* add lda model

* tweak latent dirichlet allocation

* Update README.md

* Update README.md

* update example index

* update header

* minor tweak

* add example test

* update doc

* Update README.md

* Update README.md

* add partial_fit for free

* Update examples/pytorch/lda/lda_model.py

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>

* Update examples/pytorch/lda/example_20newsgroups.py

Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>

* Update lda_model.py

* bugfix torch Gamma uses rate parameter

Co-authored-by: Yifei Ma <yifeim@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2021-05-17 20:25:36 +08:00
Minjie Wang 8a07ab7737 [Doc] Tutorials re-organization (#2683)
* reorg

* change titles

* rm some stale API doc; minor fix

* fix docs

* add warning

* rm new-tutorial run in ci

* lint
2021-02-20 17:42:45 +08:00
Quan (Andy) Gan 117dd25204 [Doc] Fix inconsistencies and GPU code (#2642)
* fix inconsistencies and GPu

* bug fixes

* fix

* trigger new tutorials
2021-02-08 16:33:39 +08:00
Quan (Andy) Gan 9a7235faf2 [Performance] Use allocator from PyTorch if possible (#2328)
* first commit

* some thoughts

* move around

* more commit

* more fixes

* now it uses torch allocator

* fix symbol export error

* fix

* fixes

* test fix

* add script

* building separate library per version

* fix for vs2019

* more fixes

* fix on windows build

* update jenkinsfile

* auto copy built dlls for windows

* lint and installation guide update

* fix

* specify conda environment

* set environment for ci

* fix

* fix

* fix

* fix again

* revert

* fix cmake

* fix

* switch to using python interpreter path

* remove scripts

* debug

* oops sorry

* Update index.rst

* Update index.rst

* copies automatically, no need for this

* do not print message if library not found

* tiny fixes

* debug on nightly

* replace add_compile_definitions to make CMake 3.5 happy

* fix linking to wrong lib for multiple pytorch envs

* changed building strategy

* fix nightly

* fix windows

* fix windows again

* setup bugfix

* address comments

* change README
2020-12-25 13:57:51 +08:00
xiang song(charlie.song) 09ec6020ac [Example] rm gcn spmv (#2044)
* rm gcn spmv

* remove useless test

* remove useless test

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-08-18 17:48:38 +08:00
Minjie Wang 444becf00a [Misc] Move many deprecated codes to the deprecate folder (#1893)
* some file movements

* move some codes to deprecated

* more deprecation

* lint

* remove useless test
2020-07-30 18:17:33 +08:00
Da Zheng 3e2f94ed20 [Distributed] Refactor distributed training to new DGLGraph. (#1874)
* fix tests in partition.

* fix DistGraph.

* fix without shared memory.

* fix sampling.

* enable distributed test.

* fix tests.

* fix a bug in shared-mem heterograph.

* print better error messages.

* fix.

* don't specify formats.

* fix.

* fix

* small fix.
2020-07-28 22:07:24 -07:00