86656a6b34
[CI] use https instead of http ( #4488 )
2022-08-31 14:59:40 +08:00
2e8ae9f980
[Dist][CI] Unit test for the new distributed partitioning pipeline ( #4394 )
...
* chunked graph data format
* Update
* Update
* Update task_distributed_test.sh
* Update
* Update
* Revert "Update"
This reverts commit 03c461870f19375fb03125b061fc853ab555577f.
* Update
* Update
* ssh-keygen
* CI
* install openssh
* openssh
* Update
* CI
* Update
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-142.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-87.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-20-21.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
2022-08-19 14:20:56 +08:00
8d3c58202a
[CI] upgrade python version to 3.7.0 ( #4406 )
...
* [CI] upgrade python version to 3.7.0
* do not upgrade for mxnet cpu due to seg fault
* fix test failure for mxnet
2022-08-17 08:02:44 +00:00
b3242e90b8
[CI] separate distributed tests from torch cpu tests ( #4313 )
...
* [CI] separate distributed tests from torch cpu tests
* remove TF related env
2022-07-30 16:32:35 +08:00
17f1432ab2
[DistTest] fix incorrect shell if statement ( #4304 )
...
* [DistTest] fix incorrect shell if statement
* fix incorrect use of dist.initialize()
2022-07-28 20:14:47 +08:00
79b0a50afc
[Unittest][Fix] Several unit tests fixes for Ampere+ and PyTorch 1.12+ ( #4213 )
...
* Fix test_csrmm for tensor core
* unset allow tf32 flag
* update test unified tensor
* skip fp16 for CPU
2022-07-14 19:59:57 +08:00
885be1784b
[Example][Refactor] Refactor GCN example ( #4160 )
...
* Refactor GCN example
* Refactor GCN based on graphsage
* Readme update
* Minor update
* update
* Remove user-defined GCN implementation
* README update
* Update
* Update CONTRIBUTORS.md
* update task_example_test
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-07-05 15:58:31 +08:00
85f281170f
[CI] Add new CI stage for testing cugraph ( #4171 )
...
* [CI] add new stage specific forcuda related features based on nvidia+pytorch
* build and test for gpu_nv
* fix build failure
* fix unit tests
* make -j
* install cython beforehand
* copy cython lib
* test cugraph tests only
* fix typo
* separate test script for cugraph
* refactor build dgl shell
2022-07-05 10:57:07 +08:00
f7dae4533a
[CI] Reduce CI workload ( #4196 )
...
* try optimize CI
* fix go test; adjust timing report
* disable certain tests for mx/tf backends
* fix ut
* add pydantic
2022-06-30 20:01:16 +08:00
652f4c0743
[Dist] Add env var for non-default SSH configs in tests ( #4098 )
...
Signed-off-by: Serge Panev <spanev@nvidia.com >
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
2022-06-15 12:38:50 +08:00
9501ed6a07
[Dist] master port should be fixed for all trainers ( #4108 )
...
* [Dist] master port should be fixed for all trainers
* add tests for tools/launch.py
2022-06-14 11:22:22 +08:00
c1ff4c9b41
[DistTest] add basic pipeline for dist test across machines ( #3984 )
...
* [DistTest] add basic pipeline for dist test across machines
* move launch remote cmd to separate file
* add test for rpc
* fix function naming rule
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-06-08 12:37:02 +08:00
7a065a9c56
[Build][Tests] Enable FP16 for GPU builds in CI ( #4030 )
...
* Enable FP16 for GPU builds in CI
* Limit default GPU archs to pascal and above
* Disable FP16 dispatching for cuda architectures less than 60
* Fix linting
* Fix typos
2022-05-26 09:48:28 -07:00
65e6b04d8b
[Build] enable USE_AVX in CI and regression tests ( #4004 )
2022-05-12 21:29:11 -07:00
cb39eae1c4
Supporting DGL build with arch specific optimization ( #3996 )
...
Based on the pull request: https://github.com/dmlc/dgl/pull/3983
2022-05-11 12:38:42 -07:00
22d7f92407
[CI] print commit/build ID into comment ( #3989 )
2022-05-10 16:04:21 +08:00
334e6434d2
[DGL-Go] CI for DGL-Go ( #3959 )
...
* Update
* Update
* Fix
* Update
* CI
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
2022-05-06 12:26:50 +08:00
3f792071f6
[BugFix] fix CI status ( #3963 )
2022-04-29 14:47:19 +08:00
56614f320f
[BugFix] fix job status in master CI ( #3962 )
...
* [BugFix] fix job status in master CI
* finalize
2022-04-29 12:54:13 +08:00
5561af4517
[Feature] Dataloader worker affinitization ( #3723 )
...
* PR3355 + CSR conversion workaround
* Remove debug code
* Fix convention errors
* Remove wrongly added code section during merge
* Update to reflect dataloading changes
* Fix missing changes
* Remove comment
* Fix linter errors
* Fix trailing whitespace
* Add wrapper around worker init function
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-04-28 13:00:04 +08:00
2eaa58e047
[CI] refine status check ( #3958 )
2022-04-27 15:28:26 +08:00
338db32d21
[Feature] Try upload report to s3 ( #3891 )
...
refactor CI report and log
2022-04-22 16:02:12 +08:00
ab50eb9b4e
Add unbuffered to distributed test ( #3732 )
...
Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com >
2022-02-15 10:23:45 +08:00
dc78e11c3f
[Dataset] Adapter to convert a dataset for link prediction task ( #3699 )
...
* add ut
* add doc link
* install dep
* add
* Revert "install dep"
This reverts commit e574a8377144749056c6849b655004df2771e179.
* add
* merge fix
* rm files
* fix
* fix
* fix
* fix
* fix typo
* fix tf
* fix
* fix
* fix
* fix
* fix
* fix dependency
* fix test
* fix
* fix
* add doc
* fix
* fix
* fix test
* fix test
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-02-11 16:59:14 +08:00
5152a87982
[Data] Utility function and class for converting a dataset for node prediction ( #3695 )
...
* add ut
* add doc link
* install dep
* fix ci
* fix ut; more comments
* remove deprecated attributes in rdf datasets; fix label feature name
* address comments
* fix ut for other frameworks
2022-01-30 14:31:37 +08:00
e4cb4a3779
[Fix] reduce error msg, refine fetch logic of available ports ( #3658 )
...
* [Fix] reduce error msg, refine fetch logic of available ports
* un-initialize client before sending shutdown request
* fix import error
* print connect failure log only in debug mode
* enable DMLC_LOG_DEBUG=1 in CI
2022-01-19 09:41:58 +08:00
95c0ff6302
[Feature] add CSVDataset to load data from csv files ( #3547 )
...
* [Feature] add CSVDataset to load data from csv files
* add CSVDataset class file
* install pyyaml when running unit tests
* install pandas for unit tests
* utilize pydantic for YAML config check
* generate yaml and csv files during test
* make more keys as optional
* remove/rename several keys in yaml config and more tets though looks a bit clumsy
* fix test failure on mxnet
* pass /path/to/dataset instead of yaml path
* code refinement
* code refine
* change several yaml field such as feat and graph_id
* merge graph generation logic
* refine code
* Refactored_first_version
* DGLCSVDataset works for single heterograph
* add more tests
* fix test failure in mxnet
* add docstring
* use list comprehension for dict
* fix version in YAML
* refine data length assert
* use dict.pop directly
* remove ambiguous variable names
* refine graph id missing logic
* refine graph create call
* separate node/edge/graph data parser
* remove separator in DefaultDataParser
* refine validation error log for yaml field
* minor check
* refine code via dict.get()
* move load_from_csv into Node/Edge/GraphData
* move _parse_node/edge/graph_data into Node/Edge/GraphData
* refine id-related fields check
* check duplicate ntypes/etypes when load yaml
* refine docstring
2022-01-10 17:21:57 +08:00
0b3a6216f5
[Test] Enable kvstore test ( #3079 )
...
* try enable kvstore test
* fix
* fix
* seperate out kvstore test
* add comment
2021-07-02 14:54:38 +08:00
9664cdffd3
[Build] Make nccl optional ( #3056 )
...
* fix
* remove nvidiasmi
* fix
* fix docs
* fix
* fix
* 1
* fix
* remove
* skip deprecated kernel
* fix
* Revert "skip deprecated kernel"
This reverts commit c5ceb7f60dbbaf065b81cc3680757fd611d90ad3.
* fix
2021-06-27 22:36:01 +08:00
ff519f98c3
[API] Standardize Subgraph APIs ( #2929 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Fix subgraph tests
* Capture stdout for distributed test
* Capture stdout for distributed test
* Update
* Update
* Update
* Update subgraph.cc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-06-21 19:53:37 +08:00
b1628f2398
[Tutorial] Distributed node classification. ( #2969 )
...
* add init version.
* fix build.
* fix format.
* fix.
* fix.
* fix format.
* update README.
* avoid running CI on distributed training tutorials.
* Update tutorials/dist/1_node_classification.py
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
* fix.
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2021-06-04 21:50:16 +08:00
6a56562a7c
[CI] Use k8s cluster ( #2957 )
...
* add
* fix
* set default
* fix
* try master
* try fix
* try
* fix
* 111
* fix
* fix
* update
* ccc
* try
* fix
* fix
* try new machine
* fix
* fix
* fix
* Revert "fix"
This reverts commit e716d66b046f92fe7ae368947a51a036a7a3188a.
* try
* more parrallel
* use k8s for all
* fix name
* try not specify instance type
* ci
* use one yaml
* Revert "use one yaml"
This reverts commit 717d8d852be39fbf2e2e45f9f224aa97907c372c.
* add timeout
* fix permission
* mount efs
* print
* fix pvc
* fix
* restrict num of gpu instances
* check
* fix
* fix
2021-06-04 18:31:11 +08:00
c018436588
[Example] add latent dirichlet allocation ( #2883 )
...
* add lda model
* tweak latent dirichlet allocation
* Update README.md
* Update README.md
* update example index
* update header
* minor tweak
* add example test
* update doc
* Update README.md
* Update README.md
* add partial_fit for free
* Update examples/pytorch/lda/lda_model.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
* Update examples/pytorch/lda/example_20newsgroups.py
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
* Update lda_model.py
* bugfix torch Gamma uses rate parameter
Co-authored-by: Yifei Ma <yifeim@amazon.com >
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-05-17 20:25:36 +08:00
8a07ab7737
[Doc] Tutorials re-organization ( #2683 )
...
* reorg
* change titles
* rm some stale API doc; minor fix
* fix docs
* add warning
* rm new-tutorial run in ci
* lint
2021-02-20 17:42:45 +08:00
117dd25204
[Doc] Fix inconsistencies and GPU code ( #2642 )
...
* fix inconsistencies and GPu
* bug fixes
* fix
* trigger new tutorials
2021-02-08 16:33:39 +08:00
9a7235faf2
[Performance] Use allocator from PyTorch if possible ( #2328 )
...
* first commit
* some thoughts
* move around
* more commit
* more fixes
* now it uses torch allocator
* fix symbol export error
* fix
* fixes
* test fix
* add script
* building separate library per version
* fix for vs2019
* more fixes
* fix on windows build
* update jenkinsfile
* auto copy built dlls for windows
* lint and installation guide update
* fix
* specify conda environment
* set environment for ci
* fix
* fix
* fix
* fix again
* revert
* fix cmake
* fix
* switch to using python interpreter path
* remove scripts
* debug
* oops sorry
* Update index.rst
* Update index.rst
* copies automatically, no need for this
* do not print message if library not found
* tiny fixes
* debug on nightly
* replace add_compile_definitions to make CMake 3.5 happy
* fix linking to wrong lib for multiple pytorch envs
* changed building strategy
* fix nightly
* fix windows
* fix windows again
* setup bugfix
* address comments
* change README
2020-12-25 13:57:51 +08:00
09ec6020ac
[Example] rm gcn spmv ( #2044 )
...
* rm gcn spmv
* remove useless test
* remove useless test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-08-18 17:48:38 +08:00
444becf00a
[Misc] Move many deprecated codes to the deprecate folder ( #1893 )
...
* some file movements
* move some codes to deprecated
* more deprecation
* lint
* remove useless test
2020-07-30 18:17:33 +08:00
3e2f94ed20
[Distributed] Refactor distributed training to new DGLGraph. ( #1874 )
...
* fix tests in partition.
* fix DistGraph.
* fix without shared memory.
* fix sampling.
* enable distributed test.
* fix tests.
* fix a bug in shared-mem heterograph.
* print better error messages.
* fix.
* don't specify formats.
* fix.
* fix
* small fix.
2020-07-28 22:07:24 -07:00
44089c8b4d
[Refactor][Graph] Merge DGLGraph and DGLHeteroGraph ( #1862 )
...
* Merge
* [Graph][CUDA] Graph on GPU and many refactoring (#1791 )
* change edge_ids behavior and C++ impl
* fix unittests; remove utils.Index in edge_id
* pass mx and th tests
* pass tf test
* add aten::Scatter_
* Add nonzero; impl CSRGetDataAndIndices/CSRSliceMatrix
* CSRGetData and CSRGetDataAndIndices passed tests
* CSRSliceMatrix basic tests
* fix bug in empty slice
* CUDA CSRHasDuplicate
* has_node; has_edge_between
* predecessors, successors
* deprecate send/recv; fix send_and_recv
* deprecate send/recv; fix send_and_recv
* in_edges; out_edges; all_edges; apply_edges
* in deg/out deg
* subgraph/edge_subgraph
* adj
* in_subgraph/out_subgraph
* sample neighbors
* set/get_n/e_repr
* wip: working on refactoring all idtypes
* pass ndata/edata tests on gpu
* fix
* stash
* workaround nonzero issue
* stash
* nx conversion
* test_hetero_basics except update routines
* test_update_routines
* test_hetero_basics for pytorch
* more fixes
* WIP: flatten graph
* wip: flatten
* test_flatten
* test_to_device
* fix bug in to_homo
* fix bug in CSRSliceMatrix
* pass subgraph test
* fix send_and_recv
* fix filter
* test_heterograph
* passed all pytorch tests
* fix mx unittest
* fix pytorch test_nn
* fix all unittests for PyTorch
* passed all mxnet tests
* lint
* fix tf nn test
* pass all tf tests
* lint
* lint
* change deprecation
* try fix compile
* lint
* update METIDS
* fix utest
* fix
* fix utests
* try debug
* revert
* small fix
* fix utests
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* trigger
* +1s
* [kernel] Use heterograph index instead of unitgraph index (#1813 )
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* trigger
* +1s
* [Graph] Mutation for Heterograph (#1818 )
* mutation add_nodes and add_edges
* Add support for remove_edges, remove_nodes, add_selfloop, remove_selfloop
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
* upd
* upd
* upd
* fix
* [Transfom] Mutable transform (#1833 )
* add nodesy
* All three
* Fix
* lint
* Add some test case
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* fix
* triger
* Fix
* fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
* [Graph] Migrate Batch & Readout module to heterograph (#1836 )
* dgl.batch
* unbatch
* fix to device
* reduce readout; segment reduce
* change batch_num_nodes|edges to function
* reduce readout/ softmax
* broadcast
* topk
* fix
* fix tf and mx
* fix some ci
* fix batch but unbatch differently
* new checkk
* upd
* upd
* upd
* idtype behavior; code reorg
* idtype behavior; code reorg
* wip: test_basics
* pass test_basics
* WIP: from nx/ to nx
* missing files
* upd
* pass test_basics:test_nx_conversion
* Fix test
* Fix inplace update
* WIP: fixing tests
* upd
* pass test_transform cpu
* pass gpu test_transform
* pass test_batched_graph
* GPU graph auto cast to int32
* missing file
* stash
* WIP: rgcn-hetero
* Fix two datasety
* upd
* weird
* Fix capsuley
* fuck you
* fuck matthias
* Fix dgmg
* fix bug in block degrees; pass rgcn-hetero
* rgcn
* gat and diffpool fix
also fix ppi and tu dataset
* Tree LSTM
* pointcloud
* rrn; wip: sgc
* resolve conflicts
* upd
* sgc and reddit dataset
* upd
* Fix deepwalk, gindt and gcn
* fix datasets and sign
* optimization
* optimization
* upd
* upd
* Fix GIN
* fix bug in add_nodes add_edges; tagcn
* adaptive sampling and gcmc
* upd
* upd
* fix geometric
* fix
* metapath2vec
* fix agnn
* fix pickling problem of block
* fix utests
* miss file
* linegraph
* upd
* upd
* upd
* graphsage
* stgcn_wave
* fix hgt
* on unittests
* Fix transformer
* Fix HAN
* passed pytorch unittests
* lint
* fix
* Fix cluster gcn
* cluster-gcn is ready
* on fixing block related codes
* 2nd order derivative
* Revert "2nd order derivative"
This reverts commit 523bf6c249bee61b51b1ad1babf42aad4167f206.
* passed torch utests again
* fix all mxnet unittests
* delete some useless tests
* pass all tf cpu tests
* disable
* disable distributed unittest
* fix
* fix
* lint
* fix
* fix
* fix script
* fix tutorial
* fix apply edges bug
* fix 2 basics
* fix tutorial
Co-authored-by: yzh119 <expye@outlook.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-42.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-1-5.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-68-185.ec2.internal >
2020-07-28 14:30:41 +08:00
168a88e570
[Sampling] NodeDataLoader for node classification ( #1635 )
...
* neighbor sampler data loader first commit
* more commit
* nodedataloader
* fix
* update RGCN example
* update OGB
* fixes
* fix minibatch RGCN crashing with self loop
* reverting gatconv test code
* fix
* change to new solution that doesn't require tf dataloader
* fix
* lint
* fix
* fixes
* change doc
* fix docstring
* docstring fixes
* return seeds and input nodes from data loader
* fixes
* fix test
* fix windows build problem
* add pytorch wrapper
* fixes
* add pytorch wrapper
* add unit test
* add -1 support to sample_neighbors & fix docstrings
* docstring fix
* lint
* add minibatch rgcn evaluations
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Tong He <hetong007@gmail.com >
2020-07-02 16:31:05 +08:00
da8632cafe
fix parallel msbuild ( #1717 )
2020-06-28 17:50:37 +08:00
6d04885381
[Backend] Turn to official dlpack for Tensorflow ( #1511 )
...
* Turn to official dlpack
* fix
* fix
2020-05-11 21:16:25 +08:00
16561a2e6b
[Test] Add tests for TensorFlow ( #1501 )
...
* add test.
* move test code.
* remvoe unnecessary test.
* fix.
* turn on tests for TF.
* Revert "move test code."
This reverts commit e7b4f36395b2121a7be030bd4364a704d0e357bf.
* fix.
* fix.
* skip test for tensorflow.
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-05-06 00:33:53 -07:00
e317f715e0
deprecate DGL-KE. ( #1432 )
...
* deprecate DGL-KE.
* Update README.md
* remove from CI.
* remove tests.
Co-authored-by: Zheng <dzzhen@3c22fb0ed81b.ant.amazon.com >
2020-04-09 01:08:33 -07:00
6111ea4665
[Model] RGCN mini-batch training; Better bipartite graph support ( #1337 )
...
* change the model to use node embedding only
* minibatch training
* add readme
* small fix
* backward-compatible
* backward compatible
* modify to_block; rgcn changed
* fix
* fix transform
* fix bug in unittest script
* docstring
* fix lint
* add tests
* address comments; fix offline eval
* gitignore
2020-03-16 18:58:06 +08:00
032a08cc32
[Test] Fix test script ( #1257 )
...
* fix script
* t
* fix weird bugs
* fix
* fix
2020-02-13 18:09:15 +08:00
ffe5898317
[Optimization][KG] Several optimizations on DGL-KG ( #1233 )
...
* Several optimizations on DGL-KG:
1. Sorted positive edges for sampling which can reduce random
memory access during positive sampling
2. Asynchronous node embedding update
3. Balanced Relation Partition that gives balanced number of
edges in each partition. When there is no cross partition
relation, relation embedding can be pin into GPU memory
4. tunable neg_sample_size instead of fixed neg_sample_size
* Fix test
* Fix test and eval.py
* Now TransR is OK
* Fix single GPU with mix_cpu_gpu
* Add app tests
* Fix test script
* fix mxnet
* Fix sample
* Add docstrings
* Fix
* Default value for num_workers
* Upd
* upd
2020-02-09 13:01:00 -08:00
eeeb52f464
[Feature] Preference to COO for "hypersparse" unit graphs & graph compaction ( #1238 )
...
* unit graph that prefers coo queries
* auto detect coo preference
* forgot some functions
* disable lint on detect_prefer_coo
* reorg
* change comment
* lint
* fix
* move array_utils.h to src
* compact graph impl
* fix redundant copying in idhashmap
* docstring
* moving preference detection to C
* lint
* fix unit test & address comments
* hypersparse autorestrict
* docstring & fix
* revert copyto and asnumbits
* fix stupid bug
* lint
* leave a TODO for sorted COO
* fixing same node type mapping to different id in different graphs
* addresses comments
* made induced nodes a feautre column
* lint?
2020-02-04 22:39:28 +08:00
a00636a02b
[Tensorflow] Several nn & example ( #1191 )
...
* several nn example
* appnp
* fix lint
* lint
* add dgi
* fix
* fix
* fix
* fff
* docs
* 111
* fix
* change init
* change result
* tiaocan+1
* fix
* fix lint
* fix
* fix
2020-01-19 22:48:40 +08:00
Rhett Ying