bd311aa68d
[doc] fix invalid link in user guide ( #4468 )
2022-08-25 23:16:44 +08:00
7a41c126f6
[Doc] Change random.py to random_partition.py in guide on distributed partition pipeline ( #4438 )
...
* Update distributed-preprocessing.rst
* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal >
2022-08-22 18:12:55 +08:00
bc2fef9c32
[Doc] Misc Fix for User Guide 7.1 Data Preprocessing ( #4433 )
...
* Update
* rollback for partition_algo/random.py
Co-authored-by: Ubuntu <ubuntu@ip-172-31-20-21.us-west-2.compute.internal >
2022-08-18 20:27:40 +08:00
3bcb268a36
[Doc] Update distributed chapter according to new pipeline ( #4275 )
...
* dist index chapter
* preproc chapter
* rst
* tools page
* partition chapter
* rst
* hetero chapter
* 7.1 step1
* add parmetis back
* changed based on feedback
* address comments
2022-08-17 13:30:04 +08:00
86c81b4e92
[Feature] Add CUDA Weighted Neighborhood Sampling ( #4064 )
...
* add weighted sampling without replacement (A-Chao)
* improve Algorithm A-Chao with block-wise prefix sum
* correctly fill out_idxs
* implement weighted sampling with replacement
* small fix
* merge host-side code of weighted/uniform sampling
* enable unit tests for cuda weighted sampling
* move thrust/cub wrapper to the cmake file
* update docs accordingly
* fix linting
* fix linting
* fix unit test
* Bump external CUB/Thrust versions
* Fix code style and update description of algorithm design
* [Feature] GPU support weighted graph neighbor sampling
commit by pengqirong(OPPO)
* merge pengqirong's implementation
* revert the change to cub and thrust
* fix linting
* use DeviceSegmentedSort for better performance
* add more comments
* add necessary notes
* add necessary notes
* resolve some comments
* define THRUST_CUB_WRAPPED_NAMESPACE
* fix doc
Co-authored-by: 彭齐荣 <657017034@qq.com >
2022-07-29 11:08:48 +08:00
7e6a6b4ac0
[Feature] Add CUDA Weighted Randomwalk Sampling ( #4243 )
...
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* [Feature] Add CUDA Weighted Randomwalk Sampling
* fix empty prob array && enable non-uniform for restart && enable unit tests
* update doc and guide for randomwalk and pinsage
* update comments
Co-authored-by: zhenliangqiu <ubuntu@ip-172-31-24-245.ap-southeast-1.compute.internal >
Co-authored-by: xiny <xiny@nvidia.com >
2022-07-26 13:42:39 +08:00
05d9d4969b
Update ( #4277 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-142.us-west-2.compute.internal >
2022-07-21 13:27:42 +08:00
92063d88f0
fix indent in minibatch-custom-sampler.rst ( #4063 )
...
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
2022-06-03 01:00:31 +08:00
d9c25521bc
[Data] AsGraphPredDataset ( #4073 )
...
* Update
* CI
* Update
* Update
* Fix
* Fix
2022-06-02 16:18:48 +08:00
ae3316c846
[Doc] Update doc of dataloading package. ( #3886 )
...
* wip: dataloading doc
* update dataloading package doc and many others
* lint
2022-03-26 21:59:02 +08:00
e963256882
[Doc] Update message-api.rst ( #3844 )
...
fix #3812
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-03-25 23:22:20 +08:00
7d4160869c
[Bug] Fix multiple issues in distributed multi-GPU GraphSAGE example ( #3870 )
...
* fix distributed multi-GPU example device
* try Join
* update version requirement in README
* use model.join
* fix docs
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2022-03-25 22:04:02 +08:00
8425c936c2
update doc for gpu&uva sampling ( #3787 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2022-02-28 21:17:32 +08:00
d41d07d0f6
[Doc and bugfix] Add docs and user guide and update tutorial for sampling pipeline ( #3774 )
...
* huuuuge update
* remove
* lint
* lint
* fix
* what happened to nccl
* update multi-gpu unsupervised graphsage example
* replace most of the dgl.mp.process with torch.mp.spawn
* update if condition for use_uva case
* update user guide
* address comments
* incorporating suggestions from @jermainewang
* oops
* fix tutorial to pass CI
* oops
* fix again
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-02-28 02:21:51 +08:00
6d9433b04f
[Transform] [Doc] Rename transform to transforms and update doc ( #3765 )
...
* Update
* Update
* Update
* Fix
* Update
* Update
* Update
* Fix
2022-02-24 21:28:15 +08:00
8db2dd3355
[Dataset][Fix] Allow non-numeric values and some fix on doc ( #3757 )
...
* [Fix] be able to parse ids if numeric and non-numeric values are used together
* add required package info and cache note into docstring
* duplicate node id is not allowed
2022-02-23 17:26:12 +08:00
6106a99d0a
refine
2022-02-17 02:00:10 +00:00
3b49370d3e
refine doc
2022-02-17 00:52:20 +00:00
8d861d9443
refine especiaaly for data parser
2022-02-17 00:52:20 +00:00
e0f054fb27
[Doc] add doc for DGLCSVDataset
2022-02-17 00:52:16 +00:00
fcd8ed9acc
[Feature] Launch Long Live Servers and Multiple Client Groups ( #3688 )
...
* enable to launch multiple client groups sequentially
* launch simultaneously is enabled
* refine docstring
* revert unnecessary change
* [DOC] add doc for long live server
* refine
* refine doc
* refine doc
2022-02-09 15:56:38 +08:00
738e8318fd
[Feature] CUDA UVA sampling for MultiLayerNeighborSampler ( #3674 )
...
* implement pin_memory/unpin_memory/is_pinned for dgl.graph
* update python docstring
* update c++ docstring
* add test
* fix the broken UnifiedTensor
* XPU_SWITCH for kDLCPUPinned
* a rough version ready for testing
* eliminate extra context parameter for pin/unpin
* update train_sampling
* fix linting
* fix typo
* multi-gpu uva sampling case
* disable new format materialization for pinned graphs
* update python doc for pin_memory_
* fix unit test
* UVA sampling for link prediction
* dispatch most csr ops
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update doc
* update examples
* change unitgraph and heterograph's PinMemory to in-place
* update examples for multi-gpu uva sampling
* update doc
* fix linting
* fix cpu build
* fix is_pinned for DistGraph
* fix is_pinned for DistGraph
* update graphsage unsupervised example
* update doc for gpu sampling
* update some check for sampling device switching
* fix linting
* adapt for new dataloader
* fix linting
* fix
* fix some name issue
* adjust device check
* add unit test for uva sampling & fix some zero_copy bug
* fix linting
* update num_threads in graphsage examples
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2022-02-09 14:06:17 +08:00
701b4fccc2
[Sampling] New sampling pipeline plus asynchronous prefetching ( #3665 )
...
* initial update
* more
* more
* multi-gpu example
* cluster gcn, finalize homogeneous
* more explanation
* fix
* bunch of fixes
* fix
* RGAT example and more fixes
* shadow-gnn sampler and some changes in unit test
* fix
* wth
* more fixes
* remove shadow+node/edge dataloader tests for possible ux changes
* lints
* add legacy dataloading import just in case
* fix
* update pylint for f-strings
* fix
* lint
* lint
* lint again
* cherry-picking commit fa9f494
* oops
* fix
* add sample_neighbors in dist_graph
* fix
* lint
* fix
* fix
* fix
* fix tutorial
* fix
* fix
* fix
* fix warning
* remove debug
* add get_foo_storage apis
* lint
2022-01-30 16:13:00 +08:00
069dc45d50
[Docs] Fix sampling tutorials ( #3683 )
...
Fix #3626
2022-01-26 18:05:41 +08:00
8f5baa15e4
[Doc] Fix spelling, references and update info on building docs ( #3682 )
...
* Fix ref to message-passing guide
* Fix pygments and spacing
* Update build documentation steps in README.md
* Use links
* Adjust parameters in SAGEConv docstring in same order as init
* Fix spelling error
* Change doc link
2022-01-25 17:22:18 +08:00
3aef467733
[Doc] User Guide for Using Edge Weights in Training ( #3509 )
...
* Update
* Update
* CI
2021-11-18 11:58:15 +08:00
863c249568
Fix User Guide EN and CN typo ( #3495 )
...
* Update graph-heterogeneous.rst
`tensor([0, 1, 2, 0, 1, 2])` should be output instead of code
* Update message-api.rst
`updata_all_example()` should be `update_all_example()`
* Update message-efficient.rst
`cat_feat` need to concatenate with `dim=1` for the # edge features to match # edges
* Update nn-construction.rst
all `max_pool` in the aggregator type of `SAGEConv` should be `pool` instead
* Update graph-heterogeneous.rst
`tensor([0, 1, 2, 0, 1, 2])` should be output instead of code
* Update message-api.rst
`updata_all_example()` should be `update_all_example()`
* Update message-efficient.rst
`cat_feat` need to concatenate with `dim=1` for the # edge features to match # edges
* Update nn-construction.rst
all `max_pool` in the aggregator type of `SAGEConv` should be `pool` instead
* Update nn-forward.rst
all `max_pool` in the aggregator type of `SAGEConv` should be `pool` instead
* Update nn-forward.rst
all `max_pool` in the aggregator type of `SAGEConv` should be `pool` instead
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-11-14 20:37:39 +08:00
31772b142e
fixed typos in docs and docstring ( #3231 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-08-16 13:58:45 +08:00
3f6f694159
[Bug fix] Various fix from bug bash ( #3133 )
...
* Update
* Update
* Update dependencies
* Update
* Update
* Fix ogbn-products gat
* Update
* Update
* Reformat
* Fix typo in node2vec_random_walk
* Specify file encoding
* Working for 6.7
* Update
* Fix subgraph
* Fix doc for sample_neighbors_biased
* Fix hyperlink
* Add example for udf cross reducer
* Fix
* Add example for slice_batch
* Replace dgl.bipartite
* Fix GATConv
* Fix math rendering
* Fix doc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-156.us-west-2.compute.internal >
2021-07-15 09:17:15 +08:00
d7390763f0
[Distributed] Deprecate old DistEmbedding impl, use synchronized embedding impl ( #3111 )
...
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* update doc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-07-14 00:15:53 +08:00
186ef59283
[Feature] apply dgl.reorder() onto several node classification datase… ( #3102 )
...
* [Feature] apply dgl.reorder() onto several node classification datasets in DGL
* rebase on latest dgl.reorder_graph()
2021-07-13 08:52:17 +08:00
ced0e443c2
[Doc] Add user guide for GPU-based sampling ( #3070 )
...
* add user guide for gpu sampling
* Update minibatch-gpu-sampling.rst
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-07-05 12:07:43 +08:00
81456af8c5
[Doc] fix inconsistent variable names ( #3065 )
...
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-06-28 13:30:08 +08:00
ff519f98c3
[API] Standardize Subgraph APIs ( #2929 )
...
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Fix subgraph tests
* Capture stdout for distributed test
* Capture stdout for distributed test
* Update
* Update
* Update
* Update subgraph.cc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-06-21 19:53:37 +08:00
895d6fff72
[Doc] changing of output dimension ( #2981 )
...
Co-authored-by: zhjwy9343 <6593865@qq.com >
2021-06-05 22:12:41 +08:00
6383e64937
[Doc] correct the matrix multiplication symbols ( #2961 )
...
* correct the matrix multiplication symbols
* modified dimensionality of linear
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-03 08:51:23 +08:00
2ae190d690
[Doc] fix shape mismatch problem in user guide ( #2938 )
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-02 12:18:10 +08:00
a90296aa9c
fix. ( #2907 )
...
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
2021-05-13 16:55:31 +08:00
5498cf051e
[Doc]Fix examples on negative graph sampler ( #2863 )
...
* Fix examples on negative graph sampler
In heterogeneous graph, the old example may build a negative graph without edges. In dgl.dataloading.dataloader line637, it shows the key of neg_edges dict should be item in g.canonical_etypes.
* poke ci
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-05-06 21:34:31 +08:00
975eb8fc5d
[Distributed] Distributed node embedding and sparse optimizer ( #2733 )
...
* Draft for sparse emb
* add some notes
* Fix
* Add sparse optim for dist pytorch
* Update test
* Fix
* upd
* upd
* Fix
* Fix
* Fix bug
* add transductive exmpale
* Fix example
* Some fix
* Upd
* Fix lint
* lint
* lint
* lint
* upd
* Fix lint
* lint
* upd
* remove dead import
* update
* lint
* update unitest
* update example
* Add adam optimizer
* Add unitest and update data
* upd
* upd
* upd
* Fix docstring and fix some bug in example code
* Update rgcn readme
Co-authored-by: Ubuntu <ubuntu@ip-172-31-57-25.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-24-210.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
2021-05-03 08:37:40 +08:00
45ec21b033
[Distributed] fix docs for distributed training. ( #2840 )
...
* update distributed training doc.
* explain data split.
* fix message passing.
* id mapping.
* fix.
* test data reshuffling.
* fix a bug.
* fix test.
* Revert "fix test."
This reverts commit 907f1dd88994894db94ffcd77b5995279f76b489.
* Revert "fix a bug."
This reverts commit ff0a4d893bbe4911a75ecb04bcbd9bc65315166b.
* Revert "test data reshuffling."
This reverts commit 99bb2f6405de67fc73652220b02e7184461c3e5e.
Co-authored-by: Zheng <dzzhen@3c22fba32af5.ant.amazon.com >
2021-04-29 14:38:41 +08:00
e36c5db614
[Distributed] Simplify distributed API ( #2775 )
...
* remove num_workers.
* remove num_workers.
* remove num_workers.
* remove num-servers.
* update error message.
* update docstring.
* fix docs.
* fix tests.
* fix test.
* fix.
* print messages in test.
* fix.
* fix test.
* fix.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal >
2021-03-29 19:19:23 -07:00
1789972c4b
[Bugfix] add default value for BlockSampler ( #2771 )
...
* [Bugfix] add default value for BlockSampler
* [Doc] modify user_guide description about MultiLayerDropoutSample
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-03-22 14:02:27 +08:00
17f863569e
fix ( #2744 )
2021-03-21 22:38:55 -07:00
99751d4960
[Doc] Rename block to message flow graph ( #2702 )
...
* rename block to mfg
* revert
* rename
2021-02-25 18:26:19 +08:00
48a1794f85
[Doc] Fix the user guide for distributed partitioning. ( #2684 )
...
* fix doc.
* explain the schema file.
* fix.
2021-02-22 18:59:15 -08:00
8a07ab7737
[Doc] Tutorials re-organization ( #2683 )
...
* reorg
* change titles
* rm some stale API doc; minor fix
* fix docs
* add warning
* rm new-tutorial run in ci
* lint
2021-02-20 17:42:45 +08:00
0fc649523b
[Doc] Distributed heterogeneous graph training ( #2654 )
...
* update DistGraph docstrings.
* add user guide.
* add doc string.
* fix.
* fix.
* fix.
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-02-19 10:48:05 -08:00
e4ff484494
[Distributed] Distributed METIS partition ( #2576 )
...
* add convert.
* fix.
* add write_mag.
* fix convert_partition.py
* write data.
* use pyarrow to read.
* update write_mag.py
* fix convert_partition.py.
* load node/edge features when necessary.
* reshuffle nodes.
* write mag correctly.
* fix a bug: inner nodes in a partition might be empty.
* fix bugs.
* add verify code.
* insert reverse edges.
* fix a bug.
* add get node/edge data.
* add instructions.
* remove unnecessary argument.
* update distributed preprocessing.
* fix readme.
* fix.
* fix.
* fix.
* fix readme.
* fix doc.
* fix.
* update readme
* update doc.
* update readme.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-132.us-west-1.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal >
2021-02-08 17:19:18 -08:00
8ccb8206bf
[Doc] Update docs to motivate the use of set_batch_num_nodes and set_batch_num_edges ( #2630 )
...
* fix doc
* update docstring for transformation on batch graph
* update cn guide
* fix format
* update rst
2021-02-08 14:43:24 +08:00
Rhett Ying