* Start on uniform GPU sampling
* Save more work
* Get cu file compiling
* Update sampling
* More changes
* Get GPU sampling for uniform probabilities solved
* Fix batch tensor migration
* Fix
* update kernels
* expand blocking
* Undo testing change
* Cut down on sampling overhead
* Fix replacement
* Update unit tests
* Add option to gpu sample in graphsage
* Copy only csc to gpu
* Add ogbn support
* Fix linting
* Remove nvtx from sample
* Improve documentation and error checking
* Expand documentation
* Update assert checking
* delete extra space
* Use standard dataloader when dataset is a dictionary
* ogb -> ogbn
* Fix edge selection determinism
* Fix typos
* Remove nvtx
* Add comment for self.fanout_arrays and assert
* Fix linting
* Migrate to scalarbatcher
* Fix indentation
* Fix batcher
* Fix indexing
* Only use databatcher for GPU
* Convert to DGL NDArray to PyTorch Tensor
* Add optimization for PyTorch's F.tensor() for list of GPU tensors
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* clean commit
* oops forgot the most important files
* use einsum
* copy feature from frontier to block
* Revert "copy feature from frontier to block"
This reverts commit 5224ec963eb6a3ef1b6ab74d8ecbd44e4e42f285.
* temp fix
* unit test
* fix
* revert jtnn
* lint
* fix win64
* docstring fixes and doc indexing
* revert einsum in sparse bidecoder
* fix some examples
* lint
* fix due to some tediousness in remove_edges
* addresses comments
* fix
* more jtnn fixes
* fix
* slice dstdata from srcdata within nn module
* a bunch of fixes
* add comment
* fix gcmc layer
* repr for blocks
* fix
* fix context
* fix
* do not copy internal columns
* docstring
* initial version from distributed training.
This is copied from multiprocessing training.
* modify for distributed training.
* it's runnable now.
* measure time in neighbor sampling.
* simplify neighbor sampling.
* fix a bug in distributed neighbor sampling.
* allow single-machine training.
* fix a bug.
* fix a bug.
* fix openmp.
* make some improvement.
* fix.
* add prepare in the sampler.
* prepare nodeflow async.
* fix a bug.
* get id.
* simplify the code.
* improve.
* fix partition.py
* fix the example.
* add more features.
* fix the example.
* allow one partition
* use distributed kvstore.
* do g2l map manually.
* fix commandline.
* a temp script to save reddit.
* fix pull_handler.
* add pytorch version.
* estimate the time for copying data.
* delete unused code.
* fix a bug.
* print id.
* fix a bug
* fix a bug
* fix a bug.
* remove redundent code.
* revert modify in sampler.
* fix temp script.
* remove pytorch version.
* fix.
* distributed training with pytorch.
* add distributed graph store.
* fix.
* add metis_partition_assignment.
* fix a few bugs in distributed graph store.
* fix test.
* fix bugs in distributed graph store.
* fix tests.
* remove code of defining DistGraphStore.
* fix partition.
* fix example.
* update run.sh.
* only read necessary node data.
* batching data fetch of multiple NodeFlows.
* simplify gcn.
* remove unnecessary code.
* use the new copy_from_kvstore.
* update training script.
* print time in graphsage.
* make distributed training runnable.
* use val_nid.
* fix train_sampling.
* add distributed training.
* add run.sh
* add more timing.
* fix a bug.
* save graph metadata when partition.
* create ndata and edata in distributed graph store.
* add timing in minibatch training of GraphSage.
* use pytorch distributed.
* add checks.
* fix a bug in global vs. local ids.
* remove fast pull
* fix a compile error.
* update and add new APIs.
* implement more methods in DistGraphStore.
* update more APIs.
* rename it to DistGraph.
* rename to DistTensor
* remove some unnecessary API.
* remove unnecessary files.
* revert changes in sampler.
* Revert "simplify gcn."
This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.
* Revert "simplify neighbor sampling."
This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.
* Revert "measure time in neighbor sampling."
This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.
* Revert "add timing in minibatch training of GraphSage."
This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.
* Revert "fix train_sampling."
This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.
* fix lint.
* add comments and small update.
* add more comments.
* add more unit tests and fix bugs.
* check the existence of shared-mem graph index.
* use new partitioned graph storage.
* fix bugs.
* print error in fast pull.
* fix lint
* fix a compile error.
* save absolute path after partitioning.
* small fixes in the example
* Revert "[kvstore] support any data type for init_data() (#1465)"
This reverts commit 87b6997bf2.
* fix a bug.
* disable evaluation.
* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""
This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.
* support set and init data.
* support set and init data.
* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""
This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.
* fix bugs.
* fix unit test.
* move to dgl.distributed.
* fix lint.
* fix lint.
* remove local_nids.
* fix lint.
* fix test.
* remove train_dist.
* revert train_sampling.
* rename funcs.
* address comments.
* address comments.
Use NodeDataView/EdgeDataView to keep track of data.
* address comments.
* address comments.
* revert.
* save data with DGL serializer.
* use the right way of getting shape.
* fix lint.
* address comments.
* address comments.
* fix an error in mxnet.
* address comments.
* add edge_map.
* add more test and fix bugs.
Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
* control variate first commit
* bug fixes
* split to single and multi GPU
* update readme
* bugfix
* bugfix
* remove push
* bugfix on multi gpu
* update README
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Quan (Andy) Gan