* enable uva for pinsage sampler
* unit test
* modify some checks on the python side
* remove legacy random walk code
* update unit test
* update unit test
* fix unit test
* adjust checks
* move some checks to c++
* move max_nodes check to cuda kernel
* fix ci for tf
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
* Feat: support randomwalk and SelectPinSageNeighbors on GPU
Port RandomWalk algorithm on GPU,
and port SelectPinSageNeighbors on GPU.
* Feat: support GPU on python APIs
* Feat: remove perf print information in FrequenchHashmap
* Fix: modified the code format
Modified the code format as task_lint.sh suggested
* Feat: let test script support PinSAGESampler on GPU
Let test script support PinSAGESampler on GPU,
minor of "restart_prob".
* Minor
* Minor
* Minor
* Refactor: use the atomic operations from the array module
* Minor: change the long lines
* Refactor: modified the get_node_types for gpu
* Feat: update the contributor date
* Perf: remove unnecessary stream sync
* Feat: support other random walk
But the non-uniform choice is still not supported.
* Fix: add CUDA switch for random walk
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* [Feature] Exclude edges in sample_neighbors
Extending sample_neighbors and sample_frontier
API to support exclude_edges parameter.
exclude_edges support tensor and dict data
Feature enable excluding certain edges
during neighborhood sampling
Exclude_edges contains EID's of edges
which will be excluded
during neighbor picking for seed nodes.
Added test case for heterograph and homograph
RFC issue id: 2944
* compatibility
* fix
* fix
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* Basic imple of heterogeneous on homogenenous sampling
* make pass
* Pass C++ test
* Add python test code
* lint
* lint
* Add MultiLayerEtypeNeighborSampler
* Add unitest for single machine dataloader
* Add dist dataloader test for edge type sampler
* Fix lint
* fix
* support for per etype sample
* Fix some bug and enable distributed training with per edge sample
* fix
* Now distributed training works
* turn off some mxnet
* turn off mxnet for some dist test
* fix
* upd
* upd according to the comments
* Fix
* Fix test and now distributed works.
* upd
* upd
* Fix
* Fix bug
* remove dead code.
* upd
* Fix
* upd
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* Start on uniform GPU sampling
* Save more work
* Get cu file compiling
* Update sampling
* More changes
* Get GPU sampling for uniform probabilities solved
* Fix batch tensor migration
* Fix
* update kernels
* expand blocking
* Undo testing change
* Cut down on sampling overhead
* Fix replacement
* Update unit tests
* Add option to gpu sample in graphsage
* Copy only csc to gpu
* Add ogbn support
* Fix linting
* Remove nvtx from sample
* Improve documentation and error checking
* Expand documentation
* Update assert checking
* delete extra space
* Use standard dataloader when dataset is a dictionary
* ogb -> ogbn
* Fix edge selection determinism
* Fix typos
* Remove nvtx
* Add comment for self.fanout_arrays and assert
* Fix linting
* Migrate to scalarbatcher
* Fix indentation
* Fix batcher
* Fix indexing
* Only use databatcher for GPU
* Convert to DGL NDArray to PyTorch Tensor
* Add optimization for PyTorch's F.tensor() for list of GPU tensors
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* improve performance of sample_neighbors
* some more improve
* test script
* benchmarks
* multi process
* update more tests
* WIP
* adding two API for state saving
* add create from state
* upd test
* missing file
* wip: pickle/unpickle
* more c apis
* find the problem of empty data array
* add null array; pickling speed is bad
* still bad perf
* still bad perf
* wip
* fix the pickle speed test; now everything looks good
* minor fix
* bugfix
* some lint fix
* address comments
* more fix
* fix lint
* add utest for random.choice
* add utest for dgl.rand_graph
* fix cpp utests
* try fix ci
* fix bug in TF backend
* upd choice docstring
* address comments
* upd
* try fix compile
* add comment
* [WIP] PinSAGE operators
* moved the edge remapping mess into C
* some docstrings
* lint
* lint x2
* lint x3
* skip gpu test on topk
* extend pinsage to any metapath
* lint x4
* addresses #1265
* add always_preserve (fixes#1266) and fix a silly bug
* disable gpu test on compaction
* lint
* fix a horrible bug and add more tests
* lint
* addresses comments
* lint
* bugfix
* addresses comments
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
* trying to refactor IndexSelect
* partial implementation
* add index select and assign for floats as well
* move to random choice source
* more updates
* fixes
* fixes
* more fixes
* adding python impl
* fixes
* unit test
* lint
* lint x2
* lint x3
* update metapath2vec
* debugging performance
* still debugging for performance
* tuning
* switching to succvec
* redo
* revert non-uniform sampler to use vector
* still not fast
* why does this crash with OpenMP???
* because there was a data race!!!
* add documentations and remove assign op
* lint
* lint x2
* lol what have i done
* lint x3
* fix and disable gpu testing
* bugfix
* generic random walk
* reorg the random walk source code
* Update randomwalks.h
* Update randomwalks_cpu.cc
* rename file
* move internal function to anonymous ns
* reorg & docstrings
* constant restart probability
* docstring fix
* more commit
* random walk with restart, tested
* some fixes
* switch to using NDArray for choice
* massive fix & docstring
* lint x?
* lint x??
* fix
* export symbols
* skip gpu test
* addresses comments
* replaces another VecToIdArray
* add randomwalks.h to include
* replace void * with template
Xin Yao