* Added graph updating method and tests. resolves#3488
* removed spaces around named args
* customizing indices for graph's idtype and ctx
* changing torch ops to generic backend ops
* changing tensors to np arrays
* created dgl merge function and tests
* Changed per-graph edge updates to single ag update
* removed update method and tests
* reformat newlines & spaces
* concatenating in one-shot instead of iteratively
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [Feature] enable TP::Receiver wait for any numbers of senders
* fix random unit test failure
* avoid endless future wait
* fix unit test failure
* fix seg fault when finalize wait in receiver
* [Feature] refactor sender connect logic and remove unnecessary sleeps in unit tests
* fix lint
* release RPCContext resources before process exits
* [Debug] TPReceiver wait start log
* [Debug] add log in get port
* [Debug] add log
* [ReDebug] revert time sleep in unit tests
* [Debug] remove sleep for test_distri,test_mp
* [debug] add more log
* [debug] add listen_booted_ flag
* [debug] restore commented code for queue
* [debug] sleep more in rpc_client
* restore change in tests
* Revert "restore change in tests"
This reverts commit 41a18926d181ec2517069389bfc41de2cc949280.
* Revert "[debug] sleep more in rpc_client"
This reverts commit a908e758eabca0a6ce62eb2e59baea02a840ac67.
* Revert "[debug] restore commented code for queue"
This reverts commit d3f993b3746e6bb6e2cc2f90204dd7e9461c6301.
* Revert "[debug] add listen_booted_ flag"
This reverts commit 244b2167d94942ff2a0acec8823b974975e52580.
* Revert "[debug] add more log"
This reverts commit 4b78447b0a575a824821dc7e25cca2246e6e30e2.
* Revert "[Debug] remove sleep for test_distri,test_mp"
This reverts commit e1df1aadcc8b1c2a0013ed77322ac391a8807612.
* remove debug code
* revert unnecessary change
* revert unnecessary changes
* always reset RPCContext when get started and reset all data
* remove time.sleep in dist tests
* fix lint
* reset envs before each dist test
* reset env properly
* add time sleep when start each server
* sleep for a while when boot server
* replace wait_thread with callback
* fix lint
* add dglconnect handshake check
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* [Feature] add CSVDataset to load data from csv files
* add CSVDataset class file
* install pyyaml when running unit tests
* install pandas for unit tests
* utilize pydantic for YAML config check
* generate yaml and csv files during test
* make more keys as optional
* remove/rename several keys in yaml config and more tets though looks a bit clumsy
* fix test failure on mxnet
* pass /path/to/dataset instead of yaml path
* code refinement
* code refine
* change several yaml field such as feat and graph_id
* merge graph generation logic
* refine code
* Refactored_first_version
* DGLCSVDataset works for single heterograph
* add more tests
* fix test failure in mxnet
* add docstring
* use list comprehension for dict
* fix version in YAML
* refine data length assert
* use dict.pop directly
* remove ambiguous variable names
* refine graph id missing logic
* refine graph create call
* separate node/edge/graph data parser
* remove separator in DefaultDataParser
* refine validation error log for yaml field
* minor check
* refine code via dict.get()
* move load_from_csv into Node/Edge/GraphData
* move _parse_node/edge/graph_data into Node/Edge/GraphData
* refine id-related fields check
* check duplicate ntypes/etypes when load yaml
* refine docstring
* Added activation function as an optional parameter
* lint fixes
* Modified the input parameters in tandem with other classes
* lint corrections
* corrected tests
* Reverting back to the old interface
* lint corrections
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* edge_softmax_hetero forwar+cpu+norm=dst
* convert eids to list
* addedunittest
* added unittest
* added backward. Not tested correctness
* minor
* changed reducer to max from sum
* bugfix
* docstring
* add GPU unittest
* output converted to dict from tuple
* lint check
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
* Feat: support randomwalk and SelectPinSageNeighbors on GPU
Port RandomWalk algorithm on GPU,
and port SelectPinSageNeighbors on GPU.
* Feat: support GPU on python APIs
* Feat: remove perf print information in FrequenchHashmap
* Fix: modified the code format
Modified the code format as task_lint.sh suggested
* Feat: let test script support PinSAGESampler on GPU
Let test script support PinSAGESampler on GPU,
minor of "restart_prob".
* Minor
* Minor
* Minor
* Refactor: use the atomic operations from the array module
* Minor: change the long lines
* Refactor: modified the get_node_types for gpu
* Feat: update the contributor date
* Perf: remove unnecessary stream sync
* Feat: support other random walk
But the non-uniform choice is still not supported.
* Fix: add CUDA switch for random walk
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* added distgnn plus libra codebase
* Dist application codes
* added comments in partition code. changed the interface of partitioning call.
* updated readme
* create libra partitioning branch for the PR
* removed disgnn files for first PR
* updated kernel.cc
* added libra_partition.cc and moved libra code from kernel.cc to libra_partition.cc
* fixed lint error; merged libra2dgl.py and main_Libra.py to libra_partition.py; added graphsage/distgnn folder and partition script.
* removed libra2dgl.py
* fixed the lint error and cleaned the code.
* revisions due to PR comments. added distgnn/tools contains partitions routines
* update 2 PR revision I
* fixed errors; also improved the runtime by 10x.
* fixed minor lint error
* fixed some more lints
* PR revision II changed the interface of libra partition function
* rewrite docstring
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Make the default value of parameter replace in docstring consistent with code
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* min/max support for forward CPU heterograph
* Added etype with each argU values
* scatter_add needs fix
* added scatter_add_hetero. Grads dont match for max reducer
* storing ntype in argX
* fixing scatter_add_hetero
* hetero matches with torch's scatter add
* works copy_e forward+cpu
* added backward for copy_rhs
* Computes gradient for all node types in one kernel
* bug fix
* unnitest for max/min on CPU
* renamed scatter_add_hetero to update_grad_minmax_hetero
* lint check and comment out cuda call for max. Code is for CPU only
* lint check
* replace inf with zero
* minor
* lint check
* removed LIBXSMM code from hetro code
* fixing backward operator of UpdateGradMinMaxHetero
* removed backward from update_grad_minmax_hetero
* docstring
* improved docstring and coding style
* Added pass by pointer for output
* typos and pass by references
* Support for copy_rhs
* Added header <string>
* fix bug in copy_u_max
* Added comments and dimension check of all etypes
* skip mxnet check
* pass by pointer output arrays
* updated docstring
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* Added SDDMMCOO_hetero support
* removed redundant CUDA kernels
* added benchmark for regression test
* fix
* fixed bug for single src node type
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [Feature] enable async transfer in NodeDataLoader for homograph
* fix lint issues
* fix device choose when creating stream
* fix test on cpu only machine
* fix pin_memory config
* support homo only
* avoid creating stream in each step and sync via event
* fix lint
* enable graph copy on non-default stream
* fix lint
* refine arg description
* fix conflicts
* squeeze node labels in FraudDataset
* fix RLModule
* update results in README.md
* fix KeyError in full graph training
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* gpu compact graph template
* cuda compact graph draft
* fix typo
* compact graphs
* pass unit test but fail in training
* example using EdgeDataLoader on the GPU
* refactor cuda_compact_graph and cuda_to_block
* update training scripts
* fix linting
* fix linting
* fix exclude_edges for the GPU
* add --data-cpu & fix copyright
Rhett Ying