f9d51fdf0b
[Feature] Add a HINT for the per edge type sampler of heterogeneous DistGraph that highlighting the etypes are sorted already. ( #3260 )
...
* pass cpp test
* distgraph use sorted edge flag.
* lint
* triger
* update test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
2021-11-04 06:02:42 +00:00
508197e807
[New Feature] Per edge type sampler for to_homogeneous graphs. ( #3131 )
...
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* Basic imple of heterogeneous on homogenenous sampling
* make pass
* Pass C++ test
* Add python test code
* lint
* lint
* Add MultiLayerEtypeNeighborSampler
* Add unitest for single machine dataloader
* Add dist dataloader test for edge type sampler
* Fix lint
* fix
* support for per etype sample
* Fix some bug and enable distributed training with per edge sample
* fix
* Now distributed training works
* turn off some mxnet
* turn off mxnet for some dist test
* fix
* upd
* upd according to the comments
* Fix
* Fix test and now distributed works.
* upd
* upd
* Fix
* Fix bug
* remove dead code.
* upd
* Fix
* upd
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal >
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-11-02 09:21:20 +00:00
526d4ac0bb
[Doc] Fix random_walk documentation inconsistency ( #3188 )
...
* Update randomwalks.py
* Update randomwalks.py
FIx lint
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2021-08-20 02:26:20 +00:00
3f6f694159
[Bug fix] Various fix from bug bash ( #3133 )
...
* Update
* Update
* Update dependencies
* Update
* Update
* Fix ogbn-products gat
* Update
* Update
* Reformat
* Fix typo in node2vec_random_walk
* Specify file encoding
* Working for 6.7
* Update
* Fix subgraph
* Fix doc for sample_neighbors_biased
* Fix hyperlink
* Add example for udf cross reducer
* Fix
* Add example for slice_batch
* Replace dgl.bipartite
* Fix GATConv
* Fix math rendering
* Fix doc
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-17.us-west-2.compute.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-156.us-west-2.compute.internal >
2021-07-15 09:17:15 +08:00
d3e4460b98
[Doc] Add docstring for missing APIs ( #3088 )
...
* add docstring for missing API; fix some docstring
* rename apis; address comments
2021-07-05 18:40:07 +08:00
a97aaa4903
fix ( #3076 )
2021-06-30 18:27:05 +08:00
427a5a96e0
[Doc] Update NodeDataLoader and EdgeDataLoader for GPU-based neighbor sampling ( #3046 )
...
* update docstrings and tidy code
* add docs
* address comments
* Update __init__.py
* address comments
2021-06-25 10:46:21 +08:00
e56bbafd25
[Feature] Biased Neighbor Sampling ( #2987 )
...
* update
* update
* update
* update
* lint
* lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* lint
* update
* clone
* update
* update
* update
* update
* replace idarray with ndarray
* refactor cpp part
* refactor python part
* debug
* refactor interface
* test and doc
* lint and test
* lint
* fix
* fix
* fix
* const
* doc
* fix
* fix
* fix
* fix
* fix & doc
* fix
* fix
* update
* update
* update
* merge
* doc
* doc
* lint
* fix
* more tests
* doc
* fix
* fix
* update
* update
* update
* fix
* fix
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-23 17:44:51 +08:00
e667545da5
[Feature] Node2vec ( #2992 )
...
* add seal example
* 1. add paper infomation in examples/README
2. adjust codes
3. option test
* use latest `to_simple` to replace coalesce graph function
* remove outdated codes
* remove useless comment
* Node2vec
1.implement node2vec random walk c++ op
2.implement node2vec model
3.implement node2vec example
* add CMakeLists file modify
* refine c++ codes
* refine c++ codes
* add missing whitespace
* refine python codes
* add codes
* add node2vec_impl.h
* fix codes
* fix code style problem
* fixes
* remove
* lots of changes
* add benchmark
* fixes
Co-authored-by: smilexuhc <smile.xuhc@gmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-06-23 14:47:56 +08:00
e70138bbe4
[Performance][GPU] Enable GPU uniform edge sampling ( #2716 )
...
* Start on uniform GPU sampling
* Save more work
* Get cu file compiling
* Update sampling
* More changes
* Get GPU sampling for uniform probabilities solved
* Fix batch tensor migration
* Fix
* update kernels
* expand blocking
* Undo testing change
* Cut down on sampling overhead
* Fix replacement
* Update unit tests
* Add option to gpu sample in graphsage
* Copy only csc to gpu
* Add ogbn support
* Fix linting
* Remove nvtx from sample
* Improve documentation and error checking
* Expand documentation
* Update assert checking
* delete extra space
* Use standard dataloader when dataset is a dictionary
* ogb -> ogbn
* Fix edge selection determinism
* Fix typos
* Remove nvtx
* Add comment for self.fanout_arrays and assert
* Fix linting
* Migrate to scalarbatcher
* Fix indentation
* Fix batcher
* Fix indexing
* Only use databatcher for GPU
* Convert to DGL NDArray to PyTorch Tensor
* Add optimization for PyTorch's F.tensor() for list of GPU tensors
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2021-04-14 18:55:09 -07:00
72a16b22e4
[bugfix] Fix the behavior of select_topk ( #2688 )
...
* fix
* lint
* lint
2021-02-21 18:52:35 +08:00
f13b9b6212
[Doc] Scan the API docs and make many changes ( #2080 )
...
* WIP: api
* dgl.sampling, dgl.data
* dgl.sampling; dgl.dataloading
* sampling packages
* convert
* subgraph
* deprecate
* subgraph APIs
* All docstrings for convert/subgraph/transform
* almost all funcs under dgl namespace
* WIP: DGLGraph
* done graph query
* message passing functions
* lint
* fix merge error
* fix test
* lint
* fix
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-08-20 23:26:49 +08:00
75e89a153f
remove debug information ( #2056 )
2020-08-19 13:19:41 +08:00
edfbee2ca8
[Feature] Make sample_neighbors copy features on demand ( #2042 )
...
* fix
* fix
* lint
* fix
* test
* fix
* fix
2020-08-18 22:25:19 +08:00
be444e52d9
[Doc/Feature] Refactor, doc update and behavior fix for graphs ( #1983 )
...
* Update graph
* Fix for dgl.graph
* from_scipy
* Replace canonical_etypes with relations
* from_networkx
* Update for hetero_from_relations
* Roll back the change of canonical_etypes to relations
* heterograph
* bipartite
* Update doc
* Fix lint
* Fix lint
* Fix test cases
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Update
* Fix test
* Fix
* Update
* Use DGLError
* Update
* Update
* Update
* Update
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Fix
* Update
* Update
* Update
* Update
* Update
* Update
* rewrite sanity checks
* delete unnecessary checks
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Fix
* Update
* Update
* Update
* Fix
* Fix
* Fix
* Update
* Fix
* Update
* Fix
* Fix
* Update
* Fix
* Update
* Fix
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2020-08-18 04:26:29 +08:00
cd48435206
[Transform] Docstring and subframes ( #1962 )
...
* update knn graph docs
* more docs
* [Doc] transform module docstrings
* remove copy_ndata and copy_edata
* fix
* lint
* fix
* fix
* fix
* clean up docstrings
* fix docstring
* dtype specifications
* addresses comments
* fix
Co-authored-by: Mufei Li <mufeili1996@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-08-13 10:41:43 +08:00
f5eb80d221
[Feature] Edge DataLoader for edge classification & link prediction ( #1828 )
...
* clean commit
* oops forgot the most important files
* use einsum
* copy feature from frontier to block
* Revert "copy feature from frontier to block"
This reverts commit 5224ec963eb6a3ef1b6ab74d8ecbd44e4e42f285.
* temp fix
* unit test
* fix
* revert jtnn
* lint
* fix win64
* docstring fixes and doc indexing
* revert einsum in sparse bidecoder
* fix some examples
* lint
* fix due to some tediousness in remove_edges
* addresses comments
* fix
* more jtnn fixes
* fix
2020-08-11 18:00:58 +08:00
22167f7203
[Refactor] Enable new kernel in all message passing APIs ( #1953 )
...
* WIP: frame refactor
* new frame
* simple update_all builtin
* move all subgraph routines into the same file
* sddmm & spmm schedule; node & edge udf
* degree bucketing
* some tricky 0deg corner cases
* bug in frame append
* merge test_hetero_basics and test_basics
* some code rearange
* fix test_heterograph
* add mean spmm
* enable all builtin combinations
* pass gpu test
* pass pytorch tests
* wip
* fix some pt debugging codes
* fix bug in mxnet backward
* pass all mxnet utests
* passed tf tests
* docstring
* lint
* lint
* fix broadcasting bugs
* add warning and clamp for mean reducer
* add test for zero-degree mean
* address comments
* lint
* small fix
2020-08-07 15:40:25 +08:00
823eb5be1b
[Doc] dgl.sampling docstring fixes ( #1928 )
2020-08-04 17:12:46 +08:00
68a978d4de
[Model] Fixes JTNN for 0.5 ( #1879 )
...
* jtnn and fixes
* make metagraph a method
* fix test
* fix
2020-07-30 13:36:08 +08:00
44089c8b4d
[Refactor][Graph] Merge DGLGraph and DGLHeteroGraph ( #1862 )
...
* Merge
* [Graph][CUDA] Graph on GPU and many refactoring (#1791 )
* change edge_ids behavior and C++ impl
* fix unittests; remove utils.Index in edge_id
* pass mx and th tests
* pass tf test
* add aten::Scatter_
* Add nonzero; impl CSRGetDataAndIndices/CSRSliceMatrix
* CSRGetData and CSRGetDataAndIndices passed tests
* CSRSliceMatrix basic tests
* fix bug in empty slice
* CUDA CSRHasDuplicate
* has_node; has_edge_between
* predecessors, successors
* deprecate send/recv; fix send_and_recv
* deprecate send/recv; fix send_and_recv
* in_edges; out_edges; all_edges; apply_edges
* in deg/out deg
* subgraph/edge_subgraph
* adj
* in_subgraph/out_subgraph
* sample neighbors
* set/get_n/e_repr
* wip: working on refactoring all idtypes
* pass ndata/edata tests on gpu
* fix
* stash
* workaround nonzero issue
* stash
* nx conversion
* test_hetero_basics except update routines
* test_update_routines
* test_hetero_basics for pytorch
* more fixes
* WIP: flatten graph
* wip: flatten
* test_flatten
* test_to_device
* fix bug in to_homo
* fix bug in CSRSliceMatrix
* pass subgraph test
* fix send_and_recv
* fix filter
* test_heterograph
* passed all pytorch tests
* fix mx unittest
* fix pytorch test_nn
* fix all unittests for PyTorch
* passed all mxnet tests
* lint
* fix tf nn test
* pass all tf tests
* lint
* lint
* change deprecation
* try fix compile
* lint
* update METIDS
* fix utest
* fix
* fix utests
* try debug
* revert
* small fix
* fix utests
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* trigger
* +1s
* [kernel] Use heterograph index instead of unitgraph index (#1813 )
* upd
* upd
* upd
* fix
* upd
* upd
* upd
* upd
* upd
* trigger
* +1s
* [Graph] Mutation for Heterograph (#1818 )
* mutation add_nodes and add_edges
* Add support for remove_edges, remove_nodes, add_selfloop, remove_selfloop
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
* upd
* upd
* upd
* fix
* [Transfom] Mutable transform (#1833 )
* add nodesy
* All three
* Fix
* lint
* Add some test case
* Fix
* Fix
* Fix
* Fix
* Fix
* Fix
* fix
* triger
* Fix
* fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
* [Graph] Migrate Batch & Readout module to heterograph (#1836 )
* dgl.batch
* unbatch
* fix to device
* reduce readout; segment reduce
* change batch_num_nodes|edges to function
* reduce readout/ softmax
* broadcast
* topk
* fix
* fix tf and mx
* fix some ci
* fix batch but unbatch differently
* new checkk
* upd
* upd
* upd
* idtype behavior; code reorg
* idtype behavior; code reorg
* wip: test_basics
* pass test_basics
* WIP: from nx/ to nx
* missing files
* upd
* pass test_basics:test_nx_conversion
* Fix test
* Fix inplace update
* WIP: fixing tests
* upd
* pass test_transform cpu
* pass gpu test_transform
* pass test_batched_graph
* GPU graph auto cast to int32
* missing file
* stash
* WIP: rgcn-hetero
* Fix two datasety
* upd
* weird
* Fix capsuley
* fuck you
* fuck matthias
* Fix dgmg
* fix bug in block degrees; pass rgcn-hetero
* rgcn
* gat and diffpool fix
also fix ppi and tu dataset
* Tree LSTM
* pointcloud
* rrn; wip: sgc
* resolve conflicts
* upd
* sgc and reddit dataset
* upd
* Fix deepwalk, gindt and gcn
* fix datasets and sign
* optimization
* optimization
* upd
* upd
* Fix GIN
* fix bug in add_nodes add_edges; tagcn
* adaptive sampling and gcmc
* upd
* upd
* fix geometric
* fix
* metapath2vec
* fix agnn
* fix pickling problem of block
* fix utests
* miss file
* linegraph
* upd
* upd
* upd
* graphsage
* stgcn_wave
* fix hgt
* on unittests
* Fix transformer
* Fix HAN
* passed pytorch unittests
* lint
* fix
* Fix cluster gcn
* cluster-gcn is ready
* on fixing block related codes
* 2nd order derivative
* Revert "2nd order derivative"
This reverts commit 523bf6c249bee61b51b1ad1babf42aad4167f206.
* passed torch utests again
* fix all mxnet unittests
* delete some useless tests
* pass all tf cpu tests
* disable
* disable distributed unittest
* fix
* fix
* lint
* fix
* fix
* fix script
* fix tutorial
* fix apply edges bug
* fix 2 basics
* fix tutorial
Co-authored-by: yzh119 <expye@outlook.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-42.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-1-5.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-68-185.ec2.internal >
2020-07-28 14:30:41 +08:00
168a88e570
[Sampling] NodeDataLoader for node classification ( #1635 )
...
* neighbor sampler data loader first commit
* more commit
* nodedataloader
* fix
* update RGCN example
* update OGB
* fixes
* fix minibatch RGCN crashing with self loop
* reverting gatconv test code
* fix
* change to new solution that doesn't require tf dataloader
* fix
* lint
* fix
* fixes
* change doc
* fix docstring
* docstring fixes
* return seeds and input nodes from data loader
* fixes
* fix test
* fix windows build problem
* add pytorch wrapper
* fixes
* add pytorch wrapper
* add unit test
* add -1 support to sample_neighbors & fix docstrings
* docstring fix
* lint
* add minibatch rgcn evaluations
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Tong He <hetong007@gmail.com >
2020-07-02 16:31:05 +08:00
18a26fcfb1
[Data] Support HeteroGraph save/load ( #1526 )
...
* 111
* add history version test
* fix
* 111
* save
* ``
* fix1
* 111
* add save heterograph
* lint
* lint
* add tests
* minor fix
* fix
* docs
* add format tets
* use unique_ptr
* fix
* fix interface
* 111
* 111
* fix
* lint
* fix
* add support to s3
* fix
* fix
* fix leak
* fix
* fix docs
* fix
* linlt
* fix
* fix
* fix
* address comment
* address comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-30 13:34:50 +08:00
7ba9cbc63e
[Docs] Fix docstrings and documentations ( #1420 )
...
* fix docs of to_block
* fix
* oops
* more fixes
2020-05-19 13:04:55 +08:00
e4cc81852d
[Windows] fix compilation issues on vs2015 ( #1405 )
...
* [Windows] fix compilation issues on vs2015
* fix test
2020-03-30 22:08:26 +08:00
4b4186f87d
[Sampler] Change argument type of fanout from list to dict ( #1403 )
2020-03-30 15:52:46 +08:00
0f40c6e49c
[Hetero] Replace card with num_nodes
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-03-20 16:56:53 +08:00
5dd35580f7
[Feature] Improve sampling speed; Better pickle/unpickle; other fixes ( #1299 )
...
* improve performance of sample_neighbors
* some more improve
* test script
* benchmarks
* multi process
* update more tests
* WIP
* adding two API for state saving
* add create from state
* upd test
* missing file
* wip: pickle/unpickle
* more c apis
* find the problem of empty data array
* add null array; pickling speed is bad
* still bad perf
* still bad perf
* wip
* fix the pickle speed test; now everything looks good
* minor fix
* bugfix
* some lint fix
* address comments
* more fix
* fix lint
* add utest for random.choice
* add utest for dgl.rand_graph
* fix cpp utests
* try fix ci
* fix bug in TF backend
* upd choice docstring
* address comments
* upd
* try fix compile
* add comment
2020-03-02 23:16:05 +08:00
c3a33407fe
[Feature] PinSAGE sampler ( #1249 )
...
* [WIP] PinSAGE operators
* moved the edge remapping mess into C
* some docstrings
* lint
* lint x2
* lint x3
* skip gpu test on topk
* extend pinsage to any metapath
* lint x4
* addresses #1265
* add always_preserve (fixes #1266 ) and fix a silly bug
* disable gpu test on compaction
* lint
* fix a horrible bug and add more tests
* lint
* addresses comments
* lint
* bugfix
* addresses comments
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu >
2020-02-28 20:46:13 +08:00
f8d4264e5b
[Feature] Neighbor-hood based sampling APIs ( #1251 )
...
* WIP: working on random choices
* light slice
* basic CPU impl
* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph
* simple test case works
* fix bug in slicing probability array
* fix bug in getting the correct relation graph
* fix bug in creating placeholder graph
* enable omp
* add cpp test
* sample topk
* add in|out_subgraph
* try fix lint; passed all unittests
* fix lint
* fix msvc compile; add sorted flag and constructors
* fix msvc
* coosort
* COOSort; CSRRowWiseSampling; CSRRowWiseTopk
* WIP: remove DType in CSR and COO; Restrict data array to be IdArray
* fix all CSR ops for missing data array
* compiled
* passed tests
* lint
* test sampling out edge
* test different per-relation fanout/k values
* fix bug in random choice
* finished cpptest
* fix compile
* Add induced edges
* add check
* fixed bug in sampling on hypersparse graph; add tests
* add ascending flag
* in|out_subgraph returns subgraph and induced eid
* address comments
* lint
* fix
2020-02-17 21:25:21 +08:00
a9dabcc769
[Feature] Random Walk for 0.5 ( #1209 )
...
* trying to refactor IndexSelect
* partial implementation
* add index select and assign for floats as well
* move to random choice source
* more updates
* fixes
* fixes
* more fixes
* adding python impl
* fixes
* unit test
* lint
* lint x2
* lint x3
* update metapath2vec
* debugging performance
* still debugging for performance
* tuning
* switching to succvec
* redo
* revert non-uniform sampler to use vector
* still not fast
* why does this crash with OpenMP???
* because there was a data race!!!
* add documentations and remove assign op
* lint
* lint x2
* lol what have i done
* lint x3
* fix and disable gpu testing
* bugfix
* generic random walk
* reorg the random walk source code
* Update randomwalks.h
* Update randomwalks_cpu.cc
* rename file
* move internal function to anonymous ns
* reorg & docstrings
* constant restart probability
* docstring fix
* more commit
* random walk with restart, tested
* some fixes
* switch to using NDArray for choice
* massive fix & docstring
* lint x?
* lint x??
* fix
* export symbols
* skip gpu test
* addresses comments
* replaces another VecToIdArray
* add randomwalks.h to include
* replace void * with template
2020-01-31 19:52:08 +08:00
xiang song(charlie.song)