870da747ea
[CUDA][Kernel] More CUDA kernels; Standardize the behavior for sorted COO/CSR ( #1704 )
...
* add cub; array cumsum
* CSRSliceRows
* fix warning
* operator << for ndarray; CSRSliceRows
* add CSRIsSorted
* add csr_sort
* inplace coosort and outplace csrsort
* WIP: coo is sorted
* mv cuda_utils
* add AllTrue utility
* csr sort
* coo sort
* coo2csr for sorted coo arrays
* CSRToCOO from sorted
* pass tests for the new kernel changes
* cannot use inplace sort
* lint
* try fix msvc error
* Fix g.copy_to and g.asnumbits; ToBlock no longer uses CSC
* stash
* revert some hack
* revert some changes
* address comments
* fix
* fix to_block unittest
* add todo note
2020-06-28 18:37:28 +08:00
c8b18b793c
[RPC] Support null NDArray in RPC communication ( #1653 )
2020-06-21 17:20:58 +08:00
f1b19a6b67
[CUDA] Many CUDA operators; Prepare for DGLGraph on CUDA ( #1660 )
...
* add cuda utils; change g.to; add g.device
* split array.h into several headers
* cuda index select
* file
* three cuda kernels
* add cuda elementwise arith and several others
* cuda CSRIsNonZero
* fix lint
* lint
* lint
* fix bug in changing ctx to property
* address comments
* remove unused codes
* address comments
2020-06-19 15:44:42 +08:00
41349dcef5
[Feature] Range partition ( #1522 )
...
* add reorder immutable graph.
* add python API.
* add reorder for csr.
* remove gk version.
* fix
* add cpp test.
* bug fixes
* fix tests.
* fix bugs and add check
* fix test.
* add omp.
* add comments.
* add coo reorder.
* fix a bug.
* handle reorder for different graph structues.
* fix lint.
* fix.
* add original ids.
* reshuffle nodes before metis partition.
* inner nodes are in contiguous Id range.
* reshuffle nodes/edges when partitioning.
* load partition return graph partition book.
* use inner_node/inner_edges
* add and test range partition book.
* count inner_edge correctly.
* fix lint.
* fix lint.
* fix lint.
* fix errors.
* fix errors.
* fix for TF.
* fix.
* fix.
* change docstring.
* support logical and.
* add comments.
* avoid copy.
* fix
* update docstring.
* fix a bug.
* add range search.
* fix
* fix a bug.
* add more tests.
* load graph partition book.
* support shared memory for range partition book.
* fix a bug.
* fix.
* fix lint.
* remove check
* fix test.
* remove num_nodes and num_edges
* fix lint.
* fix graph partition book.
* address comments.
* use makedirs.
* fix compile
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-06-15 11:43:04 -07:00
d6d517bb02
[Kernel] CUDA CSR2COO COOSort COO2CSR ( #1620 )
...
* add cuda source
* moving codes from kernel2 branch
* operator overloading
* Better error message for unsupported device
* fix c tests
* coo sort using cusparse
* move test_rpc to distributed
* lint
* address comments and add utests
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com >
2020-06-15 14:35:12 +08:00
7639b5e70c
[Fix] New StreamWithBuffer interface ( #1557 )
...
* WIP: rpc components
* client & server
* move network package to rpc
* fix include
* fix compile
* c api
* wip: test
* add basic tests
* missing file
* [RPC] Zero copy serializer (#1517 )
* zerocopy serialization
* add test for HeteroGraph
* fix lint
* remove unnecessary codes
* add comment
* lint
* lint
* disable pylint for now
* add include for win
* windows guard
* lint
* lint
* skip test on windows
* refactor
* add comment
* fix
* comment
* 1111
* fix
* Update Jenkinsfile
* [RPC] Implementation of RPC infra (#1544 )
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* remove client.cc and server.cc
* fix lint
* update
* update
* fix linr
* update
* fix lint
* update
* update
* update
* update
* update
* update
* update test
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update comment
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Refactor StreamWithBuffer (#1550 )
* refactor
* fix with new interface
* remove copy
* fix
* remove comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Chao Ma <mctt90@gmail.com >
2020-05-27 01:54:25 +08:00
3e69692233
[RPC] New RPC infrastructure. ( #1549 )
...
* WIP: rpc components
* client & server
* move network package to rpc
* fix include
* fix compile
* c api
* wip: test
* add basic tests
* missing file
* [RPC] Zero copy serializer (#1517 )
* zerocopy serialization
* add test for HeteroGraph
* fix lint
* remove unnecessary codes
* add comment
* lint
* lint
* disable pylint for now
* add include for win
* windows guard
* lint
* lint
* skip test on windows
* refactor
* add comment
* fix
* comment
* 1111
* fix
* Update Jenkinsfile
* [RPC] Implementation of RPC infra (#1544 )
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* remove client.cc and server.cc
* fix lint
* update
* update
* fix linr
* update
* fix lint
* update
* update
* update
* update
* update
* update
* update test
* update
* update test
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update comment
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* fix lint
* fix lint
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-05-22 18:48:37 +08:00
67cb7a43a0
[Feature] Deprecate multigraph ( #1389 )
...
* Deprecate multi-graph
* Handle heterograph and edge_ids
* lint
* Fix
* Remove multigraph in C++ end
* Fix lint
* Add some test and fix something
* Fix
* Fix
* upd
* Fix some test case
* Fix
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-03-28 21:31:24 +08:00
5a1ef70fd4
[Optimization] Optimize performance of COOSort ( #1349 )
...
* Fix coo_sort
* Add COOSort test
* Fix
* omp
* coo2csr can benefit from sorted coo
* Add more test code
* Add LIBCXX_ENABLE_PARALLEL_ALGORITHMS to Makefile
* upd
* Turn off LIBCXX_ENABLE_PARALLEL_ALGORITHMS by default
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal >
2020-03-13 22:39:36 +08:00
a9520f71ce
[Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API ( #1297 )
...
* remove edge and to bipartite and graphsage with sampling
* fixes
* fixes
* fixes
* reenable multigpu training
* fixes
* compatibility in DGLGraph
* rename to compact_as_bipartite
* bugfix
* lint
* add offline inference
* skip GPU tests
* fix
* addresses comments
* fix
* fix
* fix
* more tests
* more docs and unit tests
* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
9caff61777
[Feature] More serialization support ( #1295 )
...
* fix script
* t
* fix weird bugs
* fix
* fix
* upload
* fix
* fix
* lint
* fix
* tmp
* fix serialization
* fix
* fix lint
* fix message
* fix
* lint
* address comment
* fix
* lint
* fix
* fix
* fix
* Remove duplicate serialization for meta graph
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2020-03-06 21:38:56 +08:00
5dd35580f7
[Feature] Improve sampling speed; Better pickle/unpickle; other fixes ( #1299 )
...
* improve performance of sample_neighbors
* some more improve
* test script
* benchmarks
* multi process
* update more tests
* WIP
* adding two API for state saving
* add create from state
* upd test
* missing file
* wip: pickle/unpickle
* more c apis
* find the problem of empty data array
* add null array; pickling speed is bad
* still bad perf
* still bad perf
* wip
* fix the pickle speed test; now everything looks good
* minor fix
* bugfix
* some lint fix
* address comments
* more fix
* fix lint
* add utest for random.choice
* add utest for dgl.rand_graph
* fix cpp utests
* try fix ci
* fix bug in TF backend
* upd choice docstring
* address comments
* upd
* try fix compile
* add comment
2020-03-02 23:16:05 +08:00
ebca1188a9
[Feature] Support serialization for smart pointer ( #1291 )
...
* fix script
* t
* fix weird bugs
* fix
* fix
* upload
* fix
* fix
* lint
* fix
2020-02-26 17:29:27 +08:00
f8d4264e5b
[Feature] Neighbor-hood based sampling APIs ( #1251 )
...
* WIP: working on random choices
* light slice
* basic CPU impl
* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph
* simple test case works
* fix bug in slicing probability array
* fix bug in getting the correct relation graph
* fix bug in creating placeholder graph
* enable omp
* add cpp test
* sample topk
* add in|out_subgraph
* try fix lint; passed all unittests
* fix lint
* fix msvc compile; add sorted flag and constructors
* fix msvc
* coosort
* COOSort; CSRRowWiseSampling; CSRRowWiseTopk
* WIP: remove DType in CSR and COO; Restrict data array to be IdArray
* fix all CSR ops for missing data array
* compiled
* passed tests
* lint
* test sampling out edge
* test different per-relation fanout/k values
* fix bug in random choice
* finished cpptest
* fix compile
* Add induced edges
* add check
* fixed bug in sampling on hypersparse graph; add tests
* add ascending flag
* in|out_subgraph returns subgraph and induced eid
* address comments
* lint
* fix
2020-02-17 21:25:21 +08:00
032a08cc32
[Test] Fix test script ( #1257 )
...
* fix script
* t
* fix weird bugs
* fix
* fix
2020-02-13 18:09:15 +08:00
23893bb0a8
[Data format] Serialization for Immutable Graph and HeteroGraph ( #1254 )
...
* graph format
* fix lint
* lint
* fix
* unit test
* lint
* add magic num
* move serialize out of struct
* lint
* serialize
* trigger CI
* fix lint
* lint
Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com >
2020-02-11 00:14:31 +08:00
f103bbf9ee
[Data format] Serialization for UnitGraph ( #1242 )
...
* graph format
* fix lint
* lint
* fix
* unit test
* lint
* add magic num
* move serialize out of struct
* lint
Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com >
2020-02-08 20:49:34 +08:00
a9dabcc769
[Feature] Random Walk for 0.5 ( #1209 )
...
* trying to refactor IndexSelect
* partial implementation
* add index select and assign for floats as well
* move to random choice source
* more updates
* fixes
* fixes
* more fixes
* adding python impl
* fixes
* unit test
* lint
* lint x2
* lint x3
* update metapath2vec
* debugging performance
* still debugging for performance
* tuning
* switching to succvec
* redo
* revert non-uniform sampler to use vector
* still not fast
* why does this crash with OpenMP???
* because there was a data race!!!
* add documentations and remove assign op
* lint
* lint x2
* lol what have i done
* lint x3
* fix and disable gpu testing
* bugfix
* generic random walk
* reorg the random walk source code
* Update randomwalks.h
* Update randomwalks_cpu.cc
* rename file
* move internal function to anonymous ns
* reorg & docstrings
* constant restart probability
* docstring fix
* more commit
* random walk with restart, tested
* some fixes
* switch to using NDArray for choice
* massive fix & docstring
* lint x?
* lint x??
* fix
* export symbols
* skip gpu test
* addresses comments
* replaces another VecToIdArray
* add randomwalks.h to include
* replace void * with template
2020-01-31 19:52:08 +08:00
f1420d194f
[Refactor] Renaming class methods of sampler utilities to improve readability ( #1180 )
...
* upd
* upd
2020-01-07 22:02:44 +08:00
dfb10db86f
[Feature] Bag of samplers for efficient categorical sampling w/ and w/o replacement ( #1142 )
...
* upd
* upd
* lint
* upd
* upd
* lint
* upd
* refactor
* lint
* upd
* upd
* upd
* lint
* fix
* fix
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* complete test
* upd
* upd
* fix
* vir
* upd
* fix
* fuck
* fix numerical
2020-01-01 11:42:11 +08:00
65e1ba4f60
Sort csr ( #886 )
...
* sort
* sort in parallel.
* fix a bug in sorting adj
* rename.
* add more comments.
* accelerate GetData
* fix tests.
* avoid sorting multiple times.
* add test.
* change back.
* sort.
* add sort_csr.
* Fix a bug.
* fix.
* revert modifcation.
* rename
* speed up EdgeIds.
2019-09-28 10:52:12 -07:00
c3516f1a8e
[Network] Refactoring Communicator ( #679 )
...
* Refactoring Communicator
* fix lint
* change non-const reference
* add header file
* use MemoryBuffer
* update PR
* fix bug on csr shape
* zero-copy msg_queue
* fix lint
* fix lint
* fix lint
* add header file
* fix windows build error
* fix windows build error
* update
* fix lint
* update
* fix lint
* fix lint
* add more test
* fix windows test
* update windows test
* update windows test
* update windows test
* update
* fix lint
* fix lint
* update
* update
* update
* update
* use STATUS code
* update test
* remove mem_cpy
* fix lint
* update
* finish
* ConstructNFTensor
* add test for deallocator
* update
* fix lint
2019-08-05 10:49:25 +08:00
b0d9e7aa43
[Refactor] Separating graph and sparse matrix operations ( #699 )
...
* WIP: array refactoring
* WIP: implementation
* wip
* most csr part
* WIP: on coo
* WIP: coo
* finish refactoring immutable graph
* compiled
* fix undefined ndarray copy bug; add COOToCSR when coo has no data array
* fix bug in COOToCSR
* fix bug in CSR constructor
* fix bug in in_edges(vid)
* fix OutEdges bug
* pass test_graph
* pass test_graph
* fix bug in CSR constructor
* fix bug in CSR constructor
* fix bug in CSR constructor
* fix stupid bug
* pass gpu test
* remove debug printout
* fix lint
* rm biparate grpah
* fix lint
* address comments
* fix bug in Clone
* cpp utests
2019-07-17 17:49:38 -04:00
3a11d26134
update ( #593 )
2019-06-03 14:41:58 +08:00
16af6ebd3b
[Test] Add cpp test to CI ( #567 )
...
* add cpptest to ci
* try windows
* Update Jenkinsfile
* Update Jenkinsfile
* Update Jenkinsfile
* Update task_cpp_unit_test.bat
* Change localhost to 127.0.0.1
* windows socket gtest
* lint
* Update task_cpp_unit_test.bat
* fix batch script
* export all symbols
* Update Jenkinsfile
* compile once
* try mxnet compact
2019-05-30 11:01:16 -04:00
85f2d1d935
[CI] Fix CI for cpp test ( #570 )
...
* fix CI for cpp test
* update port number
2019-05-26 22:01:35 +08:00
3d7c3303ac
Add unittest for C++ RPC ( #566 )
2019-05-26 14:29:47 +08:00
7bdc16197f
add network cpp test ( #565 )
2019-05-26 13:46:37 +08:00
de54891fc8
[Test] Add gtest to project ( #547 )
...
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
2019-05-24 15:15:47 +08:00
Minjie Wang