ca2a7e1ca1
[Refactor] Nodeflow, sampling, CAPI ( #430 )
...
* enable cython
* add helper function and data structure for void_p vector return
* move sampler from graph index to contrib.sampling
* WIP
* WIP
* refactor layer sampling
* pass tests
* fix lint
* fix graphsage
* remove comments
* pickle test
* fix comments
* update dev guide for cython build
2019-03-05 14:07:22 -05:00
fb4246e564
[Feature]Uniform layer-wise sampler ( #416 )
...
* migrate to node-flow
* uniform layer sampler test cases
* more test cases
* documentations
* fix lint errors
* fix lint errors
* fix lint errors
* iota
* add asnumpy
* requested changes
* fix indptr error
* fix lint errors
* requested changes & fix lint errors
* fix lint errors
* fix LayerSampler unit test
2019-03-02 16:37:18 -08:00
7e30382e4f
[Model][MXNet] neighbor sampling & skip connection & control variate & graphsage ( #322 )
...
* neighbor sampling draft
* val/test acc
* control variate draft
* control variate
* update
* fix new_history
* maintain aggregated history while updating new history
* preprocess the first layer, change push to pull
* update
* fix subg_degree
* nodeflow
* clear
* readme
* doc and unittest for self loop
* address comments
* rename
* update
* fix
* Update node_flow.py
* Update node_flow.py
2019-02-28 13:29:51 -08:00
bfe3e9d64a
[Feature] Add switch readonly state interface ( #389 )
...
* fix rgcn tutorial
* small fix
* add switch readonly state interface
* remove trailing ws
* fix
* fix
* change signature
* change signature
* remove trailing ws
* upd
* upd interface
* upd
* rm trailing ws
* reformat
* trailing space
* upd
2019-02-20 18:51:56 -05:00
2e1cbd5d68
[Feature] Random walk traces generation ( #392 )
...
* random walk traces generation
* remove outdated comments
* oops put in the wrong place
* explicit inline
* moving rand_r to util
* moving random walk to public function
* per-thread seed and openmp support
* type cast styles
2019-02-20 10:41:59 -05:00
f370e628cd
[Feature] add NodeFlow API ( #361 )
...
* sample layer subgraphs.
* fix.
* fix.
* add layered subgraph.
* fix lint.
* fix.
* fix tutorial.
* fix.
* remove copy_to_parent.
* add num_layers
* move sampling code to sampler.cc
* fix.
* move subgraph construction out.
* Revert "move subgraph construction out."
This reverts commit 24b3d13b0d8ed5f91847ea75a7674ee8f7d45cff.
* change to NodeFlow.
* use NodeFlow in Python.
* use NodeFlowIndex.
* add node_mapping and edge_mapping.
* remove unnecessary code in SSE tutorial.
* Revert "remove unnecessary code in SSE tutorial."
This reverts commit 093f0413d5fa2e63ca5f80c46c80a126a9fb720c.
* fix tutorial.
* move to node_flow.
* update gcn cv updater.
* import NodeFlow.
* update.
* add demo code for vanilla control variate sampler.
* update.
* update.
* add neighbor sampling.
* return flow offsets.
* update node_flow.
* add test.
* fix sampler.
* fix graph index.
* fix a bug in sampler.
* fix map_to_layer_nid and map_to_flow_eid.
* fix apply_flow.
* remove model code.
* implement flow_compute.
* fix a bug.
* reverse the csr physically.
* add mini-batch test.
* add mini batch test.
* update flow_compute.
* add prop_flows
* run on specific nodes.
* test copy
* fix a bug in creating frame in NodeFlow.
* add init gcn_cv_updater.
* fix a minor bug.
* fix gcn_cv_updater.
* fix a bug.
* fix a bug in NodeFlow.
* use new h in gcn_cv_updater.
* add layer_in_degree and layer_out_degree.
* fix gcn_cv_updater for gpu.
* temp fix in NodeFlow for diff context.
* allow enabling/disabling copy back.
* add with-updater option.
* fix a bug in computing degree.
* add with-cv option.
* rename and add comments.
* fix lint complain.
* fix lint.
* avoid assert.
* remove assert.
* fix.
* fix.
* fix.
* fix.
* fix the methods in NodeFlow.
* fix lint.
* update SSE.
* remove gcn_cv_updater.
* correct comments for the schedulers.
* update comment.
* add map_to_nodeflow_nid
* address comment.
* remove duplicated test.
* fix int.
* fix comments.
* fix lint
* fix.
* replace subgraph with NodeFlow.
* move view.
* address comments.
* fix lint.
* fix lint.
* remove static_cast.
* fix docstring.
* fix comments.
* break SampleSubgraph.
* move neighbor sampling to sampler.cc
* fix comments.
* rename.
* split neighbor_list.
* address comments.
* fix.
* remove TODO.
2019-02-19 15:49:28 -08:00
220a1e68fb
[BUG] Fixing bug in pickling readonly graphs ( #397 )
...
* fixing bug in pickling readonly graphs
* multigraph test coverage
* fixing lint (??????)
* another lint
* is_readonly interface
2019-02-19 00:14:11 -05:00
40afdf7cd5
[Bugfix] Remove unnecessary assert in from_scipy_sparse_matrix ( #391 )
2019-02-13 17:45:22 -05:00
929742b588
[Feature] Reimplement Immutable graph index in DGL ( #342 )
...
* init.
* it's compiled.
* add immutable graph constructor.
* add immutable graph API.
* fix.
* impl get adjacency matrix.
* fix.
* fix graph_index from scipy matrix.
* add neighbor sampling.
* remap vertex ids.
* fix.
* move sampler test.
* fix tests.
* add comments
* remove mxnet-specific immutable graph.
* fix.
* fix lint.
* fix.
* try to fix windows compile error.
* fix.
* fix.
* add test.
* unify Graph and ImmutableGraph.
* fix bugs.
* fix compile.
* move immutable graph.
* fix.
* remove print.
* fix lint.
* fix
* fix lint.
* fix lint.
* fix test.
* fix comments.
* merge GraphIndex and ImmutableGraphIndex.
* temp fix.
* impl GetAdj.
* fix lint
* fix.
* fix.
* fix.
* fix.
* fix.
* use csr only for readonly graph.
* Revert "use csr only for readonly graph."
This reverts commit 8e24bb033af8504531b22849de5b7567b168e0d5.
* remove code.
* fix.
* fix.
* fix.
* fix.
* fix.
* fix.
* address comments.
* fix for comments.
* fix comments.
* revert.
* move test_graph_index to compute.
* fix.
* fix.
* impl GetAdj for coo.
* fix.
* fix tests.
* address comments.
* address comments.
* fix comment.
* address comments.
* use lambda.
* other comments.
* address comments.
* modify the semantics of edges.
* fix order.
* use DGLIdIter
* fix.
* remove NotImplemented.
* revert some code.
2019-01-17 09:59:53 +08:00
ddc2faa547
[Bugfix] Fix gat residual bug ( #355 )
...
* fix gat residual bug
* fix the residual addition; output heads; add some shape notations;
* minor
* fix the output head average
* add requests package in requirement
2019-01-12 08:37:01 -05:00
3a868eb04b
[Feature] Max readout and consecutive labeling for networkx ( #341 )
...
* Max readout and consecutive labeling
* Delete test_readout.py
* Delete test_basics.py
* Test case and fix
* Recover accidentally removed file
* Fix import order
* Fix test case
* Fix
* Fix
* Fix
* Fix
* Fix
* revert
* Fix
* Fix
* Fix
2019-01-10 12:58:05 -05:00
707334ce85
minor spelling tweaks ( #349 )
...
* minor spelling tweaks
* Update CONTRIBUTORS.md
2019-01-10 12:02:31 -05:00
e19cd62ecd
[Test] Unify tests for different backends ( #333 )
...
* test basics
* batched graph & filter, mxnet filter fix
* frame and function; bugfix
* test graph adj and inc matrices
* fixing start = 0 for mxnet
* test index
* inplace update & line graph
* multi send recv
* more tests
* oops
* more tests
* removing old test files; readonly graphs for mxnet still kept
* modifying test scripts
* adding a placeholder for pytorch to reserve directory
* torch 0.4.1 compat fixes
* moving backend out of compute to avoid nose detection
* tests guide
* mx sparse-to-dense/sparse-to-numpy is buggy
* oops
* contribution guide for unit tests
* printing incmat
* printing dlpack
* small push
* typo
* fixing duplicate entries that causes undefined behavior
* move equal comparison to backend
2019-01-05 23:32:33 -05:00
4bd4d6e348
[Lint] Pylint ( #330 )
...
* fix lint for graph_index.py
* pylint for base.py
* pylint for batched_graph.py
* pylint for frame.py; simplify and fix bugs in frame when index is slice type
* pylint for graph.py
* pylint for immutable_graph_index.py
* pylint for init.py
* pylint for rest files in root package
* pylint for _ffi package
* pylint for function package
* pylint for runtime package
* pylint for runtime.ir package
* add pylint to ci
* fix mx tests
* fix lint errors
* fix ci
* fix as requested
* fix lint
2019-01-01 15:15:41 -05:00
f35ac5443e
[Graph][Model] Cache adj & inc; MX GCN ( #307 )
...
* add cache to adj and incmat
* Fix bug in cached adj/inc
* mx gcn spmv runnable; acc debugging...
* fix bug in mx gcn that loss is not correctly calculated
* fix mx utest
* fix as requested
* use raw parameter tensors rather than dense layer
* fix dropout
* Add numbers in readme
2018-12-16 22:51:01 -05:00
2194b7dfa6
[Bugfix] edge order is not preserved when converting from edge list ( #262 )
2018-12-05 17:41:26 -05:00
eafcb7e7f5
[Bugfix][MXNet] Fix edge order and builtin max bug in mx ( #247 )
...
* Fix edge order and builtin max bug in mx
* fix as requested
2018-12-05 01:32:54 -08:00
57daf9c9a6
[Graph] small fix on the all index ( #219 )
2018-12-02 13:39:04 -05:00
57b07fce92
[Model] Accelerate SSE ( #172 )
...
* lazy eval edge ids.
* parallelize node id lookup.
* fix a bug.
* use mxnet for index.
* use update_all in the subgraph training.
* sample neighbors.
* Revert "parallelize node id lookup."
This reverts commit e661f69bb06cb5a7c246f0e106f245e27800e220.
* update README.
* cache subgraphs.
* support all degrees.
* cache adj in CPU.
* fix a bug in sse.
* print.
* raise error on mutable graphs.
* measure train time per epoch.
* fix a bug in graph_index.
* remove readonly in DGLSubGraph.
* cache subgraph properly.
* accelerate getting adjacency.
* split infer.
* lazy eval edges.
* specify inference mode.
* update for new sampler.
* use new mxnet sampling api.
* fix indent.
* remove profiling code.
* remove mxnet from sampler.
* return a lambda function for graph edges.
* add docs for immutable subgraph.
* Revert "return a lambda function for graph edges."
This reverts commit 0de5d7f100e230c518a3fb8976a6227f474d09ee.
* get parent_eid.
2018-12-02 12:43:25 -05:00
deb653f8dc
[Runtime] Scheduler and Executor ( #140 )
...
* executor api
* draft executor interface
* WIP
* revert changes to avoid conflict with api change
* core scheduling logic
* WIP: build graph adj
* incidence matrix for in edges
* support incidence matrix for partial recv nodes
* improve
* build adjmat in scheduler
* graph store
* get degree bucketing schedule
* connect to c++ degree bucketing
* conceptual executor creation code
* executor comments
* fix
* more executor comments
* WIP: full send_and_recv schedule
* most schedulers
* simplify scheduler
* executors
* runtime
* builtin function base class
* adj indices and shape
* completely refactor scheduler
* rename and move bundled out to function.py
* use_edge_feature in msg func
* rewrite scheduler
* node edge executor
* connect with graph api
* handle zero degree
* misc
* fix test cases
* fix a good many bugs...
* remove old scheduler
* push and pull
* fix send recv
* c++ lint
* fix batched send recv
* hot fix for mxnet
* typo
* write back executor
* apply node edge
* clean up, doc string
* fix as requested
* refactor
* fix
* WIP
* WIP
* ir draft
* more on ir
* WIP: spmv schedule
* WIP
* recv schedule
* refactor
* WIP
* snr degree bucketing
* snr scheduler
* move prog to graph.py; rename
* unittest for send/recv
* remove some legacy codes
* WIP: update_all
* pass test_basics
* passed all current utests
* more utests; fix mx utest
* WIP: fixing zero deg initial value
* some tests
* fix 0deg problem
* fix mx
* fix mx
* some notes
* fix as requested
2018-11-22 12:15:50 -05:00
79a510252b
Pickling support ( #155 )
...
* pickling support
* resorting to suggested way of pickling
* custom attribute pickling check
* working around a weird pytorch pickling bug
* including partial frame case
* pickling everything now
* fix as requested
2018-11-16 17:19:20 -05:00
048f6d7a30
[GraphIndex] refactor graph caching ( #150 )
...
* refactor graph caching
* fix mx test
* fix typo
2018-11-13 22:12:38 -05:00
0452cc3c0c
[Bug] Transpose adj is not correctly cached ( #148 )
2018-11-12 22:15:45 -05:00
7241a9c008
[Backend] backend interface ( #109 )
...
* backend interface
* small fix
* more comments to the data type dict
* WIP
* convert_to and narrow
* WIP
* pytorch and numpy backend; WIP on mxnet backend
* mxnet backend
* narrow
* Fix all usages
* fix for mx
* fix for mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* fix mx
* revert jenkins
* add sparse_matrix api
* sparse matrix api
* some fixme
* Fix as requested
2018-11-04 21:42:39 -05:00
9e9a9488f1
[GraphIndex] Immutable graph index using MXNet CSRArray ( #86 )
...
* add immutable graph index.
* update ImmutableGraphIndex.
* add benchmarks of subgraph generation.
* fix node_subgraphs.
* use scipy synthetic data in immutable graph.
* create immutable_graph.
* use spmv in gcn.
* fix immutable graph index.
* update graph index.
* create ImmutableSubgraphIndex.
* return subgraph node and edge mapping.
* fix benchmark.
* add mapping to subgraph nid.
* test in_edges and out_edges.
* update benchmark.
* Fix immutable subgraph.
* complete in_edges and out_edges.
* fix benchmarks.
* add test on node_subgraphs
* add sse.
* construct subgraphs in parallel.
* add in/out degree(s).
* make immutable graph index backend-specific.
* implement has_edge(s) and edge_id(s).
* Revert "use spmv in gcn."
This reverts commit 9cfed5f5fb3dd2ed9b98745348b1c0e9731ed7f7.
* implement node_subgraphs in GraphIndex.
* fix sse.
* address comments.
* address comments.
* Update mxnet docker.
2018-10-26 11:28:25 -07:00
70d475813c
[BUG] Fixing networkx conversion with 0 edges ( #94 )
...
* fixing networkx conversion with 0 edges
* fixes as required
* remove obsolete comment
2018-10-21 23:15:45 -04:00
fb6be9fbd7
[GraphIndex] Create graph index directly from scipy matrix. ( #87 )
...
* create synthetic data with scipy.
* changes as comments
2018-10-18 16:39:45 -04:00
bc3f852dab
[GRAPHINDEX] Multigraph support ( #79 )
...
* multigraph support on graph index
* more tests
* multigraph flag, bugfix on clear & copy
* networkx interfaces
* including graph index tests in Jenkins
* node subgraph test
* edge subgraphs
* removing duplicates in pred/succ
* more explicit test and doc
* query source and destination from edge id
* subgraphindex
* renaming has_edge to has_edge_between, apply_edges adding eid
* send_on and send_and_recv_on
* DGLGraph edge subgraph
* merged send_on and send_and_recv_on
* change request
* removing hashmap
* creating multigraph by flag; mingw support
* changes per request
* reverting networkx auto multigraph discovery
* notes on send/send_and_recv on multigraphs
* changing test reducer from sum to max
* added a fixme note in spmv scheduler
2018-10-17 21:46:10 -04:00
e7679cf2e6
implement requested changes
2018-10-13 14:24:46 +08:00
0a6d720c30
fix build failure
2018-10-10 12:43:05 +08:00
baf5906bab
(non)-backtracking line graph
2018-10-08 08:04:25 +08:00
5119a50475
sparse line graph
2018-10-08 07:38:03 +08:00
905db29286
cpp lg
2018-10-07 22:25:45 +08:00
26dcfb5bc2
Merge branch 'cpp' of https://github.com/jermainewang/dgl into cpp
...
Conflicts:
python/dgl/graph.py
2018-10-04 21:26:56 +08:00
cc372e3774
sbm mixture
2018-10-04 21:22:41 +08:00
2be55fb50a
graph batch and unbatch
2018-10-03 17:07:41 -04:00
314a75f367
WIP: batched graph
2018-09-24 17:31:50 -04:00
4aebfd7bd8
add merge graph; fix bug in from_networkx
2018-09-21 12:25:54 -04:00
7b5d4c5886
pass specialization test
2018-09-20 18:22:46 -04:00
a1038eb1ee
graph index
2018-09-19 21:02:15 -04:00
Minjie Wang