文件历史

31 次代码提交

作者 SHA1 备注 提交日期
Mufei Li 02029dce9b Sync initializers (#772) 2019-08-17 20:25:54 +08:00
Minjie Wang fc9d30fae4 [Graph] add local scope function (#735)
* add local scope function

* fix lint

* fix docstring

* change local_scope to local_var; add context manager

* address comments
2019-08-02 00:11:48 -04:00
Da Zheng cdfca992e1 [BUGFix] Improve multi-processing training (#526)
* fix.

* add comment.

* remove.

* temp fix.

* initialize for shared memory.

* fix graphsage.

* fix gcn.

* add more unit tests.

* add more tests.

* avoid creating shared-memory exclusively.

* redefine remote initializer.

* improve initializer.

* fix unit test.

* fix lint.

* fix lint.

* initialize data in the graph store server properly.

* fix test.

* fix test.

* fix test.

* small fix.

* add comments.

* cleanup server.

* test graph store with a random port.

* print.

* print to stderr.

* test1

* test2

* remove comment.

* adjust the initializer signature.
2019-05-20 14:23:19 -07:00
Da Zheng bfdd1eaa44 [Feature] Create shared memory graph store. (#468)
* accelerate gcn_ns.

* add timing.

* run infer with whole graph.

* distributed gcn_ns.

* reconstruct gcn_ns.

* minor fix.

* change graphsage_cv for numa.

* fix #OMP threads.

* accelerate graphsage_cv.

* fix a weird bug.

* add profiler in graphsage_cv.

* accelerate graphsage_cv.

manually aggregate neighbors' embeddings with pull.

* load csr directly in gcn_ns_sc.

* parallel sort for graph index.

* Revert "parallel sort for graph index."

This reverts commit 86fe2c7117fe5e56b0d481b39849c258b166945b.

* run gcn_ns_sc on GPUs.

* acc gcn_cv_sc.

* change gcn_cv for numa.

* fix gcn_cv to use numa and gpu.

* improve graphsage_cv to use numa and gpu.

* improve gcn_ns.

* improve graphsage_cv.

* init shared memory graph store.

* fix.

* enable init ndata.

* improve tests.

* add bidirectional communication.

* link to rt.

* fix compilation error.

* fix shared memory init.

* use MessageQueue for inter-process communication.

* reconstruct immutable graph csr.

* fix gcn.

* load csr to shared memory.

* fix minor bugs.

* add comments.

* refactor SharedMemory.

* fix bugs in ImmutableGraph.

* create CSR graph from shared memory.

* add more test for loading a csr graph.

* terminate graph store properly.

* allow initializing ndata in the graph store server.

* use RPC for inter-process communication.

* a script for loading a graph.

* allow customizing port.

* list all ndata and edata.

* support dtype.

* reorganize SharedMemoryGraphStore.

* fix ndata shape.

* reconstruct gcn_ns.

* print info.

* set omp in gcn_ns.

* reset sampling examples.

* fix lint.

* fix lint.

* reset gcn.

* disable shared memory in windows.

* fix.

* fix.

* reset changes.

* revert nodeflow changes.

* fix cmake.

* fix test.

* fix test.

* fix test.

* fix test.

* add comments.

* fix test.

* move vector out.

* fix lint.

* fix lint.

* move SharedMemory.

* update cmake.

* update comment.

* fix comments.

* Revert "update cmake."

This reverts commit 592445e37077f70a6e3f2e5245f9a3d086b04f3b.

* update cmake.

* add comments.

* rename.

* change the comment.

* fix a bug.

* rename.

* add comments.

* add comments.

* add init_edata.

* rewrite memory alloc.

* move vector to CSR.

* fix.

* init data.

* Revert "init data."

This reverts commit 2b217b9553911b7dd84a9f1d9b68430b5aa18e23.

* init data.

* init new columns correctly.
2019-04-08 12:09:02 -07:00
Minjie Wang 33b9c38389 [Bugfix] fix bug in pickling (#456)
* fix bug in pickling

* fix lint
2019-03-18 23:48:20 -04:00
brett koonce 707334ce85 minor spelling tweaks (#349)
* minor spelling tweaks

* Update CONTRIBUTORS.md
2019-01-10 12:02:31 -05:00
Quan (Andy) Gan e19cd62ecd [Test] Unify tests for different backends (#333)
* test basics

* batched graph & filter, mxnet filter fix

* frame and function; bugfix

* test graph adj and inc matrices

* fixing start = 0 for mxnet

* test index

* inplace update & line graph

* multi send recv

* more tests

* oops

* more tests

* removing old test files; readonly graphs for mxnet still kept

* modifying test scripts

* adding a placeholder for pytorch to reserve directory

* torch 0.4.1 compat fixes

* moving backend out of compute to avoid nose detection

* tests guide

* mx sparse-to-dense/sparse-to-numpy is buggy

* oops

* contribution guide for unit tests

* printing incmat

* printing dlpack

* small push

* typo

* fixing duplicate entries that causes undefined behavior

* move equal comparison to backend
2019-01-05 23:32:33 -05:00
Minjie Wang 4bd4d6e348 [Lint] Pylint (#330)
* fix lint for graph_index.py

* pylint for base.py

* pylint for batched_graph.py

* pylint for frame.py; simplify and fix bugs in frame when index is slice type

* pylint for graph.py

* pylint for immutable_graph_index.py

* pylint for init.py

* pylint for rest files in root package

* pylint for _ffi package

* pylint for function package

* pylint for runtime package

* pylint for runtime.ir package

* add pylint to ci

* fix mx tests

* fix lint errors

* fix ci

* fix as requested

* fix lint
2019-01-01 15:15:41 -05:00
Lingfan Yu 2664ed2dff [Bugfix] Fix multiple send recv (#320)
* fix bug: frame append should check itself

* more test case

* fix multi-send-recv bug

* remv msg graph and clean up

* test cases

* more test case

* fix for batchedgraph

* fix bugs: converting from a graph with edges

* fix

* add more operators to utils.Index

* clear frame executor

* change message indicator to a graph level index

* fix test cases

* guard the case that mxnet does not support concat zero shape tensor

* fix bug: avoid convert full slice to numpy

* test multi-send-recv after conversion

* fix as request (partially)

* add dtype, ctx to full_1d

* add slice data to utils.Index

* fix

* more doc string

* fix as requested
2018-12-23 21:35:49 -05:00
Lingfan Yu a1d50f0f53 [Refactor] Rename before release (#261)
* include/dgl/runtime

* include

* src/runtime

* src/graph

* src/scheduler

* src

* clean up CMakeLists

* further clean up in cmake

* install commands

* python/dgl/_ffi/_cython

* python/dgl/_ffi/_ctypes

* python/dgl/_ffi

* python/dgl

* some fix

* copy right
2018-12-05 16:45:36 -05:00
Minjie Wang 21255b6528 [Bugfix] tolist and dependencies in dgl.data (#239)
* change Index.tolist -> Index.tonumpy; fix bug in traversal; remove dependencies in data

* fix import

* fix __all__ and some docstring
2018-12-05 12:20:59 -05:00
Lingfan Yu 52ed09a392 [Bug] Fix inplace update (#221)
* inplace write row op and executor

* update scheduler and graph to use inplace write

* fix

* fix bug

* test case for inplace

* fix bugs for inplace apply node/edge

* fix comments

* th.allclose -> U.allclose
2018-12-02 19:14:39 -05:00
Minjie Wang b1eeb93449 [Bugfix][Runtime] Zero degree behaviors (#177)
* fix recv nodes are all 0deg; fix hybriddict does not through keyerror properly

* fallback to apply_nodes when all nodes are 0deg; WIP on pull spmv 0deg

* new 0deg behavior

* new 0deg behavior

* update mx utest for pull-0deg

* fix mx

* fix mx

* get rid of unnecessary sort-n-unique
2018-11-25 22:33:08 -05:00
Lingfan Yu deb653f8dc [Runtime] Scheduler and Executor (#140)
* executor api

* draft executor interface

* WIP

* revert changes to avoid conflict with api change

* core scheduling logic

* WIP: build graph adj

* incidence matrix for in edges

* support incidence matrix for partial recv nodes

* improve

* build adjmat in scheduler

* graph store

* get degree bucketing schedule

* connect to c++ degree bucketing

* conceptual executor creation code

* executor comments

* fix

* more executor comments

* WIP: full send_and_recv schedule

* most schedulers

* simplify scheduler

* executors

* runtime

* builtin function base class

* adj indices and shape

* completely refactor scheduler

* rename and move bundled out to function.py

* use_edge_feature in msg func

* rewrite scheduler

* node edge executor

* connect with graph api

* handle zero degree

* misc

* fix test cases

* fix a good many bugs...

* remove old scheduler

* push and pull

* fix send recv

* c++ lint

* fix batched send recv

* hot fix for mxnet

* typo

* write back executor

* apply node edge

* clean up, doc string

* fix as requested

* refactor

* fix

* WIP

* WIP

* ir draft

* more on ir

* WIP: spmv schedule

* WIP

* recv schedule

* refactor

* WIP

* snr degree bucketing

* snr scheduler

* move prog to graph.py; rename

* unittest for send/recv

* remove some legacy codes

* WIP: update_all

* pass test_basics

* passed all current utests

* more utests; fix mx utest

* WIP: fixing zero deg initial value

* some tests

* fix 0deg problem

* fix mx

* fix mx

* some notes

* fix as requested
2018-11-22 12:15:50 -05:00
Minjie Wang 0ec1a492b3 [Frame] change num rows behavior (#163)
* move initializer to column scheme; change num rows behavior

* poke mx ci

* fix mx utest in append

* fix bug in add edges

* utest for updating partial rows

* fix bug in from_networkx and from_scipy_matrix

* revert per-col initializer change

* fix pickle utest
2018-11-21 11:50:46 -05:00
Gan Quan e515f5311b [Frame] supporting partial row on first column and individual column initializers (#160)
addresses issue #156
2018-11-19 18:18:36 -05:00
Gan Quan 79a510252b Pickling support (#155)
* pickling support

* resorting to suggested way of pickling

* custom attribute pickling check

* working around a weird pytorch pickling bug

* including partial frame case

* pickling everything now

* fix as requested
2018-11-16 17:19:20 -05:00
Minjie Wang 048f6d7a30 [GraphIndex] refactor graph caching (#150)
* refactor graph caching

* fix mx test

* fix typo
2018-11-13 22:12:38 -05:00
Minjie Wang 7241a9c008 [Backend] backend interface (#109)
* backend interface

* small fix

* more comments to the data type dict

* WIP

* convert_to and narrow

* WIP

* pytorch and numpy backend; WIP on mxnet backend

* mxnet backend

* narrow

* Fix all usages

* fix for mx

* fix for mx

* fix mx

* fix mx

* fix mx

* fix mx

* fix mx

* fix mx

* fix mx

* revert jenkins

* add sparse_matrix api

* sparse matrix api

* some fixme

* Fix as requested
2018-11-04 21:42:39 -05:00
Gan Quan 2ecd2b23ec [Frame] Support slice type indexing; optimize dgl.batch (#110)
* cherry picking optimization from jtnn

* unbatch by slicing frames

* reduce pack

* oops

* support frame read/write with slices

* reverting to unbatch by splitting; slicing is unfriendly to backward

* replacing lru cache with static object factory

* replacing Scheme object with namedtuple

* remove comment

* forgot the find edges interface

* subclassing namedtuple
2018-11-01 00:03:06 -04:00
Mufei Li 9827e48176 Support for adding nodes/edges after setting representations (#114)
* Fix

1. Fix two typos in gcn.py and gcn_spmv.py
2. Update README

* Fix GCN module

1. Update the outdated graph convolution layer class
2. Fix a bug in the code where dropout never works. Modules like dropout/batch norm depend on whether we are in the training stage or inference stage.

* Fix a bug in dropout

1. dropout depends on nn.Module.training

* Update GCN module

* Fix README

* Fix dropout & remove self.msg_field

* Fix

* Align with TF implementation

* Make g an argument for forward

* Remove features from the argument of GraphConv layer

* Support for create nodes/edges after setting representations

* Remove redundant commit

* Delete test_init_repr.py

* Test case for dynamic addition

* Base 'add_rows' upon 'append'

* Move test function

* Fix

* test by assertion

* changed add_rows to adding blank rows only; adding convert_to to backend

* moving test to basics

* oops mxnet
2018-10-29 23:33:54 -04:00
Minjie Wang 2046980231 [DOC] API tutorials (#96)
* setup sphinx-gallery; work on graph tutorial

* draft dglgraph tutorial

* update readme to include document url

* rm obsolete file

* Draft the message passing tutorial
2018-10-23 16:27:15 -04:00
Minjie Wang b9631912db [Frame] Refactor frame. (#85)
* refactor frame codes

* fix unit test

* fix gcn example

* minor doc/message changes

* raise errors for non-exist columns in FrameRef; sanity check when append

* fix unittest; change error msg

* Add warning for none initializer

* fix unittest

* use warnings package
2018-10-18 15:08:21 -04:00
Da Zheng b0e02e5b52 Update the subgraph (#73)
* update subgraph.

* update subgraph API.

* keep node embedding.
2018-10-05 22:17:33 -04:00
Minjie Wang b2e4bdc03f passed basic test batching 2018-09-20 14:44:06 -04:00
Minjie Wang b24daa6607 change to rel import within dgl 2018-09-16 16:14:37 -04:00
Minjie Wang 61fa3c6cf5 Builtin function and API changes (#53)
* WIP: API renaming

* API rewrite and node function refactor

* builtin functions

* builtin functions tested

* fix test

* send and recv spmv test

* WIP: fix examples

* Fix examples using new APIs
2018-09-01 13:38:01 -04:00
Minjie Wang 6105e44142 Many fix and updates (#47)
* subgraph copy from

* WIP

* cached members

* Change all usage of id tensor to the new Index object; remove set device in DGLGraph;

* subgraph merge API tested

* add dict type reduced msg test
2018-08-22 12:25:05 -04:00
Minjie Wang 3721822e9e Subgraph API (#39)
* subgraph

* more test cases

* WIP

* new FrameRef and test

* separate nx init code

* WIP

* subgraph code and test

* line graph code and test

* adding new test for adding new features on line graphs

* no backtracking line graph

* fix inplace relabel
2018-08-18 14:14:43 -04:00
Minjie Wang 2c489fadec SPMV specialization (#32)
* fix edge list order problem in cached graph.

* minor fix

* fix bug in edge iter

* SPMV works

* gcn spmv on CPU

* change gcn style

* fix cached graph performance; fixed gcn dataset bug

* reorg dir

* non-batch spmv; partial update problem with shape change

* fix reorder problem; finish gcn-batch impl

* pop API

* GPU context
2018-08-06 14:00:06 -04:00
Minjie Wang 11e42d1094 Batching semantics and naive frame storage (#31)
* batch message_func, reduce_func and update_func

Conflicts:
	python/dgl/backend/pytorch.py

* test cases for batching

Conflicts:
	python/dgl/graph.py

* resolve conflicts

* setter/getter

Conflicts:
	python/dgl/graph.py

* test setter/getter

Conflicts:
	python/dgl/graph.py

* merge DGLGraph and DGLBGraph

Conflicts:
	python/dgl/graph.py

Conflicts:
	python/dgl/graph.py

* batchability test

Conflicts:
	python/dgl/graph.py

Conflicts:
	python/dgl/graph.py

* New interface (draft)

Conflicts:
	_reference/gat_mx.py
	_reference/molecular-gcn.py
	_reference/molecular-gcn_mx.py
	_reference/multi-gcn.py
	_reference/multi-gcn_mx.py
	_reference/mx.py
	python/dgl/graph.py

* Batch operations on graph

Conflicts:
	.gitignore
	python/dgl/backend/__init__.py
	python/dgl/backend/numpy.py
	python/dgl/graph.py

* sendto

* storage

* NodeDict

* DGLFrame/DGLArray

* scaffold code for graph.py

* clean up files; initial frame code

* basic frame tests using pytorch

* frame autograd test passed

* fix non-batched tests

* initial code for cached graph; tested

* batch sendto

* batch recv

* update routines

* update all

* anonymous repr batching

* specialize test

* igraph dep

* fix

* fix

* fix

* fix

* fix

* clean some files

* batch setter and getter

* fix utests
2018-07-30 14:10:37 -04:00