文件历史

84 次代码提交

作者 SHA1 备注 提交日期
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Tong He cbee427839 [Model] Scene Graph Extraction Model with GluonCV (#1260)
* add working scripts

* add frcnn training script

* remove redundent files

* refactor validation computation, will optimize sgdet and training

* validation finally finished

* f-rcnn training

* test reldn

* rm file

* update reldn training

* data preprocess to h5

* temp

* use coco json

* fix conflict

* new obj dataset for detection

* update training

* before cleanup

* remove abundant files

* add arg parse to train

* cleanup code file

* update

* fix

* add readme

* add ipynb as demo

* add demo pic

* update readme

* add demo script

* improve paths

* improve readme

* add docstrings

* fix args description

* update readme

* add models from s3

* update README

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 11:54:26 +08:00
Chao Ma ddf8c858e1 [KVStore] Small fix on kvstore (#1284)
* update

* str to int

* update

* update

* update

* test remove sync

* update

* update

* update

* update

* update

* update

* update

* update

* clear
2020-02-23 15:43:34 +08:00
Chao Ma b133abb82e [KVStore] New kvstore used by DGL-KE (#1263)
* new kvstore

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* test warning

* update

* update

* udpate

* update

* update

* update

* update

* small fix

* small fix

* get group count

* update

* update

* make file

* update

* use addr

* get id

* partition book

* update

* partition

* barrier

* update

* loop count

* update

* update

* update

* update

* update

* update

* update

* update

* update

* add mxnet demo

* update ip

* update

* update

* update

* random

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* fix lint
2020-02-17 13:39:13 +08:00
Zihao Ye 7a80faf120 [Bugfix] Correct the loss function in RGCN model (#1217)
* upd

* upd

* fix

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-02-13 18:35:07 +08:00
Chao Ma 6731ea3af7 [KVStore] Reduce memory cost of kvstore (#1156)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint
2019-12-30 20:25:32 +08:00
Chao Ma 4f02bb751b [KVStore] Made kvstore can support multiple NICs (#1150)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update
2019-12-30 15:17:31 +08:00
Chao Ma d57ff78da1 [Small Fix] Change default message queue size of communicator to 2GB (#1140)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1
2019-12-25 17:03:52 +08:00
Chao Ma e4ef8d1a80 [KVStore] API change of kvstore (#1058)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument
2019-12-24 17:05:15 +08:00
Zihao Ye fdc58a8996 [Hotfix] Fix GAT example and clarify the usage of early stop (#1065)
* upd

* rm redundancy:

* upd
2019-11-30 23:43:15 +08:00
Chao Ma a0193fd539 Small change for kvstore api (#981)
* Small change for kvstore api

* fix ci

* fix ci
2019-11-08 16:25:12 +08:00
Chao Ma 7897fa3b9c add init api on kvserver (#975) 2019-11-05 16:21:35 +08:00
Chao Ma cccde032f4 [kvstore] Performance improvement for distributed kvstore (#972)
* Performance improvment for distributed kvstore

* update

* update
2019-11-04 18:27:00 +08:00
Zihao Ye fdd0fe651d hotfix (#971) 2019-11-04 12:11:53 +08:00
Zihao Ye 9a0511c8e9 [NN] nn modules & examples update (#890)
* upd

* damn it

* fuck

* fuck pylint

* fudge

* remove some comments about MXNet

* upd

* upd

* damn it

* damn it

* fuck

* fuck

* upd

* upd

* pylint bastard

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd
2019-11-03 17:29:42 -05:00
xiang song(charlie.song) 82499e602b [Bug Fix] Fix package reliability bug of networkx (#949)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test for mxnet TAGCov

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix

* reproduce the bug

* Fix concurrency bug reported at #755.
Also make test_shared_mem_store.py more deterministic.

* Update test_shared_mem_store.py

* Update dmlc/core

* networkx >= 2.4 will break our examples

* Update tutorials/requirements

* fix selfloop edges

* upd version
2019-10-29 23:34:31 -04:00
Chao Ma ff9f67ee0d Send any shape of tensor rather than matrix (#942) 2019-10-21 17:27:03 +08:00
Minjie Wang 9737b2700f [Model][Hetero] GCMC using new hetero APIs (#860)
* init

* rm data

* README

* fix rating values that are not decimal

* rm stale codes

* small fix

* upd

* rewrite decoder

* fix many

* many fix; performance matched

* upd; handle sparse input

* upd

* address comments

* more docstring; download data automatically

* shared param mode
2019-09-26 17:50:38 -04:00
Chao Ma 7782276962 [KVStore] Distributed kvstore (#851)
* update

* speedup

* add some comments
2019-09-11 14:54:50 +08:00
Zihao Ye 6a4b5ae98a [Feature] Edge softmax on a subset of edges in the graph. (#842)
* upd

* add test

* fix

* upd

* merge

* hotfix

* upd

* fix
2019-09-09 13:44:56 +08:00
xiang song(charlie.song) e17add5602 [NN] Add MXNet impl for TAGCN module. (#799)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix
2019-08-28 13:19:17 +08:00
Zihao Ye 9314aabd1f [Refactor] Interface of nn modules (#798)
* refactor

* upd mpnn
2019-08-27 22:29:25 +08:00
Minjie Wang 708765f0a1 [NN] RGCN modules (#744)
* rgcn module

* support id input

* WIP: model codes

* use faster index select

* dropout

* self loop

* WIP: link prediction

* fix lint

* WIP: docs

* docstring

* docstring

* merge two child classes

* mxnet rgcn module

* fix lint

* fix lint

* fix rename bug

* add uniform edge sampler

* fix fn name

* docstring

* fix mxnet rgcn module

* fix mx rgcn

* enable test on cuda
2019-08-23 16:38:48 -04:00
VoVAllen 0f12763725 [Model] Early stop GAT (#750)
* Add early stop

* add mxnet version

* Poke ci
2019-08-21 14:58:56 +08:00
Da Zheng 94ecb8eb96 [BUGFIX] copy graph index to shared memory. (#634)
* copy graph index to shared memory.

* fix.

* fix.

* fix.

* use a diff name for in-csr and out-csr.

* fix lint.

* remove print.

* add test.

* add comments.
2019-06-12 01:32:25 -07:00
Da Zheng 411bcd7e6c [BUGFIX] extand DGLGraph graph_data. (#632)
* enable tutorial test in CI.

* extand DGLGraph graph_data.

* update doc.

* Revert "enable tutorial test in CI."

This reverts commit cd774067180922bb6ae979bde4aecbffc61c8147.

* accept DGLGraph in graph store.
2019-06-09 17:22:04 -07:00
Da Zheng fc7775a20f Define node/edge Ids in NodeFlow more clearly (#628)
* add tests.

* distinguish layer-local nid and nodeflow nid.

* use numpy assert_array_equal and assert_allclose

* fix map_from_parent_nid

* fix test

* fix test.

* renmae remap.

* update doc.

* update doc.

* update doc.

* fix test.

* fix test.
2019-06-09 12:10:07 -07:00
Minjie Wang dec8b49b5d [Examples] Run all the examples (#623)
* all pytorch examples

* scan through mxnet examples

* change reddit data

* tweak numerical range for unittest

* fix ci

* fix ci

* fix

* add seed to workaround
2019-06-09 00:26:13 -04:00
Lingfan Yu 74e13eea61 [Model] Update GAT model code (#622)
* fix gat code to use latest edge softmax module

* avoid transpose

* update README

* use edge_softmax op

* mxnet edge softmax op

* mxnet gat

* update README

* fix unittest

* fix ci

* fix mxnet nn test; relax criteria for prod reducer
2019-06-08 21:41:38 -04:00
Minjie Wang 01a4cc5b7b [Graph] Add API to convert graph to simple graph (#587)
* to simple

* WIP: multigraph flag

* graph index refactor; pass basic testing

* graph index refactor; pass basic testing

* fix bug in to_simple; pass torch test

* fix mx utest

* fix example

* fix lint

* fix ci

* poke ci

* poke ci

* WIP

* poke ci

* poke ci

* poke ci

* change ci workspace

* poke ci

* poke ci

* poke ci

* poke ci

* delete ci

* use enum for multigraph flag
2019-06-01 23:32:46 -04:00
Chao Ma e0ce190dfe [Doc] Update README.MD of demo for distributed sampler (#579)
* Update README.md for the demo of distributed sampler

* Update README.MD of demo for distributed sampler
2019-05-30 01:24:13 -04:00
Da Zheng e0004f7199 [Doc] sampler readme update. (#581) 2019-05-29 23:47:57 -04:00
Chao Ma 08d4900fd1 Update README.md for the demo of distributed sampler (#577) 2019-05-28 17:24:33 +08:00
Chao Ma 9aa5ffcab1 [DEMO] Update demo of distributed sampler (#564)
* update

* update

* update demo
2019-05-26 12:10:53 +08:00
Da Zheng 8f378d90ad [BUGFIX] fix bugs for running GCN on giant graphs. (#561)
* load mxnet csr.

* enable load large csr.

* fix

* fix.

* fix int overflow.

* fix test.
2019-05-25 11:09:33 -07:00
Da Zheng 924efc6520 [Perf] Improve performance of graph store. (#554)
* fix.

* use inplace.

* move to shared memory graph store.

* fix.

* add more unit tests.

* fix.

* fix test.

* fix test.

* disable test.

* fix.
2019-05-23 12:23:31 -07:00
Chao Ma 49c4a9e4cc [DEMO] Remove duplicate code for sampling (#557)
* update

* update

* re-use single-machine code

* update

* use relative path

* update

* update

* update

* add __init__.py

* add __init__.py

* import sys, os

* fix typo

* update
2019-05-23 16:57:35 +08:00
Chao Ma 28379f927f [DEMO] Reproduce numbers of distributed training in AMLC giant graph paper (#556)
* update

* update

* update

* update num_hops

* fix bug

* update

* report numbers of distributed training in AMLC giant graph paper
2019-05-23 14:23:14 +08:00
Chao Ma f99725adbc all demo use python-3 (#555) 2019-05-23 10:38:05 +08:00
Da Zheng b2b8be25bb [API] update graph store API. (#549)
* add init_ndata and init_edata in DGLGraph.

* adjust SharedMemoryGraph API.

* print warning.

* fix comment.

* update example

* fix.

* fix examples.

* add unit tests.

* add comments.
2019-05-21 03:27:05 -07:00
Da Zheng cdfca992e1 [BUGFix] Improve multi-processing training (#526)
* fix.

* add comment.

* remove.

* temp fix.

* initialize for shared memory.

* fix graphsage.

* fix gcn.

* add more unit tests.

* add more tests.

* avoid creating shared-memory exclusively.

* redefine remote initializer.

* improve initializer.

* fix unit test.

* fix lint.

* fix lint.

* initialize data in the graph store server properly.

* fix test.

* fix test.

* fix test.

* small fix.

* add comments.

* cleanup server.

* test graph store with a random port.

* print.

* print to stderr.

* test1

* test2

* remove comment.

* adjust the initializer signature.
2019-05-20 14:23:19 -07:00
Da Zheng 3a1392e641 [Model] add multiprocessing training with sampling. (#484)
* reorganize sampling code.

* add multi-process training.

* speed up gcn_cv

* fix graphsage_cv.

* add new API in graph store.

* update barrier impl.

* support both local and distributed training.

* fix multiprocess train.

* fix.

* fix barrier.

* add script for loading data.

* multiprocessing sampling.

* accel training.

* replace pull with spmv for speedup.

* nodeflow copy from parent with context.

* enable GPU.

* fix a bug in graph store.

* enable multi-GPU training.

* fix lint.

* add comments.

* rename to run_store_server.py

* fix gcn_cv.

* fix a minor bug in sampler.

* handle error better in graph store.

* improve graphsage_cv for distributed mode.

* update README.

* fix.

* update.
2019-05-06 23:51:56 -07:00
Chao Ma 2ff8ecfbac update test_batch_size (#510) 2019-04-25 13:28:36 +08:00
Chao Ma fe7d5e9b8b [Sampler] Change Distributed Sampler API (#499)
* Change Distributed Sampler API

* fix lint

* fix lint

* update demo

* update

* update

* update

* update demo

* update demo
2019-04-22 09:41:47 +08:00
Da Zheng 6124667f42 [Model] Refactor the sampling examples (#498)
* reorganize sampling code.

* speedup gcn_ns.

* speed up gcn_cv

* fix graphsage_cv.

* undo the modification.

* accel training.

* update readme.
2019-04-17 01:31:14 -07:00
Chao Ma b89dcce16c [RPC] Refactoring networking APIs (#496)
* Refactoring network API

* update demo

* update

* update demo

* update demo

* add num_sender

* update

* fix lint

* fix lint

* fix lint

* update
2019-04-16 16:33:29 +08:00
Da Zheng 688a9228a8 fix. (#491) 2019-04-11 15:38:35 -07:00
Hao Zhang 039a711da1 [Model] fix self-edge bug in GCN and GAT. (#482)
* Update gcn_mp.py

* Update train.py

* Update train.py

* Update train.py

* Update gat_batch.py

* Update gat_batch.py

* Update gcn_mp.py
2019-04-10 09:04:19 -04:00
Da Zheng 4ea42e3e08 fix gcn. (#469) 2019-04-05 10:56:09 -07:00