文件历史

89 次代码提交

作者 SHA1 备注 提交日期
Da Zheng 7c47d8c9b4 [Feature] Simplify shared memory graph index (#1381)
* simplify shared memory graph index.

* fix.

* remove edge_dir in SharedMemGraphStore.

* avoid creating shared-mem graph store with from_csr.

* simplify from_csr.

* add comments.

* fix lint.

* remove the test.

* fix compilation error.

* fix a bug.

* fix a bug.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-04-05 15:33:47 -07:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Chao Ma d98c71ef90 [DGL-KE] Clean up argument for DGL-KE (#1325)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* clean up args

* change args for eval

* update

* update

* update

* update num_worker=8

* add num_proc=1

* update

* update

* add eval percent

* update

* update

* update

* save emb

* update

* update

* update

* update

* update

* remove neg_chunk_size

* update

* update

* update

* remove

* update

* remove

* change num_thread
2020-03-09 23:11:56 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Jinjing Zhou c23a61bd5c fix s3 link (#1310) 2020-03-04 21:01:29 +08:00
Quan (Andy) Gan 1c4bfb62bb [Performance] Replace np.array with np.asarray (#1301)
* replace np.array with np.asarray

* fix

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-03 20:45:41 +08:00
Chao Ma 00ba409440 [DGL-KE] Distributed training of DGL-KE (#1290)
* update

* change name

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* change worker number

* update

* update

* update

* update

* update

* update

* test

* update

* update

* update

* remove barrier

* max_step

* update

* add complex

* update

* chmod +x

* update

* update

* random partition

* random partition

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* change num_test_proc

* update num_thread

* update
2020-03-02 14:17:54 +08:00
Chao Ma ddf8c858e1 [KVStore] Small fix on kvstore (#1284)
* update

* str to int

* update

* update

* update

* test remove sync

* update

* update

* update

* update

* update

* update

* update

* update

* clear
2020-02-23 15:43:34 +08:00
Chao Ma b133abb82e [KVStore] New kvstore used by DGL-KE (#1263)
* new kvstore

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* test warning

* update

* update

* udpate

* update

* update

* update

* update

* small fix

* small fix

* get group count

* update

* update

* make file

* update

* use addr

* get id

* partition book

* update

* partition

* barrier

* update

* loop count

* update

* update

* update

* update

* update

* update

* update

* update

* update

* add mxnet demo

* update ip

* update

* update

* update

* random

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* fix lint
2020-02-17 13:39:13 +08:00
Mufei Li 828a5e5bc6 [DGL-LifeSci] Migration and Refactor (#1226)
* First commit

* Update

* Update splitters

* Update

* Update

* Update

* Update

* Update

* Update

* Migrate ACNN

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Finish classification

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* trigger CI

* Fix CI

* Update

* Update

* Update

* Add default values

* Rename

* Update deprecation message
2020-02-04 01:38:09 +08:00
Da Zheng c167373e1e [Doc] Fix the doc of NeighborSampler. (#1236)
* fix.

* fix.
2020-01-31 13:36:17 -08:00
Chao Ma 38d292e541 [KVStore] Fix memory leak bug. (#1174)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint

* delete msg

* clear kv msg

* update

* update

* update

* update

* update

* update

* is not None

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-01-06 19:47:54 +08:00
Da Zheng 1022d5d64d [KG] PBG's way of constructing negative edges (#1159)
* attach positive.

* add neg_deg_sample.

* add comment.

* add neg_deg_sample for eval.

* change the edge sampler.

* rename edge sampler in KG.

* allow specifying chunk size and negative sample size separately.

* fix bugs in KG.

* add check in sampler.

* add more checks.

* fix

* add comment.

* add comments.
2020-01-04 19:25:06 -08:00
xiang song(charlie.song) 61b78e6ee7 [Bug Fix]edge sample hotfix (#1152)
* hot fix

* Fix docs

* Fix ArrayHeap float overflow bug

* Fix

* Clean some dead code

* Fix

* FIx

* Add some comments

* run test
2019-12-30 19:42:57 -08:00
Chao Ma 6731ea3af7 [KVStore] Reduce memory cost of kvstore (#1156)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint
2019-12-30 20:25:32 +08:00
Chao Ma 4f02bb751b [KVStore] Made kvstore can support multiple NICs (#1150)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update
2019-12-30 15:17:31 +08:00
Chao Ma d57ff78da1 [Small Fix] Change default message queue size of communicator to 2GB (#1140)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1
2019-12-25 17:03:52 +08:00
Chao Ma e4ef8d1a80 [KVStore] API change of kvstore (#1058)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument
2019-12-24 17:05:15 +08:00
xiang song(charlie.song) 41f8a162f4 [Feature Improvement]Edge Sampler w and w/o replacement (#1116)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description

* add replacement for edge sampler

* Now edge sampler support replacement and no-replacement

* Fix

* Fix

* change kg/app to use edge sampler with replacement config

* Update replacement algo

* Fix syntax

* Update

* Update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2019-12-24 16:25:40 +08:00
VoVAllen d30a69bf27 [Backend] TF backend (#978)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image
2019-12-20 15:56:51 +08:00
xiang song(charlie.song) 632a9af8f9 [Feature] No-Uniform Edge Sampler (#1087)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description
2019-12-16 11:26:34 -08:00
Da Zheng adea4ba1d0 fix doc. (#1100) 2019-12-13 05:24:37 -08:00
Chao Ma a0193fd539 Small change for kvstore api (#981)
* Small change for kvstore api

* fix ci

* fix ci
2019-11-08 16:25:12 +08:00
Chao Ma 7897fa3b9c add init api on kvserver (#975) 2019-11-05 16:21:35 +08:00
Chao Ma cccde032f4 [kvstore] Performance improvement for distributed kvstore (#972)
* Performance improvment for distributed kvstore

* update

* update
2019-11-04 18:27:00 +08:00
Chao Ma ff9f67ee0d Send any shape of tensor rather than matrix (#942) 2019-10-21 17:27:03 +08:00
Da Zheng 15b951d4c4 [KG][Model] Knowledge graph embeddings (#888)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test for mxnet TAGCov

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix

* init version.

* change default value of regularization.

* avoid specifying adversarial_temperature

* use default eval_interval.

* remove original model.

* remove optimizer.

* set default value of num_proc

* set default value of log_interval.

* don't need to set neg_sample_size_valid.

* remove unused code.

* use uni_weight by default.

* unify model.

* rename model.

* remove unnecessary data sampler.

* remove the code for checkpoint.

* fix eval.

* raise exception in invalid arguments.

* remove RowAdagrad.

* remove unsupported score function for now.

* Fix bugs of kg
Update README

* Update Readme for mxnet distmult

* Update README.md

* Update README.md

* revert changes on dmlc

* add tests.

* update CI.

* add tests script.

* reorder tests in CI.

* measure performance.

* add results on wn18

* remove some code.

* rename the training script.

* new results on TransE.

* remove --train.

* add format.

* fix.

* use EdgeSubgraph.

* create PBGNegEdgeSubgraph to simplify the code.

* fix test

* fix CI.

* run nose for unit tests.

* remove unused code in dataset.

* change argument to save embeddings.

* test training and eval scripts in CI.

* check Pytorch version.

* fix a minor problem in config.

* fix a minor bug.

* fix readme.

* Update README.md

* Update README.md

* Update README.md
2019-10-02 12:57:29 -07:00
Da Zheng 3e454476b6 edge sampler return head nodes and tail nodes. (#889)
* edge sampler return head nodes and tail nodes.

* edge subgraph.

* use VecToIdArray

* example.

* fix compile error.
2019-09-30 21:25:16 -07:00
Minjie Wang 1c91f460d3 [Hetero][Model] RGCN for heterogeneous input (#885)
* new hetero RGCN

* bgs running

* fix gpu

* am dataset

* fix bug in label preparation

* Fix AM training; add result

* rm sym link

* new embed layer; mutag

* mutag matched; other fix

* minor fix

* dataset refactor

* new data loading

* rm old files

* refactor

* docstring

* include literal nodes in AIFB dataset

* address comments

* docstring
2019-09-29 00:21:17 -04:00
Quan (Andy) Gan e3921d5dec [Sampler] Metapath sampler for metapath2vec (#861)
* metapath sampler

* lint & fixes

* lint x2

* lint x3

* fix windows

* remove max_cycle argument

* add todo note
2019-09-26 16:42:33 -04:00
Da Zheng 1db697ec65 Improve edge sampler (#881)
* optimizer for sampling all negative edges.

* allow to disable checking false negative.

* fix lint.

* fix docstring.

* fix for comment.

* add comments.
2019-09-25 22:56:37 -07:00
Da Zheng 2d489617dd [Feature] find the existence of negative edges. (#875)
* find the existence of negative edges.

* add comment.

* fix test.
2019-09-23 01:59:02 -07:00
Chao Ma 4cd5c19e51 use np.unique instead of loop (#858) 2019-09-16 12:01:46 +08:00
Chao Ma 7782276962 [KVStore] Distributed kvstore (#851)
* update

* speedup

* add some comments
2019-09-11 14:54:50 +08:00
Da Zheng 1c9c033919 [Doc] add figures for explanation (#807)
* fix.

* update
2019-08-29 04:05:22 +08:00
Da Zheng 30c4625171 fix. (#794) 2019-08-27 09:56:38 -07:00
Da Zheng 66971c1aa4 [Feature] Add edge sampling for link prediction (#780)
* add edge sampler.

* add test and run.

* add negative sampling.

* remap the edge subgraph vid.

* negative graph excludes edges of positive edgs.

* remove print.

* avoid sampling NodeFlow when expand_factor or num_hops is 0.

* fix a bug when excluding nodes in negative graph.

* support multigraph.

* exclude positive edges.

* fix memory leak.

* return subgraph object directly.

* fix many problems.

* add comments.

* address comments
2019-08-21 23:05:24 -07:00
Mufei Li 02029dce9b Sync initializers (#772) 2019-08-17 20:25:54 +08:00
Quan (Andy) Gan 16061925d5 [NodeFlow] Non-uniform neighbor sampling (#711)
* nonuniform sampler

* unit test

* test on out neighbors

* error checks

* lint

* fix

* clarification

* use macro switcher

* use empty array for uniform sampling

* oops

* Revert "oops"

This reverts commit a11f9ae707aaeb67fb5921c887a17d3711d5b04a.

* Revert "use empty array for uniform sampling"

This reverts commit 8526ce4cade89f2c1b09a08aca8830375ebafb31.

* re-reverting

* use a method
2019-08-07 09:42:08 +08:00
Chao Ma c3516f1a8e [Network] Refactoring Communicator (#679)
* Refactoring Communicator

* fix lint

* change non-const reference

* add header file

* use MemoryBuffer

* update PR

* fix bug on csr shape

* zero-copy msg_queue

* fix lint

* fix lint

* fix lint

* add header file

* fix windows build error

* fix windows build error

* update

* fix lint

* update

* fix lint

* fix lint

* add more test

* fix windows test

* update windows test

* update windows test

* update windows test

* update

* fix lint

* fix lint

* update

* update

* update

* update

* use STATUS code

* update test

* remove mem_cpy

* fix lint

* update

* finish

* ConstructNFTensor

* add test for deallocator

* update

* fix lint
2019-08-05 10:49:25 +08:00
Quan (Andy) Gan e9e587b6cb [Refactor] C random number generator (#729)
* rng refactor

* fix bugs

* unit test

* remove setsize

* lint

* fix test

* use explicit instantiation instead of inlining

* stricter test

* use tvm solution

* moved python interface to dgl.random

* lint

* address comments

* make getthreadid an inline function
2019-07-31 17:54:30 +08:00
Minjie Wang 67dc11971f [Refactor] Use object system for all CAPIs (#716)
* WIP: using object system for graph

* c++ side refactoring done; compiled

* remove stale apis

* fix bug in DGLGraphCreate; passed test_graph.py

* fix bug in python modify; passed utest for pytorch/cpu

* fix lint

* address comments
2019-07-23 12:47:05 -04:00
Da Zheng 8059fa6f8b print errors if users sample from mutable graph. (#649)
* give warning.

* fix.

* update error msg.
2019-06-12 17:54:02 +08:00
Da Zheng 94ecb8eb96 [BUGFIX] copy graph index to shared memory. (#634)
* copy graph index to shared memory.

* fix.

* fix.

* fix.

* use a diff name for in-csr and out-csr.

* fix lint.

* remove print.

* add test.

* add comments.
2019-06-12 01:32:25 -07:00
Da Zheng 16ec2a8b10 we need to sync for mxnet. (#648) 2019-06-11 18:22:21 -07:00
Da Zheng 411bcd7e6c [BUGFIX] extand DGLGraph graph_data. (#632)
* enable tutorial test in CI.

* extand DGLGraph graph_data.

* update doc.

* Revert "enable tutorial test in CI."

This reverts commit cd774067180922bb6ae979bde4aecbffc61c8147.

* accept DGLGraph in graph store.
2019-06-09 17:22:04 -07:00
Da Zheng 2e9949d228 [BUGFIX] Fix bugs in shared mem graph store. (#630)
* fix graph store for Pytorch.

* add test.

* fix dtype error in test

* disable test on GPU.

* test avoid windows.

* fix shared-memory test.

* use script to control testing environment.

* update test.

* enable all tests.

* fix test script.
2019-06-09 13:08:47 -07:00
Da Zheng 5da57663cc [BUGFIX] fix sampler. (#616)
* fix sampler.

* update doc.

* fix.
2019-06-06 20:47:59 -07:00
Minjie Wang 01a4cc5b7b [Graph] Add API to convert graph to simple graph (#587)
* to simple

* WIP: multigraph flag

* graph index refactor; pass basic testing

* graph index refactor; pass basic testing

* fix bug in to_simple; pass torch test

* fix mx utest

* fix example

* fix lint

* fix ci

* poke ci

* poke ci

* WIP

* poke ci

* poke ci

* poke ci

* change ci workspace

* poke ci

* poke ci

* poke ci

* poke ci

* delete ci

* use enum for multigraph flag
2019-06-01 23:32:46 -04:00
Da Zheng 372203f04a [Test] Fix tests in test_shared_mem_store. (#588)
* fix test.

* better assert.

* more asserts.

* print to stderr.

* destroy g.

* fix tests.

* add timeout in sync_barrier.

* test _sync_barrier.

* fix.

* avoid printing messages.

* fix test.

* fix test.

* fix.
2019-06-01 21:18:01 -04:00