文件历史

2070 次代码提交

作者 SHA1 备注 提交日期
Da Zheng 0e153c4baf [Feature] add metis partitioning to DGL (#1308)
* add metis.

* add test.

* construct partition id.

* link to METIS github repo.

* update metis.

* add a tool for partitioning a graph.

* update metis.

* update.

* update.

* fix metis.

* fix lint

* fix indent.

* another way of building metis.

* disable metis in windows.

* test windows

* fix.

* disable metis for windows properly.

* fix for tensorflow.

* skip test for gpu.

* make graph symmetric

* address comments.

* more comments.

* fix compile

* fix a bug.

* add test.

* change the default #hops of HALO nodes.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
2020-03-07 23:24:03 -08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Quan (Andy) Gan ce6e19f233 [Bugfix] Allows node types with no inbound/outbound edges (#1323)
* add num nodes in ctors

* fix

* lint

* addresses comments

* replace with constexpr

* remove function with rvalue reference

* address comments
2020-03-08 01:11:44 +08:00
Zihao Ye 09ade2f265 upd (#1321) 2020-03-06 14:17:48 +08:00
Jinjing Zhou 978ac41097 [NN] Fix RelGraphConv num_bases error message (#1309)
* Fix RelGraphConv when num_bases is None

* Improve error message in when num_bases is wrong

* Fix integer modulo by zero exception when num_bases is 0

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-06 01:00:48 +08:00
Jinjing Zhou e2db791312 [Feature] Import subpackge based on backend of dgl.nn (#1305)
* automatically select nn backend

* rename load_backend to private

* switch to backend_name

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-05 19:59:45 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Da Zheng ce93330e34 [Doc] Fix the docstring of builtin functions. (#1303)
* fix.

* fix.

* fix.

* fix cmake.
2020-03-04 11:38:05 -08:00
Jinjing Zhou c23a61bd5c fix s3 link (#1310) 2020-03-04 21:01:29 +08:00
Quan (Andy) Gan 1c4bfb62bb [Performance] Replace np.array with np.asarray (#1301)
* replace np.array with np.asarray

* fix

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-03 20:45:41 +08:00
Minjie Wang 5dd35580f7 [Feature] Improve sampling speed; Better pickle/unpickle; other fixes (#1299)
* improve performance of sample_neighbors

* some more improve

* test script

* benchmarks

* multi process

* update more tests

* WIP

* adding two API for state saving

* add create from state

* upd test

* missing file

* wip: pickle/unpickle

* more c apis

* find the problem of empty data array

* add null array; pickling speed is bad

* still bad perf

* still bad perf

* wip

* fix the pickle speed test; now everything looks good

* minor fix

* bugfix

* some lint fix

* address comments

* more fix

* fix lint

* add utest for random.choice

* add utest for dgl.rand_graph

* fix cpp utests

* try fix ci

* fix bug in TF backend

* upd choice docstring

* address comments

* upd

* try fix compile

* add comment
2020-03-02 23:16:05 +08:00
Chao Ma 00ba409440 [DGL-KE] Distributed training of DGL-KE (#1290)
* update

* change name

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* change worker number

* update

* update

* update

* update

* update

* update

* test

* update

* update

* update

* remove barrier

* max_step

* update

* add complex

* update

* chmod +x

* update

* update

* random partition

* random partition

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* change num_test_proc

* update num_thread

* update
2020-03-02 14:17:54 +08:00
Quan (Andy) Gan c3a33407fe [Feature] PinSAGE sampler (#1249)
* [WIP] PinSAGE operators

* moved the edge remapping mess into C

* some docstrings

* lint

* lint x2

* lint x3

* skip gpu test on topk

* extend pinsage to any metapath

* lint x4

* addresses #1265

* add always_preserve (fixes #1266) and fix a silly bug

* disable gpu test on compaction

* lint

* fix a horrible bug and add more tests

* lint

* addresses comments

* lint

* bugfix

* addresses comments

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 20:46:13 +08:00
Minjie Wang 37d992ecf6 [Bugfix] Fix no attribute num_edges bug in Nodeflow (#1289)
* fix nodeflow bug when using builtin on edge data

* fix
2020-02-26 10:05:38 +08:00
Chao Ma ddf8c858e1 [KVStore] Small fix on kvstore (#1284)
* update

* str to int

* update

* update

* update

* test remove sync

* update

* update

* update

* update

* update

* update

* update

* update

* clear
2020-02-23 15:43:34 +08:00
Minjie Wang c7f6cf6282 [Bugfix] Consistent order for cross-type stack reducer (#1267)
* WIP

* fix shape of stack reducer

* apply merge order by edge id for stack reducer
2020-02-18 13:54:11 +08:00
Minjie Wang 537d37c26c [Misc] add requests as the package dependency (#1268) 2020-02-17 23:45:17 +08:00
Minjie Wang f8d4264e5b [Feature] Neighbor-hood based sampling APIs (#1251)
* WIP: working on random choices

* light slice

* basic CPU impl

* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph

* simple test case works

* fix bug in slicing probability array

* fix bug in getting the correct relation graph

* fix bug in creating placeholder graph

* enable omp

* add cpp test

* sample topk

* add in|out_subgraph

* try fix lint; passed all unittests

* fix lint

* fix msvc compile; add sorted flag and constructors

* fix msvc

* coosort

* COOSort; CSRRowWiseSampling; CSRRowWiseTopk

* WIP: remove DType in CSR and COO; Restrict data array to be IdArray

* fix all CSR ops for missing data array

* compiled

* passed tests

* lint

* test sampling out edge

* test different per-relation fanout/k values

* fix bug in random choice

* finished cpptest

* fix compile

* Add induced edges

* add check

* fixed bug in sampling on hypersparse graph; add tests

* add ascending flag

* in|out_subgraph returns subgraph and induced eid

* address comments

* lint

* fix
2020-02-17 21:25:21 +08:00
Quan (Andy) Gan c7c0fd0e37 [Feature] Accessing node/edge types in UDFs. (#1243)
* typed udf

* docstrings
2020-02-17 16:02:56 +08:00
Chao Ma b133abb82e [KVStore] New kvstore used by DGL-KE (#1263)
* new kvstore

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* test warning

* update

* update

* udpate

* update

* update

* update

* update

* small fix

* small fix

* get group count

* update

* update

* make file

* update

* use addr

* get id

* partition book

* update

* partition

* barrier

* update

* loop count

* update

* update

* update

* update

* update

* update

* update

* update

* update

* add mxnet demo

* update ip

* update

* update

* update

* random

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* fix lint
2020-02-17 13:39:13 +08:00
xiang song(charlie.song) 77ae4d5321 [Example] Fix Rgcn hetero nonedetermistic in graph and model initialization (#1244)
* Fix model save and load problem.

Add offline inference example

* Fix test

* Fix randomness problem between test runs

* Make rdf loader deterministic
2020-02-07 11:07:09 +08:00
Quan (Andy) Gan eeeb52f464 [Feature] Preference to COO for "hypersparse" unit graphs & graph compaction (#1238)
* unit graph that prefers coo queries

* auto detect coo preference

* forgot some functions

* disable lint on detect_prefer_coo

* reorg

* change comment

* lint

* fix

* move array_utils.h to src

* compact graph impl

* fix redundant copying in idhashmap

* docstring

* moving preference detection to C

* lint

* fix unit test & address comments

* hypersparse autorestrict

* docstring & fix

* revert copyto and asnumbits

* fix stupid bug

* lint

* leave a TODO for sorted COO

* fixing same node type mapping to different id in different graphs

* addresses comments

* made induced nodes a feautre column

* lint?
2020-02-04 22:39:28 +08:00
Mufei Li 828a5e5bc6 [DGL-LifeSci] Migration and Refactor (#1226)
* First commit

* Update

* Update splitters

* Update

* Update

* Update

* Update

* Update

* Update

* Migrate ACNN

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Finish classification

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* trigger CI

* Fix CI

* Update

* Update

* Update

* Add default values

* Rename

* Update deprecation message
2020-02-04 01:38:09 +08:00
Quan (Andy) Gan e4948c5cbc fix regression in #1237 (#1239) 2020-02-03 11:50:06 +08:00
Da Zheng c167373e1e [Doc] Fix the doc of NeighborSampler. (#1236)
* fix.

* fix.
2020-01-31 13:36:17 -08:00
Quan (Andy) Gan a9dabcc769 [Feature] Random Walk for 0.5 (#1209)
* trying to refactor IndexSelect

* partial implementation

* add index select and assign for floats as well

* move to random choice source

* more updates

* fixes

* fixes

* more fixes

* adding python impl

* fixes

* unit test

* lint

* lint x2

* lint x3

* update metapath2vec

* debugging performance

* still debugging for performance

* tuning

* switching to succvec

* redo

* revert non-uniform sampler to use vector

* still not fast

* why does this crash with OpenMP???

* because there was a data race!!!

* add documentations and remove assign op

* lint

* lint x2

* lol what have i done

* lint x3

* fix and disable gpu testing

* bugfix

* generic random walk

* reorg the random walk source code

* Update randomwalks.h

* Update randomwalks_cpu.cc

* rename file

* move internal function to anonymous ns

* reorg & docstrings

* constant restart probability

* docstring fix

* more commit

* random walk with restart, tested

* some fixes

* switch to using NDArray for choice

* massive fix & docstring

* lint x?

* lint x??

* fix

* export symbols

* skip gpu test

* addresses comments

* replaces another VecToIdArray

* add randomwalks.h to include

* replace void * with template
2020-01-31 19:52:08 +08:00
Tong He f1a8f926a4 [Feature] Add environment variable to switch on/off MXNet set_np_shape (#1207)
* add env var

* Trigger CI

* simplification

* add doc
2020-01-20 13:54:14 +08:00
VoVAllen a00636a02b [Tensorflow] Several nn & example (#1191)
* several nn example

* appnp

* fix lint

* lint

* add dgi

* fix

* fix

* fix

* fff

* docs

* 111

* fix

* change init

* change result

* tiaocan+1

* fix

* fix lint

* fix

* fix
2020-01-19 22:48:40 +08:00
xiang song(charlie.song) 31a7d50964 [Bug Fix] Add boundary check for heterograph build with card as input (#1192)
* Add boundary check for heterograph build with card as input

* Fix when u or v is empty

* fix test_kernel.py error print

* Revert "fix test_kernel.py error print"

This reverts commit a71c20292549c0ac62a5326c30669ca4bde8febc.

* Turn op validation check on graph and bipartite by default

* upd

* udp

* upd

* update test
2020-01-18 17:18:40 +08:00
Quan (Andy) Gan 86fe58ebb8 [Bug] Another fix on PyTorch memory leakage issue (#1139)
* another fix

* another try

* fix

* rewriting with kernel functions

* revert mxnet softmax changes

* lint fix

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
2020-01-14 22:54:19 +08:00
Quan (Andy) Gan 28a24414c8 [BUG] Fix empty heterograph creation with list & tuple (#1188)
* [BUG] Fix empty heterograph creation with list & tuple

* more fix
2020-01-09 22:40:27 +08:00
Zihao Ye aa06e4579f [Doc] Improve the docstring of several nn modules. (#1187)
* upd

* upd

* upd

* upd

* lint

* upd

* upd

* upd

* upd

* upd

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 21:14:33 +08:00
Quan (Andy) Gan 3ef757dbff [Feature] Pickle support for heterograph (#1177)
* [Feature] Pickle support for heterograph

* fix lint

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 13:27:29 +08:00
Zihao Ye ca302a13ac [Feature] Add dgl.nn.*.Sequential for usability (#1166)
* upd

* upd

* upd

* upd

* lint

* upd

* upd

* upd

* upd

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 12:39:22 +08:00
VoVAllen 319118341d [Dataset] Change download repo link to https://data.dgl.ai (CDN) (#1175)
* change download link to cdn

* Update env_var.rst
2020-01-07 17:16:45 +08:00
Chao Ma 38d292e541 [KVStore] Fix memory leak bug. (#1174)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint

* delete msg

* clear kv msg

* update

* update

* update

* update

* update

* update

* is not None

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-01-06 19:47:54 +08:00
Chao Ma 6d0a9c4297 [Fix issue] Fix issue 1170 (#1171)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint
2020-01-05 14:31:41 +08:00
Da Zheng 1022d5d64d [KG] PBG's way of constructing negative edges (#1159)
* attach positive.

* add neg_deg_sample.

* add comment.

* add neg_deg_sample for eval.

* change the edge sampler.

* rename edge sampler in KG.

* allow specifying chunk size and negative sample size separately.

* fix bugs in KG.

* add check in sampler.

* add more checks.

* fix

* add comment.

* add comments.
2020-01-04 19:25:06 -08:00
xiang song(charlie.song) 61b78e6ee7 [Bug Fix]edge sample hotfix (#1152)
* hot fix

* Fix docs

* Fix ArrayHeap float overflow bug

* Fix

* Clean some dead code

* Fix

* FIx

* Add some comments

* run test
2019-12-30 19:42:57 -08:00
Chao Ma 6731ea3af7 [KVStore] Reduce memory cost of kvstore (#1156)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint
2019-12-30 20:25:32 +08:00
Chao Ma 4f02bb751b [KVStore] Made kvstore can support multiple NICs (#1150)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update
2019-12-30 15:17:31 +08:00
Quan (Andy) Gan 655d75688a fix deprecation warning (#1147) 2019-12-28 12:42:16 +08:00
Chao Ma d57ff78da1 [Small Fix] Change default message queue size of communicator to 2GB (#1140)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1
2019-12-25 17:03:52 +08:00
Chao Ma e4ef8d1a80 [KVStore] API change of kvstore (#1058)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument
2019-12-24 17:05:15 +08:00
xiang song(charlie.song) 41f8a162f4 [Feature Improvement]Edge Sampler w and w/o replacement (#1116)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description

* add replacement for edge sampler

* Now edge sampler support replacement and no-replacement

* Fix

* Fix

* change kg/app to use edge sampler with replacement config

* Update replacement algo

* Fix syntax

* Update

* Update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2019-12-24 16:25:40 +08:00
Da Zheng e890a899fc Partition a graph with HALO nodes (#1076)
* get subgraph with halo.

* add partition function.

* add comment.

* parallel partition.

* fix a compilation error.

* fix lint error.

* address comments.

* add comments.

* fix for TF.
2019-12-23 08:46:53 -08:00
VoVAllen d30a69bf27 [Backend] TF backend (#978)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image
2019-12-20 15:56:51 +08:00
Mufei Li cf9ba90fd4 [Chem] ACNN and various utilities (#1117)
* Add several splitting methods

* Update

* Update

* Update

* Update

* Update

* Fix

* Update

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Finally

* CI
2019-12-19 11:54:40 +08:00
Zihao Ye 6beef85b07 upd (#1113) 2019-12-17 15:24:04 +08:00
xiang song(charlie.song) 632a9af8f9 [Feature] No-Uniform Edge Sampler (#1087)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description
2019-12-16 11:26:34 -08:00