文件历史

提交图

1069 次代码提交

作者 SHA1 备注 提交日期
Zihao Ye 09ade2f265 upd (#1321) 2020-03-06 14:17:48 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Jinjing Zhou 64174583fe [Test] Regression test script update (#1313)
* regression test

* script
2020-03-05 01:17:02 +08:00
Minjie Wang 5dd35580f7 [Feature] Improve sampling speed; Better pickle/unpickle; other fixes (#1299)
* improve performance of sample_neighbors

* some more improve

* test script

* benchmarks

* multi process

* update more tests

* WIP

* adding two API for state saving

* add create from state

* upd test

* missing file

* wip: pickle/unpickle

* more c apis

* find the problem of empty data array

* add null array; pickling speed is bad

* still bad perf

* still bad perf

* wip

* fix the pickle speed test; now everything looks good

* minor fix

* bugfix

* some lint fix

* address comments

* more fix

* fix lint

* add utest for random.choice

* add utest for dgl.rand_graph

* fix cpp utests

* try fix ci

* fix bug in TF backend

* upd choice docstring

* address comments

* upd

* try fix compile

* add comment
2020-03-02 23:16:05 +08:00
Quan (Andy) Gan c3a33407fe [Feature] PinSAGE sampler (#1249)
* [WIP] PinSAGE operators

* moved the edge remapping mess into C

* some docstrings

* lint

* lint x2

* lint x3

* skip gpu test on topk

* extend pinsage to any metapath

* lint x4

* addresses #1265

* add always_preserve (fixes #1266) and fix a silly bug

* disable gpu test on compaction

* lint

* fix a horrible bug and add more tests

* lint

* addresses comments

* lint

* bugfix

* addresses comments

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 20:46:13 +08:00
Jinjing Zhou 87bca129a4 [CI] Regression Test Configuration (#1157)
* push

* 1111

* 111

* fix

* 222

* 1231231

* 111

* 45458643465

* fix

* fix

* fix

* 4156416

* fix

* path

* 111

* 1231

* 111

* 152554

* fix

* fix

* fix

* fix

* try fix

* change name

* try

* fix

* fix

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 14:43:32 +08:00
Jinjing Zhou ebca1188a9 [Feature] Support serialization for smart pointer (#1291)
* fix script

* t

* fix weird bugs

* fix

* fix

* upload

* fix

* fix

* lint

* fix
2020-02-26 17:29:27 +08:00
Minjie Wang 37d992ecf6 [Bugfix] Fix no attribute num_edges bug in Nodeflow (#1289)
* fix nodeflow bug when using builtin on edge data

* fix
2020-02-26 10:05:38 +08:00
Minjie Wang c7f6cf6282 [Bugfix] Consistent order for cross-type stack reducer (#1267)
* WIP

* fix shape of stack reducer

* apply merge order by edge id for stack reducer
2020-02-18 13:54:11 +08:00
Minjie Wang f8d4264e5b [Feature] Neighbor-hood based sampling APIs (#1251)
* WIP: working on random choices

* light slice

* basic CPU impl

* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph

* simple test case works

* fix bug in slicing probability array

* fix bug in getting the correct relation graph

* fix bug in creating placeholder graph

* enable omp

* add cpp test

* sample topk

* add in|out_subgraph

* try fix lint; passed all unittests

* fix lint

* fix msvc compile; add sorted flag and constructors

* fix msvc

* coosort

* COOSort; CSRRowWiseSampling; CSRRowWiseTopk

* WIP: remove DType in CSR and COO; Restrict data array to be IdArray

* fix all CSR ops for missing data array

* compiled

* passed tests

* lint

* test sampling out edge

* test different per-relation fanout/k values

* fix bug in random choice

* finished cpptest

* fix compile

* Add induced edges

* add check

* fixed bug in sampling on hypersparse graph; add tests

* add ascending flag

* in|out_subgraph returns subgraph and induced eid

* address comments

* lint

* fix
2020-02-17 21:25:21 +08:00
Quan (Andy) Gan c7c0fd0e37 [Feature] Accessing node/edge types in UDFs. (#1243)
* typed udf

* docstrings
2020-02-17 16:02:56 +08:00
Jinjing Zhou 032a08cc32 [Test] Fix test script (#1257)
* fix script

* t

* fix weird bugs

* fix

* fix
2020-02-13 18:09:15 +08:00
Jinjing Zhou 23893bb0a8 [Data format] Serialization for Immutable Graph and HeteroGraph (#1254)
* graph format

* fix lint

* lint

* fix

* unit test

* lint

* add magic num

* move serialize out of struct

* lint

* serialize

* trigger CI

* fix lint

* lint

Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com>
2020-02-11 00:14:31 +08:00
xiang song(charlie.song) ffe5898317 [Optimization][KG] Several optimizations on DGL-KG (#1233)
* Several optimizations on DGL-KG:
1. Sorted positive edges for sampling which can reduce random
   memory access during positive sampling
2. Asynchronous node embedding update
3. Balanced Relation Partition that gives balanced number of
   edges in each partition. When there is no cross partition
   relation, relation embedding can be pin into GPU memory
4. tunable neg_sample_size instead of fixed neg_sample_size

* Fix test

* Fix test and eval.py

* Now TransR is OK

* Fix single GPU with mix_cpu_gpu

* Add app tests

* Fix test script

* fix mxnet

* Fix sample

* Add docstrings

* Fix

* Default value for num_workers

* Upd

* upd
2020-02-09 13:01:00 -08:00
Jinjing Zhou f103bbf9ee [Data format] Serialization for UnitGraph (#1242)
* graph format

* fix lint

* lint

* fix

* unit test

* lint

* add magic num

* move serialize out of struct

* lint

Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com>
2020-02-08 20:49:34 +08:00
Quan (Andy) Gan eeeb52f464 [Feature] Preference to COO for "hypersparse" unit graphs & graph compaction (#1238)
* unit graph that prefers coo queries

* auto detect coo preference

* forgot some functions

* disable lint on detect_prefer_coo

* reorg

* change comment

* lint

* fix

* move array_utils.h to src

* compact graph impl

* fix redundant copying in idhashmap

* docstring

* moving preference detection to C

* lint

* fix unit test & address comments

* hypersparse autorestrict

* docstring & fix

* revert copyto and asnumbits

* fix stupid bug

* lint

* leave a TODO for sorted COO

* fixing same node type mapping to different id in different graphs

* addresses comments

* made induced nodes a feautre column

* lint?
2020-02-04 22:39:28 +08:00
Mufei Li 828a5e5bc6 [DGL-LifeSci] Migration and Refactor (#1226)
* First commit

* Update

* Update splitters

* Update

* Update

* Update

* Update

* Update

* Update

* Migrate ACNN

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Finish classification

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* trigger CI

* Fix CI

* Update

* Update

* Update

* Add default values

* Rename

* Update deprecation message
2020-02-04 01:38:09 +08:00
Quan (Andy) Gan a9dabcc769 [Feature] Random Walk for 0.5 (#1209)
* trying to refactor IndexSelect

* partial implementation

* add index select and assign for floats as well

* move to random choice source

* more updates

* fixes

* fixes

* more fixes

* adding python impl

* fixes

* unit test

* lint

* lint x2

* lint x3

* update metapath2vec

* debugging performance

* still debugging for performance

* tuning

* switching to succvec

* redo

* revert non-uniform sampler to use vector

* still not fast

* why does this crash with OpenMP???

* because there was a data race!!!

* add documentations and remove assign op

* lint

* lint x2

* lol what have i done

* lint x3

* fix and disable gpu testing

* bugfix

* generic random walk

* reorg the random walk source code

* Update randomwalks.h

* Update randomwalks_cpu.cc

* rename file

* move internal function to anonymous ns

* reorg & docstrings

* constant restart probability

* docstring fix

* more commit

* random walk with restart, tested

* some fixes

* switch to using NDArray for choice

* massive fix & docstring

* lint x?

* lint x??

* fix

* export symbols

* skip gpu test

* addresses comments

* replaces another VecToIdArray

* add randomwalks.h to include

* replace void * with template
2020-01-31 19:52:08 +08:00
VoVAllen a00636a02b [Tensorflow] Several nn & example (#1191)
* several nn example

* appnp

* fix lint

* lint

* add dgi

* fix

* fix

* fix

* fff

* docs

* 111

* fix

* change init

* change result

* tiaocan+1

* fix

* fix lint

* fix

* fix
2020-01-19 22:48:40 +08:00
xiang song(charlie.song) 31a7d50964 [Bug Fix] Add boundary check for heterograph build with card as input (#1192)
* Add boundary check for heterograph build with card as input

* Fix when u or v is empty

* fix test_kernel.py error print

* Revert "fix test_kernel.py error print"

This reverts commit a71c20292549c0ac62a5326c30669ca4bde8febc.

* Turn op validation check on graph and bipartite by default

* upd

* udp

* upd

* update test
2020-01-18 17:18:40 +08:00
xiang song(charlie.song) c49582c996 [Test] Fix the random seed in test_kernel.py and fix the _print_error (#1194)
* Fix the random seed and fix the _print_error

* Fix check_positive_edge_sampler sample weight.

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-17 13:48:47 +08:00
Quan (Andy) Gan 86fe58ebb8 [Bug] Another fix on PyTorch memory leakage issue (#1139)
* another fix

* another try

* fix

* rewriting with kernel functions

* revert mxnet softmax changes

* lint fix

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
2020-01-14 22:54:19 +08:00
xiang song(charlie.song) cd9599ab04 [KG][Optimization] Remove copy from parent in minibatch generation (#1193)
* remote copy from parent

* upd

* make test_sampler easier to pass
2020-01-10 15:59:09 -08:00
Quan (Andy) Gan 28a24414c8 [BUG] Fix empty heterograph creation with list & tuple (#1188)
* [BUG] Fix empty heterograph creation with list & tuple

* more fix
2020-01-09 22:40:27 +08:00
Quan (Andy) Gan 3ef757dbff [Feature] Pickle support for heterograph (#1177)
* [Feature] Pickle support for heterograph

* fix lint

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 13:27:29 +08:00
Zihao Ye ca302a13ac [Feature] Add dgl.nn.*.Sequential for usability (#1166)
* upd

* upd

* upd

* upd

* lint

* upd

* upd

* upd

* upd

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 12:39:22 +08:00
Zihao Ye f1420d194f [Refactor] Renaming class methods of sampler utilities to improve readability (#1180)
* upd

* upd
2020-01-07 22:02:44 +08:00
Da Zheng 1022d5d64d [KG] PBG's way of constructing negative edges (#1159)
* attach positive.

* add neg_deg_sample.

* add comment.

* add neg_deg_sample for eval.

* change the edge sampler.

* rename edge sampler in KG.

* allow specifying chunk size and negative sample size separately.

* fix bugs in KG.

* add check in sampler.

* add more checks.

* fix

* add comment.

* add comments.
2020-01-04 19:25:06 -08:00
Zihao Ye dfb10db86f [Feature] Bag of samplers for efficient categorical sampling w/ and w/o replacement (#1142)
* upd

* upd

* lint

* upd

* upd

* lint

* upd

* refactor

* lint

* upd

* upd

* upd

* lint

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* complete test

* upd

* upd

* fix

* vir

* upd

* fix

* fuck

* fix numerical
2020-01-01 11:42:11 +08:00
xiang song(charlie.song) 61b78e6ee7 [Bug Fix]edge sample hotfix (#1152)
* hot fix

* Fix docs

* Fix ArrayHeap float overflow bug

* Fix

* Clean some dead code

* Fix

* FIx

* Add some comments

* run test
2019-12-30 19:42:57 -08:00
Chao Ma 4f02bb751b [KVStore] Made kvstore can support multiple NICs (#1150)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update
2019-12-30 15:17:31 +08:00
xiang song(charlie.song) 913e324903 Add device check for sampler input (#1145)
current samplers only support working on CPU
2019-12-27 22:50:52 +08:00
xiang song(charlie.song) 41f8a162f4 [Feature Improvement]Edge Sampler w and w/o replacement (#1116)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description

* add replacement for edge sampler

* Now edge sampler support replacement and no-replacement

* Fix

* Fix

* change kg/app to use edge sampler with replacement config

* Update replacement algo

* Fix syntax

* Update

* Update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2019-12-24 16:25:40 +08:00
Da Zheng e890a899fc Partition a graph with HALO nodes (#1076)
* get subgraph with halo.

* add partition function.

* add comment.

* parallel partition.

* fix a compilation error.

* fix lint error.

* address comments.

* add comments.

* fix for TF.
2019-12-23 08:46:53 -08:00
VoVAllen 0d9acc9d58 hot fix (#1135) 2019-12-24 00:19:25 +08:00
VoVAllen a1b28b3954 [Hot fix] Fix build threads (#1133)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image

* try node

* try

* fix

* try new ci

* try new ci

* try new ci

* try new ci

* ci

* try

* try

* fix

* hot fix

* fix

* fix cpp test

* add comments
2019-12-23 21:42:59 +08:00
VoVAllen 7b98e899f3 [CI] Try new CI (#1124)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image

* try node

* try

* fix

* try new ci

* try new ci

* try new ci

* try new ci

* ci

* try

* try

* fix
2019-12-23 17:28:06 +08:00
VoVAllen d30a69bf27 [Backend] TF backend (#978)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image
2019-12-20 15:56:51 +08:00
xiang song(charlie.song) 632a9af8f9 [Feature] No-Uniform Edge Sampler (#1087)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description
2019-12-16 11:26:34 -08:00
VoVAllen dd65ee211e [CI] Change tests for flexibility
* change ci image

* fix

* force bash

* fix

* fix python version

* fix

* fix

* fix

* update gpu

* cuda

* jenkins

* fix build sh

* fix

* Revert "fix"

This reverts commit 6b091914b3ef6b4300fa662ee375aa4b27944f5c.

* try fix

* fix

* Revert "fix"

This reverts commit e42c3035fa4974c6b197aaba0748f7b45113ddaa.

* try fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix tests

* try fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix ctx problem

* fix many tests

* fix typo

* add backend

* move to pytorch folder

* fix?

* fix ci

* try skip

* try false

* try?

* try?

* Revert to 7d9a52f

* fix various

* fix lint

* Revert Jenkinsfile

* revert env

* revert env

* address comment

* remove file
2019-12-16 14:54:37 +08:00
Quan (Andy) Gan 6ae93e5c6a [Bug] Fix #1088 (#1089)
* [Bug] Fix #1088

* fix

* add comment
2019-12-11 10:27:57 +08:00
Zihao Ye fa0ee46a01 [hotfix] node id validity check (#1073)
* fix

* improve

* fix lint

* upd

* fix

* upd
2019-12-06 01:36:41 +08:00
Quan (Andy) Gan 365d361725 [Bug] Fix #1036 (#1037)
* fix

* unit test
2019-11-29 11:53:29 +08:00
Zhang Zhi 4f6063124b [Graph]To(device) should return updated object (#1057)
* To(device) should return updated object

* Add test case and doc

* Add another doc

* Add descriptions  and a test

* Pass test
2019-11-29 00:15:22 +08:00
Mufei Li ead64de98e [Hetero] Batching/Unbatching DGLHeteroGraph (#1017)
* Update

* Update

* Update

* Fix

* CI style fix

* CI fix style

* Fix

* Try CI

* Fix test

* Update

* Update

* Update

* Update
2019-11-26 14:40:50 +08:00
Mufei Li 6694f7b945 [Bug] Fix DGLHeteroGraph.edge_type_subgraph (#1040)
* Update

* Try CI
2019-11-25 03:35:25 +08:00
Chao Ma a0193fd539 Small change for kvstore api (#981)
* Small change for kvstore api

* fix ci

* fix ci
2019-11-08 16:25:12 +08:00
Chao Ma 0b4935d4c3 Test shared-mem on kvstore (#976) 2019-11-05 17:09:35 +08:00
Chao Ma 7897fa3b9c add init api on kvserver (#975) 2019-11-05 16:21:35 +08:00
Chao Ma cccde032f4 [kvstore] Performance improvement for distributed kvstore (#972)
* Performance improvment for distributed kvstore

* update

* update
2019-11-04 18:27:00 +08:00