文件历史

提交图

140 次代码提交

作者 SHA1 备注 提交日期
Quan (Andy) Gan 91f7ff8e52 fix #1421 (#1422) 2020-04-06 12:33:43 +08:00
Da Zheng 7c47d8c9b4 [Feature] Simplify shared memory graph index (#1381)
* simplify shared memory graph index.

* fix.

* remove edge_dir in SharedMemGraphStore.

* avoid creating shared-mem graph store with from_csr.

* simplify from_csr.

* add comments.

* fix lint.

* remove the test.

* fix compilation error.

* fix a bug.

* fix a bug.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-04-05 15:33:47 -07:00
Quan (Andy) Gan 24dc71fc95 [BUG] Fix #1409 (#1411)
* [BUG] Fix #1409

* fix test
2020-04-03 20:45:56 +08:00
Quan (Andy) Gan e4cc81852d [Windows] fix compilation issues on vs2015 (#1405)
* [Windows] fix compilation issues on vs2015

* fix test
2020-03-30 22:08:26 +08:00
Quan (Andy) Gan 4b4186f87d [Sampler] Change argument type of fanout from list to dict (#1403) 2020-03-30 15:52:46 +08:00
Quan (Andy) Gan 97b08fbba5 [BUG] fix crash in neighbor sampling w/ replacement on 0 degree nodes (#1402) 2020-03-30 14:48:23 +08:00
Quan (Andy) Gan e3a9a6bba8 add an optional include_dst_in_src argument (#1401) 2020-03-30 12:06:12 +08:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Quan (Andy) Gan bbfff8ce76 [Feature] Casting between DGLGraph and DGLHeteroGraph (#1391)
* [Feature] Casting between DGLGraph and DGLHeteroGraph

* lint

* address comments

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-27 13:34:55 +08:00
Jinjing Zhou 1e27e90e4e [MXNet] Patch mxnet unittest (#1395)
* patch

* turn off tf test

* skip test

* fix
2020-03-26 01:05:59 +08:00
Quan (Andy) Gan b9c65e91ef [BUG] Another fix on remove edges when all edges are removed (#1386)
* [BUG] Another fix on remove edges when all edges are removed

* fix mxnet
2020-03-23 19:04:52 +08:00
Quan (Andy) Gan d27b485912 [BUG] Fix remove_edges crashing with empty edge ID tensors (#1384) 2020-03-22 21:09:44 +08:00
Da Zheng 10253a5c07 [BUGFIX] Fix is_multigraph in the construction from scipy coo matrix (#1357)
* fix is_multigraph in from_coo.

* add tests for partition.

* fix.

* Revert "add tests for partition."

This reverts commit cb8c8555da3e0c70a482c2d639adce2943475bfc.

* fix everywhere from_scipy_sparse_matrix is used.
2020-03-22 00:30:27 -07:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Quan (Andy) Gan 0a51dc5435 [Bug] Fix dsttype in GraphSAGE minibatch model (#1371)
* fix for new ntype API for blocks

* adding two new interfaces
2020-03-18 23:57:03 +08:00
Minjie Wang 6111ea4665 [Model] RGCN mini-batch training; Better bipartite graph support (#1337)
* change the model to use node embedding only

* minibatch training

* add readme

* small fix

* backward-compatible

* backward compatible

* modify to_block; rgcn changed

* fix

* fix transform

* fix bug in unittest script

* docstring

* fix lint

* add tests

* address comments; fix offline eval

* gitignore
2020-03-16 18:58:06 +08:00
Da Zheng 1a584ced11 [Feature] Improve the speed of to_bidirected (#1327)
* make graph symmetric

* call gklib routine.

* use gklib version except in windows.

* minor errors.

* fix test.

* update doc.

* fix a compile error.

* fix.

* add comments for the new C API.

* fix a bug.

* address comments.

* fix compile error.

* fix comment.
2020-03-10 22:44:32 -07:00
xiang song(charlie.song) 7b3a7b1438 [Optimization] Optimize CompactGraph (#1328)
* compact

* prealloc memory for hashtable

* Fix

* upd

* Reduce memory

* upd

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-03-10 12:54:12 +08:00
Da Zheng 0e153c4baf [Feature] add metis partitioning to DGL (#1308)
* add metis.

* add test.

* construct partition id.

* link to METIS github repo.

* update metis.

* add a tool for partitioning a graph.

* update metis.

* update.

* update.

* fix metis.

* fix lint

* fix indent.

* another way of building metis.

* disable metis in windows.

* test windows

* fix.

* disable metis for windows properly.

* fix for tensorflow.

* skip test for gpu.

* make graph symmetric

* address comments.

* more comments.

* fix compile

* fix a bug.

* add test.

* change the default #hops of HALO nodes.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
2020-03-07 23:24:03 -08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Quan (Andy) Gan ce6e19f233 [Bugfix] Allows node types with no inbound/outbound edges (#1323)
* add num nodes in ctors

* fix

* lint

* addresses comments

* replace with constexpr

* remove function with rvalue reference

* address comments
2020-03-08 01:11:44 +08:00
Zihao Ye 09ade2f265 upd (#1321) 2020-03-06 14:17:48 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Minjie Wang 5dd35580f7 [Feature] Improve sampling speed; Better pickle/unpickle; other fixes (#1299)
* improve performance of sample_neighbors

* some more improve

* test script

* benchmarks

* multi process

* update more tests

* WIP

* adding two API for state saving

* add create from state

* upd test

* missing file

* wip: pickle/unpickle

* more c apis

* find the problem of empty data array

* add null array; pickling speed is bad

* still bad perf

* still bad perf

* wip

* fix the pickle speed test; now everything looks good

* minor fix

* bugfix

* some lint fix

* address comments

* more fix

* fix lint

* add utest for random.choice

* add utest for dgl.rand_graph

* fix cpp utests

* try fix ci

* fix bug in TF backend

* upd choice docstring

* address comments

* upd

* try fix compile

* add comment
2020-03-02 23:16:05 +08:00
Quan (Andy) Gan c3a33407fe [Feature] PinSAGE sampler (#1249)
* [WIP] PinSAGE operators

* moved the edge remapping mess into C

* some docstrings

* lint

* lint x2

* lint x3

* skip gpu test on topk

* extend pinsage to any metapath

* lint x4

* addresses #1265

* add always_preserve (fixes #1266) and fix a silly bug

* disable gpu test on compaction

* lint

* fix a horrible bug and add more tests

* lint

* addresses comments

* lint

* bugfix

* addresses comments

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 20:46:13 +08:00
Minjie Wang 37d992ecf6 [Bugfix] Fix no attribute num_edges bug in Nodeflow (#1289)
* fix nodeflow bug when using builtin on edge data

* fix
2020-02-26 10:05:38 +08:00
Minjie Wang c7f6cf6282 [Bugfix] Consistent order for cross-type stack reducer (#1267)
* WIP

* fix shape of stack reducer

* apply merge order by edge id for stack reducer
2020-02-18 13:54:11 +08:00
Minjie Wang f8d4264e5b [Feature] Neighbor-hood based sampling APIs (#1251)
* WIP: working on random choices

* light slice

* basic CPU impl

* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph

* simple test case works

* fix bug in slicing probability array

* fix bug in getting the correct relation graph

* fix bug in creating placeholder graph

* enable omp

* add cpp test

* sample topk

* add in|out_subgraph

* try fix lint; passed all unittests

* fix lint

* fix msvc compile; add sorted flag and constructors

* fix msvc

* coosort

* COOSort; CSRRowWiseSampling; CSRRowWiseTopk

* WIP: remove DType in CSR and COO; Restrict data array to be IdArray

* fix all CSR ops for missing data array

* compiled

* passed tests

* lint

* test sampling out edge

* test different per-relation fanout/k values

* fix bug in random choice

* finished cpptest

* fix compile

* Add induced edges

* add check

* fixed bug in sampling on hypersparse graph; add tests

* add ascending flag

* in|out_subgraph returns subgraph and induced eid

* address comments

* lint

* fix
2020-02-17 21:25:21 +08:00
Quan (Andy) Gan c7c0fd0e37 [Feature] Accessing node/edge types in UDFs. (#1243)
* typed udf

* docstrings
2020-02-17 16:02:56 +08:00
xiang song(charlie.song) ffe5898317 [Optimization][KG] Several optimizations on DGL-KG (#1233)
* Several optimizations on DGL-KG:
1. Sorted positive edges for sampling which can reduce random
   memory access during positive sampling
2. Asynchronous node embedding update
3. Balanced Relation Partition that gives balanced number of
   edges in each partition. When there is no cross partition
   relation, relation embedding can be pin into GPU memory
4. tunable neg_sample_size instead of fixed neg_sample_size

* Fix test

* Fix test and eval.py

* Now TransR is OK

* Fix single GPU with mix_cpu_gpu

* Add app tests

* Fix test script

* fix mxnet

* Fix sample

* Add docstrings

* Fix

* Default value for num_workers

* Upd

* upd
2020-02-09 13:01:00 -08:00
Quan (Andy) Gan eeeb52f464 [Feature] Preference to COO for "hypersparse" unit graphs & graph compaction (#1238)
* unit graph that prefers coo queries

* auto detect coo preference

* forgot some functions

* disable lint on detect_prefer_coo

* reorg

* change comment

* lint

* fix

* move array_utils.h to src

* compact graph impl

* fix redundant copying in idhashmap

* docstring

* moving preference detection to C

* lint

* fix unit test & address comments

* hypersparse autorestrict

* docstring & fix

* revert copyto and asnumbits

* fix stupid bug

* lint

* leave a TODO for sorted COO

* fixing same node type mapping to different id in different graphs

* addresses comments

* made induced nodes a feautre column

* lint?
2020-02-04 22:39:28 +08:00
Quan (Andy) Gan a9dabcc769 [Feature] Random Walk for 0.5 (#1209)
* trying to refactor IndexSelect

* partial implementation

* add index select and assign for floats as well

* move to random choice source

* more updates

* fixes

* fixes

* more fixes

* adding python impl

* fixes

* unit test

* lint

* lint x2

* lint x3

* update metapath2vec

* debugging performance

* still debugging for performance

* tuning

* switching to succvec

* redo

* revert non-uniform sampler to use vector

* still not fast

* why does this crash with OpenMP???

* because there was a data race!!!

* add documentations and remove assign op

* lint

* lint x2

* lol what have i done

* lint x3

* fix and disable gpu testing

* bugfix

* generic random walk

* reorg the random walk source code

* Update randomwalks.h

* Update randomwalks_cpu.cc

* rename file

* move internal function to anonymous ns

* reorg & docstrings

* constant restart probability

* docstring fix

* more commit

* random walk with restart, tested

* some fixes

* switch to using NDArray for choice

* massive fix & docstring

* lint x?

* lint x??

* fix

* export symbols

* skip gpu test

* addresses comments

* replaces another VecToIdArray

* add randomwalks.h to include

* replace void * with template
2020-01-31 19:52:08 +08:00
xiang song(charlie.song) 31a7d50964 [Bug Fix] Add boundary check for heterograph build with card as input (#1192)
* Add boundary check for heterograph build with card as input

* Fix when u or v is empty

* fix test_kernel.py error print

* Revert "fix test_kernel.py error print"

This reverts commit a71c20292549c0ac62a5326c30669ca4bde8febc.

* Turn op validation check on graph and bipartite by default

* upd

* udp

* upd

* update test
2020-01-18 17:18:40 +08:00
xiang song(charlie.song) c49582c996 [Test] Fix the random seed in test_kernel.py and fix the _print_error (#1194)
* Fix the random seed and fix the _print_error

* Fix check_positive_edge_sampler sample weight.

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-17 13:48:47 +08:00
Quan (Andy) Gan 86fe58ebb8 [Bug] Another fix on PyTorch memory leakage issue (#1139)
* another fix

* another try

* fix

* rewriting with kernel functions

* revert mxnet softmax changes

* lint fix

Co-authored-by: Zihao Ye <zihaoye.cs@gmail.com>
2020-01-14 22:54:19 +08:00
xiang song(charlie.song) cd9599ab04 [KG][Optimization] Remove copy from parent in minibatch generation (#1193)
* remote copy from parent

* upd

* make test_sampler easier to pass
2020-01-10 15:59:09 -08:00
Quan (Andy) Gan 28a24414c8 [BUG] Fix empty heterograph creation with list & tuple (#1188)
* [BUG] Fix empty heterograph creation with list & tuple

* more fix
2020-01-09 22:40:27 +08:00
Quan (Andy) Gan 3ef757dbff [Feature] Pickle support for heterograph (#1177)
* [Feature] Pickle support for heterograph

* fix lint

Co-authored-by: VoVAllen <VoVAllen@users.noreply.github.com>
2020-01-09 13:27:29 +08:00
Da Zheng 1022d5d64d [KG] PBG's way of constructing negative edges (#1159)
* attach positive.

* add neg_deg_sample.

* add comment.

* add neg_deg_sample for eval.

* change the edge sampler.

* rename edge sampler in KG.

* allow specifying chunk size and negative sample size separately.

* fix bugs in KG.

* add check in sampler.

* add more checks.

* fix

* add comment.

* add comments.
2020-01-04 19:25:06 -08:00
xiang song(charlie.song) 61b78e6ee7 [Bug Fix]edge sample hotfix (#1152)
* hot fix

* Fix docs

* Fix ArrayHeap float overflow bug

* Fix

* Clean some dead code

* Fix

* FIx

* Add some comments

* run test
2019-12-30 19:42:57 -08:00
xiang song(charlie.song) 913e324903 Add device check for sampler input (#1145)
current samplers only support working on CPU
2019-12-27 22:50:52 +08:00
xiang song(charlie.song) 41f8a162f4 [Feature Improvement]Edge Sampler w and w/o replacement (#1116)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description

* add replacement for edge sampler

* Now edge sampler support replacement and no-replacement

* Fix

* Fix

* change kg/app to use edge sampler with replacement config

* Update replacement algo

* Fix syntax

* Update

* Update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2019-12-24 16:25:40 +08:00
Da Zheng e890a899fc Partition a graph with HALO nodes (#1076)
* get subgraph with halo.

* add partition function.

* add comment.

* parallel partition.

* fix a compilation error.

* fix lint error.

* address comments.

* add comments.

* fix for TF.
2019-12-23 08:46:53 -08:00
VoVAllen 0d9acc9d58 hot fix (#1135) 2019-12-24 00:19:25 +08:00
VoVAllen a1b28b3954 [Hot fix] Fix build threads (#1133)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image

* try node

* try

* fix

* try new ci

* try new ci

* try new ci

* try new ci

* ci

* try

* try

* fix

* hot fix

* fix

* fix cpp test

* add comments
2019-12-23 21:42:59 +08:00
VoVAllen d30a69bf27 [Backend] TF backend (#978)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image
2019-12-20 15:56:51 +08:00
xiang song(charlie.song) 632a9af8f9 [Feature] No-Uniform Edge Sampler (#1087)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description
2019-12-16 11:26:34 -08:00
VoVAllen dd65ee211e [CI] Change tests for flexibility
* change ci image

* fix

* force bash

* fix

* fix python version

* fix

* fix

* fix

* update gpu

* cuda

* jenkins

* fix build sh

* fix

* Revert "fix"

This reverts commit 6b091914b3ef6b4300fa662ee375aa4b27944f5c.

* try fix

* fix

* Revert "fix"

This reverts commit e42c3035fa4974c6b197aaba0748f7b45113ddaa.

* try fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix tests

* try fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix ctx problem

* fix many tests

* fix typo

* add backend

* move to pytorch folder

* fix?

* fix ci

* try skip

* try false

* try?

* try?

* Revert to 7d9a52f

* fix various

* fix lint

* Revert Jenkinsfile

* revert env

* revert env

* address comment

* remove file
2019-12-16 14:54:37 +08:00
Quan (Andy) Gan 6ae93e5c6a [Bug] Fix #1088 (#1089)
* [Bug] Fix #1088

* fix

* add comment
2019-12-11 10:27:57 +08:00
Zihao Ye fa0ee46a01 [hotfix] node id validity check (#1073)
* fix

* improve

* fix lint

* upd

* fix

* upd
2019-12-06 01:36:41 +08:00