文件历史

93 次代码提交

作者 SHA1 备注 提交日期
Da Zheng 6a3685bea0 [BUGFIX] fix a bug in graph partitioning. (#1769)
* fix

* use utils.toindex in the right place.

* fix.

* update tensor for mxnet backend.

* fix

* fix
2020-07-09 01:16:42 -07:00
xiang song(charlie.song) 29e6c93fba [Kernel] Matrix toSimple (#1756)
* Matrix to simple

* Pass test

* new impl

* Fix test

* Fix lint

* trigger

* upd

* Fix comments

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-07-09 13:50:41 +08:00
Quan (Andy) Gan 168a88e570 [Sampling] NodeDataLoader for node classification (#1635)
* neighbor sampler data loader first commit

* more commit

* nodedataloader

* fix

* update RGCN example

* update OGB

* fixes

* fix minibatch RGCN crashing with self loop

* reverting gatconv test code

* fix

* change to new solution that doesn't require tf dataloader

* fix

* lint

* fix

* fixes

* change doc

* fix docstring

* docstring fixes

* return seeds and input nodes from data loader

* fixes

* fix test

* fix windows build problem

* add pytorch wrapper

* fixes

* add pytorch wrapper

* add unit test

* add -1 support to sample_neighbors & fix docstrings

* docstring fix

* lint

* add minibatch rgcn evaluations

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Tong He <hetong007@gmail.com>
2020-07-02 16:31:05 +08:00
Jinjing Zhou 18a26fcfb1 [Data] Support HeteroGraph save/load (#1526)
* 111

* add history version test

* fix

* 111

* save

* ``

* fix1

* 111

* add save heterograph

* lint

* lint

* add tests

* minor fix

* fix

* docs

* add format tets

* use unique_ptr

* fix

* fix interface

* 111

* 111

* fix

* lint

* fix

* add support to s3

* fix

* fix

* fix leak

* fix

* fix docs

* fix

* linlt

* fix

* fix

* fix

* address comment

* address comment

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-30 13:34:50 +08:00
Quan (Andy) Gan 2f983abe64 fix knn graph bugs (#1716) 2020-06-28 17:31:38 +08:00
Da Zheng 04d4680d5b [Distributed] add constraints for metis partitioning. (#1708)
* add constraint.

* fix bugs.

* temp fix.

* fix lint

* add balance_ntypes and balance_edges

* add comments.

* fix.

* fix
2020-06-27 21:27:18 -07:00
Da Zheng 4b96addca0 [BUGFIX] Fix bugs in range partitioning. (#1703)
* Fix bugs in partitioning.

* add comment.

* fix lint.

* fix test.

* add more tests.

* simplify the fix

* fix.
2020-06-26 15:59:45 -07:00
JryonGitHub 613df6f96b [Bugfix]fix the bug for knngraph (#1667) (#1668)
* fix the bug for knngraph (#1667)

* fix the bug for knngraph (#1667)

* fix the bug for knngraph (#1667)

* fix the bug for knngraph (#1667)

* fix the bug for knngraph (#1667)

* fix the bug for knngraph (#1667)

Co-authored-by: jiruyi <18810981229@gmail.com>
2020-06-20 16:07:28 +08:00
Minjie Wang f1b19a6b67 [CUDA] Many CUDA operators; Prepare for DGLGraph on CUDA (#1660)
* add cuda utils; change g.to; add g.device

* split array.h into several headers

* cuda index select

* file

* three cuda kernels

* add cuda elementwise arith and several others

* cuda CSRIsNonZero

* fix lint

* lint

* lint

* fix bug in changing ctx to property

* address comments

* remove unused codes

* address comments
2020-06-19 15:44:42 +08:00
Da Zheng 41349dcef5 [Feature] Range partition (#1522)
* add reorder immutable graph.

* add python API.

* add reorder for csr.

* remove gk version.

* fix

* add cpp test.

* bug fixes

* fix tests.

* fix bugs and add check

* fix test.

* add omp.

* add comments.

* add coo reorder.

* fix a bug.

* handle reorder for different graph structues.

* fix lint.

* fix.

* add original ids.

* reshuffle nodes before metis partition.

* inner nodes are in contiguous Id range.

* reshuffle nodes/edges when partitioning.

* load partition return graph partition book.

* use inner_node/inner_edges

* add and test range partition book.

* count inner_edge correctly.

* fix lint.

* fix lint.

* fix lint.

* fix errors.

* fix errors.

* fix for TF.

* fix.

* fix.

* change docstring.

* support logical and.

* add comments.

* avoid copy.

* fix

* update docstring.

* fix a bug.

* add range search.

* fix

* fix a bug.

* add more tests.

* load graph partition book.

* support shared memory for range partition book.

* fix a bug.

* fix.

* fix lint.

* remove check

* fix test.

* remove num_nodes and num_edges

* fix lint.

* fix graph partition book.

* address comments.

* use makedirs.

* fix compile

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-06-15 11:43:04 -07:00
Quan (Andy) Gan 7ba9cbc63e [Docs] Fix docstrings and documentations (#1420)
* fix docs of to_block

* fix

* oops

* more fixes
2020-05-19 13:04:55 +08:00
Da Zheng 59aefe6e31 [Doc] write warning for remove_edges. (#1509)
* write warning for remove_edges.

* fix.

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-13 18:26:59 +08:00
Jinjing Zhou dc8ca88e58 [Refactor] Explicit dtype for HeteroGraph (#1467)
* 111

* 111

* lint

* lint

* lint

* lint

* fix

* lint

* try

* fix

* lint

* lint

* test

* fix

* ttt

* test

* fix

* fix

* fix

* mxnet

* 111

* fix 64bits computation

* pylint

* roll back

* fix

* lint

* fix hetero_from_relations

* remove index_dtype in to_homo and to_hetero

* fix

* fix

* fix

* fix

* remove default

* fix

* lint

* fix

* fix error message

* fix error

* lint

* macro dispatch

* try

* lint

* remove nbits

* error message

* fix

* fix

* lint

* lint

* lint

* fix

* lint

* fix

* fix random walk

* lint

* lint

* fix

* fix

* fix

* lint

* fix

* lint

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-08 14:26:28 +08:00
Da Zheng 2190c39d67 [Feature] Distributed graph store (#1383)
* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code.

* revert modify in sampler.

* fix temp script.

* remove pytorch version.

* fix.

* distributed training with pytorch.

* add distributed graph store.

* fix.

* add metis_partition_assignment.

* fix a few bugs in distributed graph store.

* fix test.

* fix bugs in distributed graph store.

* fix tests.

* remove code of defining DistGraphStore.

* fix partition.

* fix example.

* update run.sh.

* only read necessary node data.

* batching data fetch of multiple NodeFlows.

* simplify gcn.

* remove unnecessary code.

* use the new copy_from_kvstore.

* update training script.

* print time in graphsage.

* make distributed training runnable.

* use val_nid.

* fix train_sampling.

* add distributed training.

* add run.sh

* add more timing.

* fix a bug.

* save graph metadata when partition.

* create ndata and edata in distributed graph store.

* add timing in minibatch training of GraphSage.

* use pytorch distributed.

* add checks.

* fix a bug in global vs. local ids.

* remove fast pull

* fix a compile error.

* update and add new APIs.

* implement more methods in DistGraphStore.

* update more APIs.

* rename it to DistGraph.

* rename to DistTensor

* remove some unnecessary API.

* remove unnecessary files.

* revert changes in sampler.

* Revert "simplify gcn."

This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.

* Revert "simplify neighbor sampling."

This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.

* Revert "measure time in neighbor sampling."

This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.

* Revert "add timing in minibatch training of GraphSage."

This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.

* Revert "fix train_sampling."

This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.

* fix lint.

* add comments and small update.

* add more comments.

* add more unit tests and fix bugs.

* check the existence of shared-mem graph index.

* use new partitioned graph storage.

* fix bugs.

* print error in fast pull.

* fix lint

* fix a compile error.

* save absolute path after partitioning.

* small fixes in the example

* Revert "[kvstore] support any data type for init_data() (#1465)"

This reverts commit 87b6997bf2.

* fix a bug.

* disable evaluation.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* support set and init data.

* support set and init data.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* fix bugs.

* fix unit test.

* move to dgl.distributed.

* fix lint.

* fix lint.

* remove local_nids.

* fix lint.

* fix test.

* remove train_dist.

* revert train_sampling.

* rename funcs.

* address comments.

* address comments.

Use NodeDataView/EdgeDataView to keep track of data.

* address comments.

* address comments.

* revert.

* save data with DGL serializer.

* use the right way of getting shape.

* fix lint.

* address comments.

* address comments.

* fix an error in mxnet.

* address comments.

* add edge_map.

* add more test and fix bugs.

Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
2020-05-02 19:30:51 -07:00
Minjie Wang f25bc176d0 [Hetero] Improve speed of several Hetero APIs (#1486)
* add clone function to frame

* add utest

* replace all local_var with local_scope

* fix utest

* avoid creating canonical types in __getitem__

* lint

* try another utest  appraoch for mx

* utest
2020-05-01 15:21:33 +08:00
Minjie Wang 6cd7c313f1 [Bugfix] Fix corrupted memory when using multi-processing (#1453)
* fix

* fix mapping array for empty node set
2020-04-23 18:37:32 +08:00
Mufei Li bd1e48a51e [Bug] Fix khop graph (#1433)
* Update

* Update

* Update
2020-04-10 02:18:45 +08:00
Quan (Andy) Gan e3a9a6bba8 add an optional include_dst_in_src argument (#1401) 2020-03-30 12:06:12 +08:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Quan (Andy) Gan bbfff8ce76 [Feature] Casting between DGLGraph and DGLHeteroGraph (#1391)
* [Feature] Casting between DGLGraph and DGLHeteroGraph

* lint

* address comments

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-27 13:34:55 +08:00
Quan (Andy) Gan b9c65e91ef [BUG] Another fix on remove edges when all edges are removed (#1386)
* [BUG] Another fix on remove edges when all edges are removed

* fix mxnet
2020-03-23 19:04:52 +08:00
Quan (Andy) Gan d27b485912 [BUG] Fix remove_edges crashing with empty edge ID tensors (#1384) 2020-03-22 21:09:44 +08:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Minjie Wang 6111ea4665 [Model] RGCN mini-batch training; Better bipartite graph support (#1337)
* change the model to use node embedding only

* minibatch training

* add readme

* small fix

* backward-compatible

* backward compatible

* modify to_block; rgcn changed

* fix

* fix transform

* fix bug in unittest script

* docstring

* fix lint

* add tests

* address comments; fix offline eval

* gitignore
2020-03-16 18:58:06 +08:00
Da Zheng 3938b399d9 [BUGFIX] metis partition should generate a subgraph to the original graph. (#1352)
* fix a partition.

* add comments.
2020-03-12 10:43:08 +08:00
Da Zheng 1a584ced11 [Feature] Improve the speed of to_bidirected (#1327)
* make graph symmetric

* call gklib routine.

* use gklib version except in windows.

* minor errors.

* fix test.

* update doc.

* fix a compile error.

* fix.

* add comments for the new C API.

* fix a bug.

* address comments.

* fix compile error.

* fix comment.
2020-03-10 22:44:32 -07:00
Da Zheng 0e153c4baf [Feature] add metis partitioning to DGL (#1308)
* add metis.

* add test.

* construct partition id.

* link to METIS github repo.

* update metis.

* add a tool for partitioning a graph.

* update metis.

* update.

* update.

* fix metis.

* fix lint

* fix indent.

* another way of building metis.

* disable metis in windows.

* test windows

* fix.

* disable metis for windows properly.

* fix for tensorflow.

* skip test for gpu.

* make graph symmetric

* address comments.

* more comments.

* fix compile

* fix a bug.

* add test.

* change the default #hops of HALO nodes.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
2020-03-07 23:24:03 -08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Quan (Andy) Gan c3a33407fe [Feature] PinSAGE sampler (#1249)
* [WIP] PinSAGE operators

* moved the edge remapping mess into C

* some docstrings

* lint

* lint x2

* lint x3

* skip gpu test on topk

* extend pinsage to any metapath

* lint x4

* addresses #1265

* add always_preserve (fixes #1266) and fix a silly bug

* disable gpu test on compaction

* lint

* fix a horrible bug and add more tests

* lint

* addresses comments

* lint

* bugfix

* addresses comments

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 20:46:13 +08:00
Minjie Wang f8d4264e5b [Feature] Neighbor-hood based sampling APIs (#1251)
* WIP: working on random choices

* light slice

* basic CPU impl

* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph

* simple test case works

* fix bug in slicing probability array

* fix bug in getting the correct relation graph

* fix bug in creating placeholder graph

* enable omp

* add cpp test

* sample topk

* add in|out_subgraph

* try fix lint; passed all unittests

* fix lint

* fix msvc compile; add sorted flag and constructors

* fix msvc

* coosort

* COOSort; CSRRowWiseSampling; CSRRowWiseTopk

* WIP: remove DType in CSR and COO; Restrict data array to be IdArray

* fix all CSR ops for missing data array

* compiled

* passed tests

* lint

* test sampling out edge

* test different per-relation fanout/k values

* fix bug in random choice

* finished cpptest

* fix compile

* Add induced edges

* add check

* fixed bug in sampling on hypersparse graph; add tests

* add ascending flag

* in|out_subgraph returns subgraph and induced eid

* address comments

* lint

* fix
2020-02-17 21:25:21 +08:00
Da Zheng e890a899fc Partition a graph with HALO nodes (#1076)
* get subgraph with halo.

* add partition function.

* add comment.

* parallel partition.

* fix a compilation error.

* fix lint error.

* address comments.

* add comments.

* fix for TF.
2019-12-23 08:46:53 -08:00
Mufei Li b599f6ed96 Update (#1090) 2019-12-11 13:39:18 +08:00
Mufei Li c03046a08f [Model][Hetero] HAN (#868)
* Add HAN

* Fix

* WIP; load raw ACM dataset

* DGL's own preprocessing with metapath coalescer

* various fixes

* comparison against simple logistic regression

* rename

* fix test
2019-09-27 12:53:16 +08:00
VoVAllen e16667bfb9 [Fix] Fix Transform (#876)
* add transform

* lint

* lint

* fix

* fixmx

* fix

* add test

* fix typo

* fix default num_classes

* change to non-inplace operation

* fix lint

* fix

* fix

* fix lint

* fixlint
2019-09-26 21:21:53 -04:00
VoVAllen bf8bb58f60 [Transform] Add to_self_loop, remove_self_loop, onehot_degree transform (#862)
* add transform

* lint

* lint

* fix

* fixmx

* fix

* add test

* fix typo

* fix default num_classes

* change to non-inplace operation

* fix lint

* fix
2019-09-21 00:05:38 +08:00
Quan (Andy) Gan 14bffe9728 [NN] Renaming NearestNeighborGraph to KNNGraph (#802)
* initial commit

* second commit

* another commit

* change docstring

* migrating to dgl.nn

* fixes

* docs

* lint

* multiple fixes

* doc

* renaming nearest neighbor graph
2019-08-28 12:20:35 +08:00
Quan (Andy) Gan dc19cd5687 [Example] Dynamic Graph CNN on Point Cloud (#789)
* initial commit

* second commit

* another commit

* change docstring

* migrating to dgl.nn

* fixes

* docs

* lint

* multiple fixes

* doc
2019-08-28 09:21:57 +08:00
Zihao Ye 650f6ee1e0 [NN] Add commonly used GNN models from examples to dgl.nn modules. (#748)
* gat

* upd

* upd sage

* upd

* upd

* upd

* upd

* upd

* add gmmconv

* upd ggnn

* upd

* upd

* upd

* upd

* add citation examples

* add README

* fix cheb

* improve doc

* formula

* upd

* trigger

* lint

* lint

* upd

* add test for transform

* add test

* check

* upd

* improve doc

* shape check

* upd

* densechebconv, currently not correct (?)

* fix cheb

* fix

* upd

* upd sgc-reddit

* upd

* trigger
2019-08-27 18:21:19 +08:00
Minjie Wang 67dc11971f [Refactor] Use object system for all CAPIs (#716)
* WIP: using object system for graph

* c++ side refactoring done; compiled

* remove stale apis

* fix bug in DGLGraphCreate; passed test_graph.py

* fix bug in python modify; passed utest for pytorch/cpu

* fix lint

* address comments
2019-07-23 12:47:05 -04:00
Mufei Li fb9dcc51ff [Feature] Add API to convert graph to bidirected graph (#598)
* to_bidirected

* to_bidirected

* Fix style

* Fix

* Update

* Fix

* Fix

* Update

* Add examples
2019-06-10 14:02:29 +08:00
Minjie Wang 01a4cc5b7b [Graph] Add API to convert graph to simple graph (#587)
* to simple

* WIP: multigraph flag

* graph index refactor; pass basic testing

* graph index refactor; pass basic testing

* fix bug in to_simple; pass torch test

* fix mx utest

* fix example

* fix lint

* fix ci

* poke ci

* poke ci

* WIP

* poke ci

* poke ci

* poke ci

* change ci workspace

* poke ci

* poke ci

* poke ci

* poke ci

* delete ci

* use enum for multigraph flag
2019-06-01 23:32:46 -04:00
Mufei Li 24bbdb745e [Feature] Reversed Graph and Transform Module (#331)
* reverse a graph

* Reverse a graph

* Fix

* Revert "Fix"

This reverts commit 1728826b4f3a083b76dfc0fdecd2bbf943a971b2.

* Fix

* Fix

* Delete vcs.xml

* Delete Project_Default.xml

* Fix

* Fix

* Fix

* Remove outdated test

* Reorg transform and update reverse (#2)

* Reorg transform and update reverse

* Fix doc and test

* Update test

* Resolve conflict

* CI oriented fix

* Remove outdated import

* Fix import

* Fix import

* define __all__ for wildcard imports

* Fix import

* Address circular imports

* Fix

* Fix test case

* Fix

* Fix

* Remove unused import

* Fix

* Fix

* Fix
2019-01-04 13:51:19 -05:00