文件历史

2070 次代码提交

作者 SHA1 备注 提交日期
Quan (Andy) Gan 20ec7bb0d9 [Bug?] Fixing compatibility of pickles across DGL versions (#1507)
* pickle compatibility before 0.4.2

* fixing for utils.Index

* add TODOs

* fix

* fix

* more compatibility checks for DGLGraph and DGLHeteroGraph objects

* lint

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-15 00:02:58 +08:00
Zihao Ye c2e61ce10b [Refactor] Restrict sparse format for DGLHeteroGraph (#1474)
* upd

* simplify

* further simplify

* lint

* doc

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* lint

* rename format

* upd

* lint

* upd

* upd

* upd

* udp

* debug

* upd

* upd

* upd

* upd

* upd

* 无可厚非吧

* 中三边肥

* 你一定要喊吗

* lint

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* Update unit_graph.h

Co-authored-by: Zihao Ye <yzh119@192.168.0.110>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
2020-05-14 22:11:12 +08:00
Da Zheng 59aefe6e31 [Doc] write warning for remove_edges. (#1509)
* write warning for remove_edges.

* fix.

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-13 18:26:59 +08:00
Quan (Andy) Gan e5b10b10ee [Performance] fixing performance on heterogeneous graph creation in specific cases (#1506)
* [Performance] fixing performance on heterogeneous graph creation in specific cases

* fix for mxnet
2020-05-13 17:15:06 +08:00
Jinjing Zhou 6d04885381 [Backend] Turn to official dlpack for Tensorflow (#1511)
* Turn to official dlpack

* fix

* fix
2020-05-11 21:16:25 +08:00
Mohammad Amin Khashkhashi Moghaddam 0285021235 Fix bug in GIN Dataset degree OHE logic (#1498)
In the previous code, the same vector was being assigned to all the nodes in each graph.
`:` was used incorrectly assuming that only one row will be selected for each column but all rows were selected for each column.
2020-05-09 15:11:41 +08:00
Zihao Ye 1af0d8064a [bugfix] Fix the behavior of dgl.batch when batch size is 1 (#1483)
* fix

* upd

* upd

* upd

* upd

* clone

* upd

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-08 16:12:12 +08:00
Chao Ma ef7e475051 [Distributed] Add GraphPartitionBook (#1496)
* Add graph-partition-book

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* udpate

* fix lint

* update

* update

* update

* update

* update

* update

* update

* skip test on GPU

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* update

* fix unittest

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-05-08 14:52:12 +08:00
Jinjing Zhou dc8ca88e58 [Refactor] Explicit dtype for HeteroGraph (#1467)
* 111

* 111

* lint

* lint

* lint

* lint

* fix

* lint

* try

* fix

* lint

* lint

* test

* fix

* ttt

* test

* fix

* fix

* fix

* mxnet

* 111

* fix 64bits computation

* pylint

* roll back

* fix

* lint

* fix hetero_from_relations

* remove index_dtype in to_homo and to_hetero

* fix

* fix

* fix

* fix

* remove default

* fix

* lint

* fix

* fix error message

* fix error

* lint

* macro dispatch

* try

* lint

* remove nbits

* error message

* fix

* fix

* lint

* lint

* lint

* fix

* lint

* fix

* fix random walk

* lint

* lint

* fix

* fix

* fix

* lint

* fix

* lint

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-05-08 14:26:28 +08:00
Da Zheng 16561a2e6b [Test] Add tests for TensorFlow (#1501)
* add test.

* move test code.

* remvoe unnecessary test.

* fix.

* turn on tests for TF.

* Revert "move test code."

This reverts commit e7b4f36395b2121a7be030bd4364a704d0e357bf.

* fix.

* fix.

* skip test for tensorflow.

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-05-06 00:33:53 -07:00
Da Zheng 2190c39d67 [Feature] Distributed graph store (#1383)
* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code.

* revert modify in sampler.

* fix temp script.

* remove pytorch version.

* fix.

* distributed training with pytorch.

* add distributed graph store.

* fix.

* add metis_partition_assignment.

* fix a few bugs in distributed graph store.

* fix test.

* fix bugs in distributed graph store.

* fix tests.

* remove code of defining DistGraphStore.

* fix partition.

* fix example.

* update run.sh.

* only read necessary node data.

* batching data fetch of multiple NodeFlows.

* simplify gcn.

* remove unnecessary code.

* use the new copy_from_kvstore.

* update training script.

* print time in graphsage.

* make distributed training runnable.

* use val_nid.

* fix train_sampling.

* add distributed training.

* add run.sh

* add more timing.

* fix a bug.

* save graph metadata when partition.

* create ndata and edata in distributed graph store.

* add timing in minibatch training of GraphSage.

* use pytorch distributed.

* add checks.

* fix a bug in global vs. local ids.

* remove fast pull

* fix a compile error.

* update and add new APIs.

* implement more methods in DistGraphStore.

* update more APIs.

* rename it to DistGraph.

* rename to DistTensor

* remove some unnecessary API.

* remove unnecessary files.

* revert changes in sampler.

* Revert "simplify gcn."

This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.

* Revert "simplify neighbor sampling."

This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.

* Revert "measure time in neighbor sampling."

This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.

* Revert "add timing in minibatch training of GraphSage."

This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.

* Revert "fix train_sampling."

This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.

* fix lint.

* add comments and small update.

* add more comments.

* add more unit tests and fix bugs.

* check the existence of shared-mem graph index.

* use new partitioned graph storage.

* fix bugs.

* print error in fast pull.

* fix lint

* fix a compile error.

* save absolute path after partitioning.

* small fixes in the example

* Revert "[kvstore] support any data type for init_data() (#1465)"

This reverts commit 87b6997bf2.

* fix a bug.

* disable evaluation.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* support set and init data.

* support set and init data.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* fix bugs.

* fix unit test.

* move to dgl.distributed.

* fix lint.

* fix lint.

* remove local_nids.

* fix lint.

* fix test.

* remove train_dist.

* revert train_sampling.

* rename funcs.

* address comments.

* address comments.

Use NodeDataView/EdgeDataView to keep track of data.

* address comments.

* address comments.

* revert.

* save data with DGL serializer.

* use the right way of getting shape.

* fix lint.

* address comments.

* address comments.

* fix an error in mxnet.

* address comments.

* add edge_map.

* add more test and fix bugs.

Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
2020-05-02 19:30:51 -07:00
Chao Ma 5fc334fc30 [KVStore] Fix bug of get_meta() API (#1491)
* fix get-shape

* update

* update

* update

* update

* update

* fix typo

* update

* update
2020-05-02 15:01:42 +08:00
Jinjing Zhou 30b8074af4 [Fix] Fix stream creation and empty dict in tensor serialization (#1489)
* add functions

* fix litn

* add unit test

* fix

* fix

* fix

* fix

* support empty dict

* simplify logic

* Update tensor_serialize.py
2020-05-02 14:24:03 +08:00
Minjie Wang f25bc176d0 [Hetero] Improve speed of several Hetero APIs (#1486)
* add clone function to frame

* add utest

* replace all local_var with local_scope

* fix utest

* avoid creating canonical types in __getitem__

* lint

* try another utest  appraoch for mx

* utest
2020-05-01 15:21:33 +08:00
Minjie Wang 3c4506e91e [Bugfix] Add bool data type to backend. (#1487)
* add bool to F.data_type_dict

* add utest

* skip bool test for mx
2020-04-29 20:08:36 +08:00
Mohammad Amin Khashkhashi Moghaddam f1e4f3786d Fix bug in using degree_as_nlabel for GIN dataset (#1451)
In order to make the `label2idx` dictionary, tensor values should be used and not the tensor itself.

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-04-29 17:42:45 +08:00
Jinjing Zhou 1506560e1e [Data] Add utils to save dict of tensors (#1481)
* add functions

* fix litn

* add unit test

* fix

* fix
2020-04-27 12:26:41 -07:00
Chao Ma 100ddd067e [KVStore] Bug fix (#1480)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* Update network.cc

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-04-27 09:42:06 -07:00
Jinjing Zhou aa17d78af7 Fix (#1478) 2020-04-27 20:02:34 +08:00
Chao Ma 27520bc515 [KVStore] add init_data() on client (#1466)
* add init_data on client

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update
2020-04-25 17:59:23 -07:00
Chao Ma 87b6997bf2 [kvstore] support any data type for init_data() (#1465)
* support any data type for init_data

* update

* update
2020-04-23 22:04:40 +08:00
Minjie Wang 6cd7c313f1 [Bugfix] Fix corrupted memory when using multi-processing (#1453)
* fix

* fix mapping array for empty node set
2020-04-23 18:37:32 +08:00
Chao Ma 5efd1c4ace [KVstore] Improve user infterface (#1454)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-04-20 15:50:16 -07:00
Chao Ma 338f24cfbc [KVstore] Fast-pull (#1446)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint
2020-04-16 23:14:54 +08:00
Mufei Li bd1e48a51e [Bug] Fix khop graph (#1433)
* Update

* Update

* Update
2020-04-10 02:18:45 +08:00
Adam J. Stewart aefc080e30 [Doc] Fix GraphConv math notation (#1430) 2020-04-09 11:31:31 +08:00
Quan (Andy) Gan 43d9f43a27 backend selection (#1424) 2020-04-06 20:44:29 +08:00
Da Zheng 7c47d8c9b4 [Feature] Simplify shared memory graph index (#1381)
* simplify shared memory graph index.

* fix.

* remove edge_dir in SharedMemGraphStore.

* avoid creating shared-mem graph store with from_csr.

* simplify from_csr.

* add comments.

* fix lint.

* remove the test.

* fix compilation error.

* fix a bug.

* fix a bug.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-04-05 15:33:47 -07:00
Mufei Li 1b152bf524 [Doc & Tutorial] Fix dgl.batch and GAT (#1418)
* Update doc and tutorials

* Fix
2020-04-04 01:23:22 +08:00
Jinjing Zhou 7e0893e697 [Docs] Fix doc build doesn't work for nn.tensorflow and nn.mxnet (#1413)
* fix

* lint

* fix
2020-04-02 20:31:07 +08:00
Quan (Andy) Gan e4cc81852d [Windows] fix compilation issues on vs2015 (#1405)
* [Windows] fix compilation issues on vs2015

* fix test
2020-03-30 22:08:26 +08:00
Jinjing Zhou e9440acb06 [TF] TF backend fix and new logic to choose backend (#1393)
* TF backend fix and new logic to choose backend

* fix

* fix

* fix

* fix

* fix backend

* fix

* dlpack alignment

* add flag

* flag

* lint

* lint

* remove unused

* several fixes

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-30 18:45:17 +08:00
Quan (Andy) Gan 4b4186f87d [Sampler] Change argument type of fanout from list to dict (#1403) 2020-03-30 15:52:46 +08:00
Quan (Andy) Gan e3a9a6bba8 add an optional include_dst_in_src argument (#1401) 2020-03-30 12:06:12 +08:00
Zihao Ye af61e2fbb4 [Feature] Support nn modules for bipartite graphs. (#1392)
* init gat

* fix

* gin

* 7 nn modules

* rename & lint

* upd

* upd

* fix lint

* upd test

* upd

* lint

* shape check

* upd

* lint

* address comments

* update tensorflow

Co-authored-by: Quan Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-29 16:48:38 +08:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Minjie Wang 3efb5d8ecf [NN] Add HeteroGraphConv module for cleaner module definition (#1385)
* Add HeteroGraphConv

* add custom aggregator; some docstring

* debugging

* rm print

* fix some acc bugs

* fix initialization problem in weight basis

* passed tests

* lint

* fix graphconv flag; add error message

* add mxnet heteroconv

* more fix for mx

* lint

* fix torch cuda test

* fix mx test_nn

* add exhaust test for graphconv

* add tf heteroconv

* fix comment
2020-03-27 18:47:06 +08:00
Quan (Andy) Gan bbfff8ce76 [Feature] Casting between DGLGraph and DGLHeteroGraph (#1391)
* [Feature] Casting between DGLGraph and DGLHeteroGraph

* lint

* address comments

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-27 13:34:55 +08:00
Quan (Andy) Gan b9c65e91ef [BUG] Another fix on remove edges when all edges are removed (#1386)
* [BUG] Another fix on remove edges when all edges are removed

* fix mxnet
2020-03-23 19:04:52 +08:00
Quan (Andy) Gan d27b485912 [BUG] Fix remove_edges crashing with empty edge ID tensors (#1384) 2020-03-22 21:09:44 +08:00
Da Zheng a0721405cf [BUGFIX] don’t import dgl in the package. (#1382)
* fix dgl data.

* remove more.

* fix.

* fix.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-03-22 01:29:16 -07:00
Da Zheng 10253a5c07 [BUGFIX] Fix is_multigraph in the construction from scipy coo matrix (#1357)
* fix is_multigraph in from_coo.

* add tests for partition.

* fix.

* Revert "add tests for partition."

This reverts commit cb8c8555da3e0c70a482c2d639adce2943475bfc.

* fix everywhere from_scipy_sparse_matrix is used.
2020-03-22 00:30:27 -07:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Quan (Andy) Gan 4af02022b2 [Bug] Multiple fixes (#1374)
* multiple fixes

* lint

* lint x2
2020-03-19 11:19:44 +08:00
Quan (Andy) Gan 0a51dc5435 [Bug] Fix dsttype in GraphSAGE minibatch model (#1371)
* fix for new ntype API for blocks

* adding two new interfaces
2020-03-18 23:57:03 +08:00
Minjie Wang 6111ea4665 [Model] RGCN mini-batch training; Better bipartite graph support (#1337)
* change the model to use node embedding only

* minibatch training

* add readme

* small fix

* backward-compatible

* backward compatible

* modify to_block; rgcn changed

* fix

* fix transform

* fix bug in unittest script

* docstring

* fix lint

* add tests

* address comments; fix offline eval

* gitignore
2020-03-16 18:58:06 +08:00
Da Zheng 3938b399d9 [BUGFIX] metis partition should generate a subgraph to the original graph. (#1352)
* fix a partition.

* add comments.
2020-03-12 10:43:08 +08:00
Da Zheng 1a584ced11 [Feature] Improve the speed of to_bidirected (#1327)
* make graph symmetric

* call gklib routine.

* use gklib version except in windows.

* minor errors.

* fix test.

* update doc.

* fix a compile error.

* fix.

* add comments for the new C API.

* fix a bug.

* address comments.

* fix compile error.

* fix comment.
2020-03-10 22:44:32 -07:00
Zihao Ye 988c8b20a4 [DOC] Improve the docstring of GATConv and new APIs (#1339)
* upd

* upd

* upd

* upd

* upd
2020-03-11 01:14:22 +08:00
Chao Ma d98c71ef90 [DGL-KE] Clean up argument for DGL-KE (#1325)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* clean up args

* change args for eval

* update

* update

* update

* update num_worker=8

* add num_proc=1

* update

* update

* add eval percent

* update

* update

* update

* save emb

* update

* update

* update

* update

* update

* remove neg_chunk_size

* update

* update

* update

* remove

* update

* remove

* change num_thread
2020-03-09 23:11:56 +08:00