文件历史

提交图

154 次代码提交

作者 SHA1 备注 提交日期
Da Zheng 729ff2ef38 Fix the performance issue of graph partitioning in new DGLGraph (#1934)
* fix perf.

* fix.

* accelerate metis.

* fix lint.

* use gklib.

* fix perf.

* fix.

* update metis.

* update launch script

* handle synchronized API.

* fix.

* fix example.

* fix dataloader.

* temp fix.

* temp fix omp.

* distinguish roles.

* initialize iterator of DistDataloader correctly.

* check the correctness of launch script.

* move feature copy to sampler.

* measure mem/network copy time.

* remove

* Revert "measure mem/network copy time."

This reverts commit 86cefdc14b7815fcf5aad6496af912dba48e4aa6.

* fix.

* fix

* fix.

* fix cmake.

* disable metis in windows.

* disable metis tests in windows.

* remove test for multigraph.

* fix test.

* fix.

* fix cmake.

* fix.

* revert.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-115.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>
2020-08-09 18:33:05 -07:00
Da Zheng 5454471fc8 [Distributed] Add roles (#1971)
* distinguish roles.

* add comments.

* fix lint.

* move roles to server_state.

* fix text.

* fix tests.

* fix tests.

* Revert "fix tests."

This reverts commit 5baa136b872a4550d4e612bfb1dfe363d7814adf.
2020-08-08 01:41:51 -07:00
Da Zheng bab32d5ba1 [Distributed] Fix dataloader (#1970)
* fix dataloader.

* initialize iterator of DistDataloader correctly.

* update test.
2020-08-07 18:09:21 -07:00
Jinjing Zhou 4f499c7ffb [Distributed] DistDataloader (#1901)
* 111

* 111

* fix

* 111

* fix

* 11

* fix

* lint

* Update __init__.py

* lint

* fix

* lint

* fix

* fix

* fix

* fix

* fix

* try fix

* try fix

* fix

* Revert "fix"

This reverts commit a0b954fd4e99b7df92b53db8334dcb583d6e1551.

* fixes.

* fix.

* fix test.

* fix exit.

* fix.

* fix

* fix

* lint

* lint

* lint

* fix

* Update .gitignore

* 111

* fix

* 111

* 111

* fff

* 1111

* 111

* 1325315

* ffff

* f???

* fff

* 1111

* 111

* fix

* 111

* asda

* 1111

* 11

* 123

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* spawn

* 1231231

* up

* 111

* fix

* fix

* Revert "fix"

This reverts commit 7373f95312fdcaa36d2fc330bf242339e89c045d.

* fix

* fix

* 1111

* fix

* fix tests

* start kvclient as early as possible.

* lint

* fix test

* lint

* 1111

* fix

* fix

* 111

* fix

* fix

* 1

* fix

* fix

* lint

* fix

* lint

* lint

* remove quit

* fix

* lint

* fix

* fix several

* lint

* fix minor

* fix

* lint

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-08-05 17:43:17 +08:00
Da Zheng a4c931a9bf [Distributed] Support multiple servers (#1886)
* client init graph on the backup servers.

* fix.

* test multi-server.

* fix anonymous dist tensors.

* check #parts.

* fix init_data

* add multi-server multi-client tests.

* update tests in kvstore.

* fix.

* verify the loaded partition.

* fix a bug.

* fix lint.

* fix.

* fix example.

* fix rpc.

* fix pull/push handler for backup kvstore

* fix example readme.

* change ip.

* update docstring.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>
2020-08-02 19:50:34 -07:00
xiang song(charlie.song) 56bbf9cb6d [Distributed][Example] Distributed training for unsupervised graphsage (#1853)
* Standalone can run

* fix

* Add save

* Fix

* Fix

* Fix

* Fix

* debug

* test

* test

* Fix

* Fix

* log

* Fix

* fix

* Profile

* auto sync grad

* update

* add test for unsupervised dist training

* upd

* Fix lr

* Fix update

* sync

* fix

* Revert "fix"

This reverts commit d5caa7398b36125f6d6e2c742a95c6ff4298c9e9.

* Fix

* unsupervised

* Fix

* remove debug

* Add test case for dist_graph find_edges()

* Fix

* skip tensorflow test for find_edges

* Update readme

* remove some test

* upd

* Update partition_graph.py

Co-authored-by: Ubuntu <ubuntu@ip-172-31-68-185.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-24-210.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-08-01 12:24:55 -07:00
Da Zheng 4be4b13424 [Distributed] add copy_partitions.py (#1866)
* fix bugs.

* eval on both vaidation and testing.

* add script.

* update.

* update launch.

* make train_dist.py independent.

* update readme.

* update readme.

* update readme.

* update readme.

* generate undirected graph.

* rename conf_file to part_config

* use rsync

* make train_dist independent.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-115.us-west-2.compute.internal>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-07-31 11:11:56 -07:00
Chao Ma e5e1a99d46 [KVStore] Fix bug in barrier. (#1904)
* update

* update

* fix lint

* update

* update
2020-07-31 22:01:03 +08:00
Chao Ma 5b515cf667 [KVStore] Support group barrier (#1880)
* udpate

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* update

* udpate

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-07-30 19:21:20 +08:00
Da Zheng 3e2f94ed20 [Distributed] Refactor distributed training to new DGLGraph. (#1874)
* fix tests in partition.

* fix DistGraph.

* fix without shared memory.

* fix sampling.

* enable distributed test.

* fix tests.

* fix a bug in shared-mem heterograph.

* print better error messages.

* fix.

* don't specify formats.

* fix.

* fix

* small fix.
2020-07-28 22:07:24 -07:00
Jinjing Zhou 71cf78652d Revert "[Distributed] DistDataloader (#1870)" (#1876)
This reverts commit 6557291fdc.
2020-07-28 17:57:57 +08:00
Jinjing Zhou 6557291fdc [Distributed] DistDataloader (#1870)
* 111

* 111

* fix

* 111

* fix

* 11

* fix

* lint

* Update __init__.py

* lint

* fix

* lint

* fix

* fix

* fix

* fix

* fix

* try fix

* try fix

* fix

* Revert "fix"

This reverts commit a0b954fd4e99b7df92b53db8334dcb583d6e1551.

* fixes.

* fix.

* fix test.

* fix exit.

* fix.

* fix

* fix

* lint

* lint

* lint

* fix

* Update .gitignore

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-07-28 16:36:37 +08:00
Da Zheng 015acfd2d8 [Distributed] Small fixes (#1859)
* fix node/edge_split.

* fix partition.

* support heterograph interface.

* fix test.

* fix

* fix docstring.
2020-07-27 01:39:48 -07:00
Da Zheng ff8f7082a3 [Distributed] turn off recording on embeddings in the inference. (#1861)
* turn on/off recording in sparse embedding.

* add test.
2020-07-27 00:42:40 -07:00
Da Zheng 6963d796c5 [Distributed] update Embedding API (#1847)
* update.

* update Embedding.

* add comments.

* fix lint

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-07-26 11:59:41 -07:00
Jinjing Zhou 33abd2757b [HeteroGraph] Metis partition for hetero (#1843)
* metis for hetero

* fix partition
2020-07-22 16:31:10 +08:00
Da Zheng 562871e76b [Distributed] add distributed evaluation. (#1810)
* add eval.

* extend DistTensor.

* fix.

* add barrier.

* add more print.

* add more checks in kvstore.

* fix lint.

* get all neighbors for eval.

* reorganize.

* fix.

* fix.

* fix.

* fix test.

* add reuse_if_exist.

* add test for reuse_if_exist.

* fix lint.

* fix bugs.

* fix.

* print errors of tcp socket.

* support delete tensors.

* fix lint.

* fix

* fix example

Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>
2020-07-22 00:40:14 -07:00
Chao Ma 5c92f6c218 [RPC] Rpc exit with explicit invocation (#1825)
* exit client

* update

* update

* update

* update

* update

* update

* update

* update test

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-07-20 14:56:26 +08:00
Chao Ma 65e7805e22 [KVStore] Support delete_data() api on client (#1824)
* delete tensor

* update

* update

* update

* update

* update

* update

* udpate

* update

* update

* update
2020-07-20 12:24:23 +08:00
Da Zheng cda0abf75a [Distributed] add the standalone mode in DistGraph (#1800)
* add standalone mode

* add comments.

* add tests for sampling.

* fix.

* make the code to run the standalone mode

* fix

* fix

* fix readme.

* fix.

* fix test

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-07-14 23:46:25 -07:00
Da Zheng a1472bcff6 [Distributed] Run distributed graph server inside DGL (#1801)
* run dist server in dgl.

* fix bugs.

* fix example.

* check environment variables and fix lint.

* fix lint
2020-07-14 07:41:44 -07:00
Da Zheng 167216afc8 [Distributed] add in_subgraph on DistGraph. (#1755)
* add in_subgraph on DistGraph.

* check in more.

* fix test.

* add comments.

* fix test.

* update test.

* update.

* rename.

* update comment

* fix test
2020-07-08 14:39:21 -07:00
Chao Ma bdc1e649e0 [RPC] Client will retry connection until the server starts. (#1758)
* update

* update

* update
2020-07-07 13:34:41 +08:00
Da Zheng afb7f5b839 [Distributed] fix init_data in kvstore. (#1750)
* fix

* fix.

* update.

* fix

* add assert

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-07-06 23:02:00 +08:00
Da Zheng cadcc1c2a6 [Feature] add sparse embedding. (#1497)
* add sparse embedding.

* fix

* add test.

* man fixes.

* many fixes

* fix sparse emb.

* fix.

* fix lint.

* fix lint.

* fix kvstore.

* expose DistTensor.

* test sparse embeddings.

* add attach_grad to the backends.

* remove part_id

* fix.

* move backward computation.

* move more computation to backend.

* fix a bug when applying learning rate.

* fix a few things.

* fix a few things.

* add docstring

* fix.

* apply no_grad.

* fix tests.

* fix for other frameworks.

* add examples in docstring.
2020-07-03 10:42:09 -07:00
Da Zheng 177011744d [Test] Fix test of distributed graph (#1748)
* Revert "Serialize server and client start for dist_graph_store test (#1736)"

This reverts commit da16ebf03d.

* sleep
2020-07-03 10:10:54 -07:00
xiang song(charlie.song) da16ebf03d Serialize server and client start for dist_graph_store test (#1736)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-07-02 15:28:26 +08:00
Chao Ma 9eb0efcf2a [KVStore] Test multi-server and multi-client (#1732)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-07-01 20:26:40 -07:00
Da Zheng 2c88f7c0b0 [Distributed] Split data to ensure data locality for multiple clients (#1710)
* fix.

* fix tests.

* fix

* add tests.

* fix.

* have default rank.

* add comment.

* fix test.

* remove check

* simplify code.

* add test.

* split data evenly.

* simplify the distributed training code.

* add comments.

* add comments.
2020-07-01 13:03:40 -07:00
Chao Ma dd8d5289ef [rpc] Add num_clients and sort client ID by machine-id (#1713)
* add num_clients to kvstore

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* add test

* update

* update

* update

* update

* fix test

* update

* update

* update

* update

* update

* update

* update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-06-28 18:40:38 -07:00
Da Zheng 45e1333e13 [Distributed] Optimize distributed sampling (#1700)
* optimize sampling neighbors from the local partition.

* fix minior bugs.

* fix lint

* overlap local sampling and remote sampling.

* fix.

* fix lint.

* fix a potential bug.

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-06-26 17:05:29 -07:00
Da Zheng 4b96addca0 [BUGFIX] Fix bugs in range partitioning. (#1703)
* Fix bugs in partitioning.

* add comment.

* fix lint.

* fix test.

* add more tests.

* simplify the fix

* fix.
2020-06-26 15:59:45 -07:00
Da Zheng fa1478a4bd [BUGFIX] fix a bug in partition book. (#1698)
* fix a bug.

* add msg for assert
2020-06-24 12:38:58 -07:00
Jinjing Zhou fb248b678a [RPC] Sampling over RPC (#1616)
* fix

* test1111

* 111

* 111

* fff

* lint

* 111

* lint

* lint

* 111

* fijx

* 111

* fix

* 111

* commit

* 111

* 111

* lint

* fix typo

* fix

* lint

* fix

* 111

* support mxnet

* support mxnet

* lint

* remove print

* fix

* fix test.

* fix test.

* fix test.

* try to fix an undetermistic error.

Co-authored-by: Chao Ma <mctt90@gmail.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-06-18 12:05:23 -07:00
Da Zheng ff5b5a4a10 [Distributed] support no shared memory. (#1663)
* support no shared memory.

* add test.

* add CAPI to check existence of shared memory.

* revert the change in src/runtime/ndarray.cc

* update docstring.

* fix compile.
2020-06-18 09:42:11 -07:00
Da Zheng b9ec3e36ec [BUGFIX] reshuffle node features in range partitioning. (#1659)
* fix a bug.

* fix test.

* simplify.

* fix test.

* test edge features.

* fix.
2020-06-17 22:20:17 -07:00
Chao Ma b372b3c796 [KVStore] Add fast-pull for kvstore (#1647)
* add fast-pull

* update

* add fast-pull

* update

* update

* update

* update test

* update

* update

* update

* update

* update

* update

* update

* update

* add omp

* update

* update
2020-06-17 13:50:25 +08:00
Da Zheng e8a56dc1b6 [KVStore] make pull/push handler per tensor. (#1646)
* make pull/push handler per tensor.

* update.
2020-06-15 23:22:49 -07:00
Da Zheng 41349dcef5 [Feature] Range partition (#1522)
* add reorder immutable graph.

* add python API.

* add reorder for csr.

* remove gk version.

* fix

* add cpp test.

* bug fixes

* fix tests.

* fix bugs and add check

* fix test.

* add omp.

* add comments.

* add coo reorder.

* fix a bug.

* handle reorder for different graph structues.

* fix lint.

* fix.

* add original ids.

* reshuffle nodes before metis partition.

* inner nodes are in contiguous Id range.

* reshuffle nodes/edges when partitioning.

* load partition return graph partition book.

* use inner_node/inner_edges

* add and test range partition book.

* count inner_edge correctly.

* fix lint.

* fix lint.

* fix lint.

* fix errors.

* fix errors.

* fix for TF.

* fix.

* fix.

* change docstring.

* support logical and.

* add comments.

* avoid copy.

* fix

* update docstring.

* fix a bug.

* add range search.

* fix

* fix a bug.

* add more tests.

* load graph partition book.

* support shared memory for range partition book.

* fix a bug.

* fix.

* fix lint.

* remove check

* fix test.

* remove num_nodes and num_edges

* fix lint.

* fix graph partition book.

* address comments.

* use makedirs.

* fix compile

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-06-15 11:43:04 -07:00
Jinjing Zhou 77006db209 [CI] Use spawn in unittest instead of fork (#1627)
* update

* update

* update

* update

* update

* update

* fix ci using spawn

Co-authored-by: aksnzhy <mctt90@gmail.com>
2020-06-11 18:15:40 +08:00
Chao Ma 0c313e51e6 [rpc] Move test_rpc.py to distributed and use dynamic port binding (#1623)
* update

* update

* update

* update

* update

* update

* update

* update
2020-06-11 16:01:05 +08:00
Da Zheng 50962c433f Refactor distributed graph store (#1597)
* refactor graph store for new kvstore.

* fix kvstore.

* fix.

* fix lint complains.

* add docstring.

* small fix in graph partition book.

* fix.

* remove tests

* disable sampling.

* Revert "disable sampling."

This reverts commit 70451008f61ff1481d5dadbf8bd199470aee559d.

* Revert "remove tests"

This reverts commit 1394364243bdd73b669abde6193a74e2cda5521d.
2020-06-10 18:51:32 -07:00
Chao Ma 48cc8fa803 [KVStore] Update kvstore test (#1607)
* remove freeze

* update

* update

* fix lint

* update test

* update
2020-06-09 16:25:35 +08:00
Chao Ma 64f49703af [KVStore] Re-write kvstore using DGL RPC infrastructure (#1569)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update init_data

* update server_state

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* debug init_data

* update

* update

* update

* update

* update

* update

* test get_meta_data

* update

* update

* update

* update

* update

* debug push

* update

* update

* update

* update

* update

* update

* update

* update

* update

* use F.reverse_data_type_dict

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint

* update

* fix lint

* update

* update

* update

* update

* fix test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* set random seed

* update
2020-06-04 16:48:53 +08:00
Da Zheng 96984faca4 [Feature] Split data for distributed training (#1505)
* fix tests in graph partition book.

* implement node_split and edge_split.

* fix a bug.

* add tests.

* remove unnecessary code.

* avoid storing graph structure in partition book.

* add partition book in DistGraph.

* Revert "avoid storing graph structure in partition book."

This reverts commit 55b11fbf9293efcae5f8c97e93f1e121399dadae.

* small fixes.

* add a test for boolean mask vector.

* fix test.

* fix bugs.

* make it work for all different cases.

* fix tests.

* fix boolean mask

* fix for TF

* fix for tensorflow.

* fix test for TF

* only support boolean mask for now.

* fix tests.

* make the code more readable.

* fix test.

Co-authored-by: Chao Ma <mctt90@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-05-14 12:46:37 -07:00
Chao Ma ef7e475051 [Distributed] Add GraphPartitionBook (#1496)
* Add graph-partition-book

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* udpate

* fix lint

* update

* update

* update

* update

* update

* update

* update

* skip test on GPU

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* update

* fix unittest

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2020-05-08 14:52:12 +08:00
Da Zheng 16561a2e6b [Test] Add tests for TensorFlow (#1501)
* add test.

* move test code.

* remvoe unnecessary test.

* fix.

* turn on tests for TF.

* Revert "move test code."

This reverts commit e7b4f36395b2121a7be030bd4364a704d0e357bf.

* fix.

* fix.

* skip test for tensorflow.

Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-05-06 00:33:53 -07:00
Da Zheng 2190c39d67 [Feature] Distributed graph store (#1383)
* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code.

* revert modify in sampler.

* fix temp script.

* remove pytorch version.

* fix.

* distributed training with pytorch.

* add distributed graph store.

* fix.

* add metis_partition_assignment.

* fix a few bugs in distributed graph store.

* fix test.

* fix bugs in distributed graph store.

* fix tests.

* remove code of defining DistGraphStore.

* fix partition.

* fix example.

* update run.sh.

* only read necessary node data.

* batching data fetch of multiple NodeFlows.

* simplify gcn.

* remove unnecessary code.

* use the new copy_from_kvstore.

* update training script.

* print time in graphsage.

* make distributed training runnable.

* use val_nid.

* fix train_sampling.

* add distributed training.

* add run.sh

* add more timing.

* fix a bug.

* save graph metadata when partition.

* create ndata and edata in distributed graph store.

* add timing in minibatch training of GraphSage.

* use pytorch distributed.

* add checks.

* fix a bug in global vs. local ids.

* remove fast pull

* fix a compile error.

* update and add new APIs.

* implement more methods in DistGraphStore.

* update more APIs.

* rename it to DistGraph.

* rename to DistTensor

* remove some unnecessary API.

* remove unnecessary files.

* revert changes in sampler.

* Revert "simplify gcn."

This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.

* Revert "simplify neighbor sampling."

This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.

* Revert "measure time in neighbor sampling."

This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.

* Revert "add timing in minibatch training of GraphSage."

This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.

* Revert "fix train_sampling."

This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.

* fix lint.

* add comments and small update.

* add more comments.

* add more unit tests and fix bugs.

* check the existence of shared-mem graph index.

* use new partitioned graph storage.

* fix bugs.

* print error in fast pull.

* fix lint

* fix a compile error.

* save absolute path after partitioning.

* small fixes in the example

* Revert "[kvstore] support any data type for init_data() (#1465)"

This reverts commit 87b6997bf2.

* fix a bug.

* disable evaluation.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* support set and init data.

* support set and init data.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* fix bugs.

* fix unit test.

* move to dgl.distributed.

* fix lint.

* fix lint.

* remove local_nids.

* fix lint.

* fix test.

* remove train_dist.

* revert train_sampling.

* rename funcs.

* address comments.

* address comments.

Use NodeDataView/EdgeDataView to keep track of data.

* address comments.

* address comments.

* revert.

* save data with DGL serializer.

* use the right way of getting shape.

* fix lint.

* address comments.

* address comments.

* fix an error in mxnet.

* address comments.

* add edge_map.

* add more test and fix bugs.

Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
2020-05-02 19:30:51 -07:00
Da Zheng 7c47d8c9b4 [Feature] Simplify shared memory graph index (#1381)
* simplify shared memory graph index.

* fix.

* remove edge_dir in SharedMemGraphStore.

* avoid creating shared-mem graph store with from_csr.

* simplify from_csr.

* add comments.

* fix lint.

* remove the test.

* fix compilation error.

* fix a bug.

* fix a bug.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-04-05 15:33:47 -07:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00