0.5.x_doc
4 次代码提交
| 作者 | SHA1 | 备注 | 提交日期 | |
|---|---|---|---|---|
 Qidong Su
|
40950629b2 |
[Feature] Shared memory utilities (#1807)
* update * update * update * update * fix * update * fix * update * update * win32 * update * fix * update * update * update * updat * update * update * fix * update * update * update * update * update * fix * TODO * 111 * fix * minor fix * minor fix * fox * Update shared_mem_manager.cc * update * update * update * update metis * update metis * update Co-authored-by: VoVAllen <jz1749@nyu.edu> Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com> |
||
|
Chao Ma
|
3e69692233 |
[RPC] New RPC infrastructure. (#1549)
* WIP: rpc components * client & server * move network package to rpc * fix include * fix compile * c api * wip: test * add basic tests * missing file * [RPC] Zero copy serializer (#1517) * zerocopy serialization * add test for HeteroGraph * fix lint * remove unnecessary codes * add comment * lint * lint * disable pylint for now * add include for win * windows guard * lint * lint * skip test on windows * refactor * add comment * fix * comment * 1111 * fix * Update Jenkinsfile * [RPC] Implementation of RPC infra (#1544) * update * update * update * update * update * update * update * update * update * update * update * update * update * update * remove client.cc and server.cc * fix lint * update * update * fix linr * update * fix lint * update * update * update * update * update * update * update test * update * update test * update * update * update * update * update * update * update * update * update * update * update * update comment * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * fix lint * fix lint * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com> |
||
|
Da Zheng
|
2190c39d67 |
[Feature] Distributed graph store (#1383)
* initial version from distributed training.
This is copied from multiprocessing training.
* modify for distributed training.
* it's runnable now.
* measure time in neighbor sampling.
* simplify neighbor sampling.
* fix a bug in distributed neighbor sampling.
* allow single-machine training.
* fix a bug.
* fix a bug.
* fix openmp.
* make some improvement.
* fix.
* add prepare in the sampler.
* prepare nodeflow async.
* fix a bug.
* get id.
* simplify the code.
* improve.
* fix partition.py
* fix the example.
* add more features.
* fix the example.
* allow one partition
* use distributed kvstore.
* do g2l map manually.
* fix commandline.
* a temp script to save reddit.
* fix pull_handler.
* add pytorch version.
* estimate the time for copying data.
* delete unused code.
* fix a bug.
* print id.
* fix a bug
* fix a bug
* fix a bug.
* remove redundent code.
* revert modify in sampler.
* fix temp script.
* remove pytorch version.
* fix.
* distributed training with pytorch.
* add distributed graph store.
* fix.
* add metis_partition_assignment.
* fix a few bugs in distributed graph store.
* fix test.
* fix bugs in distributed graph store.
* fix tests.
* remove code of defining DistGraphStore.
* fix partition.
* fix example.
* update run.sh.
* only read necessary node data.
* batching data fetch of multiple NodeFlows.
* simplify gcn.
* remove unnecessary code.
* use the new copy_from_kvstore.
* update training script.
* print time in graphsage.
* make distributed training runnable.
* use val_nid.
* fix train_sampling.
* add distributed training.
* add run.sh
* add more timing.
* fix a bug.
* save graph metadata when partition.
* create ndata and edata in distributed graph store.
* add timing in minibatch training of GraphSage.
* use pytorch distributed.
* add checks.
* fix a bug in global vs. local ids.
* remove fast pull
* fix a compile error.
* update and add new APIs.
* implement more methods in DistGraphStore.
* update more APIs.
* rename it to DistGraph.
* rename to DistTensor
* remove some unnecessary API.
* remove unnecessary files.
* revert changes in sampler.
* Revert "simplify gcn."
This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.
* Revert "simplify neighbor sampling."
This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.
* Revert "measure time in neighbor sampling."
This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.
* Revert "add timing in minibatch training of GraphSage."
This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.
* Revert "fix train_sampling."
This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.
* fix lint.
* add comments and small update.
* add more comments.
* add more unit tests and fix bugs.
* check the existence of shared-mem graph index.
* use new partitioned graph storage.
* fix bugs.
* print error in fast pull.
* fix lint
* fix a compile error.
* save absolute path after partitioning.
* small fixes in the example
* Revert "[kvstore] support any data type for init_data() (#1465)"
This reverts commit
|
||
|
Da Zheng
|
bfdd1eaa44 |
[Feature] Create shared memory graph store. (#468)
* accelerate gcn_ns. * add timing. * run infer with whole graph. * distributed gcn_ns. * reconstruct gcn_ns. * minor fix. * change graphsage_cv for numa. * fix #OMP threads. * accelerate graphsage_cv. * fix a weird bug. * add profiler in graphsage_cv. * accelerate graphsage_cv. manually aggregate neighbors' embeddings with pull. * load csr directly in gcn_ns_sc. * parallel sort for graph index. * Revert "parallel sort for graph index." This reverts commit 86fe2c7117fe5e56b0d481b39849c258b166945b. * run gcn_ns_sc on GPUs. * acc gcn_cv_sc. * change gcn_cv for numa. * fix gcn_cv to use numa and gpu. * improve graphsage_cv to use numa and gpu. * improve gcn_ns. * improve graphsage_cv. * init shared memory graph store. * fix. * enable init ndata. * improve tests. * add bidirectional communication. * link to rt. * fix compilation error. * fix shared memory init. * use MessageQueue for inter-process communication. * reconstruct immutable graph csr. * fix gcn. * load csr to shared memory. * fix minor bugs. * add comments. * refactor SharedMemory. * fix bugs in ImmutableGraph. * create CSR graph from shared memory. * add more test for loading a csr graph. * terminate graph store properly. * allow initializing ndata in the graph store server. * use RPC for inter-process communication. * a script for loading a graph. * allow customizing port. * list all ndata and edata. * support dtype. * reorganize SharedMemoryGraphStore. * fix ndata shape. * reconstruct gcn_ns. * print info. * set omp in gcn_ns. * reset sampling examples. * fix lint. * fix lint. * reset gcn. * disable shared memory in windows. * fix. * fix. * reset changes. * revert nodeflow changes. * fix cmake. * fix test. * fix test. * fix test. * fix test. * add comments. * fix test. * move vector out. * fix lint. * fix lint. * move SharedMemory. * update cmake. * update comment. * fix comments. * Revert "update cmake." This reverts commit 592445e37077f70a6e3f2e5245f9a3d086b04f3b. * update cmake. * add comments. * rename. * change the comment. * fix a bug. * rename. * add comments. * add comments. * add init_edata. * rewrite memory alloc. * move vector to CSR. * fix. * init data. * Revert "init data." This reverts commit 2b217b9553911b7dd84a9f1d9b68430b5aa18e23. * init data. * init new columns correctly. |