文件历史

提交图

634 次代码提交

作者 SHA1 备注 提交日期
Quan (Andy) Gan 2f6ab43a1b [Model] Pinsage example that uses sparse embedding update (#1676)
* pinsage example that uses sparse embedding update

* add difference from paper
2020-07-06 12:45:48 +08:00
Quan (Andy) Gan 4ddd477f35 [Model] GraphSAGE inductive example (#1741)
* graphsage inductive example

* fix
2020-07-03 11:32:06 +08:00
Quan (Andy) Gan 168a88e570 [Sampling] NodeDataLoader for node classification (#1635)
* neighbor sampler data loader first commit

* more commit

* nodedataloader

* fix

* update RGCN example

* update OGB

* fixes

* fix minibatch RGCN crashing with self loop

* reverting gatconv test code

* fix

* change to new solution that doesn't require tf dataloader

* fix

* lint

* fix

* fixes

* change doc

* fix docstring

* docstring fixes

* return seeds and input nodes from data loader

* fixes

* fix test

* fix windows build problem

* add pytorch wrapper

* fixes

* add pytorch wrapper

* add unit test

* add -1 support to sample_neighbors & fix docstrings

* docstring fix

* lint

* add minibatch rgcn evaluations

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Tong He <hetong007@gmail.com>
2020-07-02 16:31:05 +08:00
Da Zheng 2c88f7c0b0 [Distributed] Split data to ensure data locality for multiple clients (#1710)
* fix.

* fix tests.

* fix

* add tests.

* fix.

* have default rank.

* add comment.

* fix test.

* remove check

* simplify code.

* add test.

* split data evenly.

* simplify the distributed training code.

* add comments.

* add comments.
2020-07-01 13:03:40 -07:00
Hao Xiong c8ec6ee5cc [Example] Experimental results of deepwalk on ogb datasets (#1729)
* ogb-deepwalk

* ?

* ?

* remove pyg
2020-07-01 14:45:12 +08:00
Da Zheng 02d3197407 [Distributed] Pytorch example of distributed GraphSage. (#1495)
* add train_dist.

* Fix sampling example.

* use distributed sampler.

* fix a bug in DistTensor.

* fix distributed training example.

* add graph partition.

* add command

* disable pytorch parallel.

* shutdown correctly.

* load diff graphs.

* add ip_config.txt.

* record timing for each step.

* use ogb

* add profiler.

* fix a bug.

* add train_dist.

* Fix sampling example.

* use distributed sampler.

* fix a bug in DistTensor.

* fix distributed training example.

* add graph partition.

* add command

* disable pytorch parallel.

* shutdown correctly.

* load diff graphs.

* add ip_config.txt.

* record timing for each step.

* use ogb

* add profiler.

* add Ips of the cluster.

* fix exit.

* support multiple clients.

* balance node types and edges.

* move code.

* remove run.sh

* Revert "support multiple clients."

* fix.

* update train_sampling.

* fix.

* fix

* remove run.sh

* update readme.

* update readme.

* use pytorch distributed.

* ensure all trainers run the same number of steps.

* Update README.md

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
2020-06-28 13:09:40 +08:00
Hao Xiong 7ee72b668e [Example] Experimental results on ogbl-citation & ogbl-ppa (#1711)
* ogb-deepwalk

* update readme

* update readme

* update readme

* update readme

* ogbl-ddi

* readme

* citation&ppa

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-28 11:37:01 +08:00
Hao Xiong f9fd3107ae [Example] Experimental results over ogbl-ddi (#1707)
* ogb-deepwalk

* update readme

* update readme

* update readme

* update readme

* ogbl-ddi

* readme

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-27 14:01:44 +08:00
Hao Xiong 8d16d4bb6b [Example] Test deepwalk on ogb leaderboard (#1689)
* ogb-deepwalk

* update readme

* update readme

* update readme

* update readme

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-25 23:09:47 +08:00
Tong He 558673e139 [Model] PointNet and PointNet++ for point cloud (#1510)
* commit patch

* commit patch

* pointnet basic

* fix data

* reorg

* reorg

* temp status

* remove validate set

* add partseg data and model

* partseg miou

* clean up

* fix loss

* network definition match paper

* fix

* fix miou

* update data format

* fix

* fix

* working pointnet ssg cls

* avoid some pytorch bug

* fix script

* update hyperparams

* add msg module

* try different dataset

* update new dataset info

* quick fix to subgraph

* fix speed

* update training

* update

* fix bs

* update docstring

* update

* update

* remove parallel reduction in fps

* switch to kernel fps, training is 30% faster

Co-authored-by: Ubuntu <ubuntu@ip-172-31-20-181.us-west-2.compute.internal>
2020-06-22 19:50:40 +08:00
xiang song(charlie.song) 8192b10b08 [Example] Sample-based GAT on ogbn-product dataset (#1613)
* Add ogbn-product with GAT with GraphSAGE sampling

* Update Readme

* Add self loop

* update score

* Update score

* Fix

* Update

* upd

Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-186.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Zihao Ye <expye@outlook.com>
2020-06-20 22:42:47 +08:00
Theofilos Papapanagiotou 3ebf26607e correct PinSAGE knn example syntax (#1617)
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Chao Ma <mctt90@gmail.com>
2020-06-15 15:47:00 +08:00
xiang song(charlie.song) 8531ee6aa6 Fix input data source (#1612)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-52-181.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-06-10 16:08:43 +08:00
Mufei Li cbe4c28fea Fix (#1596) 2020-06-07 16:43:36 +08:00
Mufei Li 36c7b77167 [LifeSci] Move to Independent Repo (#1592)
* Move LifeSci

* Remove doc
2020-06-05 21:28:24 +08:00
Hao Xiong 94c67203dc [Example] Implement deepwalk by dgl and pytorch (#1503)
* deepwalk

* add docstr, etc.

* add tested version

* some doc

* some doc

* add sample and some docs, fix a bug

* update speed

* update speed

* docs

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-04 23:35:52 +08:00
Mufei Li 90d2118da5 [Dataset] Change Cora Split (#1583)
* Use new version of Cora

* Fix import
2020-06-03 15:45:38 +08:00
Lingfan Yu e979685ad9 [Model] Add SIGN model to example (#1571)
* sign model

* mini-batch GPU eval & set default hyper-parameters

* add readme for sign

* minor
2020-05-31 17:15:01 +08:00
xiang song(charlie.song) 901e0c24d6 [Example][Optimization] Performance optimization for graphsage unsupervised example (#1531)
* test

* profile

* opt

* Some fix

* upd

* upd

* Add multigpu training support for graphsage unsupervised

* Add share neg

* Fix

* Add profile

* turn on eval

* upd

* Fix

* performance opt

Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-103.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-53.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-29.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-50.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-52-181.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-87-240.ec2.internal>
2020-05-27 11:47:02 +08:00
Quan (Andy) Gan 0b902d032a [Bug][Model] Bugfix in PinSAGE code (#1533) 2020-05-18 00:14:50 +08:00
Mufei Li 7b6af87a90 Update (#1534) 2020-05-17 21:17:03 +08:00
Quan (Andy) Gan ccd4eab04e removing old pinsage implementation (#1525) 2020-05-15 11:37:27 +08:00
Quan (Andy) Gan 1e84168edd [Model] PinSAGE model with new sampler (#1334)
* another first commit

* test commit

* porting to nowplaying

* fixes

* more updates

* update readme

* add performance

* update

* switch to return_uv

* update result

* Update examples/pytorch/pinsage/process_nowplaying_rs.py

Co-authored-by: Xiagkun Hu <huxk_hit@qq.com>

* update with comments

Co-authored-by: Xiagkun Hu <huxk_hit@qq.com>
2020-05-14 14:04:44 +08:00
jianmohuo 9b34b1c22a [Model] Spatial-temporal Graph Neural Networks for Traffic Prediction (#1445)
* stgcn_wave model

* fix readme

* rm data file

* split sensors2graph

* rm dead code

* fix README

* rename class

* rm seed & dead code

* Update README.md

* rm dead code & networkx

* add num_layer papram, make model structure adjustable

* fix

* add model structure controller string, make code easier to understand and make model strcture more flexible

* Update main.py

* Update model.py

* fix

* Update README.md

Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-255.ap-northeast-1.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-05-12 07:25:39 -07:00
Quan (Andy) Gan ed60f40123 [Model] OGB products submission (#1512) 2020-05-12 16:01:28 +08:00
Quan (Andy) Gan de34e15abb fix metapath2vec on custom datasets (#1499) 2020-05-08 12:51:10 +08:00
Linfang He 6ae440dbb5 [Model] GATNE-T (#1470)
* Add an example for GATNE-T

* Update README.md

* Add links for datasets

* Update README.md with running results

* Update README.md

* Add comments

* Update main.py

* Remove node type name `user`

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-05-05 22:35:25 -07:00
Da Zheng 2190c39d67 [Feature] Distributed graph store (#1383)
* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code.

* revert modify in sampler.

* fix temp script.

* remove pytorch version.

* fix.

* distributed training with pytorch.

* add distributed graph store.

* fix.

* add metis_partition_assignment.

* fix a few bugs in distributed graph store.

* fix test.

* fix bugs in distributed graph store.

* fix tests.

* remove code of defining DistGraphStore.

* fix partition.

* fix example.

* update run.sh.

* only read necessary node data.

* batching data fetch of multiple NodeFlows.

* simplify gcn.

* remove unnecessary code.

* use the new copy_from_kvstore.

* update training script.

* print time in graphsage.

* make distributed training runnable.

* use val_nid.

* fix train_sampling.

* add distributed training.

* add run.sh

* add more timing.

* fix a bug.

* save graph metadata when partition.

* create ndata and edata in distributed graph store.

* add timing in minibatch training of GraphSage.

* use pytorch distributed.

* add checks.

* fix a bug in global vs. local ids.

* remove fast pull

* fix a compile error.

* update and add new APIs.

* implement more methods in DistGraphStore.

* update more APIs.

* rename it to DistGraph.

* rename to DistTensor

* remove some unnecessary API.

* remove unnecessary files.

* revert changes in sampler.

* Revert "simplify gcn."

This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.

* Revert "simplify neighbor sampling."

This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.

* Revert "measure time in neighbor sampling."

This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.

* Revert "add timing in minibatch training of GraphSage."

This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.

* Revert "fix train_sampling."

This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.

* fix lint.

* add comments and small update.

* add more comments.

* add more unit tests and fix bugs.

* check the existence of shared-mem graph index.

* use new partitioned graph storage.

* fix bugs.

* print error in fast pull.

* fix lint

* fix a compile error.

* save absolute path after partitioning.

* small fixes in the example

* Revert "[kvstore] support any data type for init_data() (#1465)"

This reverts commit 87b6997bf2.

* fix a bug.

* disable evaluation.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* support set and init data.

* support set and init data.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* fix bugs.

* fix unit test.

* move to dgl.distributed.

* fix lint.

* fix lint.

* remove local_nids.

* fix lint.

* fix test.

* remove train_dist.

* revert train_sampling.

* rename funcs.

* address comments.

* address comments.

Use NodeDataView/EdgeDataView to keep track of data.

* address comments.

* address comments.

* revert.

* save data with DGL serializer.

* use the right way of getting shape.

* fix lint.

* address comments.

* address comments.

* fix an error in mxnet.

* address comments.

* add edge_map.

* add more test and fix bugs.

Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
2020-05-02 19:30:51 -07:00
Quan (Andy) Gan 05b0f1ea2f [Model] Shared memory history tensor for multi-gpu training of control variate methods (#1479)
* shared memory history for multi-gpu training

* reorg

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-28 19:13:50 +08:00
Quan (Andy) Gan dc0432e737 [Model] Unsupervised learning with GraphSAGE (#1440)
* unsupervised graphsage first commit

* fix

* disable remove_edges and still got 0.90 performance

* optimize edgeids with multimap

* change hyperparams

* update README

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-28 17:19:33 +08:00
Quan (Andy) Gan 97bb85d505 [Model] GraphSAGE with control variate sampling on new sampler (#1355)
* control variate first commit

* bug fixes

* split to single and multi GPU

* update readme

* bugfix

* bugfix

* remove push

* bugfix on multi gpu

* update README

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-26 22:06:21 +08:00
xiang song(charlie.song) 0c0a897413 Fix lr (#1457)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-38-40.ec2.internal>
2020-04-25 00:00:57 +08:00
xiang song(charlie.song) 28117cd96d [Example] GCMC with sampling (#1296)
* gcmc example

* Update Readme

* Add multiprocess support

* Fix

* Multigpu + dataloader

* Delete some dead code

* Delete more

* upd

* Add README

* upd

* Upd

* combine full batch and sample GCMCLayer, use HeteroCov

* Fix

* Update Readme

* udp

* Fix typo

* Add cpu run

* some fix and docstring

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-63-71.ec2.internal>
2020-04-13 23:43:03 +08:00
Mufei Li 88c344876f [Example] Update README of transformers for multi-gpu support (#1435)
* Update README.md

* Update README.md
2020-04-09 19:47:32 +08:00
Minjie Wang 3efb5d8ecf [NN] Add HeteroGraphConv module for cleaner module definition (#1385)
* Add HeteroGraphConv

* add custom aggregator; some docstring

* debugging

* rm print

* fix some acc bugs

* fix initialization problem in weight basis

* passed tests

* lint

* fix graphconv flag; add error message

* add mxnet heteroconv

* more fix for mx

* lint

* fix torch cuda test

* fix mx test_nn

* add exhaust test for graphconv

* add tf heteroconv

* fix comment
2020-03-27 18:47:06 +08:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Quan (Andy) Gan 0a51dc5435 [Bug] Fix dsttype in GraphSAGE minibatch model (#1371)
* fix for new ntype API for blocks

* adding two new interfaces
2020-03-18 23:57:03 +08:00
Minjie Wang 6111ea4665 [Model] RGCN mini-batch training; Better bipartite graph support (#1337)
* change the model to use node embedding only

* minibatch training

* add readme

* small fix

* backward-compatible

* backward compatible

* modify to_block; rgcn changed

* fix

* fix transform

* fix bug in unittest script

* docstring

* fix lint

* add tests

* address comments; fix offline eval

* gitignore
2020-03-16 18:58:06 +08:00
Minjie Wang d876680a36 [Model][Perf] Improve sage sampling performance (#1364)
* improve speed

* fix bugs

* upd reg test
2020-03-15 23:20:30 +08:00
hgalioulline 6c7c403914 [Feature] Filtered MRR metrics for R-GCN example (#1298)
* Add filtered metrics for R-GCN example

* Add new line to end of file

* Add evaluation protocol argument option for R-GCN example

* Update README

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-03-13 16:25:03 +08:00
Quan (Andy) Gan 20e1bb455b rewrite to use dataloader (#1333)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-10 09:22:41 +08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Mufei Li 4ec8f204dd Update (#1319) 2020-03-05 22:58:29 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Jinjing Zhou c23a61bd5c fix s3 link (#1310) 2020-03-04 21:01:29 +08:00
Quan (Andy) Gan b05cb84a77 [BUG] Fixes #1262 (#1286) 2020-02-24 13:33:55 +08:00
Chao Ma ddf8c858e1 [KVStore] Small fix on kvstore (#1284)
* update

* str to int

* update

* update

* update

* test remove sync

* update

* update

* update

* update

* update

* update

* update

* update

* clear
2020-02-23 15:43:34 +08:00
Chao Ma b133abb82e [KVStore] New kvstore used by DGL-KE (#1263)
* new kvstore

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* test warning

* update

* update

* udpate

* update

* update

* update

* update

* small fix

* small fix

* get group count

* update

* update

* make file

* update

* use addr

* get id

* partition book

* update

* partition

* barrier

* update

* loop count

* update

* update

* update

* update

* update

* update

* update

* update

* update

* add mxnet demo

* update ip

* update

* update

* update

* random

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* fix lint
2020-02-17 13:39:13 +08:00
Zihao Ye 7a80faf120 [Bugfix] Correct the loss function in RGCN model (#1217)
* upd

* upd

* fix

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-02-13 18:35:07 +08:00
Mufei Li 3f0c10058a [DGL-LifeSci] Fix broken links (#1248)
* Fix

* Update
2020-02-08 18:42:51 +08:00