文件历史

提交图

622 次代码提交

作者 SHA1 备注 提交日期
xiang song(charlie.song) 8531ee6aa6 Fix input data source (#1612)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-52-181.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-06-10 16:08:43 +08:00
Mufei Li cbe4c28fea Fix (#1596) 2020-06-07 16:43:36 +08:00
Mufei Li 36c7b77167 [LifeSci] Move to Independent Repo (#1592)
* Move LifeSci

* Remove doc
2020-06-05 21:28:24 +08:00
Hao Xiong 94c67203dc [Example] Implement deepwalk by dgl and pytorch (#1503)
* deepwalk

* add docstr, etc.

* add tested version

* some doc

* some doc

* add sample and some docs, fix a bug

* update speed

* update speed

* docs

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2020-06-04 23:35:52 +08:00
Mufei Li 90d2118da5 [Dataset] Change Cora Split (#1583)
* Use new version of Cora

* Fix import
2020-06-03 15:45:38 +08:00
Lingfan Yu e979685ad9 [Model] Add SIGN model to example (#1571)
* sign model

* mini-batch GPU eval & set default hyper-parameters

* add readme for sign

* minor
2020-05-31 17:15:01 +08:00
xiang song(charlie.song) 901e0c24d6 [Example][Optimization] Performance optimization for graphsage unsupervised example (#1531)
* test

* profile

* opt

* Some fix

* upd

* upd

* Add multigpu training support for graphsage unsupervised

* Add share neg

* Fix

* Add profile

* turn on eval

* upd

* Fix

* performance opt

Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-103.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-53.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-29.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-50.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-52-181.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-87-240.ec2.internal>
2020-05-27 11:47:02 +08:00
Quan (Andy) Gan 0b902d032a [Bug][Model] Bugfix in PinSAGE code (#1533) 2020-05-18 00:14:50 +08:00
Mufei Li 7b6af87a90 Update (#1534) 2020-05-17 21:17:03 +08:00
Quan (Andy) Gan ccd4eab04e removing old pinsage implementation (#1525) 2020-05-15 11:37:27 +08:00
Quan (Andy) Gan 1e84168edd [Model] PinSAGE model with new sampler (#1334)
* another first commit

* test commit

* porting to nowplaying

* fixes

* more updates

* update readme

* add performance

* update

* switch to return_uv

* update result

* Update examples/pytorch/pinsage/process_nowplaying_rs.py

Co-authored-by: Xiagkun Hu <huxk_hit@qq.com>

* update with comments

Co-authored-by: Xiagkun Hu <huxk_hit@qq.com>
2020-05-14 14:04:44 +08:00
jianmohuo 9b34b1c22a [Model] Spatial-temporal Graph Neural Networks for Traffic Prediction (#1445)
* stgcn_wave model

* fix readme

* rm data file

* split sensors2graph

* rm dead code

* fix README

* rename class

* rm seed & dead code

* Update README.md

* rm dead code & networkx

* add num_layer papram, make model structure adjustable

* fix

* add model structure controller string, make code easier to understand and make model strcture more flexible

* Update main.py

* Update model.py

* fix

* Update README.md

Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-255.ap-northeast-1.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-05-12 07:25:39 -07:00
Quan (Andy) Gan ed60f40123 [Model] OGB products submission (#1512) 2020-05-12 16:01:28 +08:00
Quan (Andy) Gan de34e15abb fix metapath2vec on custom datasets (#1499) 2020-05-08 12:51:10 +08:00
Linfang He 6ae440dbb5 [Model] GATNE-T (#1470)
* Add an example for GATNE-T

* Update README.md

* Add links for datasets

* Update README.md with running results

* Update README.md

* Add comments

* Update main.py

* Remove node type name `user`

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2020-05-05 22:35:25 -07:00
Da Zheng 2190c39d67 [Feature] Distributed graph store (#1383)
* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code.

* revert modify in sampler.

* fix temp script.

* remove pytorch version.

* fix.

* distributed training with pytorch.

* add distributed graph store.

* fix.

* add metis_partition_assignment.

* fix a few bugs in distributed graph store.

* fix test.

* fix bugs in distributed graph store.

* fix tests.

* remove code of defining DistGraphStore.

* fix partition.

* fix example.

* update run.sh.

* only read necessary node data.

* batching data fetch of multiple NodeFlows.

* simplify gcn.

* remove unnecessary code.

* use the new copy_from_kvstore.

* update training script.

* print time in graphsage.

* make distributed training runnable.

* use val_nid.

* fix train_sampling.

* add distributed training.

* add run.sh

* add more timing.

* fix a bug.

* save graph metadata when partition.

* create ndata and edata in distributed graph store.

* add timing in minibatch training of GraphSage.

* use pytorch distributed.

* add checks.

* fix a bug in global vs. local ids.

* remove fast pull

* fix a compile error.

* update and add new APIs.

* implement more methods in DistGraphStore.

* update more APIs.

* rename it to DistGraph.

* rename to DistTensor

* remove some unnecessary API.

* remove unnecessary files.

* revert changes in sampler.

* Revert "simplify gcn."

This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.

* Revert "simplify neighbor sampling."

This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.

* Revert "measure time in neighbor sampling."

This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.

* Revert "add timing in minibatch training of GraphSage."

This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.

* Revert "fix train_sampling."

This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.

* fix lint.

* add comments and small update.

* add more comments.

* add more unit tests and fix bugs.

* check the existence of shared-mem graph index.

* use new partitioned graph storage.

* fix bugs.

* print error in fast pull.

* fix lint

* fix a compile error.

* save absolute path after partitioning.

* small fixes in the example

* Revert "[kvstore] support any data type for init_data() (#1465)"

This reverts commit 87b6997bf2.

* fix a bug.

* disable evaluation.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* support set and init data.

* support set and init data.

* Revert "Revert "[kvstore] support any data type for init_data() (#1465)""

This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.

* fix bugs.

* fix unit test.

* move to dgl.distributed.

* fix lint.

* fix lint.

* remove local_nids.

* fix lint.

* fix test.

* remove train_dist.

* revert train_sampling.

* rename funcs.

* address comments.

* address comments.

Use NodeDataView/EdgeDataView to keep track of data.

* address comments.

* address comments.

* revert.

* save data with DGL serializer.

* use the right way of getting shape.

* fix lint.

* address comments.

* address comments.

* fix an error in mxnet.

* address comments.

* add edge_map.

* add more test and fix bugs.

Co-authored-by: Zheng <dzzhen@186590dc80ff.ant.amazon.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
2020-05-02 19:30:51 -07:00
Quan (Andy) Gan 05b0f1ea2f [Model] Shared memory history tensor for multi-gpu training of control variate methods (#1479)
* shared memory history for multi-gpu training

* reorg

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-28 19:13:50 +08:00
Quan (Andy) Gan dc0432e737 [Model] Unsupervised learning with GraphSAGE (#1440)
* unsupervised graphsage first commit

* fix

* disable remove_edges and still got 0.90 performance

* optimize edgeids with multimap

* change hyperparams

* update README

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-28 17:19:33 +08:00
Quan (Andy) Gan 97bb85d505 [Model] GraphSAGE with control variate sampling on new sampler (#1355)
* control variate first commit

* bug fixes

* split to single and multi GPU

* update readme

* bugfix

* bugfix

* remove push

* bugfix on multi gpu

* update README

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-04-26 22:06:21 +08:00
xiang song(charlie.song) 0c0a897413 Fix lr (#1457)
Co-authored-by: Ubuntu <ubuntu@ip-172-31-38-40.ec2.internal>
2020-04-25 00:00:57 +08:00
xiang song(charlie.song) 28117cd96d [Example] GCMC with sampling (#1296)
* gcmc example

* Update Readme

* Add multiprocess support

* Fix

* Multigpu + dataloader

* Delete some dead code

* Delete more

* upd

* Add README

* upd

* Upd

* combine full batch and sample GCMCLayer, use HeteroCov

* Fix

* Update Readme

* udp

* Fix typo

* Add cpu run

* some fix and docstring

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-63-71.ec2.internal>
2020-04-13 23:43:03 +08:00
Mufei Li 88c344876f [Example] Update README of transformers for multi-gpu support (#1435)
* Update README.md

* Update README.md
2020-04-09 19:47:32 +08:00
Minjie Wang 3efb5d8ecf [NN] Add HeteroGraphConv module for cleaner module definition (#1385)
* Add HeteroGraphConv

* add custom aggregator; some docstring

* debugging

* rm print

* fix some acc bugs

* fix initialization problem in weight basis

* passed tests

* lint

* fix graphconv flag; add error message

* add mxnet heteroconv

* more fix for mx

* lint

* fix torch cuda test

* fix mx test_nn

* add exhaust test for graphconv

* add tf heteroconv

* fix comment
2020-03-27 18:47:06 +08:00
Mufei Li 0f40c6e49c [Hetero] Replace card with num_nodes
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-20 16:56:53 +08:00
Quan (Andy) Gan 0a51dc5435 [Bug] Fix dsttype in GraphSAGE minibatch model (#1371)
* fix for new ntype API for blocks

* adding two new interfaces
2020-03-18 23:57:03 +08:00
Minjie Wang 6111ea4665 [Model] RGCN mini-batch training; Better bipartite graph support (#1337)
* change the model to use node embedding only

* minibatch training

* add readme

* small fix

* backward-compatible

* backward compatible

* modify to_block; rgcn changed

* fix

* fix transform

* fix bug in unittest script

* docstring

* fix lint

* add tests

* address comments; fix offline eval

* gitignore
2020-03-16 18:58:06 +08:00
Minjie Wang d876680a36 [Model][Perf] Improve sage sampling performance (#1364)
* improve speed

* fix bugs

* upd reg test
2020-03-15 23:20:30 +08:00
hgalioulline 6c7c403914 [Feature] Filtered MRR metrics for R-GCN example (#1298)
* Add filtered metrics for R-GCN example

* Add new line to end of file

* Add evaluation protocol argument option for R-GCN example

* Update README

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
2020-03-13 16:25:03 +08:00
Quan (Andy) Gan 20e1bb455b rewrite to use dataloader (#1333)
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-10 09:22:41 +08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Mufei Li 4ec8f204dd Update (#1319) 2020-03-05 22:58:29 +08:00
Zihao Ye 93ac29ce34 [Refactor] Unify DGLGraph, BatchedDGLGraph and DGLSubGraph (#1216)
* upd

* upd

* upd

* lint

* fix

* fix test

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd tutorial

* upd

* upd

* fix kg

* upd doc organization

* refresh test

* upd

* refactor doc

* fix lint

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-03-05 18:03:57 +08:00
Jinjing Zhou c23a61bd5c fix s3 link (#1310) 2020-03-04 21:01:29 +08:00
Quan (Andy) Gan b05cb84a77 [BUG] Fixes #1262 (#1286) 2020-02-24 13:33:55 +08:00
Chao Ma ddf8c858e1 [KVStore] Small fix on kvstore (#1284)
* update

* str to int

* update

* update

* update

* test remove sync

* update

* update

* update

* update

* update

* update

* update

* update

* clear
2020-02-23 15:43:34 +08:00
Chao Ma b133abb82e [KVStore] New kvstore used by DGL-KE (#1263)
* new kvstore

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* test warning

* update

* update

* udpate

* update

* update

* update

* update

* small fix

* small fix

* get group count

* update

* update

* make file

* update

* use addr

* get id

* partition book

* update

* partition

* barrier

* update

* loop count

* update

* update

* update

* update

* update

* update

* update

* update

* update

* add mxnet demo

* update ip

* update

* update

* update

* random

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* fix lint
2020-02-17 13:39:13 +08:00
Zihao Ye 7a80faf120 [Bugfix] Correct the loss function in RGCN model (#1217)
* upd

* upd

* fix

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-02-13 18:35:07 +08:00
Mufei Li 3f0c10058a [DGL-LifeSci] Fix broken links (#1248)
* Fix

* Update
2020-02-08 18:42:51 +08:00
xiang song(charlie.song) 77ae4d5321 [Example] Fix Rgcn hetero nonedetermistic in graph and model initialization (#1244)
* Fix model save and load problem.

Add offline inference example

* Fix test

* Fix randomness problem between test runs

* Make rdf loader deterministic
2020-02-07 11:07:09 +08:00
Mufei Li 828a5e5bc6 [DGL-LifeSci] Migration and Refactor (#1226)
* First commit

* Update

* Update splitters

* Update

* Update

* Update

* Update

* Update

* Update

* Migrate ACNN

* Fix

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Finish classification

* Update

* Fix

* Update

* Update

* Update

* Fix

* Fix

* Fix

* Update

* Update

* Update

* trigger CI

* Fix CI

* Update

* Update

* Update

* Add default values

* Rename

* Update deprecation message
2020-02-04 01:38:09 +08:00
Quan (Andy) Gan a9dabcc769 [Feature] Random Walk for 0.5 (#1209)
* trying to refactor IndexSelect

* partial implementation

* add index select and assign for floats as well

* move to random choice source

* more updates

* fixes

* fixes

* more fixes

* adding python impl

* fixes

* unit test

* lint

* lint x2

* lint x3

* update metapath2vec

* debugging performance

* still debugging for performance

* tuning

* switching to succvec

* redo

* revert non-uniform sampler to use vector

* still not fast

* why does this crash with OpenMP???

* because there was a data race!!!

* add documentations and remove assign op

* lint

* lint x2

* lol what have i done

* lint x3

* fix and disable gpu testing

* bugfix

* generic random walk

* reorg the random walk source code

* Update randomwalks.h

* Update randomwalks_cpu.cc

* rename file

* move internal function to anonymous ns

* reorg & docstrings

* constant restart probability

* docstring fix

* more commit

* random walk with restart, tested

* some fixes

* switch to using NDArray for choice

* massive fix & docstring

* lint x?

* lint x??

* fix

* export symbols

* skip gpu test

* addresses comments

* replaces another VecToIdArray

* add randomwalks.h to include

* replace void * with template
2020-01-31 19:52:08 +08:00
VoVAllen a00636a02b [Tensorflow] Several nn & example (#1191)
* several nn example

* appnp

* fix lint

* lint

* add dgi

* fix

* fix

* fix

* fff

* docs

* 111

* fix

* change init

* change result

* tiaocan+1

* fix

* fix lint

* fix

* fix
2020-01-19 22:48:40 +08:00
Quan (Andy) Gan 3d6646288f Update README.md (#1184) 2020-01-09 12:06:48 +08:00
Quan (Andy) Gan bfc2d3030d [BUG] Fix MovieLens feature assignment (#1183)
Fixes #1182 .

This is embarrassing.
2020-01-09 11:21:18 +08:00
Quan (Andy) Gan e705118bcc some comments (#1132) 2020-01-06 10:36:11 +08:00
Chao Ma 6731ea3af7 [KVStore] Reduce memory cost of kvstore (#1156)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint
2019-12-30 20:25:32 +08:00
Chao Ma 4f02bb751b [KVStore] Made kvstore can support multiple NICs (#1150)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1

* add multiple NICs support for kvstore

* test

* fix lint

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update
2019-12-30 15:17:31 +08:00
Chao Ma d57ff78da1 [Small Fix] Change default message queue size of communicator to 2GB (#1140)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument

* change default queue size to 2GB

* OMP_NUM_THREADS=1
2019-12-25 17:03:52 +08:00
Chao Ma e4ef8d1a80 [KVStore] API change of kvstore (#1058)
* API change of kvstore

* add demo for kvstore

* update

* remove duplicated log

* change queue size

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* change name

* update

* fix lint

* update

* update

* update

* update

* change message queue size to a python argument
2019-12-24 17:05:15 +08:00
Mufei Li 2ab4d0e10f [Doc] Hotfix for README (#1119) 2019-12-23 19:22:38 +08:00