文件历史

27 次代码提交

作者 SHA1 备注 提交日期
xiang song(charlie.song) cd9599ab04 [KG][Optimization] Remove copy from parent in minibatch generation (#1193)
* remote copy from parent

* upd

* make test_sampler easier to pass
2020-01-10 15:59:09 -08:00
xiang song(charlie.song) bb6a64763e [Feature][KG] Multi-GPU training support for DGL KGE (#1178)
* multi-gpu

* Pytorch can run but test has acc problem

* pytorch train/eval can run in multi-gpu

* Fix eval

* Fix

* Fix mxnet

* trigger

* triger

* Fix mxnet score_func

* Fix

* check

* FIx default arg

* Fix train_mxnet mix_cpu_gpu

* Make relation mix_cpu_gpu

* delete some dead code

* some opt for update

* Fix cpu grad update
2020-01-08 18:07:04 +08:00
Da Zheng 1022d5d64d [KG] PBG's way of constructing negative edges (#1159)
* attach positive.

* add neg_deg_sample.

* add comment.

* add neg_deg_sample for eval.

* change the edge sampler.

* rename edge sampler in KG.

* allow specifying chunk size and negative sample size separately.

* fix bugs in KG.

* add check in sampler.

* add more checks.

* fix

* add comment.

* add comments.
2020-01-04 19:25:06 -08:00
Da Zheng 1de192f445 [KG] More fixes on eval (#1168)
* remove parallel sampling for multiprocessing.

* avoid memory copy in eval.

* remove print.
2020-01-04 13:33:13 -08:00
Da Zheng 346bc23556 [KG] Disable filter in evaluation (#1162)
* add no_eval_filter

* fix eval.
2020-01-03 13:02:34 +08:00
Da Zheng 7451bb2a01 merge eval results in all processes. (#1160) 2020-01-01 21:40:31 -08:00
Da Zheng 162dc19a5d Accelerate edge sampling (#1151)
* accel edge sampling.

* measure sampling time.

* attach positive.

* add neg_deg_sample.

* fix a minor bug.

* remove unnecessary code.

* Revert "add neg_deg_sample."

This reverts commit 45f3c08c87e33589563ca7e6a3ec91cb1239910e.

* Revert "attach positive."

This reverts commit 503d7821555b69b7ccfed55ce8b87ee24fff852d.

Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
2019-12-31 01:40:20 -08:00
Da Zheng f81841532f [KG] reduce memory consumption. (#902)
* reduce memory consumption.

* fix a bug.

* fix a bug.

* fix.
2019-12-28 21:55:50 -08:00
xiang song(charlie.song) 41f8a162f4 [Feature Improvement]Edge Sampler w and w/o replacement (#1116)
* Add weight based edge sampler

* Can run, edge weight work.
TODO: test node weight

* Fix node weight sample

* Fix y

* Update doc

* Fix syntex

* Fix

* Fix GPU test for sampler

* Fix test

* Fix

* Refactor EdgeSampler to act as class object not function that it
can record its own private states.

* clean

* Fix

* Fix

* Fix run bug on kg app

* update

* update test

* test

* Simply python API and fix some C code

* Fix

* Fix

* Fix syntex

* Fix

* Update API description

* add replacement for edge sampler

* Now edge sampler support replacement and no-replacement

* Fix

* Fix

* change kg/app to use edge sampler with replacement config

* Update replacement algo

* Fix syntax

* Update

* Update

Co-authored-by: Da Zheng <zhengda1936@gmail.com>
2019-12-24 16:25:40 +08:00
VoVAllen d30a69bf27 [Backend] TF backend (#978)
* tf

* add builtin support

* fiix

* pytest

* fix

* fix

* fix some bugs

* fix selecting

* fix todo

* fix test

* fix test fail in tf

* fix

* fix

* fix gather row

* fix gather row

* log backend

* fix gather row

* fix gather row

* fix for pytorch

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix convert

* fix

* fix

* fix

* fix inplace

* add alignment setting

* add debug option

* Revert "add alignment setting"

This reverts commit ec63fb3506ea84fff7d447a1fbdfd1d5d1fb6110.

* tf ci

* fix lint

* fix lint

* add tfdlpack

* fix type

* add env

* fix backend

* fix

* fix tests

* remove one_hot

* remove comment

* remove comment

* fix

* use pip to install all

* fix test

* fix base

* fix

* fix

* add skip

* upgrade cmake

* change version

* change ci

* fix

* fix

* fix

* fix

* fix seg fault

* fix

* fix python version

* fix

* try fix

* fix

* fix

* tf takes longer time in ci

* change py version

* fix

* fix

* fix oom

* change kg env

* change kg env

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* 我再也不搞各种乱七八糟环境了……

* use pytest

* Chang image
2019-12-20 15:56:51 +08:00
Da Zheng 1552090a24 set default backend. (#1104) 2019-12-13 08:30:14 -08:00
Da Zheng ae743dcc02 remove tqdm. (#1051) 2019-12-01 14:41:45 -08:00
xiang song(charlie.song) dca0e376e0 [KG][Score Func] Update TransE with L2 distance support. (#1059)
* Add L2 distance score for TransE

* Update README.md

* Use linalg.gemm to speedup mx l2 dist

* Fix
2019-12-01 14:03:36 -08:00
MilkshakeForReal 8b17a5c1d5 [Model] add RotatE to dgl-kg (#964)
Add RotatE support for KGE (apps/kg)
Performance Result:
Dataset FB15k:
Result from Paper:
MR: 40
MRR: 0.797
HIT@1: 74.6
HIT@3: 83.0
HIT@10: 88.4

Our Impl:
MR: 39.6
MRR: 0.725
HIT@1: 62.8
HIT@3: 80.2
HIT@10: 87.5
2019-11-14 10:39:05 +08:00
xiang song(charlie.song) 7f65199a86 [NN]Supporting TransR in app/kg score_func (#945)
* Add TransR for kge

* Now Pytorch TransR can run

* Add MXNet TransR

* Now mxnet can work with small dim size

* Add test

* Pass simple test_score

* Update test with transR score func

* Update RESCAL MXNet

* Add missing funcs

* Update init func for transR score

* Revert "Update init func for transR score"

This reverts commit 0798bb886095e7581f6675da5343376844ce45b9.

* Update score func of TransR MXNet

Make it more memory friendly and faster,
thourgh it is still very slow and memory consuming

* Update best config

* Fix ramdom seed for test

* Init score-func specific var

* Update Readme
2019-11-01 10:50:44 -07:00
Da Zheng 0bf3b6ddb3 add KG statistics. (#931)
* add KG statistics.

* add Freebase.

* fix link.
2019-10-21 22:49:46 +08:00
Chao Ma 02fb05817b [KG] Add RESCAL model to DGL-KGE (#923)
* Add RESCAL model

* update

* update

* match acc

* update

* add README.md

* fix
2019-10-11 23:45:25 -07:00
xiang song(charlie.song) 20439e1c27 Fix bug of KG train.py script. (#922)
It cannot work when only mxnet backend is installed.
2019-10-11 10:13:09 -07:00
xiang song(charlie.song) 93e3c49ddc [KG] Update CI to cover Knowledge Graph (#913)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test for mxnet TAGCov

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix

* reproduce the bug

* Fix concurrency bug reported at #755.
Also make test_shared_mem_store.py more deterministic.

* Update test_shared_mem_store.py

* Update dmlc/core

* Update Knowledge Graph CI with new Docker image

* Remove unused line_profierx

* Poke Jenkins

* Update test with exit code check and simplify docker

* Update Jenkinsfile to make app test a standalone stage

* Update kg_test

* Update Jenkinsfile

* Make some KG test parallel

* Update

* KG MXNet does not support ComplEx

* Update Jenkinsfile

* Update Jenkins file

* Change torch-1.2 to torch-1.2-cu92

* ci

* Update ubuntu_install_mxnet_cpu.sh

* Update ubuntu_install_mxnet_gpu.sh

* We only need to test train and eval script.
Delete some test code
2019-10-11 01:32:34 -07:00
xiang song(charlie.song) bde75256be [KG] ComplEx score func for MXNet (#918)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test for mxnet TAGCov

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix

* reproduce the bug

* Fix concurrency bug reported at #755.
Also make test_shared_mem_store.py more deterministic.

* Update test_shared_mem_store.py

* Update dmlc/core

* Add complEx for mxnet

* ComplEx is ready for MXNet
2019-10-10 23:01:11 -07:00
Da Zheng c26b1bae89 [KG] a little update on readme. (#914)
* update readme.

* Update README.md
2019-10-09 13:44:06 -07:00
Minjie Wang 5eae8e0543 [Fix] Fix typo in KGE readme 2019-10-08 12:14:47 -04:00
Da Zheng 3a0bbb3edd [Doc] Update kg readme (#901)
* add Freebase results.

* update.

* explain

* add DistMult.

* reproduce.
2019-10-04 15:05:06 -07:00
Da Zheng df8a7be52d [KG] save embeddings in NumPy (#900)
* fix loading and saving.

* use numpy.
2019-10-04 09:14:16 -07:00
Da Zheng 3bc3109859 use PBG sampler for testing. (#899) 2019-10-02 22:13:09 -07:00
Da Zheng 190cdbd25f [Doc] Fix KG readme (#897)
* fix.

* update readme.

* shorten format desc.
2019-10-02 17:49:09 -07:00
Da Zheng 15b951d4c4 [KG][Model] Knowledge graph embeddings (#888)
* upd

* fig edgebatch edges

* add test

* trigger

* Update README.md for pytorch PinSage example.

Add noting that the PinSage model example under
example/pytorch/recommendation only work with Python 3.6+
as its dataset loader depends on stanfordnlp package
which work only with Python 3.6+.

* Provid a frame agnostic API to test nn modules on both CPU and CUDA side.

1. make dgl.nn.xxx frame agnostic
2. make test.backend include dgl.nn modules
3. modify test_edge_softmax of test/mxnet/test_nn.py and
    test/pytorch/test_nn.py work on both CPU and GPU

* Fix style

* Delete unused code

* Make agnostic test only related to tests/backend

1. clear all agnostic related code in dgl.nn
2. make test_graph_conv agnostic to cpu/gpu

* Fix code style

* fix

* doc

* Make all test code under tests.mxnet/pytorch.test_nn.py
work on both CPU and GPU.

* Fix syntex

* Remove rand

* Add TAGCN nn.module and example

* Now tagcn can run on CPU.

* Add unitest for TGConv

* Fix style

* For pubmed dataset, using --lr=0.005 can achieve better acc

* Fix style

* Fix some descriptions

* trigger

* Fix doc

* Add nn.TGConv and example

* Fix bug

* Update data in mxnet.tagcn test acc.

* Fix some comments and code

* delete useless code

* Fix namming

* Fix bug

* Fix bug

* Add test for mxnet TAGCov

* Add test code for mxnet TAGCov

* Update some docs

* Fix some code

* Update docs dgl.nn.mxnet

* Update weight init

* Fix

* init version.

* change default value of regularization.

* avoid specifying adversarial_temperature

* use default eval_interval.

* remove original model.

* remove optimizer.

* set default value of num_proc

* set default value of log_interval.

* don't need to set neg_sample_size_valid.

* remove unused code.

* use uni_weight by default.

* unify model.

* rename model.

* remove unnecessary data sampler.

* remove the code for checkpoint.

* fix eval.

* raise exception in invalid arguments.

* remove RowAdagrad.

* remove unsupported score function for now.

* Fix bugs of kg
Update README

* Update Readme for mxnet distmult

* Update README.md

* Update README.md

* revert changes on dmlc

* add tests.

* update CI.

* add tests script.

* reorder tests in CI.

* measure performance.

* add results on wn18

* remove some code.

* rename the training script.

* new results on TransE.

* remove --train.

* add format.

* fix.

* use EdgeSubgraph.

* create PBGNegEdgeSubgraph to simplify the code.

* fix test

* fix CI.

* run nose for unit tests.

* remove unused code in dataset.

* change argument to save embeddings.

* test training and eval scripts in CI.

* check Pytorch version.

* fix a minor problem in config.

* fix a minor bug.

* fix readme.

* Update README.md

* Update README.md

* Update README.md
2019-10-02 12:57:29 -07:00