dmlc--dgl
Zihao Ye
9f32554296
* change the signature of node/edge filter * upd filter * Support multi-dimension node feature in SPMV * push transformer * remove some experimental settings * stable version * hotfix * upd tutorial * upd README * merge * remove redundency * remove tqdm * several changes * Refactor * Refactor * tutorial train * fixed a bug * fixed perf issue * upd * change dir * move un-related to contrib * tutuorial code * remove redundency * upd * upd * upd * upd * improve viz * universal done * halt norm * fixed a bug * add draw graph * fixed several bugs * remove dependency on core * upd format of README * trigger * trigger * upd viz * trigger * add transformer tutorial * fix tutorial * fix readme * small fix on tutorials * url fix in readme * fixed func link * upd
Transformer in DGL
In this example we implement the Transformer and Universal Transformer with ACT in DGL.
The folder contains training module and inferencing module (beam decoder) for Transformer and training module for Universal Transformer
Requirements
- PyTorch 0.4.1+
- networkx
- tqdm
Usage
-
For training:
python translation_train.py [--gpus id1,id2,...] [--N #layers] [--dataset DATASET] [--batch BATCHSIZE] [--universal] -
For evaluating BLEU score on test set(by enabling
--printto see translated text):python translation_test.py [--gpu id] [--N #layers] [--dataset DATASET] [--batch BATCHSIZE] [--checkpoint CHECKPOINT] [--print] [--universal]
Available datasets: copy, sort, wmt14, multi30k(default).
Test Results
Transfomer
- Multi30k: we achieve BLEU score 35.41 with default setting on Multi30k dataset, without using pre-trained embeddings. (if we set the number of layers to 2, the BLEU score could reach 36.45).
- WMT14: work in progress
Universal Transformer
- work in progress
Notes
- Currently we do not support Multi-GPU training(this will be fixed soon), you should only specifiy only one gpu_id when running the training script.