8d5d8962ad
[Refactor] Replace third_party/nccl with PyTorch's NCCL backend ( #4989 )
...
* expose GeneratePermutation
* add sparse_all_to_all_push
* add sparse_all_to_all_pull
* add unit test
* handle world_size=1
* remove python nccl wrapper
* remove the nccl dependency
* use pinned memory to speedup D2H copy
* fix lint
* resolve comments
* fix lint
* fix ut
* resolve comments
2023-03-08 12:59:10 +08:00
bf264d00fe
[Feature] (La)yer-Neigh(bor) sampling implementation ( #4668 )
...
* adding LABOR sampling
* add ladies and pladies samplers
* fix compile error after rebase
* add reference for ladies sampler
* Improve ladies implementation.
* weighted labor sampling initial implementation draft
fix indentation and small bug in ladies script
* importance_sampling currently doesn't work with weights
* fix weighted importance sampling
* move labor example into its own folder
* lint fixes
* Improve documentation
* remove examples from the main PR
* fix linting by not using c++17 features
* fix documentation of labor_sampler.py
* update documentation for labor.py
* reformat the labor.py file with black
* fix linting errors
* replace exception use with if
* fix typo in error comment
* fixing win64 build for ci
* fixing weighted implementation, works now.
* fix bug in the weighted case and importance_sampling==0
* address part of the reviews
* remove unused code paths from cuda
* remove unused code path from cpu side
* remove extra features of labor making use of random seed.
* fix exclude_edges bug
* remove pcg and seed logic from cpu implementation, seed logic should still work for cuda.
* minor style change
* refactor CPU implementation, take out the importance_sampling probability computation into a function.
* improve CUDAWorkspaceAllocator
* refactor importance_sampling part out to a function
* minor optimization
* fix linting issue
* Revert "remove pcg and seed logic from cpu implementation, seed logic should still work for cuda."
This reverts commit c250e07ac6d7e13f57e79e8a2c2f098d777378c2.
* Revert "remove extra features of labor making use of random seed."
This reverts commit 7f99034353080308f4783f27d9a08bea343fb796.
* fix the documentation
* disable NIDs
* improve the documentation in the code
* use the stream argument in pcg32 instead of skipping ahead t times, can discard the use of hashmap now since it is faster this way.
* fix linting issue
* address another round of reviews
* further optimize CPU LABOR sampling implementation
* fix linting error
* update the comment
* reformat
* rename and rephrase comment
* fix formatting according to new linting specs
* fix compile error due to renaming, fix linting.
* lint
* rename DGLHeteroGraph to DGLGraph to match master
* replace other occurrences of DGLHeteroGraph to DGLGraph
Co-authored-by: Muhammed Fatih BALIN <m.f.balin@gmail.com >
Co-authored-by: Kaan Sancak <kaansnck@gmail.com >
Co-authored-by: Quan Gan <coin2028@hotmail.com >
2022-11-22 09:03:02 +08:00
cded5b80fe
[Feature] Bump DLPack to v0.7 and decouple DLPack from the core library ( #4454 )
...
* rename `DLContext` to `DGLContext`
* rename `kDLGPU` to `kDLCUDA`
* replace DLTensor with DGLArray
* fix linting
* Unify DGLType and DLDataType to DGLDataType
* Fix FFI
* rename DLDeviceType to DGLDeviceType
* decouple dlpack from the core library
* fix bug
* fix lint
* fix merge
* fix build
* address comments
* rename dl_converter to dlpack_convert
* remove redundant comments
2022-09-19 16:02:43 +08:00
06608f847c
[Cleanup] Remove duplicate entries of CUB submodule (issue# 4395) ( #4499 )
...
* remove third_part/cub
* remove from third_party
Co-authored-by: Israt Nisa <nisisrat@amazon.com >
Co-authored-by: Xin Yao <xiny@nvidia.com >
2022-09-05 13:23:08 +08:00
86c81b4e92
[Feature] Add CUDA Weighted Neighborhood Sampling ( #4064 )
...
* add weighted sampling without replacement (A-Chao)
* improve Algorithm A-Chao with block-wise prefix sum
* correctly fill out_idxs
* implement weighted sampling with replacement
* small fix
* merge host-side code of weighted/uniform sampling
* enable unit tests for cuda weighted sampling
* move thrust/cub wrapper to the cmake file
* update docs accordingly
* fix linting
* fix linting
* fix unit test
* Bump external CUB/Thrust versions
* Fix code style and update description of algorithm design
* [Feature] GPU support weighted graph neighbor sampling
commit by pengqirong(OPPO)
* merge pengqirong's implementation
* revert the change to cub and thrust
* fix linting
* use DeviceSegmentedSort for better performance
* add more comments
* add necessary notes
* add necessary notes
* resolve some comments
* define THRUST_CUB_WRAPPED_NAMESPACE
* fix doc
Co-authored-by: 彭齐荣 <657017034@qq.com >
2022-07-29 11:08:48 +08:00
bf649d940c
[Bugfix][Performance] Update nccl to 2.11.4 ( #3761 )
...
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2022-02-23 13:51:41 +08:00
a3ce780d29
[RPC] Use tensorpipe for rpc communication ( #3335 )
...
* doesn't know whether works
* add change
* fix
* fix
* fix
* remove
* revert
* lint
* lint
* fix
* revert
* lint
* fix
* only build rpc on linux
* lint
* lint
* fix build on windows
* fix windows
* remove old test
* fix cmake
* Revert "remove old test"
This reverts commit f1ea75c777c34cdc1f08c0589676ba6aee1feb29.
* fix windows
* fix
* fix
* fix indent
* fix indent
* address comment
* fix
* fix
* fix
* fix
* fix
* lint
* fix indent
* fix lint
* add introduction
* fix
* lint
* lint
* add more logs
* fix
* update xbyak for C++14 with gcc5
* Remove channels
* fix
* add test script
* fix
* remove unused file
* fix lint
* add timeout
2021-12-06 20:03:33 +08:00
769718dfcf
Bring back thrust for backward compatibility ( #3562 )
2021-12-03 16:30:35 +08:00
bba88cd6dc
remove unused dependency ( #3504 )
2021-11-15 14:21:58 -08:00
fac75e166b
[CPU][Kernel] Single socket spmm ( #3024 )
...
* optimizations of spmm for CPU
* Added names of contributors
* Minor code cleanup
* Moved the spmm optimization code to a new header file
* Moved to DGL's logging method
* removed duplicate code between SpMMSumCsr and SpMMCmpCsr
* Changes made to follow Google coding style
* Fixed lint errors in spmm.h
* Fixed some lint errors from spmm_blocking_libxsmm.h
* Fixed lint errors from spmm_blocking_libxsmm.h
* Added comments to SpMMCreateLibxsmmKernel
* to enable building of tests, and other cosmetic changes
* disabling libxsmm on windows
* Put a condition to avoid opt impl for FP64 as libxsmm does not have FP64 support yet
* cosmetic changes and documentation
* cosmetic changes
* to pass lint tests
* replaced multiple allocations for buffers of indices and edges with a single allocation
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com >
2021-07-13 11:18:29 +08:00
66eb240d15
[Bugfix] Include NCCL as a submodule ( #2934 )
...
* Add NCCL as a submodule
* Allow using third_party/nccl or system nccl
* Add nccl_external as a dependency
* Fix conditional
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2021-05-25 09:46:16 +08:00
e83d0a8007
[Feature] Add kd-tree implementation (CPU) for kNN ( #2767 )
...
* add submodule nanoflann
* finish python API for knn
* finish ndarray adaptor
* finish cpu-kdtree version of knn
* use openmp
* add endline
* upt
* upt
* fix format and code style
* upt
* add warning for gpu-cpu copy
* avoid contiguous copy
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
Co-authored-by: Tong He <hetong007@gmail.com >
2021-04-09 17:39:34 +08:00
4208ce2b9e
[Feature] Tvm integration ( #2367 )
...
Co-authored-by: Zihao Ye <expye@outlook.com >
2020-12-31 17:40:25 +08:00
f8ebcd7f90
[Performance] Dynamic cpu kernel V3 for SpMMSumCsr all Ops ( #2309 )
...
* support AVX512
* env DGL_CPU_INTEL_KERNEL_ENABLED=1
* env DGL_CPU_INTEL_KERNEL_LOG=1
* Add unittest test_spmm.cc
Co-authored-by: Izabela Mazur <izabela.mazur@intel.com >
Co-authored-by: Michal Szarmach <michal.szarmach@intel.com >
Review patch
2020-11-17 15:12:23 +08:00
4fb0241bfb
[CUDA] Add CUDA11 support ( #2308 )
...
* add support for cuda 11
* fix inc bug in pytorch 1.8
* poke ci
* fix
* small fix
* try fix
* try fix
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com >
2020-11-07 22:16:51 +08:00
3234189b60
Fix hdfs error when writing big data ( #2186 )
...
* update dmlc-core to latest
* fix hdfs
2020-09-12 23:58:36 +08:00
628d9fc59d
Fix build error with hdfs and update dmlc-core ( #2107 )
...
* update dmlc-core for hdfs build
* add hdfs support
* default off
* trigger ci
2020-08-26 15:10:21 +08:00
729ff2ef38
Fix the performance issue of graph partitioning in new DGLGraph ( #1934 )
...
* fix perf.
* fix.
* accelerate metis.
* fix lint.
* use gklib.
* fix perf.
* fix.
* update metis.
* update launch script
* handle synchronized API.
* fix.
* fix example.
* fix dataloader.
* temp fix.
* temp fix omp.
* distinguish roles.
* initialize iterator of DistDataloader correctly.
* check the correctness of launch script.
* move feature copy to sampler.
* measure mem/network copy time.
* remove
* Revert "measure mem/network copy time."
This reverts commit 86cefdc14b7815fcf5aad6496af912dba48e4aa6.
* fix.
* fix
* fix.
* fix cmake.
* disable metis in windows.
* disable metis tests in windows.
* remove test for multigraph.
* fix test.
* fix.
* fix cmake.
* fix.
* revert.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-115.us-west-2.compute.internal >
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal >
2020-08-09 18:33:05 -07:00
0e896a921e
update. ( #1857 )
...
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-164.us-west-2.compute.internal >
2020-07-24 10:47:57 -07:00
22a6ad6d9b
[Windows] Compile METIS on Windows ( #1771 )
...
* make metis compilable on windows
* lint
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com >
2020-07-09 15:43:50 +08:00
870da747ea
[CUDA][Kernel] More CUDA kernels; Standardize the behavior for sorted COO/CSR ( #1704 )
...
* add cub; array cumsum
* CSRSliceRows
* fix warning
* operator << for ndarray; CSRSliceRows
* add CSRIsSorted
* add csr_sort
* inplace coosort and outplace csrsort
* WIP: coo is sorted
* mv cuda_utils
* add AllTrue utility
* csr sort
* coo sort
* coo2csr for sorted coo arrays
* CSRToCOO from sorted
* pass tests for the new kernel changes
* cannot use inplace sort
* lint
* try fix msvc error
* Fix g.copy_to and g.asnumbits; ToBlock no longer uses CSC
* stash
* revert some hack
* revert some changes
* address comments
* fix
* fix to_block unittest
* add todo note
2020-06-28 18:37:28 +08:00
8a20b6c19c
[Optimization] Replaced std::unordered_map by phmap::flat_hash_map ( #1662 )
...
Co-authored-by: Da Zheng <zhengda1936@gmail.com >
2020-06-21 10:27:29 -07:00
0e153c4baf
[Feature] add metis partitioning to DGL ( #1308 )
...
* add metis.
* add test.
* construct partition id.
* link to METIS github repo.
* update metis.
* add a tool for partitioning a graph.
* update metis.
* update.
* update.
* fix metis.
* fix lint
* fix indent.
* another way of building metis.
* disable metis in windows.
* test windows
* fix.
* disable metis for windows properly.
* fix for tensorflow.
* skip test for gpu.
* make graph symmetric
* address comments.
* more comments.
* fix compile
* fix a bug.
* add test.
* change the default #hops of HALO nodes.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal >
2020-03-07 23:24:03 -08:00
cccde032f4
[kvstore] Performance improvement for distributed kvstore ( #972 )
...
* Performance improvment for distributed kvstore
* update
* update
2019-11-04 18:27:00 +08:00
9a0511c8e9
[NN] nn modules & examples update ( #890 )
...
* upd
* damn it
* fuck
* fuck pylint
* fudge
* remove some comments about MXNet
* upd
* upd
* damn it
* damn it
* fuck
* fuck
* upd
* upd
* pylint bastard
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
* upd
2019-11-03 17:29:42 -05:00
0fb13f7b9d
[Feature] Data format ( #728 )
...
* Add serialization
* add serialization
* add serialization
* lalalalalalalala
* lalalalalalalala
* serialize
* serialize
* nnn
* WIP: import tvm runtime node system
* WIP: object system
* containers
* tested basic container composition
* tested custom object
* tmp
* fix setattr bug
* tested object container return
* fix lint
* some comments about get/set state
* fix lint
* fix lint
* update cython
* fix cython
* ffi doc
* fix doc
* WIP: using object system for graph
* c++ side refactoring done; compiled
* remove stale apis
* fix bug in DGLGraphCreate; passed test_graph.py
* fix bug in python modify; passed utest for pytorch/cpu
* fix lint
* Add serialization
* Add serialization
* fix
* fix typo
* serialize with new ffi
* commit
* commit
* commit
* save
* save
* save
* save
* commit
* clean
* Delete tt2.py
* fix lint
* Add serialization
* fix lint 2
* fix lint
* fix lint
* fix lint
* fix lint
* Fix Lint
* Add serialization
* Change to Macro
* fix
* fix
* fix bugs
* refactor
* refactor
* updating dmlc-core to include force flag
* trying tempfile
* delete leaked pointer
* Fix assert
* fix assert
* add comment and test case
* add graph labels
* add load labels
* lint
* lint
* add graph labels
* lint
* fix windows
* fix
* update dmlc-core to latest
* fix
* fix camel naming
2019-09-09 20:57:51 +08:00
7d61936577
update minigun to include algorithm header ( #654 )
2019-06-13 11:32:43 +08:00
e8e56b3423
[Build][Win] fix mgpu include problem ( #652 )
2019-06-12 13:22:59 -04:00
e35e860ae3
[Build] Support older CMake & OpenMP toggle ( #619 )
...
* cmake fixes for older systems
* allow specification of cuda path
* test script fixes to enable openmp & test
* update minigun; disable minigun partial frontier compile
2019-06-07 14:00:34 -04:00
653428bdc7
[Feature][Kernel] DGL kernel support ( #596 )
...
* [Kernel] Minigun integration and fused kernel support (#519 )
* kernel interface
* add minigun
* Add cuda build
* functors
* working on binary elewise
* binary reduce
* change kernel interface
* WIP
* wip
* fix minigun
* compile
* binary reduce kernels
* compile
* simple test passed
* more reducers
* fix thrust problem
* fix cmake
* fix cmake; add proper guard for atomic
* WIP: bcast
* WIP
* bcast kernels
* update to new minigun pass-by-value practice
* broadcasting dim
* add copy src and copy edge
* fix linking
* fix none array problem
* fix copy edge
* add device_type and device_id to backend operator
* cache csr adj, remove cache for adjmat and incmat
* custom ops in backend and pytorch impl
* change dgl-mg kernel python interface
* add id_mapping var
* clean up plus v2e spmv schedule
* spmv schedule & clean up fall back
* symbolic message and reduce func, remove bundle func
* new executors
* new backend interface for dgl kernels and pytorch impl
* minor fix
* fix
* fix docstring, comments, func names
* nodeflow
* fix message id mapping and bugs...
* pytorch test case & fix
* backward binary reduce
* fix bug
* WIP: cusparse
* change to int32 csr for cusparse workaround
* disable cusparse
* change back to int64
* broadcasting backward
* cusparse; WIP: add rev_csr
* unit test for kernels
* pytorch backward with dgl kernel
* edge softmax
* fix backward
* improve softmax
* cache edge on device
* cache mappings on device
* fix partial forward code
* cusparse done
* copy_src_sum with cusparse
* rm id getter
* reduce grad for broadcast
* copy edge reduce backward
* kernel unit test for broadcasting
* full kernel unit test
* add cpu kernels
* edge softmax unit test
* missing ref
* fix compile and small bugs
* fix bug in bcast
* Add backward both
* fix torch utests
* expose infershape
* create out tensor in python
* fix c++ lint
* [Kernel] Add GPU utest and kernel utest (#524 )
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* [Kernel] Update kernel branch (#550 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* Fixing typo in JTNN after interface change (#536 )
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* [Kernel] Update kernel branch (#576 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* Fixing typo in JTNN after interface change (#536 )
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* all demo use python-3 (#555 )
* [DEMO] Reproduce numbers of distributed training in AMLC giant graph paper (#556 )
* update
* update
* update
* update num_hops
* fix bug
* update
* report numbers of distributed training in AMLC giant graph paper
* [DEMO] Remove duplicate code for sampling (#557 )
* update
* update
* re-use single-machine code
* update
* use relative path
* update
* update
* update
* add __init__.py
* add __init__.py
* import sys, os
* fix typo
* update
* [Perf] Improve performance of graph store. (#554 )
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
* [BUGIFX] fix a bug in edge_ids (#560 )
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
* [DEMO] Add Pytorch demo for distributed sampler (#562 )
* update
* update
* update
* add sender
* update
* remove duplicate cpde
* [Test] Add gtest to project (#547 )
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
* [Perf] lazily create msg_index. (#563 )
* lazily create msg_index.
* update test.
* [BUGFIX] fix bugs for running GCN on giant graphs. (#561 )
* load mxnet csr.
* enable load large csr.
* fix
* fix.
* fix int overflow.
* fix test.
* [BugFix] Fix error when bfs_level = 0 in Entity Classification with RGCN (#559 )
* [DEMO] Update demo of distributed sampler (#564 )
* update
* update
* update demo
* add network cpp test (#565 )
* Add unittest for C++ RPC (#566 )
* [CI] Fix CI for cpp test (#570 )
* fix CI for cpp test
* update port number
* [Docker] update docker image (#575 )
* update docker image
* specify lint version
* rm torch import from unified tests
* [Kernel][Scheduler][MXNet] Scheduler for DGL kernels and MXNet backend support (#541 )
* [Model] add multiprocessing training with sampling. (#484 )
* reorganize sampling code.
* add multi-process training.
* speed up gcn_cv
* fix graphsage_cv.
* add new API in graph store.
* update barrier impl.
* support both local and distributed training.
* fix multiprocess train.
* fix.
* fix barrier.
* add script for loading data.
* multiprocessing sampling.
* accel training.
* replace pull with spmv for speedup.
* nodeflow copy from parent with context.
* enable GPU.
* fix a bug in graph store.
* enable multi-GPU training.
* fix lint.
* add comments.
* rename to run_store_server.py
* fix gcn_cv.
* fix a minor bug in sampler.
* handle error better in graph store.
* improve graphsage_cv for distributed mode.
* update README.
* fix.
* update.
* [Tutorial] add sampling tutorial. (#522 )
* add sampling tutorial.
* add readme
* update author list.
* fix indent in the code.
* rename the file.
* update tutorial.
* fix the last API.
* update image.
* [BUGFIX] fix the problems in the sampling tutorial. (#523 )
* add index.
* update.
* update tutorial.
* fix gpu utest
* cuda utest runnable
* temp disable test nodeflow; unified test for kernel
* cuda test kernel done
* edge softmax module
* WIP
* Fixing typo in JTNN after interface change (#536 )
* mxnet backend support
* improve reduce grad
* add max to unittest backend
* fix kernel unittest
* [BugFix] Fix getting src and dst id of ALL edges in NodeFlow.apply_block (#515 )
* lint
* lint
* win build
* [Bug Fix] Fix inplace op at backend (#546 )
* Fix inplace operation
* fix line seprator
* [Feature] Add batch and unbatch for immutable graph (#539 )
* Add batch and unbatch for immutable graph
* fix line seprator
* fix lintr
* remove unnecessary include
* fix code review
* [BUGFix] Improve multi-processing training (#526 )
* fix.
* add comment.
* remove.
* temp fix.
* initialize for shared memory.
* fix graphsage.
* fix gcn.
* add more unit tests.
* add more tests.
* avoid creating shared-memory exclusively.
* redefine remote initializer.
* improve initializer.
* fix unit test.
* fix lint.
* fix lint.
* initialize data in the graph store server properly.
* fix test.
* fix test.
* fix test.
* small fix.
* add comments.
* cleanup server.
* test graph store with a random port.
* print.
* print to stderr.
* test1
* test2
* remove comment.
* adjust the initializer signature.
* try
* fix
* fix
* fix
* fix
* fix
* try
* test
* test
* test
* try
* try
* try
* test
* fix
* try gen_target
* fix gen_target
* fix msvc var_args expand issue
* fix
* [API] update graph store API. (#549 )
* add init_ndata and init_edata in DGLGraph.
* adjust SharedMemoryGraph API.
* print warning.
* fix comment.
* update example
* fix.
* fix examples.
* add unit tests.
* add comments.
* [Refactor] Immutable graph index (#543 )
* WIP
* header
* WIP .cc
* WIP
* transpose
* wip
* immutable graph .h and .cc
* WIP: nodeflow.cc
* compile
* remove all tmp dl managed ctx; they caused refcount issue
* one simple test
* WIP: testing
* test_graph
* fix graph index
* fix bug in sampler; pass pytorch utest
* WIP on mxnet
* fix lint
* fix mxnet unittest w/ unfortunate workaround
* fix msvc
* fix lint
* SliceRows and test_nodeflow
* resolve reviews
* resolve reviews
* try fix win ci
* try fix win ci
* poke win ci again
* poke
* lazy multigraph flag; stackoverflow error
* revert node subgraph test
* lazy object
* try fix win build
* try fix win build
* poke ci
* fix build script
* fix compile
* add a todo
* fix reviews
* fix compile
* WIP
* WIP
* all demo use python-3 (#555 )
* ToImmutable and CopyTo
* [DEMO] Reproduce numbers of distributed training in AMLC giant graph paper (#556 )
* update
* update
* update
* update num_hops
* fix bug
* update
* report numbers of distributed training in AMLC giant graph paper
* [DEMO] Remove duplicate code for sampling (#557 )
* update
* update
* re-use single-machine code
* update
* use relative path
* update
* update
* update
* add __init__.py
* add __init__.py
* import sys, os
* fix typo
* update
* [Perf] Improve performance of graph store. (#554 )
* fix.
* use inplace.
* move to shared memory graph store.
* fix.
* add more unit tests.
* fix.
* fix test.
* fix test.
* disable test.
* fix.
* [BUGIFX] fix a bug in edge_ids (#560 )
* add test.
* fix compute.
* fix test.
* turn on test.
* fix a bug.
* add test.
* fix.
* disable test.
* DGLRetValue DGLContext conversion
* [DEMO] Add Pytorch demo for distributed sampler (#562 )
* update
* update
* update
* add sender
* update
* remove duplicate cpde
* [Test] Add gtest to project (#547 )
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
* Add support to convert immutable graph to 32 bits
* [Perf] lazily create msg_index. (#563 )
* lazily create msg_index.
* update test.
* fix binary reduce following new minigun template
* enable both int64 and int32 kernels
* [BUGFIX] fix bugs for running GCN on giant graphs. (#561 )
* load mxnet csr.
* enable load large csr.
* fix
* fix.
* fix int overflow.
* fix test.
* new kernel interface done for CPU
* docstring
* rename & docstring
* copy reduce and backward
* [BugFix] Fix error when bfs_level = 0 in Entity Classification with RGCN (#559 )
* [DEMO] Update demo of distributed sampler (#564 )
* update
* update
* update demo
* adapt cuda kernels to the new interface
* add network cpp test (#565 )
* fix bug
* Add unittest for C++ RPC (#566 )
* [CI] Fix CI for cpp test (#570 )
* fix CI for cpp test
* update port number
* [Docker] update docker image (#575 )
* update docker image
* specify lint version
* rm torch import from unified tests
* remove pytorch-specific test_function
* fix unittest
* fix
* fix unittest backend bug in converting tensor to numpy array
* fix
* mxnet version
* [BUGFIX] fix for MXNet 1.5. (#552 )
* remove clone.
* turn on numpy compatible.
* Revert "remove clone."
This reverts commit 17bbf76ed72ff178df6b3f35addc428048672457.
* revert format changes
* fix mxnet api name
* revert mistakes in previous revert
* roll back CI to 20190523 build
* fix unittest
* disable test_shared_mem_store.py for now
* remove mxnet/test_specialization.py
* sync win64 test script
* fix lowercase
* missing backend in gpu unit test
* transpose to get forward graph
* pass update all
* add sanity check
* passing test_specialization.py
* fix and pass test_function
* fix check
* fix pytorch softmax
* mxnet kernels
* c++ lint
* pylint
* try
* win build
* fix
* win
* ci enable gpu build
* init submodule recursively
* backend docstring
* try
* test win dev
* doc string
* disable pytorch test_nn
* try to fix windows issue
* bug fixed, revert changes
* [Test] fix CI. (#586 )
* disable unit test in mxnet tutorial.
* retry socket connection.
* roll back to set_np_compat
* try to fix multi-processing test hangs when it fails.
* fix test.
* fix.
* doc string
* doc string and clean up
* missing field in ctypes
* fix node flow schedule and unit test
* rename
* pylint
* copy from parent default context
* fix unit test script
* fix
* demo bug in nodeflow gpu test
* [Kernel][Bugfix] fix nodeflow bug (#604 )
* fix nodeflow bug
* remove debug code
* add build gtest option
* fix cmake; fix graph index bug in spmv.py
* remove clone
* fix div rhs grad bug
* [Kernel] Support full builtin method, edge softmax and unit tests (#605 )
* add full builtin support
* unit test
* unit test backend
* edge softmax
* apply edge with builtin
* fix kernel unit test
* disable mxnet test_shared_mem_store
* gen builtin reduce
* enable mxnet gpu unittest
* revert some changes
* docstring
* add note for the hack
* [Kernel][Unittest][CI] Fix MXNet GPU CI (#607 )
* update docker image for MXNet GPU CI
* force all dgl graph input and output on CPU
* fix gpu unittest
* speedup compilation
* add some comments
* lint
* add more comments
* fix as requested
* add some comments
* comment
* lint
* lint
* update pylint
* fix as requested
* lint
* lint
* lint
* docstrings of python DGL kernel entries
* disable lint warnings on arguments in kernel.py
* fix docstring in scheduler
* fix some bug in unittest; try again
* Revert "Merge branch 'kernel' of github.com:zzhang-cn/dgl into kernel"
This reverts commit 1d2299e68b004182ea6130b088de1f1122b18a49, reversing
changes made to ddc97fbf1bec2b7815c0da7c74f7ecb2f428889b.
* Revert "fix some bug in unittest; try again"
This reverts commit ddc97fbf1bec2b7815c0da7c74f7ecb2f428889b.
* more comprehensive kernel test
* remove shape check in test_specialization
2019-06-06 15:47:55 -04:00
de54891fc8
[Test] Add gtest to project ( #547 )
...
* add gtest module
* add gtest
* fix
* Update CMakeLists.txt
* Update README.md
2019-05-24 15:15:47 +08:00
2694b12725
import ffi solution from TVM
2018-09-05 10:51:31 -04:00
Xin Yao