* Enable AVX by default
* Fix linting errors
* Fix win64 build (libxsmm not linked)
Libxsmm on Win64 is not linked, should be disabled by default
* Fix clang format issues
* Change lower supported cpu version to LIBXSMM_X86_AVX2
Change lower supported cpu version to LIBXSMM_X86_AVX2 to address https://github.com/dmlc/dgl/issues/3459 issue
* Fix unit test
Remove assumption that libxsmm is enabled in the config by default (only true for intel CPUs with AVX2 instructions)
---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-15-137.us-west-2.compute.internal>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [Sparse] Support SpMM and SDDMM
* Add SDDMMAutoGrad
* Rename SpMMImpl and SDDMMImpl and move them to dgl_sparse/src/matmul.h
* Update comments
* Update
* Minor fix
* replace dgl PRNG with pcg32
* remove pcg submodule, add a simple implementation
* replace pcg32 with std::mt19937_64
* fix include order
* change RandomEngine to pcg32
* Remove custom pcg32 implementation, use the submodule provided by the original author.
* minor bug
* move include for linting
* include pcg for tests too
Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com>
* adding LABOR sampling
* add ladies and pladies samplers
* fix compile error after rebase
* add reference for ladies sampler
* Improve ladies implementation.
* weighted labor sampling initial implementation draft
fix indentation and small bug in ladies script
* importance_sampling currently doesn't work with weights
* fix weighted importance sampling
* move labor example into its own folder
* lint fixes
* Improve documentation
* remove examples from the main PR
* fix linting by not using c++17 features
* fix documentation of labor_sampler.py
* update documentation for labor.py
* reformat the labor.py file with black
* fix linting errors
* replace exception use with if
* fix typo in error comment
* fixing win64 build for ci
* fixing weighted implementation, works now.
* fix bug in the weighted case and importance_sampling==0
* address part of the reviews
* remove unused code paths from cuda
* remove unused code path from cpu side
* remove extra features of labor making use of random seed.
* fix exclude_edges bug
* remove pcg and seed logic from cpu implementation, seed logic should still work for cuda.
* minor style change
* refactor CPU implementation, take out the importance_sampling probability computation into a function.
* improve CUDAWorkspaceAllocator
* refactor importance_sampling part out to a function
* minor optimization
* fix linting issue
* Revert "remove pcg and seed logic from cpu implementation, seed logic should still work for cuda."
This reverts commit c250e07ac6d7e13f57e79e8a2c2f098d777378c2.
* Revert "remove extra features of labor making use of random seed."
This reverts commit 7f99034353080308f4783f27d9a08bea343fb796.
* fix the documentation
* disable NIDs
* improve the documentation in the code
* use the stream argument in pcg32 instead of skipping ahead t times, can discard the use of hashmap now since it is faster this way.
* fix linting issue
* address another round of reviews
* further optimize CPU LABOR sampling implementation
* fix linting error
* update the comment
* reformat
* rename and rephrase comment
* fix formatting according to new linting specs
* fix compile error due to renaming, fix linting.
* lint
* rename DGLHeteroGraph to DGLGraph to match master
* replace other occurrences of DGLHeteroGraph to DGLGraph
Co-authored-by: Muhammed Fatih BALIN <m.f.balin@gmail.com>
Co-authored-by: Kaan Sancak <kaansnck@gmail.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* add bf16 specializations
* remove SWITCH_BITS
* enable amp for bf16
* remove SWITCH_BITS for cpu kernels
* enbale bf16 based on CUDART
* fix compiling for sm<80
* fix cpu build
* enable unit tests
* update doc
* disable test for CUDA < 11.0
* address comments
* address comments
* [Dist] enable USE_EPOLL in default
* fix build issue on windows
* fix build issue on windows
* fix build issue on windows
* fix build issue on windows
* fix build issue on windows
* fix build issue
* [DistTest] add basic pipeline for dist test across machines
* move launch remote cmd to separate file
* add test for rpc
* fix function naming rule
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
With the emergence of new ISA (like ARM and RISCV) retaining USE_AVX ON default makes the default build instructions fail. Fundamentally DGL does not require the use of AVX for functional working. AVX is mainly needed when to enable optimization. So proposal is to default turn it off and then later during build instructions, folks with AVX capabilities can enable with
`cmake .. -DUSE_AVX=ON`
Co-authored-by: Zihao Ye <expye@outlook.com>
* [Bugfix] fix a compile error for Debug-BuildType on Windows Platform
When using CMakeLists.txt to build the "Debug" BuildType on the Windows Platform, it has three compile errors (C4716) in the file "dgl\src\runtime\shared_mem.cc":
'dgl::runtime::SharedMemory::CreateNew': must return a value
'dgl::runtime::SharedMemory::Open': must return a value
'dgl::runtime::SharedMemory::Exist': must return a value
* [Bugfix] cmake error "cannot find load file" when DGL as a sub_directory on Linux
When using DGL as a subdirectory in a CMake Project, the "CMAKE_SOURCE_DIR" here will return the parent cmake scope dir, which is not a expected dir.
Maybe it is better to use "CMAKE_CURRENT_SOURCE_DIR" to set "GKLIB_PATH".
* [Bugfix] cmd cmake error when DGL as a subdirectory
When DGL as a subdirectory of another project, the WORKING_DIRECTORY of "add_custom_command" will be incorrect at the line 255 of "CMakeLists.txt", such that making a cmake "setlocal" error.
* optimizations of spmm for CPU
* Added names of contributors
* Minor code cleanup
* Moved the spmm optimization code to a new header file
* Moved to DGL's logging method
* removed duplicate code between SpMMSumCsr and SpMMCmpCsr
* Changes made to follow Google coding style
* Fixed lint errors in spmm.h
* Fixed some lint errors from spmm_blocking_libxsmm.h
* Fixed lint errors from spmm_blocking_libxsmm.h
* Added comments to SpMMCreateLibxsmmKernel
* to enable building of tests, and other cosmetic changes
* disabling libxsmm on windows
* Put a condition to avoid opt impl for FP64 as libxsmm does not have FP64 support yet
* cosmetic changes and documentation
* cosmetic changes
* to pass lint tests
* replaced multiple allocations for buffers of indices and edges with a single allocation
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
* Add NCCL as a submodule
* Allow using third_party/nccl or system nccl
* Add nccl_external as a dependency
* Fix conditional
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Split NCCL wrapper from sparse optimizer and sparse embedding
* Add more unit tests for single node nccl
* Fix unit test for tf
* Switch to device histogram
* Fix histgram issues
* Finish migration to histogram
* Handle cases with zero send/recieve data
* Start on partition object
* Get compiling
* Updates
* Add unit tests
* Switch to partition object
* Fix linting issues
* Rename partition file
* Add python doc
* Fix python assert and finish doxygen comments
* Remove stubs for range based partition to satisfy pylint
* Wrap unit test in GPU only
* Wrap explicit cuda call in ifdef
* Merge with partition.py
* update docstrings
* Cleanup partition_op
* Add Workspace object
* Switch to using workspace object
* Move last remainder based function out of nccl_api
* Add error messages
* Update docs with examples
* Fix linting erros
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* first commit
* some thoughts
* move around
* more commit
* more fixes
* now it uses torch allocator
* fix symbol export error
* fix
* fixes
* test fix
* add script
* building separate library per version
* fix for vs2019
* more fixes
* fix on windows build
* update jenkinsfile
* auto copy built dlls for windows
* lint and installation guide update
* fix
* specify conda environment
* set environment for ci
* fix
* fix
* fix
* fix again
* revert
* fix cmake
* fix
* switch to using python interpreter path
* remove scripts
* debug
* oops sorry
* Update index.rst
* Update index.rst
* copies automatically, no need for this
* do not print message if library not found
* tiny fixes
* debug on nightly
* replace add_compile_definitions to make CMake 3.5 happy
* fix linking to wrong lib for multiple pytorch envs
* changed building strategy
* fix nightly
* fix windows
* fix windows again
* setup bugfix
* address comments
* change README
Daniil Sizov