Following changes are made in this PR.
1. In dataset_utils.py, when reading edges from disk we follow the order defined by the STR_EDGE_TYPE key in the metadata.json file. This order is implicitly used to assign edgeid to edge types. This same order is used to read edges from the disk as well.
2. Now the unit test framework will also randomize the order of edges read from the disk. This is done for the edges when reading from the disk for the unit tests.
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* enable sparse on windows and mac
* that was stupid
* let's see what's going on..
* [Sparse] Fix the import error on Mac OS.
When using template functions that are defined in source files from DGL,
the loader of MacOS somehow cannot find their definitions. This fix simply
avoids depending on template functions from DGL headers.
With this fix, the sparse tests all pass on the MAC environment.
* ok this is the problem
* make errors clearer
* uh
* test
* Update __init__.py
* disabling ddp on windows
---------
Co-authored-by: czkkkkkk <zekucai@gmail.com>
* add SVD positional encoding
* modify importing module
* Fixed certain problems
* Change the test unit to a nonsigular one
* Fixed typo and make accord with lintrunner
* added svd_pe into dgl.rst
* Modified dgl.rst
* use NotImplemented
* format
* extend to pytorch scalar
* sparse div scalar
* oops
* Apply suggestions from code review
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* Allow reading and writing single-column vector Parquet files.
These files are commonly produced by Spark ML's feature processing code.
* [Dist] Only write single-column vector files for Parquet in tests.
* [Dist] Add support for Parquet-formatted edges files, remove some assumptions on edge file number.
* [Dist] Add parquet edges option to unit tests.
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
* Add SpatialEncoder and SpatialEncoder3d
* Optimize the code execution efficiency
* Fixed certain problems according to Dongyu's suggestions.
* Fix an error about probability of division by zero in PathEcoder; Change certain designs in SpatialEncoder
* Fix a typo
* polish the docstring
* fix doc
* lint
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-146.ap-northeast-1.compute.internal>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>
* [Sparse] Support SpMM and SDDMM
* Add SDDMMAutoGrad
* Rename SpMMImpl and SDDMMImpl and move them to dgl_sparse/src/matmul.h
* Update comments
* Update
* Minor fix
* replace LooseVersion with packaging.version
* import packaging from setuptools
* import packaging.version in dgl.utils
* format
* revert changes to apps/kg
* revert changes to _dataloading
* [Dist] enable to chunk node/edge data into arbitrary number of chunks
* [Dist] enable to split node/edge data into arbitrary parts
* refine code
* Format boolean to uint8 forcely to avoid dist.scatter failure
* convert boolean to int8 before scatter and revert it after scatter
* refine code
* fix test
* refine code
* move test utilities into utils.py
* update comment
* fix empty data
* update
* update
* fix empty data issue
* release unnecessary mem
* release unnecessary mem
* release unnecessary mem
* release unnecessary mem
* release unnecessary mem
* remove unnecessary shuffle data
* separate array_split into standalone utility
* add example
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Quan (Andy) Gan