文件历史

99 次代码提交

作者 SHA1 备注 提交日期
Quan Gan 239142b993 update version 2020-04-07 14:22:13 +08:00
Quan Gan 8188a0c8ac Merge branch 'master' into 0.4.x 2020-04-07 14:21:42 +08:00
Quan Gan ff244df0ea post-release 2020-04-06 22:31:51 +08:00
Da Zheng 7c47d8c9b4 [Feature] Simplify shared memory graph index (#1381)
* simplify shared memory graph index.

* fix.

* remove edge_dir in SharedMemGraphStore.

* avoid creating shared-mem graph store with from_csr.

* simplify from_csr.

* add comments.

* fix lint.

* remove the test.

* fix compilation error.

* fix a bug.

* fix a bug.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
2020-04-05 15:33:47 -07:00
Quan Gan e1d90f9b5e update version 2020-03-30 16:21:04 +00:00
Quan Gan 4a281ff3e6 Merge branch 'master' into 0.4.x 2020-03-30 16:14:14 +00:00
Jinjing Zhou e9440acb06 [TF] TF backend fix and new logic to choose backend (#1393)
* TF backend fix and new logic to choose backend

* fix

* fix

* fix

* fix

* fix backend

* fix

* dlpack alignment

* add flag

* flag

* lint

* lint

* remove unused

* several fixes

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-30 18:45:17 +08:00
Quan (Andy) Gan e3a9a6bba8 add an optional include_dst_in_src argument (#1401) 2020-03-30 12:06:12 +08:00
xiang song(charlie.song) 67cb7a43a0 [Feature] Deprecate multigraph (#1389)
* Deprecate multi-graph

* Handle heterograph and edge_ids

* lint

* Fix

* Remove multigraph in C++ end

* Fix lint

* Add some test and fix something

* Fix

* Fix

* upd

* Fix some test case

* Fix

* Fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-28 21:31:24 +08:00
Quan (Andy) Gan bbfff8ce76 [Feature] Casting between DGLGraph and DGLHeteroGraph (#1391)
* [Feature] Casting between DGLGraph and DGLHeteroGraph

* lint

* address comments

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
2020-03-27 13:34:55 +08:00
Da Zheng 1a584ced11 [Feature] Improve the speed of to_bidirected (#1327)
* make graph symmetric

* call gklib routine.

* use gklib version except in windows.

* minor errors.

* fix test.

* update doc.

* fix a compile error.

* fix.

* add comments for the new C API.

* fix a bug.

* address comments.

* fix compile error.

* fix comment.
2020-03-10 22:44:32 -07:00
Da Zheng 0e153c4baf [Feature] add metis partitioning to DGL (#1308)
* add metis.

* add test.

* construct partition id.

* link to METIS github repo.

* update metis.

* add a tool for partitioning a graph.

* update metis.

* update.

* update.

* fix metis.

* fix lint

* fix indent.

* another way of building metis.

* disable metis in windows.

* test windows

* fix.

* disable metis for windows properly.

* fix for tensorflow.

* skip test for gpu.

* make graph symmetric

* address comments.

* more comments.

* fix compile

* fix a bug.

* add test.

* change the default #hops of HALO nodes.

Co-authored-by: Ubuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
2020-03-07 23:24:03 -08:00
Quan (Andy) Gan a9520f71ce [Model][Sampler] GraphSAGE model, bipartite graph conversion & remove edges API (#1297)
* remove edge and to bipartite and graphsage with sampling

* fixes

* fixes

* fixes

* reenable multigpu training

* fixes

* compatibility in DGLGraph

* rename to compact_as_bipartite

* bugfix

* lint

* add offline inference

* skip GPU tests

* fix

* addresses comments

* fix

* fix

* fix

* more tests

* more docs and unit tests

* workaround for empty slice on empty data
2020-03-08 02:38:29 +08:00
Quan (Andy) Gan ce6e19f233 [Bugfix] Allows node types with no inbound/outbound edges (#1323)
* add num nodes in ctors

* fix

* lint

* addresses comments

* replace with constexpr

* remove function with rvalue reference

* address comments
2020-03-08 01:11:44 +08:00
Jinjing Zhou 9caff61777 [Feature] More serialization support (#1295)
* fix script

* t

* fix weird bugs

* fix

* fix

* upload

* fix

* fix

* lint

* fix

* tmp

* fix serialization

* fix

* fix lint

* fix message

* fix

* lint

* address comment

* fix

* lint

* fix

* fix

* fix

* Remove duplicate serialization for meta graph

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
2020-03-06 21:38:56 +08:00
Minjie Wang 5dd35580f7 [Feature] Improve sampling speed; Better pickle/unpickle; other fixes (#1299)
* improve performance of sample_neighbors

* some more improve

* test script

* benchmarks

* multi process

* update more tests

* WIP

* adding two API for state saving

* add create from state

* upd test

* missing file

* wip: pickle/unpickle

* more c apis

* find the problem of empty data array

* add null array; pickling speed is bad

* still bad perf

* still bad perf

* wip

* fix the pickle speed test; now everything looks good

* minor fix

* bugfix

* some lint fix

* address comments

* more fix

* fix lint

* add utest for random.choice

* add utest for dgl.rand_graph

* fix cpp utests

* try fix ci

* fix bug in TF backend

* upd choice docstring

* address comments

* upd

* try fix compile

* add comment
2020-03-02 23:16:05 +08:00
Quan (Andy) Gan c3a33407fe [Feature] PinSAGE sampler (#1249)
* [WIP] PinSAGE operators

* moved the edge remapping mess into C

* some docstrings

* lint

* lint x2

* lint x3

* skip gpu test on topk

* extend pinsage to any metapath

* lint x4

* addresses #1265

* add always_preserve (fixes #1266) and fix a silly bug

* disable gpu test on compaction

* lint

* fix a horrible bug and add more tests

* lint

* addresses comments

* lint

* bugfix

* addresses comments

Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>
2020-02-28 20:46:13 +08:00
Jinjing Zhou ebca1188a9 [Feature] Support serialization for smart pointer (#1291)
* fix script

* t

* fix weird bugs

* fix

* fix

* upload

* fix

* fix

* lint

* fix
2020-02-26 17:29:27 +08:00
Minjie Wang f8d4264e5b [Feature] Neighbor-hood based sampling APIs (#1251)
* WIP: working on random choices

* light slice

* basic CPU impl

* add python binding; fix CreateFromCOO and CreateFromCSR returning unitgraph

* simple test case works

* fix bug in slicing probability array

* fix bug in getting the correct relation graph

* fix bug in creating placeholder graph

* enable omp

* add cpp test

* sample topk

* add in|out_subgraph

* try fix lint; passed all unittests

* fix lint

* fix msvc compile; add sorted flag and constructors

* fix msvc

* coosort

* COOSort; CSRRowWiseSampling; CSRRowWiseTopk

* WIP: remove DType in CSR and COO; Restrict data array to be IdArray

* fix all CSR ops for missing data array

* compiled

* passed tests

* lint

* test sampling out edge

* test different per-relation fanout/k values

* fix bug in random choice

* finished cpptest

* fix compile

* Add induced edges

* add check

* fixed bug in sampling on hypersparse graph; add tests

* add ascending flag

* in|out_subgraph returns subgraph and induced eid

* address comments

* lint

* fix
2020-02-17 21:25:21 +08:00
Jinjing Zhou 23893bb0a8 [Data format] Serialization for Immutable Graph and HeteroGraph (#1254)
* graph format

* fix lint

* lint

* fix

* unit test

* lint

* add magic num

* move serialize out of struct

* lint

* serialize

* trigger CI

* fix lint

* lint

Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com>
2020-02-11 00:14:31 +08:00
Jinjing Zhou f103bbf9ee [Data format] Serialization for UnitGraph (#1242)
* graph format

* fix lint

* lint

* fix

* unit test

* lint

* add magic num

* move serialize out of struct

* lint

Co-authored-by: zhoujinjing09 <zhoujinjing09@users.noreply.github.com>
2020-02-08 20:49:34 +08:00
Quan (Andy) Gan eeeb52f464 [Feature] Preference to COO for "hypersparse" unit graphs & graph compaction (#1238)
* unit graph that prefers coo queries

* auto detect coo preference

* forgot some functions

* disable lint on detect_prefer_coo

* reorg

* change comment

* lint

* fix

* move array_utils.h to src

* compact graph impl

* fix redundant copying in idhashmap

* docstring

* moving preference detection to C

* lint

* fix unit test & address comments

* hypersparse autorestrict

* docstring & fix

* revert copyto and asnumbits

* fix stupid bug

* lint

* leave a TODO for sorted COO

* fixing same node type mapping to different id in different graphs

* addresses comments

* made induced nodes a feautre column

* lint?
2020-02-04 22:39:28 +08:00
Quan (Andy) Gan a9dabcc769 [Feature] Random Walk for 0.5 (#1209)
* trying to refactor IndexSelect

* partial implementation

* add index select and assign for floats as well

* move to random choice source

* more updates

* fixes

* fixes

* more fixes

* adding python impl

* fixes

* unit test

* lint

* lint x2

* lint x3

* update metapath2vec

* debugging performance

* still debugging for performance

* tuning

* switching to succvec

* redo

* revert non-uniform sampler to use vector

* still not fast

* why does this crash with OpenMP???

* because there was a data race!!!

* add documentations and remove assign op

* lint

* lint x2

* lol what have i done

* lint x3

* fix and disable gpu testing

* bugfix

* generic random walk

* reorg the random walk source code

* Update randomwalks.h

* Update randomwalks_cpu.cc

* rename file

* move internal function to anonymous ns

* reorg & docstrings

* constant restart probability

* docstring fix

* more commit

* random walk with restart, tested

* some fixes

* switch to using NDArray for choice

* massive fix & docstring

* lint x?

* lint x??

* fix

* export symbols

* skip gpu test

* addresses comments

* replaces another VecToIdArray

* add randomwalks.h to include

* replace void * with template
2020-01-31 19:52:08 +08:00
Quan Gan 55e056fbae update to 0.4.2 2020-01-20 17:35:24 +08:00
Quan Gan 10d2b8408d Merge branch 'master' into 0.4.x 2020-01-20 17:33:42 +08:00
Quan (Andy) Gan 64f6f3c1a8 [Bug?] Fix non-determinism when setting the same seed across different runs with single thread (#1205)
* fix multithread random seed value

* more fix

* typo

* what did i wrote
2020-01-16 15:03:26 +08:00
Zihao Ye f1420d194f [Refactor] Renaming class methods of sampler utilities to improve readability (#1180)
* upd

* upd
2020-01-07 22:02:44 +08:00
Zihao Ye dfb10db86f [Feature] Bag of samplers for efficient categorical sampling w/ and w/o replacement (#1142)
* upd

* upd

* lint

* upd

* upd

* lint

* upd

* refactor

* lint

* upd

* upd

* upd

* lint

* fix

* fix

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* complete test

* upd

* upd

* fix

* vir

* upd

* fix

* fuck

* fix numerical
2020-01-01 11:42:11 +08:00
Da Zheng e890a899fc Partition a graph with HALO nodes (#1076)
* get subgraph with halo.

* add partition function.

* add comment.

* parallel partition.

* fix a compilation error.

* fix lint error.

* address comments.

* add comments.

* fix for TF.
2019-12-23 08:46:53 -08:00
Quan Gan 263656f893 bump to 0.4.1 2019-11-04 11:06:05 +08:00
Da Zheng 3e454476b6 edge sampler return head nodes and tail nodes. (#889)
* edge sampler return head nodes and tail nodes.

* edge subgraph.

* use VecToIdArray

* example.

* fix compile error.
2019-09-30 21:25:16 -07:00
Da Zheng 65e1ba4f60 Sort csr (#886)
* sort

* sort in parallel.

* fix a bug in sorting adj

* rename.

* add more comments.

* accelerate GetData

* fix tests.

* avoid sorting multiple times.

* add test.

* change back.

* sort.

* add sort_csr.

* Fix a bug.

* fix.

* revert modifcation.

* rename

* speed up EdgeIds.
2019-09-28 10:52:12 -07:00
VoVAllen 54e1ef2e06 [Fix] Fix serialize NDArray when shape is 0 (#884)
* fix

* fix bugs
2019-09-28 19:21:38 +08:00
Quan (Andy) Gan e3921d5dec [Sampler] Metapath sampler for metapath2vec (#861)
* metapath sampler

* lint & fixes

* lint x2

* lint x3

* fix windows

* remove max_cycle argument

* add todo note
2019-09-26 16:42:33 -04:00
Da Zheng 2d489617dd [Feature] find the existence of negative edges. (#875)
* find the existence of negative edges.

* add comment.

* fix test.
2019-09-23 01:59:02 -07:00
Minjie Wang 9b4d60799a [Hetero] New syntax (#824)
* WIP. remove graph arg in NodeBatch and EdgeBatch

* refactor: use graph adapter for scheduler

* WIP: recv

* draft impl

* stuck at bipartite

* bipartite->unitgraph; support dsttype == srctype

* pass test_query

* pass test_query

* pass test_view

* test apply

* pass udf message passing tests

* pass quan's test using builtins

* WIP: wildcard slicing

* new construct methods

* broken

* good

* add stack cross reducer

* fix bug; fix mx

* fix bug in csrmm2 when the CSR is not square

* lint

* removed FlattenedHeteroGraph class

* WIP

* prop nodes, prop edges, filter nodes/edges

* add DGLGraph tests to heterograph. Fix several bugs

* finish nx<->hetero graph conversion

* create bipartite from nx

* more spec on hetero/homo conversion

* silly fixes

* check node and edge types

* repr

* to api

* adj APIs

* inc

* fix some lints and bugs

* fix some lints

* hetero/homo conversion

* fix flatten test

* more spec in hetero_from_homo and test

* flatten using concat names

* WIP: creators

* rewrite hetero_from_homo in a more efficient way

* remove useless variables

* fix lint

* subgraphs and typed subgraphs

* lint & removed heterosubgraph class

* lint x2

* disable heterograph mutation test

* docstring update

* add edge id for nx graph test

* fix mx unittests

* fix bug

* try fix

* fix unittest when cross_reducer is stack

* fix ci

* fix nx bipartite bug; docstring

* fix scipy creation bug

* lint

* fix bug when converting heterograph from homograph

* fix bug in hetero_from_homo about ntype order

* trailing white

* docstring fixes for add_foo and data views

* docstring for relation slice

* to_hetero and to_homo with feature support

* lint

* lint

* DGLGraph compatibility

* incidence matrix & docstring fixes

* example string fixes

* feature in hetero_from_relations

* deduplication of edge types in to_hetero

* fix lint

* fix
2019-09-17 15:41:57 -04:00
VoVAllen 0fb13f7b9d [Feature] Data format (#728)
* Add serialization

* add serialization

* add serialization

* lalalalalalalala

* lalalalalalalala

* serialize

* serialize

* nnn

* WIP: import tvm runtime node system

* WIP: object system

* containers

* tested basic container composition

* tested custom object

* tmp

* fix setattr bug

* tested object container return

* fix lint

* some comments about get/set state

* fix lint

* fix lint

* update cython

* fix cython

* ffi doc

* fix doc

* WIP: using object system for graph

* c++ side refactoring done; compiled

* remove stale apis

* fix bug in DGLGraphCreate; passed test_graph.py

* fix bug in python modify; passed utest for pytorch/cpu

* fix lint

* Add serialization

* Add serialization

* fix

* fix typo

* serialize with new ffi

* commit

* commit

* commit

* save

* save

* save

* save

* commit

* clean

* Delete tt2.py

* fix lint

* Add serialization

* fix lint 2

* fix lint

* fix lint

* fix lint

* fix lint

* Fix Lint

* Add serialization

* Change to Macro

* fix

* fix

* fix bugs

* refactor

* refactor

* updating dmlc-core to include force flag

* trying tempfile

* delete leaked pointer

* Fix assert

* fix assert

* add comment and test case

* add graph labels

* add load labels

* lint

* lint

* add graph labels

* lint

*  fix windows

* fix

* update dmlc-core to latest

* fix

* fix camel naming
2019-09-09 20:57:51 +08:00
Quan (Andy) Gan 52d4535b61 [Hetero][RFC] Heterogeneous graph Python interfaces & Message Passing (#752)
* moving heterograph index to another file

* node view

* python interfaces

* heterograph init

* bug fixes

* docstring for readonly

* more docstring

* unit tests & lint

* oops

* oops x2

* removed node/edge addition

* addressed comments

* lint

* rw on frames with one node/edge type

* homograph with underlying heterograph demo

* view is not necessary

* bugfix

* replace

* scheduler, builtins not working yet

* moving bipartite.h to header

* moving back bipartite to bipartite.h

* oops

* asbits and copyto for bipartite

* tested update_all and send_and_recv

* lightweight node & edge type retrieval

* oops

* sorry

* removing obsolete code

* oops

* lint

* various bug fixes & more tests

* UDF tests

* multiple type number_of_nodes and number_of_edges

* docstring fixes

* more tests

* going for dict in initialization

* lint

* updated api as per discussions

* lint

* bug

* bugfix

* moving back bipartite impl to cc

* note on views

* fix
2019-08-23 15:45:37 -04:00
Da Zheng be936da849 use FFI for subgraph. (#781) 2019-08-21 01:04:11 -07:00
Quan (Andy) Gan fd3d54ba98 bump up version for prerelease (#768) 2019-08-16 10:32:54 +08:00
Quan (Andy) Gan 16061925d5 [NodeFlow] Non-uniform neighbor sampling (#711)
* nonuniform sampler

* unit test

* test on out neighbors

* error checks

* lint

* fix

* clarification

* use macro switcher

* use empty array for uniform sampling

* oops

* Revert "oops"

This reverts commit a11f9ae707aaeb67fb5921c887a17d3711d5b04a.

* Revert "use empty array for uniform sampling"

This reverts commit 8526ce4cade89f2c1b09a08aca8830375ebafb31.

* re-reverting

* use a method
2019-08-07 09:42:08 +08:00
Quan (Andy) Gan e9e587b6cb [Refactor] C random number generator (#729)
* rng refactor

* fix bugs

* unit test

* remove setsize

* lint

* fix test

* use explicit instantiation instead of inlining

* stricter test

* use tvm solution

* moved python interface to dgl.random

* lint

* address comments

* make getthreadid an inline function
2019-07-31 17:54:30 +08:00
Minjie Wang 7ad663c3a9 [Hetero] Heterograph C++ implementation; Bipartite and Python wrapper (#725)
* finish bipartite graph implementation; compiled

* finished heterograph implementation; compiled

* WIP: apis

* C API codes

* compiled

* WIP: python

* HeteroGraphIndex

* WIP: test

* add DGLContext support in ffi

* fix bug in has edge

* unittests except edge subgraph

* edge subgraph

* fix lint

* address comments

* poke ci

* try fix

* fix msvc
2019-07-27 15:22:56 -04:00
Minjie Wang 67dc11971f [Refactor] Use object system for all CAPIs (#716)
* WIP: using object system for graph

* c++ side refactoring done; compiled

* remove stale apis

* fix bug in DGLGraphCreate; passed test_graph.py

* fix bug in python modify; passed utest for pytorch/cpu

* fix lint

* address comments
2019-07-23 12:47:05 -04:00
Minjie Wang b0d9e7aa43 [Refactor] Separating graph and sparse matrix operations (#699)
* WIP: array refactoring

* WIP: implementation

* wip

* most csr part

* WIP: on coo

* WIP: coo

* finish refactoring immutable graph

* compiled

* fix undefined ndarray copy bug; add COOToCSR when coo has no data array

* fix bug in COOToCSR

* fix bug in CSR constructor

* fix bug in in_edges(vid)

* fix OutEdges bug

* pass test_graph

* pass test_graph

* fix bug in CSR constructor

* fix bug in CSR constructor

* fix bug in CSR constructor

* fix stupid bug

* pass gpu test

* remove debug printout

* fix lint

* rm biparate grpah

* fix lint

* address comments

* fix bug in Clone

* cpp utests
2019-07-17 17:49:38 -04:00
Minjie Wang 90e78c5854 [FFI] FFI container support, custom structure extension via Object (#693)
* WIP: import tvm runtime node system

* WIP: object system

* containers

* tested basic container composition

* tested custom object

* fix setattr bug

* tested object container return

* fix lint

* some comments about get/set state

* fix lint

* fix lint

* update cython

* fix cython

* ffi doc

* fix doc
2019-07-01 14:40:07 -04:00
Quan (Andy) Gan 31b1c49433 [Hetero] C++ Heterogeneous Graph Interfaces (#663)
* c++ heterograph interfaces

* addressing comments

* update hetero api

* missing interfaces

* fix lint
2019-06-26 22:12:56 -04:00
Quan (Andy) Gan 059b1a6d6f [Release] Bump up version (#636)
* bump up version

* conda+cuda trial

* switch conda branch

* revert

* disable cudnn
2019-06-12 16:50:05 +08:00
Da Zheng 94ecb8eb96 [BUGFIX] copy graph index to shared memory. (#634)
* copy graph index to shared memory.

* fix.

* fix.

* fix.

* use a diff name for in-csr and out-csr.

* fix lint.

* remove print.

* add test.

* add comments.
2019-06-12 01:32:25 -07:00
Mufei Li fb9dcc51ff [Feature] Add API to convert graph to bidirected graph (#598)
* to_bidirected

* to_bidirected

* Fix style

* Fix

* Update

* Fix

* Fix

* Update

* Add examples
2019-06-10 14:02:29 +08:00