* init
* init
* working cublasGemm
* benchmark high-mem/low-mem, err gather_mm output
* cuda kernel for bmm like kernel
* removed cpu copy for E_per_Rel
* benchmark code from Minjie
* fixed cublas results in gathermm sorted
* use GPU shared mem in unsorted gather mm
* minor
* Added an optimal version of gather_mm_unsorted
* lint
* init gather_mm_scatter
* cublas transpose added
* fixed h_offset for multiple rel
* backward unittest
* cublas support to transpose W
* adding missed file
* forgot to add header file
* lint
* lint
* cleanup
* lint
* docstring
* lint
* added unittest
* lint
* lint
* unittest
* changed err type
* skip cpu test
* skip CPU code
* move in-len loop inside
* lint
* added check different dim length for B
* w_per_len is optional now
* moved gather_mm to pytorch/backend with backward support
* removed a_/b_trans support
* transpose op inside GEMM call
* removed out alloc from API, changed W 2D to 3D
* Added se_gather_mm, Separate API for sortedE
* Fixed gather_mm (unsorted) user interface
* unsorted gmm backward + separate CAPI for un/sorted A
* typecast to float to support atomicAdd
* lint typecast
* lint
* added gather_mm_scatter
* minor
* const
* design changes
* Added idx_a, idx_b support gmm_scatter
* dgl doc
* lint
* adding gather_mm in ops
* lint
* lint
* minor
* removed benchmark files
* minor
* empty commit
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
* implement pin_memory/unpin_memory/is_pinned for dgl.graph
* update python docstring
* update c++ docstring
* add test
* fix the broken UnifiedTensor
* XPU_SWITCH for kDLCPUPinned
* a rough version ready for testing
* eliminate extra context parameter for pin/unpin
* update train_sampling
* fix linting
* fix typo
* multi-gpu uva sampling case
* disable new format materialization for pinned graphs
* update python doc for pin_memory_
* fix unit test
* UVA sampling for link prediction
* dispatch most csr ops
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update doc
* update examples
* change unitgraph and heterograph's PinMemory to in-place
* update examples for multi-gpu uva sampling
* update doc
* fix linting
* fix cpu build
* fix is_pinned for DistGraph
* fix is_pinned for DistGraph
* update graphsage unsupervised example
* update doc for gpu sampling
* update some check for sampling device switching
* fix linting
* adapt for new dataloader
* fix linting
* fix
* fix some name issue
* adjust device check
* add unit test for uva sampling & fix some zero_copy bug
* fix linting
* update num_threads in graphsage examples
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* add ut
* add doc link
* install dep
* fix ci
* fix ut; more comments
* remove deprecated attributes in rdf datasets; fix label feature name
* address comments
* fix ut for other frameworks
* Fix ref to message-passing guide
* Fix pygments and spacing
* Update build documentation steps in README.md
* Use links
* Adjust parameters in SAGEConv docstring in same order as init
* Fix spelling error
* Change doc link
* implement pin_memory/unpin_memory/is_pinned for dgl.graph
* update python docstring
* update c++ docstring
* add test
* fix the broken UnifiedTensor
* eliminate extra context parameter for pin/unpin
* fix linting
* fix typo
* disable new format materialization for pinned graphs
* update python doc for pin_memory_
* fix unit test
* update doc
* change unitgraph and heterograph's PinMemory to in-place
* update comments for NDArray's PinMemory_ and PinData
* update doc
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* Added graph updating method and tests. resolves#3488
* removed spaces around named args
* customizing indices for graph's idtype and ctx
* changing torch ops to generic backend ops
* changing tensors to np arrays
* created dgl merge function and tests
* Changed per-graph edge updates to single ag update
* removed update method and tests
* reformat newlines & spaces
* concatenating in one-shot instead of iteratively
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* [Feature] enable TP::Receiver wait for any numbers of senders
* fix random unit test failure
* avoid endless future wait
* fix unit test failure
* fix seg fault when finalize wait in receiver
* [Feature] refactor sender connect logic and remove unnecessary sleeps in unit tests
* fix lint
* release RPCContext resources before process exits
* [Debug] TPReceiver wait start log
* [Debug] add log in get port
* [Debug] add log
* [ReDebug] revert time sleep in unit tests
* [Debug] remove sleep for test_distri,test_mp
* [debug] add more log
* [debug] add listen_booted_ flag
* [debug] restore commented code for queue
* [debug] sleep more in rpc_client
* restore change in tests
* Revert "restore change in tests"
This reverts commit 41a18926d1.
* Revert "[debug] sleep more in rpc_client"
This reverts commit a908e758ea.
* Revert "[debug] restore commented code for queue"
This reverts commit d3f993b374.
* Revert "[debug] add listen_booted_ flag"
This reverts commit 244b2167d9.
* Revert "[debug] add more log"
This reverts commit 4b78447b0a.
* Revert "[Debug] remove sleep for test_distri,test_mp"
This reverts commit e1df1aadcc.
* remove debug code
* revert unnecessary change
* revert unnecessary changes
* always reset RPCContext when get started and reset all data
* remove time.sleep in dist tests
* fix lint
* reset envs before each dist test
* reset env properly
* add time sleep when start each server
* sleep for a while when boot server
* replace wait_thread with callback
* fix lint
* add dglconnect handshake check
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* [Feature] add CSVDataset to load data from csv files
* add CSVDataset class file
* install pyyaml when running unit tests
* install pandas for unit tests
* utilize pydantic for YAML config check
* generate yaml and csv files during test
* make more keys as optional
* remove/rename several keys in yaml config and more tets though looks a bit clumsy
* fix test failure on mxnet
* pass /path/to/dataset instead of yaml path
* code refinement
* code refine
* change several yaml field such as feat and graph_id
* merge graph generation logic
* refine code
* Refactored_first_version
* DGLCSVDataset works for single heterograph
* add more tests
* fix test failure in mxnet
* add docstring
* use list comprehension for dict
* fix version in YAML
* refine data length assert
* use dict.pop directly
* remove ambiguous variable names
* refine graph id missing logic
* refine graph create call
* separate node/edge/graph data parser
* remove separator in DefaultDataParser
* refine validation error log for yaml field
* minor check
* refine code via dict.get()
* move load_from_csv into Node/Edge/GraphData
* move _parse_node/edge/graph_data into Node/Edge/GraphData
* refine id-related fields check
* check duplicate ntypes/etypes when load yaml
* refine docstring
* Added activation function as an optional parameter
* lint fixes
* Modified the input parameters in tandem with other classes
* lint corrections
* corrected tests
* Reverting back to the old interface
* lint corrections
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* edge_softmax_hetero forwar+cpu+norm=dst
* convert eids to list
* addedunittest
* added unittest
* added backward. Not tested correctness
* minor
* changed reducer to max from sum
* bugfix
* docstring
* add GPU unittest
* output converted to dict from tuple
* lint check
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
* Feat: support randomwalk and SelectPinSageNeighbors on GPU
Port RandomWalk algorithm on GPU,
and port SelectPinSageNeighbors on GPU.
* Feat: support GPU on python APIs
* Feat: remove perf print information in FrequenchHashmap
* Fix: modified the code format
Modified the code format as task_lint.sh suggested
* Feat: let test script support PinSAGESampler on GPU
Let test script support PinSAGESampler on GPU,
minor of "restart_prob".
* Minor
* Minor
* Minor
* Refactor: use the atomic operations from the array module
* Minor: change the long lines
* Refactor: modified the get_node_types for gpu
* Feat: update the contributor date
* Perf: remove unnecessary stream sync
* Feat: support other random walk
But the non-uniform choice is still not supported.
* Fix: add CUDA switch for random walk
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* added distgnn plus libra codebase
* Dist application codes
* added comments in partition code. changed the interface of partitioning call.
* updated readme
* create libra partitioning branch for the PR
* removed disgnn files for first PR
* updated kernel.cc
* added libra_partition.cc and moved libra code from kernel.cc to libra_partition.cc
* fixed lint error; merged libra2dgl.py and main_Libra.py to libra_partition.py; added graphsage/distgnn folder and partition script.
* removed libra2dgl.py
* fixed the lint error and cleaned the code.
* revisions due to PR comments. added distgnn/tools contains partitions routines
* update 2 PR revision I
* fixed errors; also improved the runtime by 10x.
* fixed minor lint error
* fixed some more lints
* PR revision II changed the interface of libra partition function
* rewrite docstring
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Make the default value of parameter replace in docstring consistent with code
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
* min/max support for forward CPU heterograph
* Added etype with each argU values
* scatter_add needs fix
* added scatter_add_hetero. Grads dont match for max reducer
* storing ntype in argX
* fixing scatter_add_hetero
* hetero matches with torch's scatter add
* works copy_e forward+cpu
* added backward for copy_rhs
* Computes gradient for all node types in one kernel
* bug fix
* unnitest for max/min on CPU
* renamed scatter_add_hetero to update_grad_minmax_hetero
* lint check and comment out cuda call for max. Code is for CPU only
* lint check
* replace inf with zero
* minor
* lint check
* removed LIBXSMM code from hetro code
* fixing backward operator of UpdateGradMinMaxHetero
* removed backward from update_grad_minmax_hetero
* docstring
* improved docstring and coding style
* Added pass by pointer for output
* typos and pass by references
* Support for copy_rhs
* Added header <string>
* fix bug in copy_u_max
* Added comments and dimension check of all etypes
* skip mxnet check
* pass by pointer output arrays
* updated docstring
Co-authored-by: Israt Nisa <nisisrat@amazon.com>
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
