* adding LABOR sampling
* add ladies and pladies samplers
* fix compile error after rebase
* add reference for ladies sampler
* Improve ladies implementation.
* weighted labor sampling initial implementation draft
fix indentation and small bug in ladies script
* importance_sampling currently doesn't work with weights
* fix weighted importance sampling
* move labor example into its own folder
* lint fixes
* Improve documentation
* remove examples from the main PR
* fix linting by not using c++17 features
* fix documentation of labor_sampler.py
* update documentation for labor.py
* reformat the labor.py file with black
* fix linting errors
* replace exception use with if
* fix typo in error comment
* fixing win64 build for ci
* fixing weighted implementation, works now.
* fix bug in the weighted case and importance_sampling==0
* address part of the reviews
* remove unused code paths from cuda
* remove unused code path from cpu side
* remove extra features of labor making use of random seed.
* fix exclude_edges bug
* remove pcg and seed logic from cpu implementation, seed logic should still work for cuda.
* minor style change
* refactor CPU implementation, take out the importance_sampling probability computation into a function.
* improve CUDAWorkspaceAllocator
* refactor importance_sampling part out to a function
* minor optimization
* fix linting issue
* Revert "remove pcg and seed logic from cpu implementation, seed logic should still work for cuda."
This reverts commit c250e07ac6d7e13f57e79e8a2c2f098d777378c2.
* Revert "remove extra features of labor making use of random seed."
This reverts commit 7f99034353080308f4783f27d9a08bea343fb796.
* fix the documentation
* disable NIDs
* improve the documentation in the code
* use the stream argument in pcg32 instead of skipping ahead t times, can discard the use of hashmap now since it is faster this way.
* fix linting issue
* address another round of reviews
* further optimize CPU LABOR sampling implementation
* fix linting error
* update the comment
* reformat
* rename and rephrase comment
* fix formatting according to new linting specs
* fix compile error due to renaming, fix linting.
* lint
* rename DGLHeteroGraph to DGLGraph to match master
* replace other occurrences of DGLHeteroGraph to DGLGraph
Co-authored-by: Muhammed Fatih BALIN <m.f.balin@gmail.com>
Co-authored-by: Kaan Sancak <kaansnck@gmail.com>
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* enable uva for pinsage sampler
* unit test
* modify some checks on the python side
* remove legacy random walk code
* update unit test
* update unit test
* fix unit test
* adjust checks
* move some checks to c++
* move max_nodes check to cuda kernel
* fix ci for tf
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
* fix uva sampling with features
* fix
* add is_listlike function to distinguish strings from sequence
* fix
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>
* implement pin_memory/unpin_memory/is_pinned for dgl.graph
* update python docstring
* update c++ docstring
* add test
* fix the broken UnifiedTensor
* XPU_SWITCH for kDLCPUPinned
* a rough version ready for testing
* eliminate extra context parameter for pin/unpin
* update train_sampling
* fix linting
* fix typo
* multi-gpu uva sampling case
* disable new format materialization for pinned graphs
* update python doc for pin_memory_
* fix unit test
* UVA sampling for link prediction
* dispatch most csr ops
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update graphsage example to combine uva sampling and UnifiedTensor
* update doc
* update examples
* change unitgraph and heterograph's PinMemory to in-place
* update examples for multi-gpu uva sampling
* update doc
* fix linting
* fix cpu build
* fix is_pinned for DistGraph
* fix is_pinned for DistGraph
* update graphsage unsupervised example
* update doc for gpu sampling
* update some check for sampling device switching
* fix linting
* adapt for new dataloader
* fix linting
* fix
* fix some name issue
* adjust device check
* add unit test for uva sampling & fix some zero_copy bug
* fix linting
* update num_threads in graphsage examples
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
* Feat: support randomwalk and SelectPinSageNeighbors on GPU
Port RandomWalk algorithm on GPU,
and port SelectPinSageNeighbors on GPU.
* Feat: support GPU on python APIs
* Feat: remove perf print information in FrequenchHashmap
* Fix: modified the code format
Modified the code format as task_lint.sh suggested
* Feat: let test script support PinSAGESampler on GPU
Let test script support PinSAGESampler on GPU,
minor of "restart_prob".
* Minor
* Minor
* Minor
* Refactor: use the atomic operations from the array module
* Minor: change the long lines
* Refactor: modified the get_node_types for gpu
* Feat: update the contributor date
* Perf: remove unnecessary stream sync
* Feat: support other random walk
But the non-uniform choice is still not supported.
* Fix: add CUDA switch for random walk
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* Feat: support API "randomwalk_topk" in library
* Feat: use the new API "randomwalk_topk" for PinSAGESampler
* Minor
* Minor
* Refactor: modified codes as checker required
* Minor
* Minor
* Minor
* Minor
* Fix: checking errors in RandomWalkTopk
* Refactor: modified the docstring for randomwalk_topk
* change randomwalk_topk to internal
* fix
* rename
* Minor for pinsage.py
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* [Feature] Exclude edges in sample_neighbors
Extending sample_neighbors and sample_frontier
API to support exclude_edges parameter.
exclude_edges support tensor and dict data
Feature enable excluding certain edges
during neighborhood sampling
Exclude_edges contains EID's of edges
which will be excluded
during neighbor picking for seed nodes.
Added test case for heterograph and homograph
RFC issue id: 2944
* compatibility
* fix
* fix
Co-authored-by: Quan Gan <coin2028@hotmail.com>
* fix.
* fix.
* fix.
* fix.
* Fix test
* Deprecate old DistEmbedding impl, use synchronized embedding impl
* Basic imple of heterogeneous on homogenenous sampling
* make pass
* Pass C++ test
* Add python test code
* lint
* lint
* Add MultiLayerEtypeNeighborSampler
* Add unitest for single machine dataloader
* Add dist dataloader test for edge type sampler
* Fix lint
* fix
* support for per etype sample
* Fix some bug and enable distributed training with per edge sample
* fix
* Now distributed training works
* turn off some mxnet
* turn off mxnet for some dist test
* fix
* upd
* upd according to the comments
* Fix
* Fix test and now distributed works.
* upd
* upd
* Fix
* Fix bug
* remove dead code.
* upd
* Fix
* upd
* Fix
Co-authored-by: Ubuntu <ubuntu@ip-172-31-71-112.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-66.ec2.internal>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* Start on uniform GPU sampling
* Save more work
* Get cu file compiling
* Update sampling
* More changes
* Get GPU sampling for uniform probabilities solved
* Fix batch tensor migration
* Fix
* update kernels
* expand blocking
* Undo testing change
* Cut down on sampling overhead
* Fix replacement
* Update unit tests
* Add option to gpu sample in graphsage
* Copy only csc to gpu
* Add ogbn support
* Fix linting
* Remove nvtx from sample
* Improve documentation and error checking
* Expand documentation
* Update assert checking
* delete extra space
* Use standard dataloader when dataset is a dictionary
* ogb -> ogbn
* Fix edge selection determinism
* Fix typos
* Remove nvtx
* Add comment for self.fanout_arrays and assert
* Fix linting
* Migrate to scalarbatcher
* Fix indentation
* Fix batcher
* Fix indexing
* Only use databatcher for GPU
* Convert to DGL NDArray to PyTorch Tensor
* Add optimization for PyTorch's F.tensor() for list of GPU tensors
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
* clean commit
* oops forgot the most important files
* use einsum
* copy feature from frontier to block
* Revert "copy feature from frontier to block"
This reverts commit 5224ec963eb6a3ef1b6ab74d8ecbd44e4e42f285.
* temp fix
* unit test
* fix
* revert jtnn
* lint
* fix win64
* docstring fixes and doc indexing
* revert einsum in sparse bidecoder
* fix some examples
* lint
* fix due to some tediousness in remove_edges
* addresses comments
* fix
* more jtnn fixes
* fix
Rhett Ying