-
[Refactor]: Introduce GPUKVFormat (#2567)
frostbyte_neo 发布于
2026-02-17 19:44:30 +00:00 - initial commit
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add changes to multiprocess mode
Signed-off-by: Samuel Shen slshen@uchciago.edu
- unit tests for new format
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix circular import in gpu_connector/utils.py
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix circular import in gpu_connector/utils.py
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add forward declaration
Signed-off-by: Samuel Shen slshen@uchciago.edu
- missing semicolons
Signed-off-by: Samuel Shen slshen@uchciago.edu
- missing semicolons
Signed-off-by: Samuel Shen slshen@uchciago.edu
- remove trailing comma
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix check_paged_kv_cache_equal
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add flash infer correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add keyword bindings and remove flashinfer from correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add more backends to single request correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- ifx layerwise gpu connector tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix non cuda UT
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix UT x2
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix GPUKVFormat enum naming
Signed-off-by: Samuel Shen slshen@uchciago.edu
- template the gpu_kv_format
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix the correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix flipped TransferDirection enum in layerwiseGPU connector
Signed-off-by: Samuel Shen slshen@uchciago.edu
- call get gpu kv format on unwrapped tensors
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix sglang unit tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix sglang flattening order
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix test_multi_layer_kernel
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix test_multi_layer_kernel
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add logging to correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix curl in correctness tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- actually don't need clone
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add contiguous
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fixed unit tests
Signed-off-by: Samuel Shen slshen@uchciago.edu
- revert correctness tests since they all need BATCH INVARIANCE
Signed-off-by: Samuel Shen slshen@uchciago.edu
- fix gpu picking in ci running on ie-users
Signed-off-by: Samuel Shen slshen@uchciago.edu
- loosen comprehensive from 1.1 to 1.2
Signed-off-by: Samuel Shen slshen@uchciago.edu
- add non-contiguous kv cache registration
Signed-off-by: Samuel Shen slshen@tensormesh.ai
Signed-off-by: Samuel Shen slshen@uchciago.edu
Signed-off-by: Samuel Shen slshen@tensormesh.ai
Co-authored-by: Samuel Shen slshen@uchciago.edu下载附件