github-featured/lightseekorg--tokenspeed
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
918 Commits 135 Branches 1 Tag
main
Commit Graph

477 Commits

Author SHA1 Message Date
Nicolas Perrin 619836d450 feat(mla): serve a block drafter from the CuteDSL MLA backend (#1222) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
 2026-08-24 18:17:08 -07:00
Nicolas Perrin 86886c3d5e perf(metadata): scrub the padded input tail in one launch (#1218) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
 2026-08-24 15:38:20 -07:00
FC.Li 0da906d046 refactor(runtime): finish the control-plane / data-plane split (#1224) 
Signed-off-by: FC-Li <robert_lexis@163.com>
Co-authored-by: longcat-ia-team <longcat_ia_team@meituan.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-25 00:40:19 +08:00
Pengzhan Zhao 7cd7ca0b30 feat(deepseek-v4): add AMD MI350 support (#1201) 
Signed-off-by: Pengzhan Zhao <borontion@gmail.com>
 2026-08-23 19:41:46 -07:00
FC.Li 5f4e37bbb1 feat(runtime): prefill chunk pipeline + event-loop cohesion refactor (#1214) 
Signed-off-by: FC-Li <robert_lexis@163.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: longcat-ia-team <longcat_ia_team@meituan.com>
 2026-08-23 21:59:55 +08:00
Chen Hongtao 103120bf77 perf(k3): extend latent-tail fusion to M64 with split collectives (#1174) 
Signed-off-by: chenht2022 <chenht2022@gmail.com>
 2026-08-23 02:12:13 -07:00
LightSeek Foundation 2b23fd3c99 Update tokenspeed-smg* pins to 1.9.0.post20260823 / 0.4.14.post20260823 / 0.8.0.post20260823 (#1213) 
Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com>
 2026-08-23 01:14:36 -07:00
Simo Lin 22e70d63ce feat(runtime): push scheduler load snapshots (#1210) 
Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
 2026-08-23 00:41:53 -07:00
Nicolas Perrin ec94552941 perf(metadata): fuse the speculative round's KDA metadata into single launches (#1209) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
 2026-08-23 02:28:16 -04:00
Nicolas Perrin 1e084e57c2 perf(kda): v-major decode and split the verify megafusion for NVIDIA (#1204) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
 2026-08-22 20:03:00 -07:00
torchspec-bot a2daa17d1a feat(kimi-k3): serve DSpark drafts (fc_norm + AttnRes tap) (#1031) 
Signed-off-by: torchspec-bot <262938024+torchspec-bot@users.noreply.github.com>
 2026-08-22 17:42:20 -07:00
LightSeek Foundation 2706143a86 Upgrade TokenSpeed Scheduler to 0.1.9 (#1208) 2026-08-22 13:03:26 -07:00
Simo Lin 0683caf969 fix(cli): disable smg load monitoring by default (#1205) 
Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
 2026-08-22 12:48:37 -07:00
Simo Lin 9f1c9cf016 fix(runtime): make load queries cancellation-safe (#1206) 
Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
 2026-08-22 12:47:33 -07:00
Nicolas Perrin f4d8ec6e03 perf(gemm): route the unrouted K3 decode projections (#1200) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
 2026-08-22 11:03:35 -04:00
Nicolas Perrin 22d02440f1 perf(metadata): fuse the paged write-location and page-table expansions (#1198) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
 2026-08-22 11:02:54 -04:00
LightSeek Foundation a9f200ecbd fix(runtime): serialize concurrent load queries (#1197) 
Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com>
 2026-08-21 19:36:17 -07:00
Max191 8ed1a67b3b fix(kernel): Correct DSA scoring and add GFX950 standard-cache kernel (#1196) 
Signed-off-by: Max Dawkins <max.dawkins@gmail.com>
 2026-08-21 16:27:49 -07:00
LightSeek Foundation 0358f7c509 Update tokenspeed-smg* pins to 1.9.0.post20260821 / 0.4.14.post20260821 / 0.8.0.post20260821 (#1195) 
Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com>
 2026-08-21 16:01:22 -07:00
Zhewen Yu 7a86c5c54e perf(gfx1250): port strided KDA decode and fused AttnRes (#1192) 
Signed-off-by: Yu-Zhewen <zhewenyu@amd.com>
 2026-08-21 10:49:17 -07:00
Yue Weng 739c0194d9 feat(m3): support M3 + DSpark (#1190) 2026-08-21 03:12:15 -07:00
Nicolas Perrin 138b48aba2 perf(attn_res): NVIDIA online-v2 fused kernel + combine load reorder with PDL (#1186) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
 2026-08-20 22:57:51 -07:00
HEJIAN SANG eaf9e503bb refactor(rl): remove unused RL weight-transfer APIs, consolidate on a single control plane (#1183) 
Signed-off-by: Hejian Sang <sanghj0923@gmail.com>
 2026-08-20 19:16:30 -07:00
Zhewen Yu 074049d432 perf(gfx950): use V-major recurrent state for KDA (#1176) 
Signed-off-by: Yu-Zhewen <zhewenyu@amd.com>
 2026-08-20 12:42:27 -07:00
FC.Li c731cb889d Remove CachePoolSpec.layer_group_ids; derive state groups from the plan (#1178) 
Signed-off-by: FC-Li <robert_lexis@163.com>
Co-authored-by: longcat-ia-team <longcat_ia_team@meituan.com>
 2026-08-20 11:00:40 -07:00
Nicolas Perrin 70f692669a chore(graph): deliver enable_cudagraph_gc to the capture context (#1177) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
 2026-08-20 10:42:03 -04:00
Nicolas Perrin 3d596a856d perf(k3): route decode GEMV per shape (#1157) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
 2026-08-20 01:57:56 -07:00
LightSeek Foundation d262066f89 Bump setuptools to 83.0.0 (#1166) 2026-08-20 01:16:40 -07:00
LightSeek Foundation 12bdf12f9c Upgrade TokenSpeed Scheduler to 0.1.8 (#1164) 2026-08-19 23:22:56 -07:00
Nicolas Perrin fdf3aae748 feat(kda): eager replay commit for speculative target verify (#1156) 
Signed-off-by: Nicolas Perrin <nperrin.ai@gmail.com>
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Co-authored-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2026-08-19 23:03:16 -07:00
dongjiyingdjy dc1d49b6da refactor(kda): disallow None host extend lengths — raise instead of silent fallback (#1161) 
Signed-off-by: dongjiyingdjy <87510204+dongjiyingdjy@users.noreply.github.com>
 2026-08-19 22:40:12 -07:00
Nicolas Perrin 9bf1529a98 perf(mla): build the FP8 query and write the latent KV in one launch (#1136) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
 2026-08-19 16:13:31 -07:00
Nicolas Perrin 082e2c1312 feat(loader): add the InstantTensor weight loader, opt-in only (#1153) 
Signed-off-by: nperrin-fr <nperrin.ai@gmail.com>
Co-authored-by: gongwei-130 <weigong28@gmail.com>
 2026-08-19 17:22:31 -04:00
Sanket Pandit 235a1d760a perf(kda): consume strided decode projections (#1133) 
Signed-off-by: Sanket Pandit <sanket.pandit@amd.com>
 2026-08-19 14:20:56 -07:00
Xurui Yang 978ed2cfdc perf(cache): sparsify and budget Kimi-K3 state cache (#1147) 
Signed-off-by: Xurui Yang <50316220+OftenDream@users.noreply.github.com>
Co-authored-by: longcat-ia-team <312376714+longcat-ia-team@users.noreply.github.com>
 2026-08-19 02:39:22 -07:00
Quinn Dawkins 94f4d07cf6 perf(kimi3): avoid packed KDA QKV copies (#1129) 
Signed-off-by: Quinn Dawkins <quinn.dawkins@gmail.com>
 2026-08-18 23:33:20 -07:00
dongjiyingdjy cb1b5ed9fd perf(kda): forward host cu_seqlens to the CuteDSL prefill wrapper (#1120) 
Signed-off-by: dongjiyingdjy <87510204+dongjiyingdjy@users.noreply.github.com>
 2026-08-18 23:32:00 -07:00
dongjiyingdjy 1e4a7946ca Support nvidia/Kimi-K3-NVFP4: ModelOpt FP8_PB_WO attention + NVFP4 SiTU MoE (#1128) 
Signed-off-by: dongjiyingdjy <87510204+dongjiyingdjy@users.noreply.github.com>
 2026-08-18 23:30:29 -07:00
Awshesh 6e18ae2ed3 fix(sampling): import Self from typing_extensions (#1149) 
Signed-off-by: Awshesh12 <asharma@es.iitr.ac.in>
 2026-08-18 23:12:36 -07:00
sjqgogogogo 42f8f4cc85 fix dp timeline (#1122) 
Signed-off-by: sjqgogogogo <sjqgogogogo@gmail.com>
 2026-08-17 21:06:47 -07:00
minedec 1e740b7c20 feat(quant): add block-wise fp8 online quantization support (#1097) 2026-08-17 21:02:14 -07:00
Nicolas Perrin 365db3db03 fix(comm): stop forcing NCCL's NVLS and cuMem paths off (#1127) 
Signed-off-by: Nicolas <nperrin.ai@gmail.com>
 2026-08-17 21:26:21 -04:00
Xiangyi Zhang ceaf86ae17 perf(v4): avoid DSpark prefill chunk synchronizations (#1116) 
Signed-off-by: Xiangyi Zhang <xiangyiz@nvidia.com>
 2026-08-17 02:10:45 -07:00
dongjiyingdjy b99ca12fca perf(kimi3): fold the MoE finalize into the multicast latent tail (+ extract the K3 comm layer) (#1062) 
Signed-off-by: dongjiyingdjy <87510204+dongjiyingdjy@users.noreply.github.com>
 2026-08-16 23:23:59 -07:00
FC.Li 6083537ed3 refactor(cache): one pipeline per model family — layers, group, pack, bind (#1117) 
Signed-off-by: FC-Li <robert_lexis@163.com>
Co-authored-by: longcat-ia-team <longcat_ia_team@meituan.com>
 2026-08-16 13:18:16 -07:00
Xiangyi Zhang 18e5fd4bd4 perf(v4): avoid prefill host synchronizations (#1115) 
Signed-off-by: Xiangyi Zhang <xiangyiz@nvidia.com>
 2026-08-16 00:43:42 -07:00
LightSeek Foundation f5e06fc1fc fix(hf): load metadata from resolved snapshots (#1114) 
Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com>
 2026-08-15 19:25:19 -07:00
Xurui Yang 40205cee1f refactor(pd): clarify cache transfer contract ownership (#1108) 
Signed-off-by: Xurui Yang <50316220+OftenDream@users.noreply.github.com>
Co-authored-by: longcat-ia-team <longcat_ia_team@meituan.com>
 2026-08-15 13:37:03 -07:00
Enwei Zhu 21cfc7c419 perf(inkling): eliminate per-step elementwise/copy kernels on the decode path (#1110) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2026-08-15 13:28:21 -07:00
tuanzhangCS cd78b4e0b2 feat: add Qwen3.5 GDN ReplaySSM (#1096) 
Signed-off-by: tuanzhangCS <tuan@lightseek.org>
 2026-08-15 12:59:41 -07:00