Commit 43478d0

authored and

committed

FA3 Spec Decoding to support top k = 1 and add cuda graph support (sgl-project#5050)

Co-authored-by: Qingquan Song <ustcsqq@gmail.com> Co-authored-by: Chunan Zeng <zcnrex@gmail.com>

1 parent 4f79ccd commit 43478d0Copy full SHA for 43478d0

2 files changed

+270

-119

lines changed

python/sglang/srt
- layers/attention
  - flashattention_backend.py
- model_executor
  - forward_batch_info.py

+270

-119

lines changed

Comments

(0)