Commit ca8d02a

authored

FA3 Spec Decoding to support top k = 1 and add cuda graph support (#5050)

Co-authored-by: Qingquan Song <ustcsqq@gmail.com> Co-authored-by: Chunan Zeng <zcnrex@gmail.com>

1 parent 3f287b8 commit ca8d02aCopy full SHA for ca8d02a

2 files changed

+270

-119

lines changed

python/sglang/srt
- layers/attention
  - flashattention_backend.py
- model_executor
  - forward_batch_info.py

+270

-119

lines changed

Comments

(0)