Skip to content

Commit 4d1e52a

Browse files
authored
Add an assertion to enhance the robustness of the operator (#5736)
1 parent 155890e commit 4d1e52a

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

python/sglang/srt/layers/attention/vision.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,6 +271,8 @@ def forward(
271271
Returns:
272272
[b * s, h, head_size]
273273
"""
274+
if self.flatten_batch:
275+
assert bsz == 1, "flatten_batch is True, bsz must be 1"
274276

275277
s = q.shape[0] // bsz
276278

0 commit comments

Comments
 (0)