[FixBug] compute early stopping with real batch size #3418
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
yuanlehome
previously approved these changes
Aug 15, 2025
yuanlehome
approved these changes
Aug 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pcard-71500
repetition early stopping实现时，还没有real batch size的机制，每次传入的batch_size是固定的max-num-seqs。但引入real batch size之后，windows的batch size可能会与probs的batch size不一致, 导致triton kernel的计算出现CUDA 700。triton的BUG已经在#3375 进行修复。
本PR主要修复小算子的实现方式，同时新增单测test_consistency_with_real_batch_size，对real batch size的情况进行测试。