Add custom op declaration for all_reduce
#3473
Open
+30
−23
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
SigureMo
approved these changes
Aug 19, 2025
Comment on lines
+169
to
+172
| .Inputs({"inp", | ||
| "out"}) | ||
| .Outputs({"new_out"}) | ||
| .Attrs({"_fa: int64_t", "_reg_buffer: int64_t", "reg_buffer_sz_bytes: int64_t"}) |
zhink
reviewed
Aug 20, 2025
|
|
||
| except: | ||
| tensor_model_parallel_all_reduce = None | ||
| @paddle.jit.marker.unified |
There was a problem hiding this comment.
这里能解释下为什么 def 一个函数是有可能报错的么？以我浅薄的认知，这里不会有问题，如果有问题麻烦提供下报错呢，按我理解一定会有更合适的处理方式
There was a problem hiding this comment.
是不是因为旧版本Paddle版本没有 paddle.jit.marker.unified 才加的 try except 语句呀
或者可以这样：
if hasattr(paddle.jit, "marker") and (hasattr, paddle.jit.marker, "unified"):
mark_as_unified = paddle.jit.marker.unified
else:
# do-nothing for PaddlePaddle 3.1-
mark_as_unified = lamdba fn: fn
tensor_model_parallel_all_reduce = mark_as_unified(tensor_model_parallel_all_reduce)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Deepseek V3 开启SOT动转静推理后，存在这样一个打断：
排查后发现，目前的
all_reduce只是 cpp_extensions （PyCapsule表明是C++拓展），而不是 Paddle 自定义算子，所以需要给all_reduce添加自定义算子声明：另外，由于自定义算子要求 Tensor 参数在前，其他 attr 参数在后，所以也调整了 all_reduce 的参数顺序
另外由于SOT的dataclass目前存在这样的问题——直接这样设置会存在属性设置不生效的问题：
所以提前设置一下，后续SOT会修复这个问题：
cc @SigureMo @zyfncg