You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
For many use cases, such as scientific research, debugging, or regulated environments, having reproducible model outputs is a critical requirement. The PyTorch backend currently lacks a direct mechanism to enforce deterministic algorithm execution, which can lead to slight variations in output even with the same inputs.
Describe the solution you'd like
We propose introducing a new model configuration parameter to enable deterministic operations within the PyTorch backend. This parameter would trigger at::Context::setDeterministicAlgorithms(true) during model loading or execution, ensuring that PyTorch uses deterministic algorithms whenever possible.
This would provide users with a simple and effective way to guarantee reproducibility for their models served via Triton.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Is your feature request related to a problem? Please describe.
For many use cases, such as scientific research, debugging, or regulated environments, having reproducible model outputs is a critical requirement. The PyTorch backend currently lacks a direct mechanism to enforce deterministic algorithm execution, which can lead to slight variations in output even with the same inputs.
Describe the solution you'd like
We propose introducing a new model configuration parameter to enable deterministic operations within the PyTorch backend. This parameter would trigger
at::Context::setDeterministicAlgorithms(true)
during model loading or execution, ensuring that PyTorch uses deterministic algorithms whenever possible.This would provide users with a simple and effective way to guarantee reproducibility for their models served via Triton.
Example
config.pbtxt
:Describe alternatives you've considered
N/A
Additional context
I’ve opened a small PR to add this feature. Would appreciate a review: triton-inference-server/pytorch_backend#150
Beta Was this translation helpful? Give feedback.
All reactions