PyTorch+ROCm for AMD Linux maybe?

[reaperx7]

Is there anyway we can get PyTorch with ROCm support added for AMD GPUs as an experimental alternative to Tensor-RT to help reduce CPU usage for AMD GPUs?

Many thanks.

[umireon]

@royshil

[royshil]

rocm in general is supported by ONNX runtime: https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html
i just don't have a solid way of testing this
also we would need to build onnxruntime with rocm support: https://onnxruntime.ai/docs/build/eps.html#amd-rocm
since the linux release for ORT doesn't include the rocm EP

only CUDA and TRT
so i'm not sure... we could certainly use some help w this

[umireon]

@royshil We can provide build instructions with ROCm rather than provide pre-built binaries.

@reaperx7 Are you so kind as to develop a ROCm-enabled version of our plugin? We will appreciate your contributions!

[reaperx7]

I would if I could, but I can't read code worth a penny.

[umireon]

@reaperx7 Do you have an environment to test our ROCm-enabled plugin if we provide you with build instructions?

[reaperx7]

I have an up to date ArchLinux system with the ROCm libraries, amdgpu-pro installed, and an RX 5700XT. If it's buildable, and all I need to do is drag and drop it into the plugin folder for OBS, then... Yes. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Saturday, January 27, 2024 7:02:44 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> Do you have an environment to test our ROCm-enabled plugin if we provide you with build instructions? — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6SHLL4RGKOXGWN7RV3YQUJJJAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTGE4DAOBQHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[payom]

Just dropping in to confirm I was able to build the plugin using the C/C++ API of the ROCm Execution Provider. All I needed to do was to modify ort-session-utils.cpp to add the ROCm EP to the ONNX Runtime session instead of the TensorRT EP, and to then to set the appropriate flag in CMake to use the system's own ONNX Runtime presumably built for ROCm.

[umireon]

@reaperx7 @payom I have implemented the initial ROCm support. Can you test this?
#545

[reaperx7]

I can try and test it if a tarball is available. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Friday, February 9, 2024 12:06:03 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> @payom<https://github.com/payom> I have implemented the initial ROCm support. Can you test this? #545<#545> — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6VBBOI5PD4UXITLOELYSXKGXAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZVGQ4DQNZVG4>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[umireon]

@reaperx7 You must build the binary by yourself. We will not provide any pre-built things. Thank you!

[reaperx7]

That's fine. I just need the sources either in tarball or the exact git repository. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Friday, February 9, 2024 4:18:16 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> You must build the binary by yourself. We will not provide any pre-built things. — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6TG7HWWFAFJL7ICHY3YSYHYRAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZVHAZDQNRVHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>