Skip to content

Custom properties

Custom properties allow you to decorate your repository with information such as compliance frameworks, data sensitivity, or project details.

vLLM
any new collections. should be added manually as an option
A100
available gpu's are T4, A100, A10
tinyllama-1-1b-chat-vllm-gguf
name of the template
inferless-runtime-config.yaml
the runtime file name
PYTORCH
source framework type options PYTORCH, ONNX, TENSORFLOW