PVC usage in serverConfig in mlserver

[VInozemtsev]

Hi,

I've noticed that the PVC used in serverConfig might not be as critical as it seems. When the server starts, there's a function UnloadAllModels, and in that function, there's a case like this:

if isStartup {
    // Unload any existing models on server to ensure we start in a clean state
}

So during startup, it deletes all downloaded models from the /mnt/agent folder.

If I use an RWX PVC for the StatefulSet, so that it shares a volume between the pods, and the models are already downloaded, then when I scale the replicas to 2, it ends up deleting all files in that folder.

Is there any way to preserve the downloaded models across replicas to speed things up and avoid re-downloading them on the second replica?

If we use an emptyDir instead of a standard PVC, the result is the same — the second replica starts and downloads all the models again. So what's the actual benefit of using a PVC in this case?

Thanks