BM-1473: Zeroecco/bento

[zeroecco]

🎉 move bento over to boundless
🎉 start redis before starting bento stuff
🎉 update nvidia to 12.9.1
🐛 drop sm_code=90 as it messes with mixed envs

[willpote]

LGTM, but would get @austinabell 's eyes on it before merge

[austinabell]

The nvidia compat is the only thing I think is blocking, other comments are somewhat opinionated

[austinabell]

Where does the 2gb number come from? Wondering the reason for this default? This seems like it would cause issues for provers locking a lot of large orders

[austinabell]

Noting that this change will add a lot of friction for provers building. It is already incredibly time consuming to build just the broker and we have been getting a lot of complaints.

Seems like this could be done in two ways, either also including the image in the declaration, so if you don't pass --build it uses prebuilt, and if you do it would rebuild. Alternative to have a compose.local.yml to look something like:

services:
  exec_agent0:
    image: null
    build:
      context: .
      dockerfile: dockerfiles/agent.dockerfile

  exec_agent1:
    image: null
    build:
      context: .
      dockerfile: dockerfiles/agent.dockerfile

  aux_agent:
    image: null
    build:
      context: .
      dockerfile: dockerfiles/agent.dockerfile

  gpu_prove_agent0:
    image: null
    build:
      context: .
      dockerfile: dockerfiles/agent.dockerfile

  snark_agent:
    image: null
    build:
      context: .
      dockerfile: dockerfiles/agent.dockerfile 

(haven't extensively tested both, there could be a reason why one might be worse)

[zeroecco]

It is my suggestion that we have really two options here:

1. pre build the binaries and pull them from somewhere directly into the container
2. the baking of images.

I favor the first option I am presenting. if this compiling is a larger issue we should pre compile the binaries and post them in releases IMHO.

[zeroecco]

if we agree on my suggestion I am fine to begin setting up the binaries build process

[austinabell]

Why remove this instead of updating to 12.9?

[zeroecco]

not needed in my testing with 12.9 on ampere and blackwell cards

[austinabell]

Seems like this is the case, although the specific use case is being able to run these on machines without a CUDA GPU, and not certain it works on all machines -- harder to test. Perhaps fine for this to be removed and see if anyone complains

[aridder]

I had to remove this line to make it work with 5090