Failing docker pull on M series Macbook

[JoelLucaAdams]

When attempting to download the docker image using docker pull ghcr.io/fusion-energy/neutronics-workshop I get the following error on my M3 Macbook:

Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found
Unable to find image 'ghcr.io/fusion-energy/neutronics-workshop:latest' locally
docker: Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found.
See 'docker run --help'.

I can however circumvent this issue by using the docker link located under the OS/Arch section here https://github.com/fusion-energy/neutronics-workshop/pkgs/container/neutronics-workshop

I'm not familiar with much of docker images or how to add "support" for arm chips but a rudimentary search reveals that adding platforms: linux/amd64,linux/arm64 to the end of the docker-publish.yaml file would fix the issue

[shimwell]

We did take a quick look into this but in addition to the platform other changes would be needed

The amount of other changes to the dockerfile is unknown as of yet and quite a risk at this late stage before the York.

Will certainly look into this after the York workshop but currently don't want to risk breaking things for the non mac users

[JoelLucaAdams]

No worries, just wanted to make this issue aware. Thankfully there is the fix I suggested above which should work for the workshop at York.

[JessicaRehak]

If you have docker desktop installed and access to the more advanced drivers, it's easy to build and push containers for both amd64 and arm64 (for the new mac M-series chips) with one command. This would probably work, and it will push both of the versions and the manifest, and avoid that hanging unknown build architecture:

docker build --push --platform linux/arm64,linux/amd64 --tag ghcr.io/fusion-energy/neutronics-workshop:buildx --provenance=false --label "org.opencontainers.image.source=https://github.com/fusion-energy/neutronics-workshop" -f .devcontainer/Dockerfile .

I changed the tag from latest to buildx so it won't break any existing containers that have been pushed if you wanted to try it.

[shimwell]

Thanks for the suggest I can try that command, did it work at your end?

As I think it is equivalent to the recent attempt that failed in ci

Ross did make this PR recently

#313

We did try the platform option but it failed in CI

https://github.com/fusion-energy/neutronics-workshop/actions/runs/13242354397

[JessicaRehak]

I am working on building the base repository hopefully for both platforms right now, and then I'll try to build the workshop container. When I'm finished I can upload them to my dockerhub if that works at least. Then folks can at least pull it if they need it in the mean time.

I'm running into an issue building it though with openmc-dagmc-wrapper required vtk. I'm not sure if this is an obvious issue (I was just passing through the repo and stopped to help 😅 ), but up until there this command worked just fine

docker build --platform linux/arm64,linux/amd64 --tag neutronics-workshop-base:buildx --provenance=false -f .devcontainer/base.Dockerfile .

[shimwell]

openmc-dagmc-wrapper can be removed from the container if that helps

[JessicaRehak]

Issues along the way:

• openmc-dagmc-wrapper as mentioned above, removed.
• cadquery-ocp==7.7.1 removed. I even tried building directly from github and that didn't work. It couldn't find any versions
• nlopt removed because the wheel failed to build, I'm not sure why.

I'm running into more pip install issues as I go, so I'm not sure this would actually work. I'm not sure why pip isn't finding the packages; there may not be arm64 versions. To run on arm64 it may require building some of the packages from source, which would be quite an overhaul to the Dockerfile.

Sorry 😞

[shimwell]

Many thanks for the effort. I am sure this will be useful in the future. Your efforts are much appreciated

[JessicaRehak]

Ok, looking into it much further than I had anticipated it looks like vtk specifically doesn't have arm64/aarch64 installs available. For when you folks want to update the package a user released a wheel here for both python 3.10 and python 3.12: https://github.com/finsberg/vtk-aarch64/

Unfortunately, the bookworm image your container is using comes shipped with python 3.11 which isn't one of the supported wheels. I got around this by updating to:

FROM mcr.microsoft.com/devcontainers/python:3.10-bookworm as dependencies

to downgrade the overall installation to python3.10. I think this will cause some follow-on issues because it looks like for MOAB you are hard linking to the 3.11 site package directories.

BUT, this also means installing vtk from that wheel worked using

RUN pip install "https://github.com/finsberg/vtk-aarch64/releases/download/vtk-9.2.6-cp310/vtk-9.2.6.dev0-cp310-cp310-linux_aarch64.whl"

This enabled openmc-dagmc-wrapper to install correctly. It looks like something similar may be going on with cadquery-ocp.

There are a ton of packages being installed here, and it seems like only recently folks are catching up to building on arm64/aarch64. It may therefore be worthwhile to make a second dockerfile for arm64 builds and pare down the packages included significantly.

[shimwell]

Nice work.

I've also been looking at the dependency stack and made some mac wheels for moab here https://github.com/shimwell/wheels/tree/main/moab

I also removed openmc-dagmc-wrapper from the dockerfile as we really don't need that one anymore #319