[SYSTEMDS-3877] Docker image size reduction and safety improvement

[alexanderschmi]

Purpose

Update Docker Images to run on Alpine Linux

What Changed

1. Switched from Ubuntu to Alpine
2. Added multi stage build to reduce image size

[Baunsgaard]

Thanks for the PR, @alexanderschmi

While looking into the docker images could we also look at security vulnerabilities:

https://hub.docker.com/repository/docker/apache/systemds/general

[Baunsgaard]

An example of B security:

https://hub.docker.com/layers/apache/spark-connect-swift/pi/images/sha256-ff987ef8350ee3032268af288c2479728d7c6b6cfca626741bf1216093d57382

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.95%. Comparing base (a914c0d) to head (b33283c).
⚠️ Report is 35 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2274      +/-   ##
============================================
- Coverage     72.96%   72.95%   -0.01%     
- Complexity    46090    46095       +5     
============================================
  Files          1479     1479              
  Lines        172655   172654       -1     
  Branches      33797    33796       -1     
============================================
- Hits         125972   125965       -7     
- Misses        37188    37209      +21     
+ Partials       9495     9480      -15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[alexanderschmi]

Size Reduction

Before

After

We reduced the size of the main docker image by 1.11 GB.

[Baunsgaard]

Size Reduction

Before

After

We reduced the size of the main docker image by 1.11 GB.

Hi @alexanderschmi

Thanks for the PR, it is a great improvement!

Based on your pictures, did we move the SystemDS folder to the root? If so, would it not be better if we move it back to the previous position?

Also, which image is it referring too?

Best regards
Sebastian

[alexanderschmi]

Dockerfile Refactoring: latest & testing-latest

Summary

This pull request refactors the two main Docker images of SystemDS — latest and testing-latest — with the goal of reducing image size, improving build efficiency, and modernizing the base images.

---

Changes in Detail

latest

• Switched from Ubuntu to Alpine
  → The base operating system was changed to Alpine Linux to significantly reduce image size.
  → All package installations were adapted to use apk, and unnecessary dependencies were removed.
• Minimal, production-oriented setup
  → Only essential runtime dependencies are included
  → Usage of jre instead of jdk in final image because of multi-stage approach
  → Integration of native hadoop library

testing-latest

• Switched from Ubuntu to debian:bullseye-slim
  → The operating system was changed to reduce image size.
• Migrated to a multi-stage build
  → The previous single-stage build was replaced with a multi-stage setup
  → The final testing image is smaller
• Fully functional test environment
  → All unit and integration tests run as before
  → CI integration remains fully compatible

---

Results

• latest:
  • smaller due to Alpine base (from 1.5 GiB to 460 MiB → reduced by 1.1 GB)
  • more secure through Alpine base and a default non-root user
• testing-latest:
  • smaller due to bullseye-slim base (from 1.7 GiB to 1.4 GiB → reduced by 325 MB)

[Baunsgaard]

I made a quick pass, it looks great.

Please fix the remaining comments.

[Baunsgaard]

LGTM, will merge once the images are built.