Exposing object storage to clients

[cgwalters]

See coreos/rpm-ostree#5386 (comment)

The default overlay driver is a "layer store", it doesn't identify individual files. When zstd:chunked is enabled, files can be identified by their individual sha256 digest and also deduplicated across layers.

However, in neither case is the underlying storage model exposed to higher levels at all to what can be reached from podman|buildah build or really even Go clients AFAIK.

An object store as a socket API

Imagine that inside a default podman build there was something like a socket /run/objectstore that speaks DBus|varlink|whatever and has just two core primitives:

(queue bikeshed for $digest; fsverity or full-file sha256|sha512 etc)

• Put an object by $digest (any space limitations would be restricted based on any limits in place for writing to the layer)
• Link an object named by $digest into a specific place in the target filesystem tree (and gracefully returns an ENOENT equivalent if the object isn't present)

The latter operation is the key one - it would operate underneath the overlayfs mount, and that would allow avoiding copying and re-checksumming.

[cgwalters]

This came up in a live chat; it relates to containers/image#2599 and bootc-dev/bootc#20

Basically even outside of a container image build, such an API would make sense as a way to get access to the underlying files of a c/storage instance (and actually a composefs-rs instance too).

So if we added such an API to c/storage for example, I think it'd be interesting to have it expressed via this socket API to start (but wrapped with Go bindings of course).

And we would also have e.g. a podman image object-store command that would operate like skopeo experimental-image-proxy and speak this protocol over fd 0 by default.