Add support for Docker's credential stores and helpers

[gbaso]

Running spring-boot:build-image fails when the builder configured in the spring-boot-maven-plugin is not in a public repository, with message:

Execution default-cli of goal org.springframework.boot:spring-boot-maven-plugin:3.4.2:build-image failed: Docker API call to '/var/run/docker.sock/v1.41/images/create?fromImage=my-registry/my-builder:latest' failed with status code 500 "Internal Server Error" and message "Head "https://my-registry/v2/my-builder/manifests/latest": no basic auth credentials"

Docker is configured with a credential helper (ecr-login + aws sso login for me) so docker pull/push work fine, but I suspect that direct API calls to the socket don't interact with the credential helper. Using docker login also doesn't work, nor does pulling the image beforehand.

pack build works fine, but I'd rather use a maven plugin than coordinate the build between maven and an external tool.

I need a different builder because of #43716.

Possibly related to #25898.

Steps to reproduce:

• create a private registry (e.g. on aws ecr)
• download the appropriate credential helper and put it on your PATH
• configure docker to use the credential helper (credHelpers section of .docker/config.json)
• login if required (e.g. aws sso login)
• build and push a builder to the private registry
• create an empty spring boot project and override the image builder parameter of the spring boot maven plugin with your builder
• run mvn spring-boot:build-image

[wilkinsona]

Thanks for raising this. The title suggests that authenticating with a builder registry doesn't work at all. As far as we know, that's not the case. Your description reads like the problem only occurs when using a credential helper. Does that match what you're experiencing?

[gbaso]

Hello @wilkinsona, sorry about the confusion. I also tried with docker login with a similar error:

Execution default-cli of goal org.springframework.boot:spring-boot-maven-plugin:3.4.2:build-image failed: Docker API call to '/var/run/docker.sock/v1.41/images/create?fromImage=my-registry/my-builder:latest' failed with status code 404 "Not Found"

I don't know if the different error is due to a difference in docker login vs credential helper or because I'm using a different registry here (dockerhub, as I don't have an account where I can docker login on ecr), but the end result is the same.

[mhalbritter]

If you have pulled the image beforehand, you could workaround by setting the pullPolicy of the spring-boot-plugin to NEVER. Then it should use the already pulled image.

I guess the existing authentication options are no help because you don't have a username / password / token?

[gbaso]

I guess the existing authentication options are no help because you don't have a username / password / token?

Even with a valid username / password they are not used for pulling the builder, as mentioned in the docs they are only used for publishing. I tried anyway and it doesn't work.

If you have pulled the image beforehand, you could workaround by setting the pullPolicy of the spring-boot-plugin to NEVER. Then it should use the already pulled image.

This is a valid workaround, although it introduces fragilities into the build and doesn't guarantee the builder is fresh, which has security concerns.

[mhalbritter]

I can't find the section where it mentions that the credentials are not used when pulling the builder.

If the Docker images specified by the builder or runImage properties are stored in a private Docker image registry that requires authentication, the authentication credentials can be provided using docker.builderRegistry properties.

I haven't read the code, but this reads like authenticating when pulling builder images is possible. Did you try

tasks.named("bootBuildImage") {
	docker {
		builderRegistry {
			username = "user"
			password = "secret"
			url = "https://docker.example.com/v1/"
			email = "user@example.com"
		}
	}
}

?

[gbaso]

Apologies, I was misremembering the documentation and only taking into consideration docker.publishRegistry. Everything works correctly when providing username / password to docker.builderRegistry.

@wilkinsona you were correct, the issue only sussists when using a credential helper.

[mhalbritter]

We'd have to implement support for Docker's credentials store / helpers. Documentation for that is here: https://docs.docker.com/reference/cli/docker/login/#credential-stores

[wilkinsona]

A few notes from experimenting with this. The user's Docker config identifies a store:

{
        "auths": {},
        "credsStore": "desktop",
        "currentContext": "desktop-linux",
        "plugins": {
                "-x-cli-hints": {
                        "enabled": "true"
                }
        },
        "features": {
                "hooks": "true"
        }
}

In the above, the store is named desktop. Prefixing this with docker-credential- gives the name of a binary that implements the credentials helper protocol. This binary should be on the path.

The list command can be used to list credentials in the store:

$ docker-credential-desktop list                                   
{"https://index.docker.io/v1/":"some-username","https://index.docker.io/v1/access-token":"some-username","https://index.docker.io/v1/refresh-token":"some-username"}

The get command can then be used to retrieve a credential from the store. It reads the server address from standard in, for example:

echo "https://index.docker.io/v1/" | docker-credential-desktop get
{"ServerURL":"https://index.docker.io/v1/","Username":"some-username","Secret":"dckr_pat_some_token"}

AIUI, the Username and Secret from this json can then be used to authenticate with https://index.docker.io/v1/.

[nosan]

I have several questions regarding this issue:

• Should the credential helper be used by default when credentials are not provided, or should it be enabled explicitly through a property?
• What should be done if the credential helper is not available in the $PATH for some reason?

[nosan]

I've implemented default Docker authentication in my branch, leveraging credential helpers, the credential store, and static credentials. I have not updated the documentation yet as I wanted to get some feedback on whether it's worth proceeding further. Implementation details were taken from the official Docker CLI.

By default, if users do not provide credentials, instead of using empty authentication (which was the previous behavior), a new DefaultDockerRegistryAuthentication is used. It reads the Docker configuration file and, based on it, attempts to provide authentication details for the requested ImageReference.

This change should resolve this issue, and additionally #25898 and #24547

[philwebb]

@nosan Are you able to submit a PR from your branch? I'd like to see if we can try to squeeze something for this into 3.5

[philwebb]

For reference, the following Testcontainers code might also help:

• AuthDelegatingDockerClientConfig

• RegistryAuthLocator

[nosan]

Sure, @philwebb

#45269