[BUG] Secrets not loading from .env file

[ollipa]

Description

Previously in version v2.27.1 this worked:

In .env:

MY_SECRET=very_secret

In docker compose yaml:

secrets:
  MY_SECRET:
    environment: MY_SECRET

However in after updating to v2.28.1 the secret is not loaded correctly from .env file anymore causing builds that rely on the secret to fail.

Steps To Reproduce

Define a secret in .env file and try to use it in docker compose.

Compose Version

Docker Compose version v2.28.1

Docker Environment

Client: Docker Engine - Community                                               
 Version:    27.0.2                                                             
 Context:    default                                                            
 Debug Mode: false                                                              
 Plugins:                                                                       
  buildx: Docker Buildx (Docker Inc.)                                           
    Version:  v0.15.1                                                           
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx                     
  compose: Docker Compose (Docker Inc.)                                         
    Version:  v2.28.1                                                           
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Anything else?

No response

[ndeloof]

I ran a quick test:

$ cat compose.yaml 
services:
  toto:
   image: alpine
   command: cat /run/secrets/test
   secrets:
    - test

secrets:
  test:
    environment: MY_SECRET

$ docker compose version
Docker Compose version v2.28.1-11-g11d5ecdc7

$ docker compose run toto
very_secret 

So, AFAICT this works as intended

[ollipa]

@ndeloof, I'm sorry I wasn't clear enough in my bug report. Indeed it seems that the secrets are accessible from the running container but not in the build phase. Here is a simple way to reproduce the issue based on your example:

docker-compose.yaml

services:                                                                                                                   
  toto:                                                                                                                     
   build:                                                                                                                   
     context: .                                                                                                             
     dockerfile: Dockerfile.toto                                                                                            
     secrets:                                                                                                               
       - test                                                                                                               
   command: cat /run/secrets/test                                                                                           
   secrets:                                                                                                                 
    - test                                                                                                                  
                                                                                                                            
secrets:                                                                                                                    
  test:                                                                                                                     
    environment: MY_SECRET

Dockerfile.toto

FROM debian:latest                                                                                                          
                                                                                                                            
RUN --mount=type=secret,id=test \                                                                                           
   test "$(cat /run/secrets/test)" = "very_secret"

.env

MY_SECRET=very_secret

With Docker Compose v2.27.1 this is successful:

> docker compose version                                                                                                    
Docker Compose version v2.27.1

> docker compose build --no-cache toto
[+] Building 1.7s (6/6) FINISHED                                                                             docker:default 
 => [toto internal] load build definition from Dockerfile.toto                                                         0.0s 
 => => transferring dockerfile: 148B                                                                                   0.0s 
 => [toto internal] load metadata for docker.io/library/debian:latest                                                  1.2s 
 => [toto internal] load .dockerignore                                                                                 0.0s 
 => => transferring context: 2B                                                                                        0.0s 
 => CACHED [toto stage-0 1/2] FROM docker.io/library/debian:latest@sha256:1dc55ed6871771d4df68d393ed08d1ed9361c577cfe  0.0s 
 => [toto stage-0 2/2] RUN --mount=type=secret,id=test    test "$(cat /run/secrets/test)" = "very_secret"              0.3s 
 => [toto] exporting to image                                                                                          0.1s 
 => => exporting layers                                                                                                0.1s 
 => => writing image sha256:a3f1ddc73e364763212e3951331cca2781a5c703318732a0102ffaca946a21e4                           0.0s 
 => => naming to docker.io/library/dc-bug-toto                                                                         0.0s 

With Docker Compose v2.28.1 the build fails:

> docker compose version                                                                                                    
Docker Compose version v2.28.1

> docker compose build --no-cache toto
[+] Building 2.6s (5/5) FINISHED                                                                             docker:default 
 => [toto internal] load build definition from Dockerfile.toto                                                         0.0s 
 => => transferring dockerfile: 148B                                                                                   0.0s 
 => [toto internal] load metadata for docker.io/library/debian:latest                                                  2.2s 
 => [toto internal] load .dockerignore                                                                                 0.0s 
 => => transferring context: 2B                                                                                        0.0s 
 => CACHED [toto stage-0 1/2] FROM docker.io/library/debian:latest@sha256:1dc55ed6871771d4df68d393ed08d1ed9361c577cfe  0.0s 
 => ERROR [toto stage-0 2/2] RUN --mount=type=secret,id=test    test "$(cat /run/secrets/test)" = "very_secret"        0.3s 
------                                                                                                                      
 > [toto stage-0 2/2] RUN --mount=type=secret,id=test    test "$(cat /run/secrets/test)" = "very_secret":                   
------                                                                                                                      
failed to solve: process "/bin/sh -c test \"$(cat /run/secrets/test)\" = \"very_secret\"" did not complete successfully: exi
t code: 1

[croth1-liveeo]

@ndeloof, FYI we are running into the same issue:

• secret key-value pair defined in the .env file
• passed via

secrets:
  MY_SECRET:
    environment: MY_SECRET

• secret string ends up empty in docker compose build with v2.28.1.
• passing secret explicitly via MYSECRET=secret docker compose build works.
• after downgrading to v2.27.x docker compose build also works.

Did not test whether this is already fixed in main.

[ndeloof]

@croth1-liveeo see #11974