[BUG] Setting environment and env_file service attributes

[wray27]

Description

The environment variables under env_file are ignored. When environment variables are set under the environment attribute.

Related issue on stack overflow

This was working on Docker Compose version 2.28.1

Steps To Reproduce

1. Running on M1 Mac
2. docker-compose version 2.34.0, docker version 28.0.4

.env

FRUIT=APPLE

---
docker-compose.yml

my-service: 
    image: python:3.13.2-alpine3.21
    env_file:
      - .env
    environment:
      TZ: Europe/London

4. In a directory running - dcp run my-service. The environment variables in .env are ignored. The environment variables set under the environment are there.

What I expect is for the environemnt variables to be interpolated so that they all exist.

Compose Version

docker-compose version 2.34.0

Docker Environment

docker version 28.0.4

Anything else?

No response

[glours]

Hello @wray27

Do you have more details to share because I'm unable to reproduce the issue with your example

~/tmp/issue-12706 > bat --decorations=never .env                                                                                                                                                                                      15:41:50
FRUIT=APPLE

~/tmp/issue-12706 > bat --decorations=never compose.yaml                                                                                                                                                                              16:00:17
services:
  my-service:
    image: python:3.13.2-alpine3.21
    env_file:
      - .env
    environment:
      TZ: Europe/London

~/tmp/issue-12706 > docker compose version                                                                                                                                                                                            16:00:25
Docker Compose version v2.34.0-desktop.1
~/tmp/issue-12706 > docker compose config                                                                                                                                                                                             16:00:33
name: issue-12706
services:
  my-service:
    environment:
      FRUIT: APPLE
      TZ: Europe/London
    image: python:3.13.2-alpine3.21
    networks:
      default: null
networks:
  default:
    name: issue-12706_default

~/tmp/issue-12706 > docker compose run my-service /bin/sh                                                                                                                                                                             16:00:37
[+] Creating 1/1
 ✔ Network issue-12706_default  Created                                                                                                                                                                                                   0.0s
[+] Running 4/4
 ✔ my-service Pulled                                                                                                                                                                                                                      2.9s
   ✔ 45ccf01a8316 Pull complete                                                                                                                                                                                                           1.2s
   ✔ 5f200fdec155 Pull complete                                                                                                                                                                                                           0.7s
   ✔ f88e0568d87b Pull complete                                                                                                                                                                                                           0.4s
/ # env
HOSTNAME=d5f730a0aff3
SHLVL=1
HOME=/root
GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
PYTHON_SHA256=d984bcc57cd67caab26f7def42e523b1c015bbc5dc07836cf4f0b63fa159eb56
TERM=xterm
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PYTHON_VERSION=3.13.2
FRUIT=APPLE
PWD=/
TZ=Europe/London

[studskula]

I’m experiencing a similar issue after updating today. The difference is that when using a profile, the env_file is not loaded. Without a profile, it works as expected.

Environment: macOS, Docker Compose v2.34.0.

Does not work (with profile):

run_migrations:
    build:
      context: .
      dockerfile: Dockerfile
      target: run_migrations
    networks:
      - services-network
    depends_on:
      - db
    profiles: ["run_migrations"]
    command: ["alembic", "upgrade", "head"]
    env_file:
      - ".env.local"

Does work (without profile):

run_migrations:
    build:
      context: .
      dockerfile: Dockerfile
      target: run_migrations
    networks:
      - services-network
    depends_on:
      - db
    command: ["alembic", "upgrade", "head"]
    env_file:
      - ".env.local"

Also, when running:

docker compose run --env-from-file .env.local run_migrations env

It works fine without profile. With the profile, I get panic: assignment to entry in nil map

[pflipp]

in my case it seems to relate to the profiles option of the service:
.env:

BAR: baz
FOO: bar

compose.yaml:

services:
  app:
    profiles:
      - explicit
    image: golang:alpine
    environment:
      BAR: foo
    env_file:
      - .env

.env is ignored completely

docker compose run --rm app printenv

PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=16c665e09ec4
TERM=xterm
BAR=foo
GOLANG_VERSION=1.23.2
GOTOOLCHAIN=local
GOPATH=/go
HOME=/root

.env is used with --env-from-file but overriding in service environment is not possible

docker compose run --env-from-file .env --rm app printenv

PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=3b42f7d1c9ee
TERM=xterm
BAR=baz
FOO=bar
GOLANG_VERSION=1.23.2
GOTOOLCHAIN=local
GOPATH=/go
HOME=/roo

compose.yaml without service profile:

services:
  app:
    image: golang:alpine
    environment:
      BAR: foo
    env_file:
      - .env

.env works as expected (BAR is overridden in service)

docker compose run --rm app printenv

PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=7f50410e6063
TERM=xterm
FOO=bar
BAR=foo
GOLANG_VERSION=1.23.2
GOTOOLCHAIN=local
GOPATH=/go
HOME=/root

[stefanfoulis]

I can confirm the behavior @pflipp describes. As soon as I add profiles: ["something"] to a service, I don't see any of the expected env vars inside the container anymore.
It seems to work ok with docker compose 2.33.1 and be broken in 2.34.0.

[AlwinGarside]

I'm running into this issue as well.

From what I've deduced, having profiles: defined for a service, while not having a profile active (either through --profile or COMPOSE_PROFILES) and then specifically targeting that service with docker compose will cause the env_files of that service to no longer be parsed.

Here's an easily reproducible example:

bar.env

BAR=bar

docker-compose.yaml:

services:
  service-with-environment:
    image: hello-world
    environment:
      FOO: 'foo'
    profiles: [ my-profile ]

  service-with-env_file:
    image: hello-world
    env_file:
      - bar.env
    profiles: [ my-profile ]

  service-with-both:
    image: hello-world
    env_file:
      - bar.env
    environment:
      FOO: 'foo'
    profiles: [ my-profile ]

$ docker compose config service-with-environment
name: test
services:
  service-with-environment:
    profiles:
      - my-profile
    environment:
      FOO: foo
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: test_default

$ docker compose config service-with-env_file   
name: test
services:
  service-with-env_file:
    profiles:
      - my-profile
    env_file:
      - /Users/alwin/test/bar.env
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: test_default

$ docker compose config service-with-both    
name: test
services:
  service-with-both:
    profiles:
      - my-profile
    environment:
      FOO: foo
    env_file:
      - /Users/alwin/test/bar.env
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: test_default

As you can see in the output above, service-with-env_file and service-with-both both retain the env_file: key. The expected behavior is for the referenced .env file to be loaded and it's environment variables to be injected in the environment: key.

Declaring an active profile, either by providing --profile or exporting COMPOSE_PROFILES, will cause behavior as expected:

$ docker compose --profile=my-profile config service-with-env_file
name: test
services:
  service-with-env_file:
    profiles:
      - my-profile
    environment:
      BAR: bar
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: test_default

$ export COMPOSE_PROFILES=my-profile

$ docker compose config service-with-env_file
name: test
services:
  service-with-env_file:
    profiles:
      - my-profile
    environment:
      BAR: bar
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: test_default

Directly targeting a service with profiles: set, should cause those profiles to always become active. This still works, except for the env_files: part.

It seems that whatever is responsible for including the environment variables from the env_files: no longer gets the memo that a certain profile has dynamically been activated.

[AlwinGarside]

This may be a regression caused by #12611.

[AlwinGarside]

@glours Is it possible to revert #12611 for the time being?

This issue is now causing quite the headache for me and the dev teams that I provide DevOps for, a workaround is to remove all profile: references, but that is not really feasible in our setup.

[glours]

That must have already been resolved by this PR and the fix will be part of the next Compose release coming very soon
If you can't wait, building Compose from the main branch should create a Compose binary with the fix

My test with your repo case and a Compose version built from main branch

~/tmp/issue-12706 > bat --decorations=never bar.env                                                                                                                                                                                   10:53:30
BAR=bar
~/tmp/issue-12706 > bat --decorations=never compose.yaml                                                                                                                                                                              10:53:42
services:
  service-with-environment:
    image: hello-world
    environment:
      FOO: 'foo'
    profiles: [ my-profile ]

  service-with-env_file:
    image: hello-world
    env_file:
      - bar.env
    profiles: [ my-profile ]

  service-with-both:
    image: hello-world
    env_file:
      - bar.env
    environment:
      FOO: 'foo'
    profiles: [ my-profile ]
~/tmp/issue-12706 > docker compose version                                                                                                                                                                                            10:53:49
Docker Compose version v2.34.0-30-gee3314302
~/tmp/issue-12706 > docker compose config service-with-env_file                                                                                                                                                                       10:53:56
name: issue-12706
services:
  service-with-env_file:
    profiles:
      - my-profile
    environment:
      BAR: bar
    image: hello-world
    networks:
      default: null
networks:
  default:
    name: issue-12706_default

[glours]

@Yogarine @wray27 @stefanfoulis @pflipp can you check the latest Compose release v2.35.0 fixed the issue for you?

[Drauna]

Hello @glours

I had some interests in this fix (cause me some troubles past few days). So, I've made tests on my own on this new release and it fixed this issue.

Thanks for the time/work !