Cannot propagate alembic revision from container to file tree with docker watch

[PierrickLP]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched in the documentation/README.
• I already searched in Google "How to do X" and didn't find any information.
• I already read and followed all the tutorial in the docs/README and didn't find an answer.

Commit to Help

• I commit to help with one of those options 👆

Example Code

N/A

Description

With the new recommended docker compose watch command to start the project, any db revision files generated by alembic inside the container will not be replicated in the host system.

Operating System

Linux, Windows

Operating System Details

wsl2 on w11

Python Version

3.12.6

Additional Context

No response

[yohaoquan]

I am facing the same issue. As suggested by Docker documentation, sync only works one way: from host to container. With the current watch setup and not mounting the directory as a volume, generated alembic revision files are not synced back to the host.

I tried adding back the volume mapping but it resulted in the following error:
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "fastapi": executable file not found in $PATH: unknown

[yohaoquan]

I have found a working workaround (should have thought of it when I made the original comment):

Warning: This kinda breaks docker compose watch in that it keeps re-adding those files back to the container, which then gets synced back to the host because of volume mapping, then triggers watch's sync again... Only do this when you are creating a revision and revert the change once you are done.

Change the backend volume to include the alembic folder:

    volumes:
      - ./backend/htmlcov:/app/htmlcov
      - ./backend/app/alembic:/app/app/alembic

You could drill down more to only the versions folder.

[josephna76]

I'm having this same issue. When I added those backend volumes, my docker didn't build so I still wasn't able to get it to work - see error message below. Can we make this into an issue?

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "fastapi": executable file not found in $PATH: unknown (app)

[yohaoquan]

@josephna76 It's been a while since I did that but I believe adding only the alembic folder (and not the whole backend folder) is how I solved it when I encountered that issue.

The drawback of this is that live reload of fastapi will not work. I am developing it locally in the meantime but yes, I would want this to be an issue as well. Looks like the maintainers are not actively looking at discussions anyway.

[meme8383]

Try excluding the alembic folder from watch to fix the refresh

[josephna76]

Great, thanks yohaoquan and meme8383! I tried your solutions and it seemed to work, but I went with the documented solution of using SQLModel directly without Alembic while this gets resolved.

[willhama]

As @yohaoquan mentioned correctly docker compose watch or docker compose -v only works one way. So all actions taken in the docker containers are not transferred back to the host system. This was recently introduced in this commit.

I would suggest to update the documentation where the alembic revisions are still advised to be run in the container. Actually actions should be done from now on the host. As well adding packages should be done on the host (This actually is adjusted in the docs, thx @tiangolo :)).

Again run the following actions on your HOST not in the docker container.

1. To run the alembic revisions in the host container make sure your docker containers are running (Especially the db is needed so you can also just run docker compose up db.

2. Make sure you are in your virtual environment from uv. Run source .venv/bin/activate in the /backend folder to activate it.

3. go to the /backend folder and run alembic revision --autogenerate -m "Add column last_name to User model"

4. And run the migration. alembic upgrade head

I was in the lucky position to delete all my databases and start from scratch so I had no issues after doing it accidentally in the docker container.

[Sam1320]

I created this small script to expose the vars in .env so that the settings instance can work even when running locally.

# db_revision.sh
export PYTHONPATH=$(pwd)

# turn on auto-export
set -a
# expose env vars in .env
source ../.env
# turn off auto-export
set +a

# revision
alembic revision --autogenerate -m "$1"

then u can do ./script/db_revision "add column"

[adham404]

I did the steps you mentioned but I am getting this error
sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: connection to server at "127.0.0.1", port 5432 failed: FATAL: password authentication failed for user "postgres" (Background on this error at: https://sqlalche.me/e/20/e3q8)

have you by any chance faced the same issue? I am sure of the credentials I checked them several times. Also the credentials work properly in the adminer login.

[stevleibelt]

@adham404

best guess, you've missed to start the shell in the docker container?
docker compose exec -it backend bash.

Your script tries to connect to a database on your local machine on port 5432, but as far as I get the docker-compose.yml, the port 5432 is never exposed outside the docker stack.

[Suaralanre]

For anyone still having issues. Firstly, the issue seems to arise mostly among SELinux distros. Secondly, @yohaoquan above is nearly there with his solution, however there is the "z" flag missing as shown below:

volumes:
      - ./migrations:/app/migrations:z
      - ./migrations/versions:/app/migrations/versions:z

I wrote a comprehensive solution here that allows you to have your alembic files in your container showing in your local files whilst not breaking your compose watch feature as in the solution by @yohaoquan