Optimize build cache

[terrykong]

We currently add in the project before running uv sync as opposed to adding just the uv.lock/pyproject.toml and syncing the environment first.

This makes poor use of the build cache, but needed to be done since doing a uv pip install --no-deps -e . after the uv syncs to warm the cache, still caused a uv sync at runtime. This suggests something about uv pip is not congruent with the uv sync and we should explore other ways to be more efficient with the build cache.

One option is just to naively copy after the final uv sync since the project is installed editable, but we need to think about that carefully since it may not be correct to copy all things into the container without a reinstall, e.g., the egg info/editable metadata

[chtruong814]

@terrykong could you share the build where you saw a uv sync at runtime? Our docker build is minimally including the pyproject.toml and the package info files because those are needed. Previously we could get away with only the pyproject.toml but because of the versioning being fetched dynamically, it has to include those additional files which shouldn't change much.

https://github.com/NVIDIA/reinforcer/blob/main/docker/Dockerfile#L21

[terrykong]

Yea, here's a CI run where you can see it downloading again https://github.com/NVIDIA/reinforcer/actions/runs/14233565053/job/39890706499#step:5:154

there seemed to be two problems with the dockerfile on main:

1. the multistage build resulted in a target image that installed all the dependencies in .venv but if you uv pip list, you'll only see reinforcer. this triggers a re-download from uv run/sync
2. the cache was not copied from the deps stage so when it had to re-download, it was slow

So in #102 I'm removing the multistage build to retain the cache and ensure uv run/sync can run without re syncing, but I wasn't able to COPY over just the minimal project metadata + uv pip install at the end w/o causing uv run/sync to partially re-sync.

[terrykong]

Here's a modified version of the dockerfile in #102 where i pip install at the end

ARG BASE_IMAGE=anyscale/ray:2.43.0-py312-cu125
FROM ${BASE_IMAGE} AS base
# base is just ray + uv with minimal installs so it is a very lightweight container

# It is more convenient for users to run as root
USER root

RUN apt-get update && sudo apt-get install -y jq

RUN pip install uv
RUN echo "unset RAY_RUNTIME_ENV_HOOK" >> /home/ray/.bashrc
# Disable usage stats by default for users who are sensitive to sharing usage.
# Users are encouraged to enable if the wish.
ENV RAY_USAGE_STATS_ENABLED=0

RUN chmod 755 /home/ray/.cache

FROM base AS hermetic

WORKDIR /opt/reinforcer
COPY nemo_reinforcer/package_info.py nemo_reinforcer/package_info.py
COPY nemo_reinforcer/__init__.py nemo_reinforcer/__init__.py
COPY pyproject.toml .
COPY uv.lock .
RUN <<"EOF"
uv venv .venv
# uv sync has a more reliable resolver than simple uv pip install which can fail

# Sync each training + inference backend one at a time (since they may conflict)
# to warm the uv cache, then at the end just sync the default dependencies.
# Do everything in one layer to prevent large layers.

uv sync --locked --extra vllm --no-install-project
uv sync --locked --all-groups --no-install-project
EOF

COPY --chown=ray --chmod=755 . /opt/reinforcer

RUN uv pip install --no-deps --editable .

ENV VIRTUAL_ENV=/opt/reinforcer/.venv
ENV PATH="/opt/reinforcer/.venv/bin:$PATH"

# The ray images automatically activate the anaconda venv. We will
# comment this out of the .bashrc to give the same UX between docker
# and other clusters like slurm.
RUN <<"EOF"
cp ~/.bashrc ~/.bashrc.backup  # backup existing .bashrc

# Comment out the conda initialize block
sed -i '/# >>> conda initialize >>>/,/# <<< conda initialize <<</ { /^[^#]/ s/^/# / }' ~/.bashrc

# Comment out any line that explicitly exports the anaconda3 PATH
sed -i '/export PATH=\$HOME\/anaconda3\/bin:\$PATH/ s/^/# /' ~/.bashrc
EOF

and you'll see that it still tries to download

[terrykong]

fixed in #203