Skip to content

Significantly improve standalone installer#17022

Merged
efrazer-oai merged 23 commits intomainfrom
edwarf/codex/native-installer-rework
Apr 15, 2026
Merged

Significantly improve standalone installer#17022
efrazer-oai merged 23 commits intomainfrom
edwarf/codex/native-installer-rework

Conversation

@efrazer-oai
Copy link
Copy Markdown
Contributor

@efrazer-oai efrazer-oai commented Apr 7, 2026

Summary

This PR significantly improves the standalone installer experience.

The main changes are:

  1. We now install the codex binary and other dependencies in a subdirectory under CODEX_HOME. (CODEX_HOME/packages/standalone/releases/...)

  2. We replace the codex.js launcher that npm/bun rely on with logic in the Rust binary that automatically resolves its dependencies (like ripgrep)

Motivation

A few design constraints pushed this work.

  1. Currently, the entrypoint to codex is through codex.js, which forces a node dependency to kick off our rust app. We want to move away from this so that the entrypoint to codex does not rely on node or external package managers.
  2. Right now, the native script adds codex and its dependencies directly to user PATH. Given that codex is likely to add more binary dependencies than ripgrep, we want a solution which does not add arbitrary binaries to user PATH -- the only one we want to add is the codex command itself.
  3. We want upgrades to be atomic. We do not want scenarios where interrupting an upgrade command can move codex into undefined state (for example, having a new codex binary but an old ripgrep binary). This was ~possible with the old script.
  4. Currently, the Rust binary uses heuristics to determine which installer created it. These heuristics are flaky and are tied to the codex.js launcher. We need a more stable/deterministic way to determine how the binary was installed for standalone.
  5. We do not want conflicting codex installations on PATH. For example, the user installing via npm, then installing via brew, then installing via standalone would make it unclear which version of codex is being launched and make it tough for us to determine the right upgrade command.

Design

Standalone package layout

Standalone installs now live under CODEX_HOME/packages/standalone:

$CODEX_HOME/
  packages/
    standalone/
      current -> releases/0.111.0-x86_64-unknown-linux-musl
      releases/
        0.111.0-x86_64-unknown-linux-musl/
          codex
          codex-resources/
            rg

where standalone/current is a symlink to a release directory.

On Windows, the release directory has the same shape, with .exe names and Windows helpers in codex-resources:

%CODEX_HOME%\
  packages\
    standalone\
      current -> releases\0.111.0-x86_64-pc-windows-msvc
      releases\
        0.111.0-x86_64-pc-windows-msvc\
          codex.exe
          codex-resources\
            rg.exe
            codex-command-runner.exe
            codex-windows-sandbox-setup.exe

This gives us:

  • atomic upgrades because we can fully stage a release before switching standalone/current
  • a stable way for the binary to recognize a standalone install from its canonical current_exe() path under CODEX_HOME
  • a clean place for binary dependencies like rg, Windows sandbox helpers, and, in the future, our custom zsh etc

Command location

On Unix, we add a symlink at ~/.local/bin/codex which points directly to the $CODEX_HOME/packages/standalone/current/codex binary. This becomes the main entrypoint for the CLI.

On Windows, we store the link at %LOCALAPPDATA%\Programs\OpenAI\Codex\bin.

PATH persistence

This is a tricky part of the PR, as there's no ~super reliable way to ensure that we end up on PATH without significant tradeoffs.

Most Unix variants will have ~/.local/bin on PATH already, which means we should be fine simply registering the command there in most cases. However, there are cases where this is not the case. In these cases, we directly edit the profile depending on the shell we're in.

  • macOS zsh: ~/.zprofile
  • macOS bash: ~/.bash_profile
  • Linux zsh: ~/.zshrc
  • Linux bash: ~/.bashrc
  • fallback: ~/.profile

On Windows, we update the User Path environment variable directly and we don't need to worry about shell profiles.

Standalone runtime detection

This PR adds a new shared crate, codex-install-context, which computes install ownership once per process and caches it in a OnceLock.

That context includes:

  • install manager (Standalone, Npm, Bun, Brew, Other)
  • the managed standalone release directory, when applicable
  • the managed standalone codex-resources directory, when present
  • the resolved rg_command

The standalone path is detected by canonicalizing current_exe(), canonicalizing CODEX_HOME via find_codex_home(), and checking whether the binary is running from under $CODEX_HOME/packages/standalone/releases.

We intentionally do not use a release metadata file. The binary path is the source of truth.

Dependency resolution

For standalone installs, grep_files now resolves bundled rg from codex-resources next to the Codex binary.

For npm/bun/brew/other installs, grep_files falls back to resolving rg from PATH.

For Windows standalone installs, Windows sandbox helpers are still found as direct siblings when present. If they are not direct siblings, the lookup also checks the sibling codex-resources directory.

TUI update path

The TUI now has UpdateAction::StandaloneUnix and UpdateAction::StandaloneWindows, which rerun the standalone install commands.

Unix update command:

sh -c "curl -fsSL https://chatgpt.com/codex/install.sh | sh"

Windows update command:

powershell -c "irm https://chatgpt.com/codex/install.ps1|iex"

The Windows updater runs PowerShell directly. We do this because cmd /C would parse the |iex as a cmd pipeline instead of passing it to PowerShell.

Additional installer behavior

  • standalone installs now warn about conflicting npm/bun/brew-managed codex installs and offer to uninstall them
  • same-version reruns do not redownload the release if it is already staged locally

Testing

Installer smoke tests run:

  • macOS: fresh install into isolated HOME and CODEX_HOME with scripts/install/install.sh --release latest
  • macOS: reran the installer against the same isolated install to verify the same-version/update path and PATH block idempotence
  • macOS: verified the installed codex --version and bundled codex-resources/rg --version
  • Windows: parsed scripts/install/install.ps1 with PowerShell via [scriptblock]::Create(...)
  • Windows: verified the standalone update action builds a direct PowerShell command and does not route the irm ...|iex command through cmd /C

@efrazer-oai efrazer-oai marked this pull request as draft April 7, 2026 16:50
efrazer-oai added a commit that referenced this pull request Apr 7, 2026
efrazer-oai added a commit that referenced this pull request Apr 7, 2026
@efrazer-oai efrazer-oai force-pushed the edwarf/codex/native-installer-rework branch from a7c45b2 to 88740a3 Compare April 7, 2026 17:12
efrazer-oai and others added 15 commits April 7, 2026 11:09
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Validate existing native release directories before reusing them, rewrite the shell PATH block when the install dir changes, and refuse destructive PowerShell junction replacement for normal directories.

Co-authored-by: Codex <noreply@openai.com>
Offer an immediate launch prompt after install and print clearer instructions for the current session and future terminals.

Co-authored-by: Codex <noreply@openai.com>
Use the shared CODEX_HOME helper in install-context, tighten standalone install handling, and improve installer launch UX for shell and PowerShell. Also make the shell installer quiet in non-interactive environments and verify the standalone install path with smoke tests.

Co-authored-by: Codex <noreply@openai.com>
Import the shared resources-dir constant into the windows sandbox test module and remove unused install-context dependencies so cargo shear passes again.

Co-authored-by: Codex <noreply@openai.com>
@efrazer-oai efrazer-oai force-pushed the edwarf/codex/native-installer-rework branch from af4a8f2 to 319edba Compare April 7, 2026 18:14
@efrazer-oai efrazer-oai changed the title Significantly improve native installer Significantly improve standalone installer Apr 7, 2026
@efrazer-oai efrazer-oai marked this pull request as ready for review April 7, 2026 21:10
@efrazer-oai efrazer-oai requested a review from bolinfest April 7, 2026 21:11
@bolinfest bolinfest requested a review from viyatb-oai April 13, 2026 21:57
Comment thread scripts/install/install.ps1 Outdated
Copy link
Copy Markdown
Collaborator

@viyatb-oai viyatb-oai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@efrazer-oai a couple items from my review:

  1. Unix current symlink replacement is not portable

install.sh creates a temporary symlink in update_current_link, then runs mv -f "$tmp_link" "$CURRENT_LINK".

Because CURRENT_LINK points to a directory, BSD/macOS mv can move the temporary symlink inside the existing target directory instead of replacing the current symlink. In that case, current remains pointed at the old release and .current.$$ ends up inside the old release directory.

This can break same-version reruns and real upgrades on macOS. We should use a platform-aware replacement helper here, such as GNU mv -T, BSD/macOS mv -h, or a carefully scoped fallback.

  1. Windows migration from the old standalone layout likely fails

The previous Windows installer created a real non-empty directory at %LOCALAPPDATA%\Programs\OpenAI\Codex\bin containing codex.exe, rg.exe, and helper binaries. This PR now tries to replace that same path with a junction at install.ps1:363.

Ensure-Junction refuses to replace non-empty directories, so existing standalone Windows users can hit a hard failure unless they manually delete the old install directory first. We need a migration path for the known old layout, or we should keep the visible Windows bin directory as a real directory and update the visible executable/shim inside it.

  1. Concurrent installs can corrupt or temporarily break current

There is no cross-process installer lock. Two installs of the same version can both decide the release is incomplete, both download, and then race while creating/removing the same release_dir. The risky delete is in install.sh:396.

That can temporarily leave current pointing at a release directory another installer is deleting or replacing. We should add a lock around install/update activation, covering release staging, release activation, and current/visible command updates.

  1. Reliability Issues: staging happens under /tmp

Unix staging currently uses stage_release="$tmp_dir/release" and then moves it into $RELEASES_DIR. If /tmp and CODEX_HOME are on different filesystems, the final move is not a simple atomic rename.

We should stage under $RELEASES_DIR, for example $RELEASES_DIR/.staging.<release>.<pid>, then rename into the final release directory on the same filesystem.

  1. Conflicting installs are removed before the new standalone install succeeds

handle_conflicting_install runs before the new standalone release has been downloaded, staged, and activated. If the old npm/bun/brew uninstall succeeds but the standalone download or install fails afterward, the user loses their working codex.

Safer ordering would be:

  1. Download, stage, and activate the standalone install.

  2. Verify the visible codex command works.

  3. Then offer to remove the old manager-owned install.

  4. No explicit checksum or manifest verification

The installer downloads GitHub release tarballs and extracts them directly. GitHub release transport is protected by TLS, but the installer does not verify an expected digest before unpacking/installing. For a standalone installer, especially one run through curl | sh or irm | iex, we should consider publishing a manifest with SHA-256 digests and verifying the archive before extraction.

  1. The TUI update command loses installer intent

Standalone updates rerun the installer command from update_action.rs, but that does not preserve choices such as a custom CODEX_INSTALL_DIR, custom CODEX_HOME, or future installer preferences unless those environment variables happen to still be set.

A small state file under $CODEX_HOME/packages/standalone/install.json could record user-facing installer choices for future updates. Runtime detection can still use the current executable path as source of truth; the state file would only preserve installer rerun behavior.

@efrazer-oai
Copy link
Copy Markdown
Contributor Author

Thanks! I went back through each issue and addressed in implementation.

Unix current symlink replacement is not portable

Fixed.

On Unix, the installer now replaces current in a way that works on both GNU/Linux and macOS.

Before, it created a temporary symlink and ran mv -f. On macOS, that can move the new symlink into the old release directory instead of replacing current.

Now we:

  • create a temporary symlink
  • try mv -Tf on systems that support it
  • try mv -hf on macOS/BSD
  • only fall back to remove-then-move while the installer lock is held

That keeps current pointing at the new release instead of leaving users on the old one.

Windows migration from the old standalone layout likely fails

Fixed for the older Windows Codex layout we shipped before this change.

Older Windows installs used a real bin directory at %LOCALAPPDATA%\\Programs\\OpenAI\\Codex\\bin. This PR changed that path to a junction, and the old code would fail if that directory was non-empty.

Now we:

  • detect that specific older Codex layout
  • ask the user before replacing it
  • move the old directory aside
  • create the new visible bin link
  • run codex.exe --version through that visible path
  • delete the backup only after that check passes

We still refuse to replace unknown non-empty directories.

Concurrent installs can corrupt or temporarily break current

Fixed.

The installer now takes a lock before it touches release directories or switches users to a new release.

On Unix we use flock when available, otherwise an atomic mkdir lock.
On Windows we use an exclusive lock file.

That lock covers:

  • checking whether a release is complete
  • staging a new release
  • moving it into place
  • updating current
  • updating the visible command path

So two installers cannot race each other through the same install.

Reliability issue: staging happens under /tmp

Fixed.

The downloaded archive can still live in a temp directory, but the release directory that will become live is now built under releases/.staging... and then renamed into place.

That means the final move happens on the same filesystem as the destination, which is the safe case we want.

Conflicting installs are removed before the new standalone install succeeds

Fixed.

We still detect npm, bun, and brew installs early so we can warn clearly, but we do not offer to remove them until after standalone install succeeds and the visible codex command works.

So if standalone install fails, we do not remove the user’s working npm, bun, or brew install first.

No explicit checksum or manifest verification

We are treating this one as a fast follow.

We agree that the installer should verify the downloaded archive before unpacking it. The remaining work here is mostly release-pipeline work. We need to decide the exact checksum shape we want to publish and add that cleanly to the release workflow.

We did not want to invent a partial solution in the installer first and then back into the release process afterward.

The TUI update command loses installer intent

We looked at this one closely and intentionally did not add an installer state file.

The examples here were CODEX_HOME and CODEX_INSTALL_DIR. Those are environment overrides, not stable product settings. Saving them would make later updates reuse values that may only have been set for one shell session.

So we kept the simpler behavior:

  • the TUI reruns the latest standalone installer
  • install detection still comes from the actual executable path at runtime

Windows junction swap is not atomic

We changed the Windows update path so it no longer swaps whole junction directories with temporary pending and backup names.

Instead, it keeps the junction path in place and rewrites only where that junction points. That means current and the visible bin path do not disappear during an update.

We also tightened the guard around that rewrite. The installer now only retargets junctions that already belong to this standalone install:

  • current must already point under standalone releases/
  • the visible bin path must already point under the standalone root

If the path is some other junction or some other kind of reparse point, the installer now refuses to rewrite it.

Validation:

  • Windows real filesystem smoke tests passed for:
    • fresh junction creation
    • retargeting an existing installer-owned junction
    • visible bin -> current while current moves
    • empty directory replacement
    • non-empty directory refusal
    • foreign junction refusal
  • Unix real installer smoke tests passed for:
    • fresh install
    • same-version rerun
    • upgrade
  • cargo test -p codex-install-context passed

Reference points for the Windows junction rewrite:

We also looked at a few alternatives here.

  1. Keep a stable codex.exe launcher in bin and have it resolve the current release at runtime.
  2. Keep bin as a real directory and copy codex.exe and helpers into it on every update.
  3. Delete and recreate the junction path on every update.
  4. Keep the junction path in place and retarget the existing junction.

We chose the fourth option.

That keeps the visible path stable during updates, gives us the atomicity benefit we wanted in practice, and lets us keep the same current-based release layout on Windows and Unix instead of forking the design.

Copy link
Copy Markdown
Collaborator

@viyatb-oai viyatb-oai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non blocking but one tiny race condition

On macOS the installer commonly takes this fallback path because flock is not available by default. If the process dies before release_install_lock runs, for example from a killed terminal or machine restart, the lock directory remains and every future installer invocation spins forever in this loop with no recovery path. Store owner/timestamp metadata and break stale locks, or use a lock mechanism that is released by the OS when the process exits.

@efrazer-oai
Copy link
Copy Markdown
Contributor Author

Fixed!

non blocking but one tiny race condition

On macOS the installer commonly takes this fallback path because flock is not available by default. If the process dies before release_install_lock runs, for example from a killed terminal or machine restart, the lock directory remains and every future installer invocation spins forever in this loop with no recovery path. Store owner/timestamp metadata and break stale locks, or use a lock mechanism that is released by the OS when the process exits.

On Unix, the installer already used a real OS lock when flock was available. But on macOS that often is not the default path, so we could fall back to a mkdir lock directory instead.

That fallback had one bad failure mode:

  • the installer creates install.lock.d
  • the process dies before cleanup runs
  • the lock directory stays behind
  • every future install loops forever waiting for that directory to disappear

The fix is:

  • on macOS, use lockf by default
  • on other Unix systems, keep using flock when it exists
  • only use the mkdir lock as a last fallback
  • when we do use that fallback, write owner metadata and break stale locks instead of waiting forever

So now the normal Unix paths use OS-managed locks that are released when the process exits, and the last-resort fallback can recover if a previous installer died and left a stale lock directory behind.

@efrazer-oai
Copy link
Copy Markdown
Contributor Author

/merge

@efrazer-oai efrazer-oai merged commit 9d1bf00 into main Apr 15, 2026
37 of 39 checks passed
@efrazer-oai efrazer-oai deleted the edwarf/codex/native-installer-rework branch April 15, 2026 21:44
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants