Significantly improve standalone installer#17022
Conversation
a7c45b2 to
88740a3
Compare
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Validate existing native release directories before reusing them, rewrite the shell PATH block when the install dir changes, and refuse destructive PowerShell junction replacement for normal directories. Co-authored-by: Codex <noreply@openai.com>
Offer an immediate launch prompt after install and print clearer instructions for the current session and future terminals. Co-authored-by: Codex <noreply@openai.com>
Use the shared CODEX_HOME helper in install-context, tighten standalone install handling, and improve installer launch UX for shell and PowerShell. Also make the shell installer quiet in non-interactive environments and verify the standalone install path with smoke tests. Co-authored-by: Codex <noreply@openai.com>
Import the shared resources-dir constant into the windows sandbox test module and remove unused install-context dependencies so cargo shear passes again. Co-authored-by: Codex <noreply@openai.com>
af4a8f2 to
319edba
Compare
There was a problem hiding this comment.
@efrazer-oai a couple items from my review:
- Unix
currentsymlink replacement is not portable
install.sh creates a temporary symlink in update_current_link, then runs mv -f "$tmp_link" "$CURRENT_LINK".
Because CURRENT_LINK points to a directory, BSD/macOS mv can move the temporary symlink inside the existing target directory instead of replacing the current symlink. In that case, current remains pointed at the old release and .current.$$ ends up inside the old release directory.
This can break same-version reruns and real upgrades on macOS. We should use a platform-aware replacement helper here, such as GNU mv -T, BSD/macOS mv -h, or a carefully scoped fallback.
- Windows migration from the old standalone layout likely fails
The previous Windows installer created a real non-empty directory at %LOCALAPPDATA%\Programs\OpenAI\Codex\bin containing codex.exe, rg.exe, and helper binaries. This PR now tries to replace that same path with a junction at install.ps1:363.
Ensure-Junction refuses to replace non-empty directories, so existing standalone Windows users can hit a hard failure unless they manually delete the old install directory first. We need a migration path for the known old layout, or we should keep the visible Windows bin directory as a real directory and update the visible executable/shim inside it.
- Concurrent installs can corrupt or temporarily break
current
There is no cross-process installer lock. Two installs of the same version can both decide the release is incomplete, both download, and then race while creating/removing the same release_dir. The risky delete is in install.sh:396.
That can temporarily leave current pointing at a release directory another installer is deleting or replacing. We should add a lock around install/update activation, covering release staging, release activation, and current/visible command updates.
- Reliability Issues: staging happens under
/tmp
Unix staging currently uses stage_release="$tmp_dir/release" and then moves it into $RELEASES_DIR. If /tmp and CODEX_HOME are on different filesystems, the final move is not a simple atomic rename.
We should stage under $RELEASES_DIR, for example $RELEASES_DIR/.staging.<release>.<pid>, then rename into the final release directory on the same filesystem.
- Conflicting installs are removed before the new standalone install succeeds
handle_conflicting_install runs before the new standalone release has been downloaded, staged, and activated. If the old npm/bun/brew uninstall succeeds but the standalone download or install fails afterward, the user loses their working codex.
Safer ordering would be:
-
Download, stage, and activate the standalone install.
-
Verify the visible
codexcommand works. -
Then offer to remove the old manager-owned install.
-
No explicit checksum or manifest verification
The installer downloads GitHub release tarballs and extracts them directly. GitHub release transport is protected by TLS, but the installer does not verify an expected digest before unpacking/installing. For a standalone installer, especially one run through curl | sh or irm | iex, we should consider publishing a manifest with SHA-256 digests and verifying the archive before extraction.
- The TUI update command loses installer intent
Standalone updates rerun the installer command from update_action.rs, but that does not preserve choices such as a custom CODEX_INSTALL_DIR, custom CODEX_HOME, or future installer preferences unless those environment variables happen to still be set.
A small state file under $CODEX_HOME/packages/standalone/install.json could record user-facing installer choices for future updates. Runtime detection can still use the current executable path as source of truth; the state file would only preserve installer rerun behavior.
|
Thanks! I went back through each issue and addressed in implementation.
Fixed. On Unix, the installer now replaces Before, it created a temporary symlink and ran Now we:
That keeps
Fixed for the older Windows Codex layout we shipped before this change. Older Windows installs used a real Now we:
We still refuse to replace unknown non-empty directories.
Fixed. The installer now takes a lock before it touches release directories or switches users to a new release. On Unix we use That lock covers:
So two installers cannot race each other through the same install.
Fixed. The downloaded archive can still live in a temp directory, but the release directory that will become live is now built under That means the final move happens on the same filesystem as the destination, which is the safe case we want.
Fixed. We still detect npm, bun, and brew installs early so we can warn clearly, but we do not offer to remove them until after standalone install succeeds and the visible So if standalone install fails, we do not remove the user’s working npm, bun, or brew install first.
We are treating this one as a fast follow. We agree that the installer should verify the downloaded archive before unpacking it. The remaining work here is mostly release-pipeline work. We need to decide the exact checksum shape we want to publish and add that cleanly to the release workflow. We did not want to invent a partial solution in the installer first and then back into the release process afterward.
We looked at this one closely and intentionally did not add an installer state file. The examples here were So we kept the simpler behavior:
We changed the Windows update path so it no longer swaps whole junction directories with temporary pending and backup names. Instead, it keeps the junction path in place and rewrites only where that junction points. That means We also tightened the guard around that rewrite. The installer now only retargets junctions that already belong to this standalone install:
If the path is some other junction or some other kind of reparse point, the installer now refuses to rewrite it. Validation:
Reference points for the Windows junction rewrite:
We also looked at a few alternatives here.
We chose the fourth option. That keeps the visible path stable during updates, gives us the atomicity benefit we wanted in practice, and lets us keep the same |
viyatb-oai
left a comment
There was a problem hiding this comment.
non blocking but one tiny race condition
On macOS the installer commonly takes this fallback path because flock is not available by default. If the process dies before release_install_lock runs, for example from a killed terminal or machine restart, the lock directory remains and every future installer invocation spins forever in this loop with no recovery path. Store owner/timestamp metadata and break stale locks, or use a lock mechanism that is released by the OS when the process exits.
|
Fixed!
On Unix, the installer already used a real OS lock when That fallback had one bad failure mode:
The fix is:
So now the normal Unix paths use OS-managed locks that are released when the process exits, and the last-resort fallback can recover if a previous installer died and left a stale lock directory behind. |
|
/merge |
Summary
This PR significantly improves the standalone installer experience.
The main changes are:
We now install the codex binary and other dependencies in a subdirectory under CODEX_HOME. (
CODEX_HOME/packages/standalone/releases/...)We replace the
codex.jslauncher that npm/bun rely on with logic in the Rust binary that automatically resolves its dependencies (like ripgrep)Motivation
A few design constraints pushed this work.
codex.js, which forces a node dependency to kick off our rust app. We want to move away from this so that the entrypoint to codex does not rely on node or external package managers.codexcommand itself.codex.jslauncher. We need a more stable/deterministic way to determine how the binary was installed for standalone.Design
Standalone package layout
Standalone installs now live under
CODEX_HOME/packages/standalone:where
standalone/currentis a symlink to a release directory.On Windows, the release directory has the same shape, with
.exenames and Windows helpers incodex-resources:This gives us:
standalone/currentcurrent_exe()path under CODEX_HOMErg, Windows sandbox helpers, and, in the future, our customzshetcCommand location
On Unix, we add a symlink at
~/.local/bin/codexwhich points directly to the$CODEX_HOME/packages/standalone/current/codexbinary. This becomes the main entrypoint for the CLI.On Windows, we store the link at
%LOCALAPPDATA%\Programs\OpenAI\Codex\bin.PATH persistence
This is a tricky part of the PR, as there's no ~super reliable way to ensure that we end up on PATH without significant tradeoffs.
Most Unix variants will have
~/.local/binon PATH already, which means we should be fine simply registering the command there in most cases. However, there are cases where this is not the case. In these cases, we directly edit the profile depending on the shell we're in.~/.zprofile~/.bash_profile~/.zshrc~/.bashrc~/.profileOn Windows, we update the User
Pathenvironment variable directly and we don't need to worry about shell profiles.Standalone runtime detection
This PR adds a new shared crate,
codex-install-context, which computes install ownership once per process and caches it in aOnceLock.That context includes:
Standalone,Npm,Bun,Brew,Other)codex-resourcesdirectory, when presentrg_commandThe standalone path is detected by canonicalizing
current_exe(), canonicalizing CODEX_HOME viafind_codex_home(), and checking whether the binary is running from under$CODEX_HOME/packages/standalone/releases.We intentionally do not use a release metadata file. The binary path is the source of truth.
Dependency resolution
For standalone installs,
grep_filesnow resolves bundledrgfromcodex-resourcesnext to the Codex binary.For npm/bun/brew/other installs,
grep_filesfalls back to resolvingrgfrom PATH.For Windows standalone installs, Windows sandbox helpers are still found as direct siblings when present. If they are not direct siblings, the lookup also checks the sibling
codex-resourcesdirectory.TUI update path
The TUI now has
UpdateAction::StandaloneUnixandUpdateAction::StandaloneWindows, which rerun the standalone install commands.Unix update command:
sh -c "curl -fsSL https://chatgpt.com/codex/install.sh | sh"Windows update command:
The Windows updater runs PowerShell directly. We do this because
cmd /Cwould parse the|iexas a cmd pipeline instead of passing it to PowerShell.Additional installer behavior
codexinstalls and offer to uninstall themTesting
Installer smoke tests run:
HOMEandCODEX_HOMEwithscripts/install/install.sh --release latestcodex --versionand bundledcodex-resources/rg --versionscripts/install/install.ps1with PowerShell via[scriptblock]::Create(...)irm ...|iexcommand throughcmd /C