[BUG] Using npm run-script to call npm install or npm ci in ephemeral environment fails with EACCES

[Hossy]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

In our build environment, we use docker to perform specific build steps without needing to manage an endless amount of tools on our build agents. The build agents and the steps they execute do not run as root. However, when executing docker run, the user within the ephemeral container is typically root (normal for Docker). We use npm run-script to execute a series of npm commands as part of the build.

The build agent executes docker run --rm -v $PWD:/app -v ~/.npmrc:/root/.npmrc -w /app node:16 sh -c 'npm run-script build'

$PWD contains the checked-out contents from source. We also pass a pre-configured .npmrc shared across all our builds. The contents of $PWD and .npmrc are owned by a non-root uid/gid.

build.sh:

#!/bin/sh

npm install \
  && npm run lint \
  && npm run autotest \
  && rm -rf node_modules \
  && npm install --production

When npm run-script is first executed within the node:16 container, it is running as root (normal for Docker). Once the entrypoint script determines it isn't a node command, npm run-script checks for the existence of the cache folder, which it obviously doesn't find and so creates it (owned by the current user, root) at /root/.npm.

npm then executes build.sh whose first task is to execute npm install (we also tried npm ci). The first thing npm install does is change the user context to the owner of the working directory (/app), which is the non-root user the build agent is using.
npm then checks to see if it can access /root/.npmrc and /root/.npm, which it cannot because the user context has changed.
This user does not exist in node:16's passwd file obviously, so the uid/gid numbers are shown in the error.

npm ERR! code EACCES
npm ERR! syscall lstat
npm ERR! path /root/.npm/_cacache/content-v2/sha1/f2/bb/2a7e83cbc87bb95c8e572828a06c9add6e0d
npm ERR! errno -13
npm ERR!
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
npm ERR!
npm ERR! To permanently fix this problem, please run:
npm ERR!   sudo chown -R 900:900 "/root/.npm"

This behavior is not observed in versions earlier than node:16.
node:16 comes with npm 8.3.1, but this behavior is still observed when upgrading npm to 8.5.1.

Expected Behavior

Either:

1. npm run-script shouldn't be messing with .npmrc or the cache directory since it doesn't need it
   or
2. npm run-script should be following the same user context switching code path that npm install and npm ci are following.

Steps To Reproduce

1. On a machine with Docker installed and running...
2. Create a folder containing the following two files. Ensure the folder and files are owned by a non-root user.
   package.json:

{
  "name": "test-app",
  "scripts": {
    "build": "./build.sh"
  },
  "dependencies": {
    "config": "^3.1.0"
  }
}

build.sh (+x):

#!/bin/sh

id
ls -la
ls -la $HOME
npm install \
&& echo SUCCESS!

3. Open a shell as a non-root user and change directories to the folder you just created. Not sure if it would interfere, but try not to use uid/gid 1000 which would map to the node user within node:16.
4. Run docker run --rm -v $PWD:/app -w /app node:16 sh -c 'npm -v; node -v; npm config ls; echo -----------; npm run-script build'

Output:

bash-5.1$ docker run --rm -v $PWD:/app -w /app node:16 sh -c 'npm -v; node -v; npm config ls; echo -----------; npm run-script build'
8.3.1
v16.14.0
; node bin location = /usr/local/bin/node
; cwd = /app
; HOME = /root
; Run `npm config ls -l` to show all defaults.
-----------

> build
> ./build.sh

uid=900 gid=900 groups=900
total 8
drwxr-xr-x 2  900  900  42 Feb 21 20:01 .
drwxr-xr-x 1 root root  29 Feb 21 20:28 ..
-rwxr-xr-x 1  900  900  66 Feb 21 20:26 build.sh
-rw-r--r-- 1  900  900 119 Feb 21 18:48 package.json
ls: cannot open directory '/root': Permission denied
glob error [Error: EACCES: permission denied, scandir '/root/.npm/_logs'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'scandir',
  path: '/root/.npm/_logs'
}
npm WARN logfile Error: EACCES: permission denied, scandir '/root/.npm/_logs'
npm WARN logfile  error cleaning log files [Error: EACCES: permission denied, scandir '/root/.npm/_logs'] {
npm WARN logfile   errno: -13,
npm WARN logfile   code: 'EACCES',
npm WARN logfile   syscall: 'scandir',
npm WARN logfile   path: '/root/.npm/_logs'
npm WARN logfile }
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /root/.npm/_cacache/tmp
npm ERR! errno EACCES
npm ERR! 
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
npm ERR! 
npm ERR! To permanently fix this problem, please run:
npm ERR!   sudo chown -R 900:900 "/root/.npm"

npm ERR! A complete log of this run can be found in:
npm notice 
npm notice New minor version of npm available! 8.3.1 -> 8.5.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.5.1>
npm notice Run `npm install -g npm@8.5.1` to update!
npm notice 

Output with npm upgrade to 8.5.1:

bash-5.1$ docker run --rm -v $PWD:/app -w /app node:16 sh -c 'npm install -g npm; npm -v; node -v; npm config ls; echo -----------; npm run-script build'

removed 1 package, changed 44 packages, and audited 218 packages in 13s

11 packages are looking for funding
  run `npm fund` for details

3 moderate severity vulnerabilities

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
npm notice 
npm notice New minor version of npm available! 8.3.1 -> 8.5.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.5.1>
npm notice Run `npm install -g npm@8.5.1` to update!
npm notice 
8.5.1
v16.14.0
; node bin location = /usr/local/bin/node
; cwd = /app
; HOME = /root
; Run `npm config ls -l` to show all defaults.
-----------

> build
> ./build.sh

uid=900 gid=900 groups=900
total 8
drwxr-xr-x 2  900  900  42 Feb 21 20:01 .
drwxr-xr-x 1 root root  40 Feb 21 20:29 ..
-rwxr-xr-x 1  900  900  66 Feb 21 20:26 build.sh
-rw-r--r-- 1  900  900 119 Feb 21 18:48 package.json
ls: cannot open directory '/root': Permission denied
glob error [Error: EACCES: permission denied, scandir '/root/.npm/_logs'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'scandir',
  path: '/root/.npm/_logs'
}
npm WARN logfile Error: EACCES: permission denied, scandir '/root/.npm/_logs'
npm WARN logfile  error cleaning log files [Error: EACCES: permission denied, scandir '/root/.npm/_logs'] {
npm WARN logfile   errno: -13,
npm WARN logfile   code: 'EACCES',
npm WARN logfile   syscall: 'scandir',
npm WARN logfile   path: '/root/.npm/_logs'
npm WARN logfile }
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /root/.npm/_cacache/tmp
npm ERR! errno EACCES
npm ERR! 
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
npm ERR! 
npm ERR! To permanently fix this problem, please run:
npm ERR!   sudo chown -R 900:900 "/root/.npm"

npm ERR! A complete log of this run can be found in:

Environment

• npm: 8.3.1 (part of node:16), but we tried 8.5.1 as well
• Node.js: 16.14.0
• OS Name: Alpine
• System Model Name:
• npm config:

; node bin location = /usr/local/bin/node
; cwd = /app
; HOME = /root
; Run `npm config ls -l` to show all defaults.

[fritzy]

Arborist behavior under root is pretty aggressive in trying to prevent file access issues. Issues related to this behavior most often comes up in container use cases due to the practice of using root.

[Hossy]

Arborist behavior under root is pretty aggressive in trying to prevent file access issues. Issues related to this behavior most often comes up in container use cases due to the practice of using root.

Is there something I should be doing differently? Currently, my workaround is to create /tmp/.npm, chown it to uid/gid 900, map .npmrc to /tmp/.npmrc and then pass NPM_CONFIG_userconfig=/tmp/.npmrc NPM_CONFIG_cache=/tmp/.npm when executing npm run-script. /tmp was the only reliable folder that root and all other users could access. This way, I preempt npm's logic of creating the missing cache folder so it doesn't mess up the ownership.

[huidaoweilai2024]

In a termux environment,minyami can be installed using nodejs-lts/npm8.11.0 or nodejs/npm7.24.2.
Using nodejs -lts/npm8.17.0 or nodejs/npm8.11.0 or nodejs/npm8.17.0 returns error 'your cache folder contains root-owned files'.

[robbytx]

Is there something I should be doing differently?

I just spent most of yesterday trying to answer this same question, as I have a remarkably similar scenario where my primary build spawns a sibling node container to compile some JavaScript assets residing in a subfolder shared via a bind mount. The problem arose upon upgrading to NPM v8 (from v6), and the user/ownership mismatch led NPM to fail with various errors:

• Error: EPERM: operation not permitted, lchown '/app/node_modules'
• npm WARN logfile Error: EACCES: permission denied, scandir '/root/.npm/_logs'
• Error: EACCES: permission denied, open '/root/.config/configstore/ember-cli.json'
• npm ERR! code EACCES syscall mkdir path /home/node/.npm/_cacache errno -13

After I observed:

• This issue describing a very similar scenario and failures,
• [BUG] cannot run npm scripts as root as long as the directory's owner is a regular user #4095 (comment) describing NPM's behavior to changing users as intentional and documented,
• [RRFC] Clean up file ownership story rfcs#546 discussing various complaints surrounding NPM's propensity (to attempt) to change file ownership permissions,

I finally came to the realization that I could workaround these issues by simply not running NPM as root in the container. I realized that the Node images ship with a regular user called node, so I attempted various forms of /bin/su node -c 'npm ...' that failed because it still wanted to access directories like /root/.npm and /root/.config.

Finally I realized that - as long as I wasn't attempting to install anything globally in the container - I could simply run the container using docker run --user node ..., while also changing any mounts from /root/.npmrc to /home/node/.npmrc and from /root/.npm/_logs to /home/node/logs, which I then symlinked into ~/.npm (rather than mounting to /home/node/.npm/_logs directly because that left ~/.npm owned by root and npm was unable to create the _cacache directory).

My final working command line looks like:

docker run --rm --user node --volume ${PWD}:/app --volume ${PWD}/target:/home/node/logs -w /app node:16 \
  /bin/bash -c 'mkdir ~/.npm && ln -s ~/logs ~/.npm/_logs && npm version && npm ci && npm run test'

I believe the equivalent command line from this issue's description would be:

docker run --rm --user node -v $PWD:/app -v ~/.npmrc:/home/node/.npmrc -w /app node:16 sh -c 'npm run-script build'

Addendum:
Note that the shared folder /app is writable by node because of the read-write bind mount, but any existing files and subdirectories (like /app/node_modules) are not (unless there is an exact UID/GID match). Therefore it is necessary to make those world-writable before running the container using something like chmod -R +w ${PWD}/node_modules to avoid permission errors. I already had this workaround in place in my NPM v6 build, so that's why I failed to mention it above.