copy-node-modules.sh fails on Linux during LMS provisioning due to permission error #1138
Description
Developers with a previous devstack installation cannot re-provision due to file permission issues.
This has only been observed on Linux; it may or may not occur on Mac, since we've observed inconsistent behavior in whether files end up as root-owned there.
Acceptance criteria
- Short-term: Communicate this issue to devs so they know what to do if they run into it (add to troubleshooting guide, with link back to ticket so we know when to remove the section)
- More permanently: Fix the script so that the copying happens as root (maybe a change to paver to explicitly call the copy script rather than having npm do it)
- Fix npm installation (permissions, caching) openedx/openedx-platform#32946 -- the fix proposed here is to just delete the files as root before any copying is done
Symptoms
When the provisioning script runs npm ci, npm ends up running the postinstall script scripts/copy-node-modules.sh (as of PR openedx/openedx-platform#32767). On Linux, this fails with the following:
Copying studio-frontend JS & CSS from node_modules into vendor directories...
+ read -r -d '' src_file
++ find node_modules/@edx/studio-frontend/dist -type f -print0
+ [[ node_modules/@edx/studio-frontend/dist/accessibilityPolicy.min.css = *.css ]]
+ cp --force node_modules/@edx/studio-frontend/dist/accessibilityPolicy.min.css common/static/common/css/vendor
cp: cannot remove 'common/static/common/css/vendor/accessibilityPolicy.min.css': Permission denied
Cause
The files in common/static/common/css/vendor are owned by the user root, whereas the script is running as the user app (as determined by adding a call to whoami.) This still happens if I just run npm install from inside make lms-shell -- even though I'm running that command as root, by default.
npm performs this change-of-user for all of its scripts, with no apparent workaround:
When npm is run as root, scripts are always run with the effective uid and gid of the working directory owner.
As far as I can tell, there's no workaround (see npm/rfcs#546) so maybe we can just switch to calling the script explicitly after the npm ci call completes.
Workaround
In edx-platform:
sudo rm -rf common/static/common/css/
sudo rm -rf common/static/common/js/
(More thorough version that will also clear out root-owned pycache files that keep cropping up: sudo find -user 0 -group 0 -prune -exec rm -rf '{}' \;)
Once this is done, npm install inside lms-shell will succeed, and the re-created files in those directories will have the UID and GID of the desktop user.