`v8-canary` builds are not promoted

[targos]

12:03:04 scp -p node-v20.0.0-v8-canary20230118b1865daaaf-linux-x64.tar.xz node-www:nodejs/custom/v20.0.0-v8-canary20230118b1865daaaf/node-v20.0.0-v8-canary20230118b1865daaaf-linux-x64.tar.xz

The last build to be in the correct folder is https://nodejs.org/download/v8-canary/v20.0.0-v8-canary202212266b2b946a63/

[targos]

I don't know how it's supposed to work. The build from 20221226 was also copied to this custom folder, so the problem is somewhere else.

https://ci-release.nodejs.org/job/iojs+release/9034/nodes=rhel8-x64-release/console

[richardlau]

@targos It's supposed to be done by the promotion scripts which run on the web server:

build/ansible/www-standalone/tools/promote/promote_nightly.sh

Lines 40 to 44 in 5c30993

	srcdir=$v8_canary_srcdir
	dstdir=$v8_canary_dstdir
	dirmatch=$v8_canary_dirmatch
	. ${__dirname}/_promote.sh $site

build/ansible/www-standalone/tools/promote/settings

Lines 16 to 18 in 5c30993

	v8_canary_srcdir=${staging_rootdir}custom
	v8_canary_dstdir=${dist_rootdir}v8-canary
	v8_canary_dirmatch="-v8-canary[0-9]+"

[targos]

@richardlau thanks! I just tried to run the promotion script manually and it gave me this error:

Promoting v20.0.0-v8-canary20230106a9a748cca4/node-v20.0.0-v8-canary20230106a9a748cca4-x86.msi...
Promoting v20.0.0-v8-canary20230106a9a748cca4/win-x64...
cp: cannot create regular file '/home/dist/nodejs/v8-canary/v20.0.0-v8-canary20230106a9a748cca4/win-x64/node.lib': File exists
cp: cannot create regular file '/home/dist/nodejs/v8-canary/v20.0.0-v8-canary20230106a9a748cca4/win-x64/node_pdb.7z': File exists

So I deleted /home/dist/nodejs/v8-canary/v20.0.0-v8-canary20230106a9a748cca4/

Running the script again fails with /home/dist/nodejs/v8-canary/v20.0.0-v8-canary20230112987ab62287 (it didn't try to promote the release from 20230106 this time...)

Should I continue to remove the problematic folders and rerun?

[richardlau]

@targos Do you know if the script is already running in the background (it's on a cron timer)? File exists sounds like something already tried to promote the build.

[richardlau]

On the other hand I don't suppose it's the end of the world if we do (accidentally) remove v8 canary builds 🙂.

FWIW we have another script which changes the owner of the files under /home/dist that are older than a week #2020. This may account for some permission issues as the promote script is run as dist.

[targos]

File exists sounds like something already tried to promote the build.

Yeah, but the weird thing is that these folders aren't visible on the website.

[targos]

# ps aux | grep promote | grep -v grep
dist     20194  0.0  0.0   4500   556 ?        Ss    2021   0:00 /bin/sh -c    /home/staging/tools/promote/promote_nightly.sh nodejs
dist     20196  0.0  0.0  13104  2092 ?        S     2021   0:00 bash /home/staging/tools/promote/promote_nightly.sh nodejs
dist     23099  0.0  0.0  12812  1664 ?        S     2021   0:00 bash /home/staging/tools/promote/_resha.sh nodejs /home/dist/nodejs/v8-canary v18.0.0-v8-canary202112276385a63910

I killed these but being from 2021 they're certainly unrelated

[richardlau]

@targos Direct or via the CloudFlare CDN (i.e. nodejs.org)? There's a five minute delay for CF as we invalidate the CF CDN cache on a five minute timer (if we queue a CDN purge).

[targos]

@richardlau via the CDN. Direct shows them 👍🏻

I ran the promotion script again (without deleting anything else) and it finished without errors 🤷🏻

[richardlau]

I ran the promotion script again (without deleting anything else) and it finished without errors 🤷🏻

🤞 it stays that way. My guess would be the error(s) were related to when we ran of disk space.

One thing we might look into is somehow logging the promote script output from the scheduled runs. I think (from memory) that /var/log/syslog logged that the script was run by cron but not the output.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.