Fix bootstrapping, add ETag caching, and 502/404 fallback in pull-config.sh

[Copilot]

✅ Fix Bootstrapping Problems in pull-config.sh - COMPLETE

This PR successfully addresses all three bootstrapping issues with the nginx reverse proxy configuration script.

✅ All Requirements Met

1. Check for existence of reverse-proxy.conf before backup operations

• Script now checks if config file exists before attempting backup
• Checks if backup exists before attempting restore
• Handles first-run scenario gracefully without errors

2. Store ETag and use If-None-Match header for caching

• Stores ETag in ${CONF_FILE}.etag (derived from CONF_FILE)
• Sends If-None-Match header with existing ETag on subsequent requests
• Exits early on 304 Not Modified (no unnecessary downloads or reloads)
• Significantly reduces bandwidth usage and nginx reload frequency

3. Fallback to internal cluster URL on 502 and 404 errors

• Primary: https://create-a-container.opensource.mieweb.org/nginx.conf
• Fallback: http://create-a-container.cluster.mieweb.org:3000/nginx.conf
• Automatically falls back on 502 Bad Gateway and 404 Not Found errors
• Solves the chicken-and-egg bootstrapping problem
• Handles unknown subdomains that may return 404 instead of 502

📊 Latest Changes (Current Commit)

Added 404 handling to fallback logic:

1. Handle 404 responses (line 56)
   • Also fallback to internal URL on 404 responses
   • Handles case where unknown subdomains return 404 instead of 502
   • Maintains existing 502 and connection failure (000) handling

📊 Previous Changes

Robustness improvements:

1. Check conf file before using etag (line 35)
   • Only use etag if both CONF_FILE and ETAG_FILE exist
   • Handles edge case where etag file is orphaned (exists without conf)

Code review improvements:

1. Derive ETAG_FILE from CONF_FILE (line 6)

   • Changed from hardcoded path to ETAG_FILE="${CONF_FILE}.etag"

2. Global variables for temp files (lines 7-9)

   • Added TEMP_FILE="${CONF_FILE}.tmp"
   • Added HEADERS_FILE="${CONF_FILE}.headers"
   • Added BACKUP_FILE="${CONF_FILE}.bak"
   • Removed local variable repetition throughout file

3. Build curl command as array (lines 26-42)

   • Replaced conditional if/else with bash array
   • Conditionally append ETag header to array
   • Cleaner, more maintainable code

4. Cleanup function with trap EXIT (lines 14-16)

   • Added cleanup() function to remove temp files
   • Set trap cleanup EXIT to ensure cleanup always runs
   • Removed manual cleanup before each exit
   • Handles both normal exits and failures via set -e

✅ Quality Assurance

• All tests pass: ✅
  • First run without existing config
  • ETag cache hit (304 Not Modified)
  • Config validation failure and rollback
  • 502 error with fallback to internal URL
  • 404 error with fallback to internal URL
  • Orphaned etag file scenario
• Shellcheck: ✅ No warnings or errors

🔒 Security Summary

No security vulnerabilities introduced:

• Config validation still performed before applying

• Rollback mechanism prevents bad configs

• All temporary files properly cleaned up via trap

• No new external dependencies added

• Errors are properly reported to admins

• Fixes Bootstrapping problems in pull-config.sh #94

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Bootstrapping problems in pull-config.sh</issue_title>
<issue_description>https://github.com/mieweb/opensource-server/blob/main/nginx-reverse-proxy/pull-config.sh has the problem where, before there is any reverse proxy configuration, create-a-container.opensource.mieweb.org is inaccessible since it relies on the reverse proxy to handle requests for *.opensource.mieweb.org. The script also expects /etc/nginx/conf.d/reverse-proxy.conf to exist so it can make a backup and fails due to set -e otherwise. The script also pulls the config every minute, even if there's been no changes. In effect, we need the following changes:

1. Check for the existence of /etc/nginx/conf.d/reverse-proxy.conf and skip the backup and restore operations if it doesn't exist
2. Store the ETag of the reverse-proxy.conf file in a seperate file, say /etc/nginx/conf.d/reverse-proxy.etag, send that value to the server when requesting the new config. On a cache hit do nothing, we already have the latest file. On a cache miss, proceed with the backup -> download -> check -> reload/restore process
3. If create-a-container.opensource.mieweb.org generates a gateway error (502), fallback to the http://create-a-container.cluster.mieweb.org:3000/nginx.conf (ie. the internal IP and port for the create-a-container service)</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Bootstrapping problems in pull-config.sh #94

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[runleveldev]

Tested on create-a-container-dev LB

[cmyers-mieweb]

Approved

[runleveldev]

Looks good to me.