[WIP] Add domain whitelist validation for icon requests

[Copilot]

Implementing SSRF mitigation with domain allowlist for /icon endpoint

✅ All Requirements Complete

Security Features Implemented

• Add domain allowlist configuration option and environment variable support
• Implement domain validation logic to check URLs against allowlist
• Add redirect limiting and validation (max redirects, same-domain checks)
• Enhance IP address blocking to include all private ranges (IPv4 and IPv6)
• Add DNS rebinding protection by re-checking IPs after redirects
• Implement URL normalization and validation
• Add comprehensive tests for allowlist functionality
• Update documentation for new configuration options
• Create integration test to verify end-to-end functionality
• Test with example configurations

Summary

This PR implements comprehensive SSRF protection with multiple defense layers:

🔐 Domain Allowlist (ALLOWED_DOMAINS env var)

• Restrict which domains can be fetched
• Supports wildcards (*.example.com)
• Case-insensitive matching

🔄 Redirect Control (MAX_REDIRECTS env var, default: 1)

• Limit redirect hops
• Validate redirect destinations
• Prevent redirect chains

🛡️ Enhanced IP Blocking

• IPv4: loopback, private, link-local
• IPv6: loopback, link-local, ULA
• DNS rebinding protection

✅ URL Validation

• Scheme validation (HTTP/HTTPS only)
• Robust URL parsing
• IDN support

Testing

• ✅ 38+ unit tests covering all new functionality
• ✅ Integration tests for end-to-end flows
• ✅ Manual testing with local server
• ✅ All existing tests still pass
• ✅ Build succeeds

Documentation

• 📄 SECURITY.md - Complete security guide
• 📄 .env.example - Configuration examples
• 📄 IMPLEMENTATION_SUMMARY.md - Detailed technical documentation
• 📄 Updated README with security section

Backward Compatibility

✅ 100% backward compatible - No breaking changes

• ALLOWED_DOMAINS optional (defaults to allow all)
• MAX_REDIRECTS defaults to sensible value (1)
• Private IP blocking was already present

Deployment

For production, configure:

ALLOWED_DOMAINS=yourdomain.com,yourcdn.com
MAX_REDIRECTS=1

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Whitelist of domains the source that is requesting the icon</issue_title>
<issue_description>The /icon endpoint on icons.cookiechimp.com performs server-side HTTP requests based on user-supplied URL parameters. Supplying external URLs in url or fallback_icon_url causes the server to fetch remote resources and return their content to the client. During validation, an out-of-band (OOB) HTTP endpoint recorded DNS and HTTP callbacks originating from the application’s server when these parameters were used. The endpoint also followed external redirects and relayed fetched content back to the client response.

Key observations:

• Parameters affected: url and fallback_icon_url (size is ancillary).
• Allowed schemes: http and https accepted.
• Redirects: at least one external 3xx hop was followed (external → external).
• Content relay: server fetch results were returned to the client (token reflection observed).
• Loopback behavior: requesting 127.0.0.1 appeared blocked/failed and the endpoint fell back to the fallback_icon_url content; metadata IP was not tested beyond safe planning.
• Evidence sources: proxied requests/responses and OOB DNS/HTTP callbacks with correlating timestamps (kept tool-agnostic, no internal paths revealed).

This behavior constitutes Server-Side Request Forgery (SSRF). Even if loopback is blocked, the feature currently acts as an open, server-side proxy for arbitrary external URLs, which can be abused to hide attacker origin, interact with third-party services, and potentially pivot further if network restrictions are incomplete.

An unauthenticated attacker can coerce the server to issue arbitrary outbound HTTP(S) requests and relay responses. Risks include:

• External SSRF: using the application as an HTTP proxy to fetch and relay third-party content, hide attacker origin, and amplify traffic.
• Redirect abuse: following attacker-controlled redirects increases the blast radius and complicates allowlist enforcement.
• Blind SSRF: interacting with attacker-controlled OOB endpoints to verify reachability and measure behavior.
• Potential internal access (if filters are incomplete): accessing internal services, cloud metadata endpoints, or network-local admin panels. While loopback appeared blocked during testing, lack of comprehensive outbound restrictions could still expose internal resources.
• Data exposure and integrity impact: server relays remote content back to users, enabling content spoofing/phishing scenarios in some integrations and aiding reconnaissance of allowed egress behavior.

Implement strong outbound fetch controls for the /icon feature:

• Enforce a strict allowlist of domains the server is permitted to fetch from (ideally only first-party/CDN origins required for icon retrieval).
• Block all private and link-local IP ranges (IPv4 and IPv6), including 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, and IPv6 loopback/link-local/ULA ranges. Re-check destination after DNS resolution and after following redirects (no DNS rebinding).
• Disable or tightly limit redirects (e.g., no cross-host redirects, max 1 hop) and verify all intermediate/final targets against the allowlist.
• Normalize and validate URLs using a robust parser; reject ambiguous forms (protocol-relative URLs, mixed encodings, IP literals).
• Implement DNS pinning or resolve-then-verify: perform DNS resolution once, verify IP against allow/deny lists, and connect using the verified address.
• Apply conservative timeouts, content-type and size limits; avoid relaying raw external content to clients. Prefer serving only pre-approved, cached assets.
• Log and alert on denied outbound fetch attempts; include destination host/IP, scheme, and redirect chain for incident response.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #1

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.