Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 41 additions & 1 deletion docs/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Using `--env-all` passes all host environment variables to the container, which

**Excluded variables** (even with `--env-all`): `PATH`, `PWD`, `OLDPWD`, `SHLVL`, `_`, `SUDO_*`

**Proxy variables:** `HTTP_PROXY`, `HTTPS_PROXY`, `https_proxy` (and their lowercase/uppercase variants) from the host are ignored when using `--env-all` because the firewall always sets these to point to Squid. Host proxy settings cannot be passed through as they would conflict with the firewall's traffic routing.
**Proxy variables:** `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, `https_proxy`, `NO_PROXY`, `no_proxy`, `ALL_PROXY`, and `FTP_PROXY` (all case variants) from the host are **excluded from container passthrough** when using `--env-all`. The firewall sets its own proxy variables pointing to Squid inside the container. However, host proxy variables **are read** for upstream proxy auto-detection — if the host has `https_proxy`/`http_proxy` set, AWF configures Squid to chain outbound traffic through that corporate proxy (see [Upstream Proxy Support](#upstream-corporate-proxy-support)).

## `--env-file` Support

Expand Down Expand Up @@ -246,6 +246,46 @@ The DinD TCP address (e.g., `tcp://localhost:2375`) typically refers to the runn
- **`--enable-host-access`** — allows the agent to reach `host.docker.internal` and set `DOCKER_HOST=tcp://host.docker.internal:2375` inside the agent.
- **`--enable-dind`** — mounts the local Docker socket (`/var/run/docker.sock`) directly into the agent container (only works when using the local daemon, not a remote DinD TCP socket).

## Upstream (Corporate) Proxy Support

When running on self-hosted runners behind a corporate proxy, AWF can chain Squid
through the upstream proxy using the `cache_peer` directive.

### Auto-detection

If the host has `https_proxy`/`HTTPS_PROXY` or `http_proxy`/`HTTP_PROXY` set, AWF
automatically configures Squid to route outbound traffic through that proxy.
`no_proxy`/`NO_PROXY` domain suffixes are honored as bypass rules (`always_direct`).
Comment on lines +249 to +258

```bash
# Auto-detected — no flags needed when host proxy env vars are set
export https_proxy=http://proxy.corp.com:3128
export no_proxy=.internal.corp.com,localhost
awf --allow-domains github.com 'curl https://api.github.com'
```

### Explicit override

Use `--upstream-proxy <url>` to specify the proxy explicitly (overrides auto-detection):

```bash
awf --upstream-proxy http://proxy.corp.com:3128 --allow-domains github.com 'curl https://api.github.com'
```

### Limitations (v1)

- **HTTP proxies only** — Squid `cache_peer` requires an HTTP proxy (HTTPS tunneling uses CONNECT)
- **No proxy credentials** — `user:pass@proxy` URLs are rejected; configure auth on the proxy server
- **No loopback** — `localhost`/`127.0.0.1` proxies are rejected (Squid is in a container)
- **Single proxy** — If `http_proxy` and `https_proxy` differ, use `--upstream-proxy` to disambiguate
- **Domain-only bypass** — `no_proxy` IPs, CIDRs, and wildcards are ignored (only domain suffixes work)

### Proxy environment variable exclusion

Host proxy environment variables (`HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, `https_proxy`,
`ALL_PROXY`, `NO_PROXY`, etc.) are **always excluded** from container passthrough, even with
`--env-all`. AWF sets its own proxy variables pointing to Squid (`172.30.0.10:3128`).

## Troubleshooting

**Variable not accessible:** Use `sudo -E` or pass explicitly with `--env VAR="$VAR"`
Expand Down
34 changes: 34 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import { redactSecrets } from './redact-secrets';
import { validateDomainOrPattern, SQUID_DANGEROUS_CHARS } from './domain-patterns';
import { loadAndMergeDomains } from './rules';
import { detectHostDnsServers } from './dns-resolver';
import { detectUpstreamProxy, parseProxyUrl, parseNoProxy } from './upstream-proxy';
import { OutputFormat } from './types';
import { version } from '../package.json';

Expand Down Expand Up @@ -1236,6 +1237,7 @@ const optionGroupHeaders: Record<string, string> = {
'build-local': 'Image Management:',
'env': 'Container Configuration:',
'dns-servers': 'Network & Security:',
'upstream-proxy': 'Network & Security:',
'enable-api-proxy': 'API Proxy:',
'log-level': 'Logging & Debug:',
};
Expand Down Expand Up @@ -1430,6 +1432,12 @@ program
'--dns-over-https [resolver-url]',
'Enable DNS-over-HTTPS via sidecar proxy (default: https://dns.google/dns-query)'
)
.option(
'--upstream-proxy <url>',
'Upstream (corporate) proxy URL for Squid to chain through.\n' +
' Auto-detected from host https_proxy/http_proxy if not set.\n' +
' Example: http://proxy.corp.com:3128'
)
.option(
'--enable-host-access',
'Enable access to host services via host.docker.internal',
Expand Down Expand Up @@ -1785,6 +1793,31 @@ program
logger.info(`DNS-over-HTTPS enabled: ${dnsOverHttps}`);
}

// Detect or parse upstream proxy configuration
let upstreamProxy: import('./types').UpstreamProxyConfig | undefined;
if (options.upstreamProxy) {
// Explicit --upstream-proxy flag
try {
const { host, port } = parseProxyUrl(options.upstreamProxy);
// Parse no_proxy from environment even when --upstream-proxy is explicit
const noProxyStr = (process.env.no_proxy || process.env.NO_PROXY || '').trim();
const noProxy = noProxyStr ? parseNoProxy(noProxyStr) : [];
upstreamProxy = { host, port, ...(noProxy.length > 0 ? { noProxy } : {}) };
logger.info(`Upstream proxy (explicit): ${host}:${port}`);
} catch (error) {
logger.error(`Invalid --upstream-proxy: ${error instanceof Error ? error.message : error}`);
process.exit(1);
}
} else {
// Auto-detect from host environment variables
try {
upstreamProxy = detectUpstreamProxy();
} catch (error) {
logger.error(`Upstream proxy auto-detection failed: ${error instanceof Error ? error.message : error}`);
process.exit(1);
}
}

// Parse --allow-urls for SSL Bump mode
let allowedUrls: string[] | undefined;
if (options.allowUrls) {
Expand Down Expand Up @@ -1919,6 +1952,7 @@ program
githubToken: process.env.GITHUB_TOKEN || process.env.GH_TOKEN,
diagnosticLogs: options.diagnosticLogs || false,
awfDockerHost: options.dockerHost,
upstreamProxy,
};

// Apply --docker-host override for AWF's own container operations.
Expand Down
29 changes: 29 additions & 0 deletions src/docker-manager.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1616,6 +1616,35 @@ describe('docker-manager', () => {
}
});

it('should exclude host proxy env vars from env-all passthrough to prevent routing conflicts', () => {
const saved: Record<string, string | undefined> = {};
const proxyVars = ['HTTP_PROXY', 'HTTPS_PROXY', 'http_proxy', 'https_proxy', 'NO_PROXY', 'no_proxy'];

for (const v of proxyVars) {
saved[v] = process.env[v];
process.env[v] = `http://host-proxy.corp.com:3128`;
}

try {
const configWithEnvAll = { ...mockConfig, envAll: true };
const result = generateDockerCompose(configWithEnvAll, mockNetworkConfig);
const env = result.services.agent.environment as Record<string, string>;

// Host proxy vars must not leak — AWF sets its own proxy vars pointing to Squid
for (const v of proxyVars) {
// The value should either be absent or overwritten to Squid's address
if (env[v] !== undefined) {
expect(env[v]).not.toBe('http://host-proxy.corp.com:3128');
}
}
} finally {
for (const v of proxyVars) {
if (saved[v] !== undefined) process.env[v] = saved[v];
else delete process.env[v];
}
}
});

it('should auto-inject GH_HOST from GITHUB_SERVER_URL when envAll is true', () => {
const prevServerUrl = process.env.GITHUB_SERVER_URL;
const prevGhHost = process.env.GH_HOST;
Expand Down
6 changes: 6 additions & 0 deletions src/docker-manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { logger } from './logger';
import { generateSquidConfig, generatePolicyManifest } from './squid-config';
import { generateSessionCa, initSslDb, CaFiles, parseUrlPatterns, cleanupSslKeyMaterial, unmountSslTmpfs } from './ssl-bump';
import { DEFAULT_DNS_SERVERS } from './dns-resolver';
import { PROXY_ENV_VARS } from './upstream-proxy';

const SQUID_PORT = 3128;

Expand Down Expand Up @@ -640,6 +641,10 @@ export function generateDockerCompose(
// Actions runner itself, not by the agent.
'ACTIONS_RUNTIME_TOKEN',
'ACTIONS_RESULTS_URL',
// Proxy environment variables — excluded to prevent host proxy settings from
// conflicting with AWF's internal routing (agent → Squid → internet).
// AWF sets its own HTTP_PROXY/HTTPS_PROXY pointing to Squid.
...PROXY_ENV_VARS,
]);

// When api-proxy is enabled, exclude API keys from agent environment
Expand Down Expand Up @@ -2132,6 +2137,7 @@ export async function writeConfigs(config: WrapperConfig): Promise<void> {
allowHostPorts: config.allowHostPorts,
enableDlp: config.enableDlp,
dnsServers: config.dnsServers,
upstreamProxy: config.upstreamProxy,
});
const squidConfigPath = path.join(config.workDir, 'squid.conf');
fs.writeFileSync(squidConfigPath, squidConfig, { mode: 0o644 });
Expand Down
51 changes: 51 additions & 0 deletions src/squid-config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1868,4 +1868,55 @@ describe('generatePolicyManifest', () => {
const denyRule = manifest.rules.find(r => r.id === 'deny-default');
expect(httpRule!.order).toBeLessThan(denyRule!.order);
});

describe('Upstream Proxy Configuration', () => {
it('generates cache_peer directive for upstream proxy', () => {
const config: SquidConfig = {
domains: ['github.com'],
port: defaultPort,
upstreamProxy: { host: 'proxy.corp.com', port: 3128 },
};
const result = generateSquidConfig(config);
expect(result).toContain('cache_peer proxy.corp.com parent 3128 0 no-query default');
expect(result).toContain('never_direct allow all');
});

it('generates always_direct bypass for noProxy domains', () => {
const config: SquidConfig = {
domains: ['github.com'],
port: defaultPort,
upstreamProxy: {
host: 'proxy.corp.com',
port: 3128,
noProxy: ['.corp.com', 'internal.example.com'],
},
};
const result = generateSquidConfig(config);
expect(result).toContain('acl upstream_bypass dstdomain .corp.com');
expect(result).toContain('acl upstream_bypass dstdomain internal.example.com');
expect(result).toContain('acl upstream_bypass dstdomain .internal.example.com');
expect(result).toContain('always_direct allow upstream_bypass');
expect(result).toContain('never_direct allow all');
});

it('omits upstream proxy section when not configured', () => {
const config: SquidConfig = {
domains: ['github.com'],
port: defaultPort,
};
const result = generateSquidConfig(config);
expect(result).not.toContain('cache_peer');
expect(result).not.toContain('never_direct');
});

it('generates upstream proxy with custom port', () => {
const config: SquidConfig = {
domains: ['github.com'],
port: defaultPort,
upstreamProxy: { host: '10.0.0.50', port: 8080 },
};
const result = generateSquidConfig(config);
expect(result).toContain('cache_peer 10.0.0.50 parent 8080 0 no-query default');
});
});
});
45 changes: 42 additions & 3 deletions src/squid-config.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { SquidConfig, PolicyManifest, PolicyRule } from './types';
import { SquidConfig, PolicyManifest, PolicyRule, UpstreamProxyConfig } from './types';
import {
parseDomainList,
isDomainMatchedByPattern,
Expand All @@ -9,6 +9,45 @@ import {
import { generateDlpSquidConfig } from './dlp';
import { DEFAULT_DNS_SERVERS } from './dns-resolver';

/**
* Generates Squid cache_peer / always_direct / never_direct directives for
* upstream (corporate) proxy chaining.
*
* When an upstream proxy is configured, ALL outbound traffic goes through
* the parent proxy except domains in the no_proxy bypass list.
*/
function generateUpstreamProxySection(upstream: UpstreamProxyConfig): string {
const lines: string[] = [
'# Upstream corporate proxy — route outbound traffic through parent proxy',
'# Required for self-hosted runners where direct egress is blocked',
`cache_peer ${upstream.host} parent ${upstream.port} 0 no-query default`,
];

// Generate always_direct ACL for no_proxy bypass domains
if (upstream.noProxy && upstream.noProxy.length > 0) {
lines.push('');
lines.push('# Bypass upstream proxy for these domains (from host no_proxy)');
for (const domain of upstream.noProxy) {
// All entries are treated as suffix matches (domain + subdomains),
// matching standard no_proxy semantics:
// .corp.com → *.corp.com
// internal.corp.com → internal.corp.com AND *.internal.corp.com
const squidDomain = domain.startsWith('.') ? domain : `.${domain}`;
lines.push(`acl upstream_bypass dstdomain ${squidDomain}`);
// For non-dot entries, also add the exact domain for Squid dstdomain matching
if (!domain.startsWith('.')) {
lines.push(`acl upstream_bypass dstdomain ${domain}`);
}
}
lines.push('always_direct allow upstream_bypass');
}

// Force all non-bypass traffic through the parent proxy
lines.push('never_direct allow all');

return lines.join('\n');
}

/**
* Ports that should never be allowed, even with --allow-host-ports
* These ports are blocked for security reasons to prevent access to sensitive services
Expand Down Expand Up @@ -265,7 +304,7 @@ ${urlAclSection}${urlAccessRules}`;
* // Blocked: internal.example.com -> acl blocked_domains dstdomain .internal.example.com
*/
export function generateSquidConfig(config: SquidConfig): string {
const { domains, blockedDomains, port, sslBump, caFiles, sslDbPath, urlPatterns, enableHostAccess, allowHostPorts, enableDlp, dnsServers } = config;
const { domains, blockedDomains, port, sslBump, caFiles, sslDbPath, urlPatterns, enableHostAccess, allowHostPorts, enableDlp, dnsServers, upstreamProxy } = config;

// Parse, deduplicate, and group domains by protocol (shared logic)
const { domainsByProto, patternsByProto } = parseDomainConfig(domains);
Expand Down Expand Up @@ -609,7 +648,7 @@ cache deny all

# DNS settings - Squid resolves all domains for HTTP/HTTPS traffic
dns_nameservers ${(dnsServers && dnsServers.length > 0) ? dnsServers.join(' ') : DEFAULT_DNS_SERVERS.join(' ')}

${upstreamProxy ? '\n' + generateUpstreamProxySection(upstreamProxy) : ''}
# Forwarded headers
forwarded_for delete
via off
Expand Down
40 changes: 40 additions & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -920,6 +920,38 @@ export interface WrapperConfig {
* @example 45
*/
agentTimeout?: number;

/**
* Upstream (corporate) proxy for Squid to route outbound traffic through.
*
* When set, Squid uses `cache_peer` to forward all outbound HTTP/HTTPS
* traffic through this parent proxy instead of connecting directly to the
* internet. This is required on self-hosted runners behind corporate proxies
* where direct egress is blocked.
*
* Auto-detected from host `https_proxy`/`HTTPS_PROXY`/`http_proxy`/`HTTP_PROXY`
* environment variables, or explicitly set via `--upstream-proxy <url>`.
*
* @example { host: 'proxy.corp.com', port: 3128 }
*/
upstreamProxy?: UpstreamProxyConfig;
}

/**
* Upstream proxy configuration for Squid cache_peer routing
*/
export interface UpstreamProxyConfig {
/** Hostname or IP of the upstream proxy (e.g., 'proxy.corp.com') */
host: string;
/** Port of the upstream proxy (e.g., 3128) */
port: number;
/**
* Domains that should bypass the upstream proxy and connect directly.
* Parsed from host `no_proxy`/`NO_PROXY`. Only domain suffixes are
* supported (e.g., '.corp.com', 'internal.example.com').
* IPs, CIDRs, and wildcards are ignored with a warning.
*/
noProxy?: string[];
}

/**
Expand Down Expand Up @@ -1067,6 +1099,14 @@ export interface SquidConfig {
* @default ['8.8.8.8', '8.8.4.4']
*/
dnsServers?: string[];

/**
* Upstream (corporate) proxy for Squid to chain outbound traffic through.
*
* When set, generates `cache_peer` / `never_direct` / `always_direct`
* directives so Squid forwards traffic through the parent proxy.
*/
upstreamProxy?: UpstreamProxyConfig;
}

/**
Expand Down
Loading
Loading