feat: add integration test for rostbuness #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Mossaka
commented
Oct 23, 2025
Mossaka
commented
- Update installation instructions and wrapper script for awf
- clean up unused github actions
- refactor: remove end-to-end tests for firewall and add unit tests for CLI, Docker manager, and host iptables
- feat: add comprehensive firewall robustness test suite and GitHub Actions workflow
Signed-off-by: Jiaxiao (mossaka) Zhou <duibao55328@gmail.com>
Signed-off-by: Jiaxiao (mossaka) Zhou <duibao55328@gmail.com>
… CLI, Docker manager, and host iptables Signed-off-by: Jiaxiao (mossaka) Zhou <duibao55328@gmail.com>
…ions workflow Signed-off-by: Jiaxiao (mossaka) Zhou <duibao55328@gmail.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+15
to
+114
| runs-on: ubuntu-latest | ||
| timeout-minutes: 60 | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '20' | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| npm install | ||
| npm run build | ||
|
|
||
| - name: Make wrapper available globally with sudo | ||
| run: | | ||
| # Create sudo wrapper script for iptables manipulation | ||
| sudo tee /usr/local/bin/awf > /dev/null <<EOF | ||
| #!/bin/bash | ||
| exec $(which node) $GITHUB_WORKSPACE/dist/cli.js "\$@" | ||
| EOF | ||
| sudo chmod +x /usr/local/bin/awf | ||
| which awf | ||
| awf --version | ||
|
|
||
| - name: Pull required Docker images | ||
| run: | | ||
| echo "Pulling images to avoid timeouts during tests..." | ||
| docker pull curlimages/curl:latest | ||
| docker pull alpine:latest | ||
| docker pull dannydirect/tinyproxy:latest | ||
|
|
||
| - name: Run robustness test suite | ||
| run: | | ||
| chmod +x scripts/ci/test-firewall-robustness.sh | ||
| sudo ./scripts/ci/test-firewall-robustness.sh | ||
|
|
||
| - name: Cleanup awf resources | ||
| if: always() | ||
| run: | | ||
| chmod +x scripts/ci/cleanup.sh | ||
| sudo ./scripts/ci/cleanup.sh || true | ||
|
|
||
| - name: Upload test logs | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: robustness-test-logs | ||
| path: | | ||
| /tmp/firewall-test-*.log | ||
| /tmp/firewall-obs-test.log | ||
| /tmp/copilot-logs-* | ||
| /tmp/squid-logs-* | ||
| if-no-files-found: warn | ||
|
|
||
| - name: Test Summary | ||
| if: always() | ||
| run: | | ||
| echo "==========================================" | ||
| echo "Firewall Robustness Test Suite" | ||
| echo "==========================================" | ||
| echo "" | ||
| echo "This test validates comprehensive security scenarios:" | ||
| echo "" | ||
| echo "Test Categories:" | ||
| echo " 1. Happy-path basics (exact domains, subdomains, case insensitivity)" | ||
| echo " 2. Deny cases (IP literals, non-standard ports, SSH blocking)" | ||
| echo " 3. Redirect behavior (cross-domain vs same-domain)" | ||
| echo " 4. Protocol edges (HTTP/2, bypass attempts, DoH, metadata endpoints)" | ||
| echo " 5. IPv4/IPv6 parity" | ||
| echo " 6. Git operations (HTTPS allowed, SSH blocked)" | ||
| echo " 7. Security corner cases (SNI/Host mismatch, multicast)" | ||
| echo " 8. Docker container egress:" | ||
| echo " - Basic egress control" | ||
| echo " - Network modes (bridge, host, none)" | ||
| echo " - DNS controls" | ||
| echo " - Proxy pivot attempts" | ||
| echo " - Container-to-container bounce" | ||
| echo " - UDP, QUIC, multicast" | ||
| echo " - Metadata protection" | ||
| echo " - Privilege abuse (NET_ADMIN, privileged)" | ||
| echo " - SNI/Host mismatch" | ||
| echo " - Custom networks" | ||
| echo " - Build-time egress" | ||
| echo " - IPv6 from containers" | ||
| echo " 9. Observability (audit log validation)" | ||
| echo "" | ||
| echo "Key security validations:" | ||
| echo " - L7 filtering for HTTP/HTTPS traffic" | ||
| echo " - Host header spoofing protection" | ||
| echo " - Container network isolation" | ||
| echo " - Metadata endpoint blocking (AWS/GCP)" | ||
| echo " - Privilege escalation prevention" | ||
| echo " - Comprehensive audit logging" | ||
| echo "" | ||
| echo "Test logs uploaded as artifacts (if generated)" | ||
| echo "==========================================" |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 months ago
The most effective fix is to add a permissions block to this workflow, preferably at the root level (just below the workflow on: block) so that it applies to all jobs unless a job-specific override is specified. For this workflow, the minimum required permission is usually contents: read, since it checks out code and uploads artifacts but does not need to write to repository contents, issues, or pull requests. This can be done in .github/workflows/test-firewall-robustness.yml by inserting typically:
permissions:
contents: readon a new line after the on: section (after line 12). This limits the GITHUB_TOKEN permissions for all jobs, adhering to least privilege.
No new library imports or definitions are needed; this is a YAML configuration change only.
Suggested changeset
1
.github/workflows/test-firewall-robustness.yml
| @@ -10,6 +10,9 @@ | ||
| # Run daily at 2 AM UTC to catch any regressions | ||
| - cron: '0 2 * * *' | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| robustness-tests: | ||
| runs-on: ubuntu-latest |
Copilot is powered by AI and may make mistakes. Always verify output.
There was a problem hiding this comment.
Pull Request Overview
This PR adds a comprehensive integration test suite for firewall robustness and refactors the test infrastructure. The changes remove end-to-end tests in favor of unit tests for core components, add a new robustness test script covering extensive security scenarios, clean up unused GitHub Actions workflows, and update documentation to reflect the new testing approach.
Key changes:
- Added unit tests for CLI, Docker manager, and host iptables components
- Introduced comprehensive firewall robustness test suite with 50+ test scenarios
- Removed obsolete end-to-end and integration test files
- Cleaned up 6 unused GitHub Actions workflows
Reviewed Changes
Copilot reviewed 19 out of 21 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/host-iptables.test.ts | New unit tests for host iptables functionality |
| src/docker-manager.test.ts | New unit tests for Docker manager and compose generation |
| src/cli.test.ts | New unit tests for CLI argument parsing and validation |
| src/tests/firewall.e2e.test.ts | Removed end-to-end firewall tests |
| src/tests/cli.test.ts | Removed old CLI integration tests |
| src/tests/cli.integration.test.ts | Removed old CLI integration tests |
| scripts/ci/test-firewall-robustness.sh | New comprehensive robustness test script with 50+ scenarios |
| scripts/ci/README.md | Updated documentation for new test script |
| docs/QUICKSTART.md | Updated documentation references |
| README.md | Updated installation instructions |
| CLAUDE.md | Updated wrapper script setup with improved clarity |
| .github/workflows/test-firewall-robustness.yml | New workflow for robustness tests |
| .github/workflows/*.yml | Removed 6 obsolete workflow files |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| if [[ $exit_code -ne 0 ]]; then | ||
| # Check for test setup errors first (these should cause the test to fail) | ||
| # Exclude matches from the "[entrypoint] Executing command:" line to avoid false positives | ||
| if grep -v "^\[entrypoint\] Executing command:" "$log_file" 2>/dev/null | grep -qiE "Failed to resolve IP|Couldn't parse CURLOPT_RESOLVE|command not found"; then |
There was a problem hiding this comment.
[nitpick] The complex grep pipeline with regex patterns should be extracted into a helper function for better readability and maintainability. Consider creating a check_test_setup_error function.
|
|
||
| # Verify it was blocked (not a different error) | ||
| # Note: HTTP 400 errors from Squid often indicate blocked IP literal access | ||
| if grep -qiE "denied|forbidden|403|ERR_ACCESS_DENIED|connection.*refused|proxy.*error|timeout|timed out|Empty reply|Failed to connect|Connection reset|Could not resolve host|error: 400|returned error: 400" "$log_file" 2>/dev/null; then |
There was a problem hiding this comment.
[nitpick] The lengthy regex pattern for detecting blocked requests is hard to maintain. Consider defining these patterns as an array at the top of the script for easier modification and documentation of each pattern's purpose.
Collaborator
Author
|
@copilot the integration test failed with can you please fix it |
Contributor
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.