Fix bandit warnings#344
Conversation
|
let's hold off on this PR for now, I will rebase atop |
15aa3a4 to
5265270
Compare
|
I rebased this branch atop main and ran into a test failure: This is happening because I added a timeout in tools.py to make it safe to run in production, and this test is doing an actual HTTP GET which happened to run slowly just now and trigger the timeout. The solution here is to mock the call to |
|
I re-ran the test job and it succeeded. So this is an intermittent error, we should expect it to reoccur sometimes when github tests run slowly. This will keep happening until we fix the test to use a mock. I will create a github issue to address this. @20001LastOrder please proceed on reviewing this PR. I have rebased it on main and it no longer depends on #343 . |
Run `poetry self update && poetry install` after picking up this commit. Changes: - Change all make steps to depend on poetry being installed - Add mypy for static type checking, but leave disabled for now - Use more sophisticated lint options - Include Slack app in `make test` - Include Slack tests in `make lint` - Fail tests if coverage drops below 75% mypy gives us static type checking. It triggers many warnings, so I am disabling mypy for now and will create an issue to resolve mypy warnings.
Change code to resolve these lint warnings: - F821: Undefined names - F841: Local variable name is assigned to but never used - F811: Redefinition of unused name from line name
Change code to resolve these warnings: - F811: Redefinition of unused name from line name - W291: Trailing whitespace `make lint` will now warn and fail if files contain trailing whitespace.
flake8 is currently configured to ignore these errors: W503: Line break occurred before a binary operator E501: Line too long (82 > 79 characters) E402: Module level import not at top of file This change makes the behavior consistent for both `make lint` and `poetry run flake8`. Also removing E402 as that warning was already resolved.
d9d9746 to
a7719bf
Compare
Create a Makefile just for Slackapp so we can run `make test` within /sherpa/src/apps/slackapp using Slackapp's own Poetry environment. Remove knowledge of Slackapp from Sherpa's makefile. Change make to install Spacy model dependency.
Run `poetry self update && poetry install` to pick this up.
Fixes the following Bandit errors seen with `make lint`: 1. Issue: [B104:hardcoded_bind_all_interfaces] Possible binding to all interfaces. Severity: Medium Confidence: Medium CWE: CWE-605 (https://cwe.mitre.org/data/definitions/605.html) More Info: https://bandit.readthedocs.io/en/1.7.8/plugins/b104_hardcoded_bind_all_interfaces.html Location: apps/slackapp/slackapp/bolt_app.py:346:13 Change: suppress the warning with #nosec B104, since our use of host '0.0.0.0' is intentional. Also add a security note. 2. Issue: [B113:request_without_timeout] Requests call without timeout Location: apps/slackapp/slackapp/bolt_app.py:340:23 Location: sherpa_ai/scrape/extract_github_readme.py:75:19 Location: sherpa_ai/utils.py:83:15 Location: sherpa_ai/scrape/extract_github_readme.py:52:19 Location: sherpa_ai/scrape/file_scraper.py:69:19 Change: add timeouts for each of these. Adjustment may be needed. 3. Issue: [B310:blacklist] Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected. Severity: Medium Confidence: High CWE: CWE-22 (https://cwe.mitre.org/data/definitions/22.html) More Info: https://bandit.readthedocs.io/en/1.7.8/blacklists/blacklist_calls.html#b310-urllib-urlopen Location: sherpa_ai/tools.py:59:15 Location: sherpa_ai/utils.py:294:12 Change: add logic to ensure URLs are HTTP or HTTPS. Add tests.
a7719bf to
e0a2c6e
Compare
Your checklist for this pull request
Thank you for submitting a pull request! To speed up the review process, please follow this checklist:
make format)/docs)pytest tests(offline mode)Additional steps for code with networking dependencies:
pytest tests --external_api(online mode, making network calls)Description
Add Bandit to provide security checks in
make lintand address Bandit security warnings.This change improves the security of our code.
Depends on #343 .