-
Notifications
You must be signed in to change notification settings - Fork 17.4k
fix: adds the ability to disallow SQL functions per engine #28639
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,7 +22,7 @@ | |
| import time | ||
| from typing import Any, TYPE_CHECKING | ||
|
|
||
| from flask import current_app | ||
| from flask import current_app, Flask | ||
| from sqlalchemy.engine.reflection import Inspector | ||
| from sqlalchemy.engine.url import URL | ||
| from sqlalchemy.exc import NoSuchTableError | ||
|
|
@@ -218,19 +218,22 @@ def execute_with_cursor( | |
| execute_result: dict[str, Any] = {} | ||
| execute_event = threading.Event() | ||
|
|
||
| def _execute(results: dict[str, Any], event: threading.Event) -> None: | ||
| def _execute( | ||
| results: dict[str, Any], event: threading.Event, app: Flask | ||
| ) -> None: | ||
| logger.debug("Query %d: Running query: %s", query_id, sql) | ||
|
|
||
| try: | ||
| cls.execute(cursor, sql, query.database) | ||
| with app.app_context(): | ||
| cls.execute(cursor, sql, query.database) | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. threads don't have an app context: https://stackoverflow.com/questions/72541670/why-flask-app-context-is-lost-in-child-thread-when-application-factory-pattern-i
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, interesting. |
||
| except Exception as ex: # pylint: disable=broad-except | ||
| results["error"] = ex | ||
| finally: | ||
| event.set() | ||
|
|
||
| execute_thread = threading.Thread( | ||
| target=_execute, | ||
| args=(execute_result, execute_event), | ||
| args=(execute_result, execute_event, current_app._get_current_object()), # pylint: disable=protected-access | ||
| ) | ||
| execute_thread.start() | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,6 +39,7 @@ | |
| from sqlparse import keywords | ||
| from sqlparse.lexer import Lexer | ||
| from sqlparse.sql import ( | ||
| Function, | ||
| Identifier, | ||
| IdentifierList, | ||
| Parenthesis, | ||
|
|
@@ -223,6 +224,19 @@ def get_cte_remainder_query(sql: str) -> tuple[str | None, str]: | |
| return cte, remainder | ||
|
|
||
|
|
||
| def check_sql_functions_exist( | ||
| sql: str, function_list: set[str], engine: str | None = None | ||
| ) -> bool: | ||
| """ | ||
| Check if the SQL statement contains any of the specified functions. | ||
|
|
||
| :param sql: The SQL statement | ||
| :param function_list: The list of functions to search for | ||
| :param engine: The engine to use for parsing the SQL statement | ||
| """ | ||
| return ParsedQuery(sql, engine=engine).check_functions_exist(function_list) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We (probably me) will have to convert this to use
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Happy to do it myself, can I just not use sqlparse? implement something using the same pattern as |
||
|
|
||
|
|
||
| def strip_comments_from_sql(statement: str, engine: str | None = None) -> str: | ||
| """ | ||
| Strips comments from a SQL statement, does a simple test first | ||
|
|
@@ -743,6 +757,34 @@ def tables(self) -> set[Table]: | |
| self._tables = self._extract_tables_from_sql() | ||
| return self._tables | ||
|
|
||
| def _check_functions_exist_in_token( | ||
| self, token: Token, functions: set[str] | ||
| ) -> bool: | ||
| if ( | ||
| isinstance(token, Function) | ||
| and token.get_name() is not None | ||
| and token.get_name().lower() in functions | ||
| ): | ||
| return True | ||
| if hasattr(token, "tokens"): | ||
| for inner_token in token.tokens: | ||
| if self._check_functions_exist_in_token(inner_token, functions): | ||
| return True | ||
| return False | ||
|
|
||
| def check_functions_exist(self, functions: set[str]) -> bool: | ||
| """ | ||
| Check if the SQL statement contains any of the specified functions. | ||
|
|
||
| :param functions: A set of functions to search for | ||
| :return: True if the statement contains any of the specified functions | ||
| """ | ||
| for statement in self._parsed: | ||
| for token in statement.tokens: | ||
| if self._check_functions_exist_in_token(token, functions): | ||
| return True | ||
| return False | ||
|
|
||
| def _extract_tables_from_sql(self) -> set[Table]: | ||
| """ | ||
| Extract all table references in a query. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm unsure where the best place for this deny list, i.e., here in the configuration or within the extra JSON payload of the database.
Additionally should this be engine (dialect) specific or database specific? If it's the later then maybe the extra JSON payload field is preferable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JSON payload at the database level is more dynamic and would avoid having to change the config to add remove disallowed functions. But on the other hand the user that actually registers the db could have intentions to "abuse" these functions.