Avoid non-recommended usage of logging

[Taragolis]

Enable two ruff checks for the logging and usages:

• flake8-logging-format (G), skip two rules for now since it required to make more changes, which could make review longer
  • "Logging statement uses +" G003
  • "Logging statement uses f-string" G004
• flake8-logging (LOG)

Example of rule violations before manual + auto fixes

Run 'ruff' for extremely fast Python linting.............................Failed
- hook id: ruff
- exit code: 1
- files were modified by this hook

airflow/jobs/backfill_job_runner.py:684:42: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/jobs/backfill_job_runner.py:989:22: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/models/variable.py:246:29: G001 Logging statement uses `str.format`
airflow/providers/amazon/aws/operators/sagemaker.py:352:64: G202 Logging statement has redundant `exc_info`
airflow/providers/amazon/aws/operators/sagemaker.py:358:66: G202 Logging statement has redundant `exc_info`
airflow/providers/amazon/aws/operators/sagemaker.py:780:22: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/providers/amazon/aws/operators/sagemaker.py:788:22: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/providers/amazon/aws/operators/sagemaker.py:816:66: G202 Logging statement has redundant `exc_info`
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:434:30: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/providers/google/cloud/operators/dataproc.py:838:26: G201 Logging `.exception(...)` should be used instead of `.error(..., exc_info=True)`
airflow/utils/db_cleanup.py:52:28: LOG002 Use `__name__` with `logging.getLogger()`
Found 13 errors (2 fixed, 11 remaining).
No fixes available (1 hidden fix can be enabled with the `--unsafe-fixes` option).

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[Taragolis]

I've also thought about ban implicit usage of root logger because I thought it made by a mistake in most case if not at all.

E.g. by add this rules into the banned api

# Logging root logger
"logging.debug".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.info".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.warning".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.error".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.exception".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.fatal".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.critical".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.log".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"

❯ pre-commit run ruff --all-files
Run 'ruff' for extremely fast Python linting.............................Failed
- hook id: ruff
- exit code: 1

airflow/example_dags/example_python_decorator.py:60:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/example_dags/example_python_operator.py:63:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/metrics/otel_logger.py:308:9: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/metrics/otel_logger.py:403:5: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:84:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:85:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:86:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/sensors/base.py:260:21: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/settings.py:469:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:41:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:55:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:67:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:72:13: TID251 `logging.exception` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:85:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:90:13: TID251 `logging.exception` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:134:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:136:9: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_processor_handler.py:127:21: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_processor_handler.py:133:17: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_task_handler.py:541:17: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/task_handler_with_custom_formatter.py:61:9: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/extensions/init_jinja_globals.py:54:9: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1361:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1480:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1487:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4009:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4510:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4897:17: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4904:21: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4910:21: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:5844:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:905:17: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:914:17: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:946:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:949:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:950:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:952:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dags/test_task_view_type_check.py:50:1: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
Found 38 errors.
tests/providers/apache/hive/transfers/test_s3_to_hive.py:210:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/amazon/aws/example_sagemaker.py:165:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/amazon/aws/example_sagemaker.py:457:9: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/github/example_github.py:84:39: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/github/example_github.py:95:39: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/utils/__init__.py:36:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/utils/__init__.py:51:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:381:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:385:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:387:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:390:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:394:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
Found 12 errors.

Any thoughts? I do not want to include this change into this PR, just want to implement and fix it as follow up

[vincbeck]

I've also thought about ban implicit usage of root logger because I thought it made by a mistake in most case if not at all.

E.g. by add this rules into the banned api

# Logging root logger
"logging.debug".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.info".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.warning".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.error".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.exception".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.fatal".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.critical".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"
"logging.log".msg = "Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger"

❯ pre-commit run ruff --all-files
Run 'ruff' for extremely fast Python linting.............................Failed
- hook id: ruff
- exit code: 1

airflow/example_dags/example_python_decorator.py:60:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/example_dags/example_python_operator.py:63:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/metrics/otel_logger.py:308:9: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/metrics/otel_logger.py:403:5: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:84:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:85:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/providers/amazon/aws/auth_manager/views/auth.py:86:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/sensors/base.py:260:21: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/settings.py:469:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:41:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:55:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:67:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:72:13: TID251 `logging.exception` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:85:5: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:90:13: TID251 `logging.exception` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:134:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/cli_action_loggers.py:136:9: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_processor_handler.py:127:21: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_processor_handler.py:133:17: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/file_task_handler.py:541:17: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/utils/log/task_handler_with_custom_formatter.py:61:9: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/extensions/init_jinja_globals.py:54:9: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1361:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1480:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:1487:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4009:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4510:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4897:17: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4904:21: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:4910:21: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
airflow/www/views.py:5844:13: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:905:17: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:914:17: TID251 `logging.debug` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:946:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:949:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:950:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dag_processing/test_job_runner.py:952:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/dags/test_task_view_type_check.py:50:1: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
Found 38 errors.
tests/providers/apache/hive/transfers/test_s3_to_hive.py:210:13: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/amazon/aws/example_sagemaker.py:165:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/amazon/aws/example_sagemaker.py:457:9: TID251 `logging.error` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/github/example_github.py:84:39: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/providers/github/example_github.py:95:39: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/utils/__init__.py:36:13: TID251 `logging.warning` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/system/utils/__init__.py:51:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:381:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:385:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:387:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:390:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
tests/task/task_runner/test_standard_task_runner.py:394:9: TID251 `logging.info` is banned: Instantiate new `logger = logging.getLogger(__name__)` and use it instead of root logger
Found 12 errors.

Any thoughts? I do not want to include this change into this PR, just want to implement and fix it as follow up

I'd be inclined to add this rule too. I really think it is used by mistake