Fix module loading in logging config #54555

viiccwen · 2025-08-15T17:32:49Z

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

logging_class_path.rsplit(".", 1)[0] produces "log_config"
import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

in airflow-core/tests/unit/core/test_logging_config.py
Added: Parametrized test for different module path structures
Added: Test for graceful fallback when remote logging vars are missing
Added: Helper method to reduce test code duplication

Related Issues

resolves #54414

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

- Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures

boring-cyborg · 2025-08-15T17:32:52Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

- Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures

viiccwen · 2025-08-16T03:37:08Z

Hello @jason810496 ,
Could you help me to review?
Thx!

jason810496

Nice! Thanks for the PR! LGTM overall.

airflow-core/src/airflow/logging_config.py

airflow-core/tests/unit/core/test_logging_config.py

…ule-loading

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

- Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures

…ule-loading

Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`

Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

jason810496

Thanks for the fix!

airflow-core/src/airflow/logging_config.py

airflow-core/tests/unit/core/test_logging_config.py

- Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`

airflow-core/tests/unit/core/test_logging_config.py

1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call`

Lee-W · 2025-08-19T10:07:47Z

I think we're good to merge once CI passes 🙂

boring-cyborg · 2025-08-19T22:11:34Z

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

* fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * refactor(logging): move `import_module` to the top level * test(logging): optimize test constants to reduce code duplication Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE` * test(logging): use shared helper method for logging config validation Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests. * refactor(logging_config): simplify, improve test config vars - Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS` * refactor(logging): imporve & simplify logging testing file 1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call` (cherry picked from commit 462ce9a) Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>

github-actions · 2025-08-19T22:12:33Z

Backport successfully created: v3-0-test

Status	Branch	Result
✅	v3-0-test

* fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * refactor(logging): move `import_module` to the top level * test(logging): optimize test constants to reduce code duplication Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE` * test(logging): use shared helper method for logging config validation Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests. * refactor(logging_config): simplify, improve test config vars - Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS` * refactor(logging): imporve & simplify logging testing file 1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call` (cherry picked from commit 462ce9a) Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>

* fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * fix(logging): fix module loading by using `import_module()` - Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports * test(logging): add tests for logging config module path handling - Add parametrized test for simple and nested module paths - Add test for graceful fallback when remote logging vars are missing - Create helper method to reduce test code duplication - Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios Ensures logging config works with various module path structures * refactor(logging): move `import_module` to the top level * test(logging): optimize test constants to reduce code duplication Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE` * test(logging): use shared helper method for logging config validation Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests. * refactor(logging_config): simplify, improve test config vars - Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS` * refactor(logging): imporve & simplify logging testing file 1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call`

viiccwen added 2 commits August 16, 2025 01:12

fix(logging): fix module loading by using import_module()

fcfba5f

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

boring-cyborg bot added the area:logging label Aug 15, 2025

viiccwen added 2 commits August 16, 2025 01:34

fix(logging): fix module loading by using import_module()

79b7553

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

viiccwen force-pushed the fix-module-loading branch from fa2fcce to f90682c Compare August 15, 2025 17:34

jason810496 self-requested a review August 16, 2025 03:06

jason810496 reviewed Aug 16, 2025

View reviewed changes

viiccwen added 3 commits August 16, 2025 16:51

Merge remote-tracking branch 'origin/fix-module-loading' into fix-mod…

5fd50dd

…ule-loading

fix(logging): fix module loading by using import_module()

4faf032

- Replace `import_string()` with `import_module()` in `load_logging_config()` - Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed - Add `importlib` import inside function to avoid circular imports

viiccwen force-pushed the fix-module-loading branch from f90682c to 347f8b1 Compare August 16, 2025 08:52

viiccwen added 4 commits August 16, 2025 16:52

Merge remote-tracking branch 'origin/fix-module-loading' into fix-mod…

42ca62e

…ule-loading

refactor(logging): move import_module to the top level

f7c2972

test(logging): optimize test constants to reduce code duplication

18c04a2

Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`

test(logging): use shared helper method for logging config validation

2caedce

Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

viiccwen requested a review from jason810496 August 16, 2025 09:20

jason810496 reviewed Aug 17, 2025

View reviewed changes

airflow-core/src/airflow/logging_config.py Outdated Show resolved Hide resolved

airflow-core/tests/unit/core/test_logging_config.py Outdated Show resolved Hide resolved

refactor(logging_config): simplify, improve test config vars

23d891f

- Remove redundant `SETTINGS_FILE_NESTED_MODULE` - Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`

viiccwen requested a review from jason810496 August 18, 2025 09:01

jason810496 approved these changes Aug 18, 2025

View reviewed changes

jason810496 requested a review from Lee-W August 19, 2025 09:36

Lee-W reviewed Aug 19, 2025

View reviewed changes

refactor(logging): imporve & simplify logging testing file

1be9758

1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var 2. add return typo with `_verify_basic_logging_config` 3. add unittest.mock `call`

Lee-W approved these changes Aug 19, 2025

View reviewed changes

potiuk added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Aug 19, 2025

potiuk merged commit 462ce9a into apache:main Aug 19, 2025
58 checks passed

potiuk mentioned this pull request Aug 22, 2025

Mishandling of logging_config_class when loading REMOTE_TASK_LOG #54784

Closed

2 tasks

Fix module loading in logging config #54555

Fix module loading in logging config #54555

Uh oh!

Conversation

viiccwen commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

Changes

Code

Test

Related Issues

Uh oh!

boring-cyborg bot commented Aug 15, 2025

Uh oh!

viiccwen commented Aug 16, 2025

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Lee-W commented Aug 19, 2025

Uh oh!

Uh oh!

boring-cyborg bot commented Aug 19, 2025

Uh oh!

github-actions bot commented Aug 19, 2025

Backport successfully created: v3-0-test

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

viiccwen commented Aug 15, 2025 •

edited

Loading