Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Dec 27, 2024

Importing all modules did not include importing trigger module, and this caused strange errors when running tests that never imported Trigger separately.

Similaarly Xcom and Variables have not been imported


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk
Copy link
Member Author

potiuk commented Dec 27, 2024

Example errors appearing in xdist tests here:

https://github.com/apache/airflow/actions/runs/12515046642/job/34913595134?pr=44126#step:7:3627

FAILED providers/tests/openlineage/utils/test_utils.py::test_get_user_provided_run_facets_with_multiple_function_definition - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/microsoft/azure/triggers/test_msgraph.py::TestMSGraphTrigger::test_run_when_response_is_bytes - AssertionError: assert equals failed
  'failure'  'success'
FAILED providers/tests/openlineage/utils/test_utils.py::test_get_user_provided_run_facets_with_duplicate_facet_keys - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/openlineage/utils/test_utils.py::test_get_user_provided_run_facets_with_invalid_function_definition - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/openlineage/utils/test_utils.py::test_get_user_provided_run_facets_with_wrong_return_type_function - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/microsoft/azure/triggers/test_msgraph.py::TestMSGraphTrigger::test_serialize - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/microsoft/azure/triggers/test_powerbi.py::TestPowerBITrigger::test_powerbi_trigger_serialization - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.
FAILED providers/tests/openlineage/utils/test_utils.py::test_get_user_provided_run_facets_with_exception - sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Triggering mapper: 'mapped class AssetModel->asset'. Original exception was: When initializing mapper mapped class AssetModel->asset, expression 'Trigger' failed to locate a name ('Trigger'). If this is a class name, consider adding this relationship() to the <class 'airflow.models.asset.AssetModel'> class after both dependent classes have been defined.

@potiuk potiuk changed the title Import the trigger module when importing all modules Import missing modules when importing all DB modules Dec 27, 2024
@potiuk potiuk changed the title Import missing modules when importing all DB modules Import missing modules when importing all DB models Dec 27, 2024
Importing all modules did not include importing trigger module, and
this caused strange errors when running tests that never imported
Trigger separately.

Similaarly Xcom and Variables have not been imported
@potiuk
Copy link
Member Author

potiuk commented Dec 27, 2024

lso it is likely that this one was impacting some other ways Airflow DB is initialized: #45236

@potiuk
Copy link
Member Author

potiuk commented Dec 27, 2024

I am not sure if we are missing other models here, but generally, depending on what sequence imports happened our DB scripts could "forget" to create certain tables, and our tests were failing with missing associations if referred DB models were not imported in earlier tests.

This is somewhat a weakness of our "imports" doing a lot of work implicitly (and sometimes forgetting to do it).

@potiuk
Copy link
Member Author

potiuk commented Dec 27, 2024

Though @uranusjr @ashb -> I am not sure if this is right - I also saw that there are those _getrr calls for all lazy imports, so I am not at all sure if this is the right approach what I am doing - this mechanism of sometimes lazy, sometimes not importing of modules is .... complex ... and I am not sure what the intentions were to have some of the modules lazy and some not... So I am not all sure if what I am doing here is fine.

@potiuk
Copy link
Member Author

potiuk commented Dec 27, 2024

Nope. It does not help;

@potiuk potiuk closed this Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants