Skip to content

Enable AIP-44 and AIP-52 by default for development and CI on main #30521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
potiuk merged 2 commits into from
Apr 7, 2023
Merged

Enable AIP-44 and AIP-52 by default for development and CI on main #30521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
570fadd
Enable AIP-44 and AIP-52 by default for development and CI on main
potiuk Apr 7, 2023
07e2ae0
Update scripts/ci/docker-compose/devcontainer.env
potiuk Apr 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions dev/breeze/src/airflow_breeze/utils/docker_command_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,8 @@ def update_expected_environment_variables(env: dict[str, str]) -> None:
set_value_to_default_if_not_set(env, "AIRFLOW_CONSTRAINTS_MODE", "constraints-source-providers")
set_value_to_default_if_not_set(env, "AIRFLOW_CONSTRAINTS_REFERENCE", "constraints-source-providers")
set_value_to_default_if_not_set(env, "AIRFLOW_EXTRAS", "")
set_value_to_default_if_not_set(env, "AIRFLOW_ENABLE_AIP_44", "true")
set_value_to_default_if_not_set(env, "AIRFLOW_ENABLE_AIP_52", "true")
set_value_to_default_if_not_set(env, "ANSWER", answer if answer is not None else "")
set_value_to_default_if_not_set(env, "BASE_BRANCH", "main")
set_value_to_default_if_not_set(env, "BREEZE", "true")
Expand Down
2 changes: 2 additions & 0 deletions scripts/ci/docker-compose/_docker.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
# under the License.
AIRFLOW_CI_IMAGE
AIRFLOW_EXTRAS
AIRFLOW_ENABLE_AIP_44
AIRFLOW_ENABLE_AIP_52
AIRFLOW_CONSTRAINTS_REFERENCE
ANSWER
BACKEND
Expand Down
2 changes: 2 additions & 0 deletions scripts/ci/docker-compose/base.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ services:
# With env files and there are many problems with it:
- AIRFLOW_CI_IMAGE=${AIRFLOW_CI_IMAGE}
- AIRFLOW_EXTRAS=${AIRFLOW_EXTRAS}
- AIRFLOW_ENABLE_AIP_44=${AIRFLOW_ENABLE_AIP_44}
- AIRFLOW_ENABLE_AIP_52=${AIRFLOW_ENABLE_AIP_52}
- AIRFLOW_CONSTRAINTS_REFERENCE=${AIRFLOW_CONSTRAINTS_REFERENCE}
- ANSWER=${ANSWER}
- BACKEND=${BACKEND}
Expand Down
2 changes: 2 additions & 0 deletions scripts/ci/docker-compose/devcontainer.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
HOME=
AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.7:latest"
ANSWER=
AIRFLOW_ENABLE_AIP_44="true"
AIRFLOW_ENABLE_AIP_52="true"
PYTHON_MAJOR_MINOR_VERSION="3.7"
AIRFLOW_EXTRAS=
BASE_BRANCH="main"
Expand Down
2 changes: 2 additions & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@
os.environ["AIRFLOW__CORE__UNIT_TEST_MODE"] = "True"
os.environ["AWS_DEFAULT_REGION"] = os.environ.get("AWS_DEFAULT_REGION") or "us-east-1"
os.environ["CREDENTIALS_DIR"] = os.environ.get("CREDENTIALS_DIR") or "/files/airflow-breeze-config/keys"
os.environ["AIRFLOW_ENABLE_AIP_44"] = os.environ.get("AIRFLOW_ENABLE_AIP_44") or "true"
Copy link
Member Author

@potiuk potiuk Apr 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default if not set when our tests are run. This way the user wil not have to set the variables in their tests when they run in from the IDE (cc: @ephraimbuddy @jedcunningham @eladkal @pierrejeambrun ). I found it highly annoying that I had to set it in main - but still in airflow they will be disabled by default.

In CI in branch 2.6 we will set the vars to be "false" by default, so the tests will be skipped there - even if the change here will be part of 2.6.

os.environ["AIRFLOW_ENABLE_AIP_52"] = os.environ.get("AIRFLOW_ENABLE_AIP_52") or "true"

from airflow import settings # noqa: E402
from airflow.models.tasklog import LogTemplate # noqa: E402
Expand Down