Skip to content

Conversation

@sunank200
Copy link
Collaborator

@sunank200 sunank200 commented Nov 11, 2024

Motivation

This PR renames execution_date to logical_date across the codebase. The shift towards logical_date helps move away from the limitations of execution_date, particularly with dynamic DAG runs and cases where multiple runs occur at the same time.

Key Changes

  • Replaced all instances of execution_date with logical_date including models.
  • Updated references in database models, templates, and functions.
  • Removed unique constraints on execution_date in the database to allow multiple DAG runs with the same logical time.

How execution_date and logical_date Work

  1. Logical date is equivalent to execution date: The two are just different names for the same value.
  2. Timetable controls logical date: The logical date can be set to any value, not necessarily tied to the data interval's start or end.
  3. Schedules dictate behavior: For value-based schedules (like cron), the logical date is set by the timetable class used.

Transitioning from execution_date

  • Airflow 3.0 will fully remove execution_date.
  • For uniquely identifying runs:
    • Use run_id for human-readable identifiers.
    • Use data_interval_start/data_interval_end for data partitioning in future.
    • id (auto-incremented) is recommended for ordering DAG runs.
  • If your existing logic relies on execution_date, switch to using data_interval_start or data_interval_end for identifying time ranges.

Testing

  • Updated unit tests to reflect the changes.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:CLI area:Scheduler including HA (high availability) scheduler provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Nov 11, 2024
@sunank200 sunank200 added legacy api Whether legacy API changes should be allowed in PR legacy ui Whether legacy UI change should be allowed in PR labels Nov 11, 2024
@sunank200 sunank200 force-pushed the rename-execution-date-to-logical-date branch 9 times, most recently from ea36b6c to 65f9b8e Compare November 12, 2024 02:54
@sunank200 sunank200 requested review from Lee-W and phanikumv November 12, 2024 02:54
@uranusjr
Copy link
Member

This needs a news fragment. Rewording the above summary would work fine. See existing files in newsfragments (the significant ones) for some examples.

@uranusjr
Copy link
Member

uranusjr commented Nov 12, 2024

For templates, replace {{ ds }} with {{ data_interval_start | ds }}.

This should probably use logical_date for max compatibility. Also should mention ts (same rewrite).

Oh wait, we’re not removing those in this PR yet, so this should not be mentioned at all.

@sunank200 sunank200 added the AIP-83 Remove Execution Date Unique Constraint from DAG Run label Nov 25, 2024
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit to astronomer/airflow that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kaxil added a commit that referenced this pull request Jan 12, 2025
We removed all the deprecated keys in #43902 so we no longer need this code.

In preparation of #45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
karenbraganz pushed a commit to karenbraganz/airflow that referenced this pull request Jan 13, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
HariGS-DB pushed a commit to HariGS-DB/airflow that referenced this pull request Jan 16, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
dauinh pushed a commit to dauinh/airflow that referenced this pull request Jan 24, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
We removed all the deprecated keys in apache#43902 so we no longer need this code.

In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 28, 2025
We removed all the deprecated keys in apache/airflow#43902 so we no longer need this code.

In preparation of apache/airflow#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

GitOrigin-RevId: 5597bb296106648fda48c768814f5ae6f3eb7a53
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 23, 2025
We removed all the deprecated keys in apache/airflow#43902 so we no longer need this code.

In preparation of apache/airflow#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

GitOrigin-RevId: 5597bb296106648fda48c768814f5ae6f3eb7a53
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 21, 2025
We removed all the deprecated keys in apache/airflow#43902 so we no longer need this code.

In preparation of apache/airflow#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

GitOrigin-RevId: 5597bb296106648fda48c768814f5ae6f3eb7a53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIP-83 Remove Execution Date Unique Constraint from DAG Run airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes area:API Airflow's REST/HTTP API area:CLI area:Scheduler including HA (high availability) scheduler legacy api Whether legacy API changes should be allowed in PR legacy ui Whether legacy UI change should be allowed in PR provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants