Skip to content

[AIRFLOW-92] Avoid unneeded upstream_failed session commits#1485

Closed
underyx wants to merge 1 commit intoapache:masterfrom
underyx:AIRFLOW-92
Closed

[AIRFLOW-92] Avoid unneeded upstream_failed session commits#1485
underyx wants to merge 1 commit intoapache:masterfrom
underyx:AIRFLOW-92

Conversation

@underyx
Copy link
Contributor

@underyx underyx commented May 10, 2016 •
edited
Loading

Resolves: https://issues.apache.org/jira/browse/AIRFLOW-92

It looks like this is the only place where set_state is used, so it should be safe to move the commit there I think. If the commit is done where it was before, then there will be a commit even if state updates are disabled (flag_upstream_failed=False), expiring objects in the session, and causing the issue referenced in the Jira ticket.

@underyx underyx changed the title Avoid unneeded upstream_failed session commits [AIRFLOW-92] Avoid unneeded upstream_failed session commits May 10, 2016
@bolkedebruin
Copy link
Contributor

bolkedebruin commented May 10, 2016

This will not work, as now the necessary commits are not taking place.

@bolkedebruin
Copy link
Contributor

bolkedebruin commented May 10, 2016

LGTM, for reference sake:

set_state is only called when flag_upstream_failed=True this will be always false for are_dependencies_met as it defaults to False. This means no "session.commit()" will be called anymore.

So this will resolve the issue. However the core issue remains and that is that a getter should not change the internal state of the object. That is another issue, so will merge.

@asfgit asfgit closed this in dddfd3b May 10, 2016
asfgit pushed a commit that referenced this pull request May 10, 2016
@bolkedebruin @aoen
AIRFLOW-92 Avoid unneeded upstream_failed session closes #1485
4a5f4a0
yiqingj pushed a commit to yiqingj/airflow that referenced this pull request May 27, 2016
Merge branch 'master' into dev
66598f3 
* master:
  AIRFLOW-92 Avoid unneeded upstream_failed session closes apache#1485
  AIRFLOW-52 Warn about overwriting tasks in a DAG
  Add logic to lock DB and avoid race condition
  Handle queued tasks from multiple jobs/executors
  [AIRFLOW-80] Move example_twitter dag to contrib/example_dags as it requires hive
  [AIRFLOW-75] Fix bug in S3 config file parsing
  Use getfqdn to make sure urls are fully qualified
  [AIRFLOW-52] Fix bottlenecks when working with many tasks
  Add bulk_dump abstract method to DbApiHook (apache#1471)
  Fix corner case with joining processes/queues (apache#1473)
  [AIRFLOW-53] Adding DagBag stats report to CLI's list_dags (apache#1468)
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 22, 2019
@bolkedebruin @leahecole
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
8e4487c 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Dec 5, 2019
@bolkedebruin @leahecole
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
6c81a0e 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jan 16, 2020
@bolkedebruin @leahecole
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
8577d7e 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jan 23, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
0a653f0 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 7, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
fab2579 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 7, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
87b17a0 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 8, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
f980c94 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 9, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a44c3a5 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 10, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a9cd208 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 11, 2020
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
5f72273 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 10, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
d6abeaf 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 18, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
e4a26a6 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 18, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a4b8b6e 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 23, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
6b2875d 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 24, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
e0998a1 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 1, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
4b06bf7 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 2, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a491bcf 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 3, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
8f5889b 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 3, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
41fe86d 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 9, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a10d0f5 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 14, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
9b5368f 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 15, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
c5f8d6a 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 16, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
3460618 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 16, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
cbaa8b4 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 23, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
26bdac7 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 14, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
e0a59b7 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 16, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
fbd59e7 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 22, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
5fceb40 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 24, 2021
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
b61cfd9 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 3, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
961225f 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Mar 1, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
552bca5 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 3, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
f4bead2 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
hankehly pushed a commit to hankehly/airflow that referenced this pull request Jun 13, 2022
@blag
Merge pull request apache#1485 from tirkarthi/cli-roles
950bd16 
Convert sync-perm command to click and make tests compatible.
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jul 8, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
ec34fba 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Aug 26, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
ef3d146 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 3, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
4aec6c9 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
aglipska pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 6, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
2d701ed 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Dec 6, 2022
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
fea741b 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jan 26, 2023
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
76b823a 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
ahidalgob pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 24, 2023
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
c1e7900 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this pull request Sep 11, 2024
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
8caae3e 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this pull request Sep 12, 2024
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
0a4eb5a 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 16, 2024
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
5763021 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this pull request Sep 17, 2024
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
4a2f89d 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 6, 2024
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
3b741cf 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Apr 30, 2025
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
c141d53 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 20, 2025
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
96c4d40 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 15, 2025
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
bb074a5 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 14, 2025
@bolkedebruin
AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airfl…
a5cf413 
…ow#1485

GitOrigin-RevId: dddfd3b5bf2cabaac6eec123dfa3cb59e73a56f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@underyx @bolkedebruin