Skip to content

Please ignore this #43917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
uranusjr wants to merge 182 commits into from
Closed

Please ignore this #43917

uranusjr wants to merge 182 commits into from

Conversation

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Nov 12, 2024

See #43915. The warning message also already says this will be a hard error in Airflow 3.

airflow/airflow/datasets/__init__.py

Lines 91 to 96 in 0380160

warnings.warn(
"A dataset URI should not contain auth info (e.g. username or "
"password). It has been automatically dropped.",
UserWarning,
stacklevel=3,
)

ephraimbuddy and others added 30 commits August 9, 2024 07:24
@ephraimbuddy
Update default branches for 2-10
ac5c3e0
@TJaniF
Typo fix dataset guide (apache#41353)
ac40b27
@ephraimbuddy
regenerate command hashes
1002266
@bbovenzi @ephraimbuddy
Fix Gantt Task Tries (apache#41342)
f2a15d4 
(cherry picked from commit cb80bda)
@ephraimbuddy
Revert "Send context using in venv operator (apache#41039)" (apache#4…
0ee7aff 
…1362)

This reverts commit da55393.

(cherry picked from commit 0a06cc6)
@ephraimbuddy
Retain the function "resource_name_for_dag" for backwards compatibili…
6b6224a 
…ty (apache#41382)

When using older FAB providers on the new airflow, this function is called
in the old provider and is no longer available in the new airflow. This
PR brings this back to fix issue in main and v2-10-test branch where all
DAGs fail because of lack of this function

(cherry picked from commit 0576f55)
@ephraimbuddy
Update version to 2.10.0
0d87d27
@ephraimbuddy @utkarsharma2
Update RELEASE_NOTES.rst
4fce874
@jscheffl @potiuk
Fix tests/models/test_taskinstance.py for Database Isolation Tests (a…
a7d48cb 
…pache#41344)

(cherry picked from commit f811ac3)
@jscheffl @potiuk
Fix TriggerDagRunOperator Tests for Database Isolation Tests (apache#…
dc0548e 
…41298)

* Attempt to fix TriggerDagRunOperator for Database Isolation Tests

* Finalize making tests run for triggerdagrunoperator in db isolation mode

* Adjust query count assert for adjustments to serialization

* Review feedback

(cherry picked from commit 6b810b8)
@jscheffl @potiuk
Fix core tests from start to SkipMixin for Database Isolation Mode (a…
2740c1c 
…pache#41369)

* Skip core tests from start to SkipMixin for Database Isolation Mode

* Skip core tests from start to SkipMixin for Database Isolation Mode, uups

* Skip core tests from start to SkipMixin for Database Isolation Mode, uups

(cherry picked from commit b87f987)
@bugraoz93 @potiuk
Fix Variable and KubernetesJobOperator Tests for Database Isolation T…
9f3c1ae 
…ests (apache#41370)

Fixing remaining Variable tests for db isolation mode, also fixing secret backend haven't called from EnvironmentVariablesBackend, Metastore and custom ones. This caused side effect to move the Variable.get() method to internal API

(cherry picked from commit c98d1a1)
@jscheffl @potiuk
Fix pytests for Core except Variable for DB Isolation Mode (apache#41375
ef147dd 
)

(cherry picked from commit 60cbea5)
@dirrao @potiuk
bump uv version to 0.2.34 (apache#41334)
34e2c9c 
(cherry picked from commit b4a92f8)
@potiuk
Skip docs publishing on non-main brnaches (apache#41385)
09567dc 
(cherry picked from commit 54c165c)
@potiuk
Fix mypy checks for new azure libraries (apache#41386)
6c6797c 
(cherry picked from commit 68a6a05)
@bugraoz93 @potiuk
Fix tests/decorators/test_python.py for database isolation tests (apa…
8ea4eb1 
…che#41387)

* Pass serialized parameter for dag_maker

* Serialisation of object is on __exit__ moving out the dag definition out of dag_maker context

(cherry picked from commit 278f3c4)
@Lee-W @ephraimbuddy @utkarsharma2
fix DagPriorityParsingRequest unique constraint error when dataset al…
e001b88 
…iases are resolved into new datasets (apache#41398)

* fix(datasets/manager): fix DagPriorityParsingRequest unique constraint error when dataset aliases are resolved into new datasets

this happens when dynamic task mapping is used

* refactor(dataset/manager): reword debug log

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

* refactor(dataset/manager): remove unnecessary logging

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

---------

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
(cherry picked from commit bf64cb6)
@potiuk
Enable pull requests to be run from v*test branches (apache#41474) (a…
b211936 
…pache#41476)

Since we switch from direct push of cherry-picking to open PRs
against v*test branch, we should enable PRs to run for the target
branch.

(cherry picked from commit a9363e6)
@potiuk
Prevent provider lowest-dependency tests to run in non-main branch (a…
5934272 
…pache#41478) (apache#41481)

When running tests in v2-10-test branch, lowest depenency tests
are run for providers - because when calculating separate tests,
the "skip_provider_tests" has not been used to filter them out.

This PR fixes it.

(cherry picked from commit 75da507)
@potiuk
Make PROD image building works in non-main PRs (apache#41480) (apache…
bf0d412 
…#41484)

The PROD image building fails currently in non-main because it
attempts to build source provider packages rather than use them from
PyPi when PR is run against "v-test" branch.

This PR fixes it:

* PROD images in non-main-targetted build will pull providers from
  PyPI rather than build them
* they use PyPI constraints to install the providers
* they use UV - which should speed up building of the images

(cherry picked from commit 4d5f1c4)
@bbovenzi @molcay
Add WebEncoder for trigger page rendering to avoid render failure (ap…
5f8230b 
…ache#41350) (apache#41485)

Co-authored-by: M. Olcay Tercanlı <muhammed_tercanli@epam.com>
@potiuk @howardyoo
Incorrect try number subtraction producing invalid span id for OTEL a…
30cfb73 
…irflow (issue apache#41501) (apache#41502) (apache#41535)

* Fix for issue apache#39336

* removed unnecessary import

(cherry picked from commit dd3c3a7)

Co-authored-by: Howard Yoo <32691630+howardyoo@users.noreply.github.com>
@potiuk
Fix failing pydantic v1 tests (apache#41534) (apache#41541)
fa4ee68 
We need to exclude some versions of Pydantic v1 because it conflicts
with aws provider.

(cherry picked from commit a033c5f)
@potiuk
Fix Non-DB test calculation for main builds (apache#41499) (apache#41543
e36e521 
)

Pytest has a weird behaviour that it will not collect tests
from parent folder when subfolder of it is specified after the
parent folder. This caused some non-db tests from providers folder
have been skipped during main build.

The issue in Pytest 8.2 (used to work before) is tracked at
pytest-dev/pytest#12605

(cherry picked from commit d489826)
@utkarsharma2
Add changelog for airflow python client 2.10.0 (apache#41583) (apache…
3ea764f 
…#41584)

* Add changelog for airflow python client 2.10.0

* Update client version

(cherry picked from commit 317a28e)
@potiuk
Make all test pass in Database Isolation mode (apache#41567)
6441563 
This adds dedicated "DatabaseIsolation" test to airflow v2-10-test
branch..

The DatabaseIsolation test will run all "db-tests" with enabled
DB isolation mode and running `internal-api` component - groups
of tests marked with "skip-if-database-isolation" will be skipped.
@utkarsharma2 @potiuk
Upgrade build and chart dependencies (apache#41570) (apache#41588)
e76dba6 
(cherry picked from commit c88192c)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@bbovenzi
Fix try selector refresh (apache#41483) (apache#41503)
3de7597
@potiuk
Remove debian bullseye support (apache#41568) (apache#41569)
49fc179 
(cherry picked from commit 5c323a9)
pierrejeambrun and others added 7 commits November 5, 2024 16:28
@pierrejeambrun @kaxil
Disable XCom list ordering by execution_date (apache#43680) (apache#4…
39355b8 
…3696)

* Disable XCom list ordering by execution_date

* Update airflow/www/views.py

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

---------

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
(cherry picked from commit c96b618)
@potiuk
Handle FileNotFound Error returned by missing uv or pipx (apache#43714)…
0a7a675 
… (apache#43715)

Subprocess.run raises FileNotFound when uv or pipx are not installed
at all. This PR will handle it.

(cherry picked from commit ed3accb)
@eladkal
Remove note about MySQL 5 (apache#43729)
5b114cc
@shahar1
Prevent using trigger_rule="always" in a dynamic mapped task (apache#…
c9436b4 
…43810)

(cherry picked from commit c753ca2)
@jscheffl @enisnazif
apache#43252 Disable extra links button if link is null or empty (apa…
20b6ab7 
…che#43844) (apache#43851)

* Disable button if link is null or empty

* Fix space

---------

Co-authored-by: Enis Nazif <enis@Eniss-MacBook-Pro.local>
(cherry picked from commit de88182)

Co-authored-by: enisnazif <enisnazif@gmail.com>
@jscheffl @xitep
Correct mime-type (apache#43879) (apache#43901)
0380160 
(cherry picked from commit 45cbad7)

Co-authored-by: xitep <xitep@users.noreply.github.com>
@uranusjr
Tweak strict_dataset_uri_validation documentation wording
df32c63
@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:dev-tools area:production-image Production image improvements and fixes labels Nov 12, 2024
@uranusjr
Copy link
Member Author

uranusjr commented Nov 12, 2024

Oops, wrong branch 🤦

@uranusjr uranusjr closed this Nov 12, 2024
@uranusjr uranusjr changed the title Dataset validation removal message Please ignore this Nov 12, 2024
@uranusjr uranusjr deleted the dataset-validation-removal-message branch November 12, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bolkedebruin bolkedebruin Awaiting requested review from bolkedebruin

@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner

@dstandish dstandish Awaiting requested review from dstandish

@eladkal eladkal Awaiting requested review from eladkal

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@pierrejeambrun pierrejeambrun Awaiting requested review from pierrejeambrun

@potiuk potiuk Awaiting requested review from potiuk potiuk is a code owner

@ashb ashb Awaiting requested review from ashb ashb is a code owner

@hussein-awala hussein-awala Awaiting requested review from hussein-awala hussein-awala is a code owner

@vincbeck vincbeck Awaiting requested review from vincbeck

@ryanahamilton ryanahamilton Awaiting requested review from ryanahamilton

@bbovenzi bbovenzi Awaiting requested review from bbovenzi

@jscheffl jscheffl Awaiting requested review from jscheffl jscheffl is a code owner

@XD-DENG XD-DENG Awaiting requested review from XD-DENG

@o-nikolas o-nikolas Awaiting requested review from o-nikolas

Assignees

No one assigned

Labels

area:API Airflow's REST/HTTP API area:dev-tools area:production-image Production image improvements and fixes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.