Skip to content

Conversation

@kkulczak
Copy link
Contributor

@kkulczak kkulczak commented Mar 21, 2025

…esent

Checking modification timestamps has sagnificant impact on performance of sensors in ase of large amount of files.
This whole block of code do not need to be executed, when newer_than=None


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg
Copy link

boring-cyborg bot commented Mar 21, 2025

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@kkulczak kkulczak changed the title Checking modification timestamps only when newer_than parameter is pr… Checking modification timestamps only when newer_than is present Mar 21, 2025
@kkulczak
Copy link
Contributor Author

Done. Please consider merging.
If merged when can I expect the release with this change? I need to inform my customer

@potiuk
Copy link
Member

potiuk commented Mar 23, 2025

NIT: Is it also possible to have a test that shows that it IS called when newer_than is set?

Done. Please consider merging. If merged when can I expect the release with this change? I need to inform my customer

With the next provider's release. We usually release every 2 weeks or so (see the docs about providers) but this is on a best-effort base and we are about to release Airlfow 3 so that might be skewed. This is open-source, no promises can be made with those as most people work here as volunteers.

@potiuk
Copy link
Member

potiuk commented Mar 23, 2025

And you are not blocked. You are absolutely free to build your own provider version. Providers are absolutely standard packages now - with their own pyproject.toml files and you can easily build your own distribution packages.

Also breeze release-management prepare-providers-distributions allows you to build providers locally easily - and you can have your customer to install such providers prepared by you without waiting for release if you do not want to wait.

@kkulczak kkulczak force-pushed the patch-1 branch 2 times, most recently from d20b281 to d6094d2 Compare March 23, 2025 15:23
@kkulczak
Copy link
Contributor Author

kkulczak commented Mar 23, 2025

@potiuk

NIT: Is it also possible to have a test that shows that it IS called when newer_than is set?

Done

Regarding release, thanks for letting me know :)

@kkulczak
Copy link
Contributor Author

Cheked tests passing on github codespace.
Cannot
Screenshot 2025-03-24 at 18 07 15

@kkulczak
Copy link
Contributor Author

Could you re-run pipeline please?

…esent

Checking modification timestamps has sagnificant impact on performance of sensors in ase of large amount of files.
@kkulczak
Copy link
Contributor Author

Are there any actions needed from my side?

@potiuk potiuk merged commit ac3090d into apache:main Mar 31, 2025
60 checks passed
@boring-cyborg
Copy link

boring-cyborg bot commented Mar 31, 2025

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@potiuk
Copy link
Member

potiuk commented Mar 31, 2025

Just delays due to Airflow 3 :)

@kkulczak kkulczak deleted the patch-1 branch March 31, 2025 08:46
shubham-pyc pushed a commit to shubham-pyc/airflow that referenced this pull request Apr 2, 2025
…esent (apache#48063)

Checking modification timestamps has sagnificant impact on performance of sensors in ase of large amount of files.
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request Apr 4, 2025
…esent (apache#48063)

Checking modification timestamps has sagnificant impact on performance of sensors in ase of large amount of files.
simonprydden pushed a commit to simonprydden/airflow that referenced this pull request Apr 8, 2025
…esent (apache#48063)

Checking modification timestamps has sagnificant impact on performance of sensors in ase of large amount of files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants