Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Mar 15, 2023

In some circumstances, changing the permission of a parent folder for the log might not be possible - for example when it was created by another user (with impersonation) or when the filesystem does not allow for permission change.

Fixes: #29112


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk potiuk requested review from dstandish and uranusjr March 15, 2023 14:06
@potiuk potiuk added this to the Airflow 2.5.3 milestone Mar 15, 2023
@potiuk
Copy link
Member Author

potiuk commented Mar 15, 2023

cc: @SamWheating

@potiuk potiuk closed this Mar 15, 2023
@potiuk potiuk reopened this Mar 15, 2023
@potiuk
Copy link
Member Author

potiuk commented Mar 15, 2023

I am also marking it as 2.5.3 - although it will not be super-easy to cherry-pick that one due to earlier changes from #27758 and related chnges, but it should be possible to apply the same workaround in the same cod (cc: @pierrejeambrun @ephraimbuddy ) . This seems to affect some common cloud cluster cases by default, so it might be worth the hassle (happy to help with cherry-picking if needed then).

In some circumstances, changing the permission of a parent folder
for the log might not be possible - for example when it was
created by another user (with impersonation) or when the filesystem
does not allow for permission change.

Fixes: apache#29112
@potiuk potiuk force-pushed the fix-permission-error-when-modifying-log-folder-permission branch from 4e75731 to 0a6ae89 Compare March 15, 2023 17:17
@potiuk
Copy link
Member Author

potiuk commented Mar 15, 2023

BTW. It also could be that @SamWheating's change from #29886 would also address the problem in some cases - maybe simply after checking the mask, the chmod would not have happened at all (when the filesystem is such that it has "open write for all" by default.

@potiuk potiuk merged commit b87cbc3 into apache:main Mar 15, 2023
@potiuk potiuk deleted the fix-permission-error-when-modifying-log-folder-permission branch March 15, 2023 20:44
@pierrejeambrun pierrejeambrun added the type:bug-fix Changelog: Bug Fixes label Mar 22, 2023
@thesuperzapper
Copy link
Contributor

@potiuk I am still seeing many users report errors in airflow 2.5.3 as this change was not back-ported.

The error is always associated with this line airflow/utils/log/file_task_handler.py#L358, perhaps we can put a temporary fix for 2.5.4 which puts a try/catch around that (and any other chmod attempts).

@potiuk
Copy link
Member Author

potiuk commented Apr 7, 2023

There won't be a 2.5.4 most likely - we are just about to release 2.6.0 and preparing for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:logging type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Operation not permitted" error when chmod on log folder

4 participants