Skip served logs for non-running task try

[Khrol]

If the task is running but is retried, the previous attempt logs opening tries to go to served logs. But the logs from there can be already moved to remote storage.

We should skip served logs for non-running task try.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

cc: @dstandish ?

[pankajkoti]

Looks correct to me

[kahlstrm]

This has caused a regression in our use case, as we currently store our logs on the worker with a Persistent Volume with a 2 week retention period. Now when looking at a DAG in our configuration with 2 fails and one retry running, logs of tries number 1 and 2 aren't visible. This happens because the current flow doesn't fetch logs from worker for a running task instance when the try number is not the most recent one.

Sure, we could refactor to use remotes but I think there is a way of resolving this that works for both use cases.

From what I understand the original problem @Khrol had was that they have their logs in remote and then the server sends an "unnecessary" request trying to fetch logs from the worker. This can be also achieved with just checking if not remote_logs prior to trying to fetch from worker instead of never trying to fetch from the worker.

[potiuk]

From what I understand the original problem @Khrol had was that they have their logs in remote and then the server sends an "unnecessary" request trying to fetch logs from the worker. This can be also achieved with just checking if not remote_logs prior to trying to fetch from worker instead of never trying to fetch from the worker.

Surely, If you would like to propose a PR for that where it could be discussed, and take a lead on that - that's the best way to move it forward.