Filter Datasets by associated dag_ids (GET /datasets)

[Satoshi-Sh]

Description

Added dag_ids query to the GET /datasets endpoint. Updated document and unit test accordingly.

Related Issue

closes #37423

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[uranusjr]

I wonder if it’s clear enough to call this just dag_ids, or should a more descriptive name be used, say comsuming_dag_ids.

[Satoshi-Sh]

dag_ids could be from consuming_dags and producing_tasks.

We could have 2 queries for consuming_dags and producing_tasks separately. For now, I put them together as dag_ids.

[bbovenzi]

I wonder if it’s clear enough to call this just dag_ids, or should a more descriptive name be used, say comsuming_dag_ids.

My use case is that I want any datasets connected to a single dag. But I am indifferent if that is a single param dag_ids or if I need to pass the dag_id twice in consuming_dag_ids and producing_dag_ids. I guess the later is most flexible.

[bbovenzi]

Marking as "request changes" to make sure we don't accidentally merge.

[jedcunningham]

My use case is that I want any datasets connected to a single dag. But I am indifferent if that is a single param dag_ids or if I need to pass the dag_id twice in consuming_dag_ids and producing_dag_ids. I guess the later is most flexible.

I think having both makes sense. In your use case, if we only had consuming/producing, you'd have to hit the endpoint twice (they'd be AND'd together, not OR'd).

So maybe we start with the simple dag_ids and be clear in the description it filters for both consuming or producing dags. Leave the more granular filters for another PR / future need?

[bbovenzi]

Tested locally, works great. Thanks for picking this up!