Skip to content

Conversation

@LefterisXefteris
Copy link

@LefterisXefteris LefterisXefteris commented Dec 23, 2024

Description

I implemented data fetching for pool slots using React Query. The data is successfully retrieved but I need help with displaying it in the required format.

Closes: #43328

Current Status

  • ✅ Successfully fetching pool slots data using React Query
  • ✅ Basic data transformation implemented
  • ⚠️ Display format needs adjustment to match design requirements

Implementation Details

  • Added usePoolServiceGetPools hook for data fetching
  • Implemented data reduction to calculate total slots by type:
    • running
    • queued
    • deferred
    • scheduled
    • open

Questions/Help Needed

  • Need guidance on transforming the display to match the horizontal stacked bar design
  • Looking for feedback on the current data transformation approach

@boring-cyborg boring-cyborg bot added the area:UI Related to UI/UX. For Frontend Developers. label Dec 23, 2024
@tirkarthi
Copy link
Contributor

Related issue #43328

@jscheffl
Copy link
Contributor

jscheffl commented Jan 1, 2025

Note: As PR #45312 has been merged, the code formatting rules have changed for new UI. Please rebase and re-run pre-commit checks to ensure that formatting in folder airflow/ui is adjusted.

gopidesupavan and others added 23 commits January 5, 2025 00:37
* Fix task id validation in BaseOperator

* add additional tests to check task id length

* fix assert statement
* #44019 - Set autocomplete to off for username and password login form

* fixed static check
* Send multiple queries on dataproc sql job

DataProcJobBuilder.add_query(query) is misleading, cause it can make you think that you can call this function multiple times with different queries and then execute and it will send all queries, but it fact it will sent the last one since its override the queries.

I've added set_queries function which takes a list of strings and send it.
Dataproc supports queries list.

* Append query to the queries sent to dataproc instead of replacing

Requested by reviewer, instead of adding a function that sets a list of queries, just fix the original function add_query that will actually append query to the queries list send to dataproc job.

* Send multiple queries on dataproc sql job

DataProcJobBuilder.add_query(query) is misleading, cause it can make you think that you can call this function multiple times with different queries and then execute and it will send all queries, but it fact it will sent the last one since its override the queries.

I've added set_queries function which takes a list of strings and send it.
Dataproc supports queries list.

* Append query to the queries sent to dataproc instead of replacing

Requested by reviewer, instead of adding a function that sets a list of queries, just fix the original function add_query that will actually append query to the queries list send to dataproc job.

* formatter

---------

Co-authored-by: Amir Mor <amir.mor26@gmail.com>
New hatchling version was released.
I suspect we no longer need this since alias resolution now happens very
late in the scheduling process. A DAG *should* be able to automatically
understand the alias is resolved without a reparse after refactorings
introduced in cccc933.
* Refactor PostConnection handling insert duplicate

* Add parsing error detail to error handler with test

* Add parsing multiple columns case in unique constraint

* Add multiple columns, parsing error cases in test

* Fix public API already existed test cases

* Refactor: remove parsing, return statement and orig

* Refactor test_exceptions with statement and orign

* Fix unique constriant violation response in api test
Co-authored-by: Marco Küttelwesch <marco.kuettelwesch@de.bosch.com>
…44962)

In #33901 this was moved to a new file and mistakenly changed to
flask_session.

It's not clear to me why this didn't cause errors, but it was highlighted in
my editor as an invalid type 🤷
Part of #44481

This commit augments the TI context available in the Task Execution Interface with the one from the Execution API Server.

In future PRs the following will be added:

- More methods on TI like ti.xcom_pull, ti.xcom_push etc
- Lazy fetching of connections, variables
- Verifying the "get_current_context" is working
* Structure endpoint add asset-condition

* Fix CI
* issue 44559

removed depricated code from alibaba provider

* Apply suggestions from code review

Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

---------

Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
This was initially added in response to #168. However, we now have `ti.log_url` that is used for that; example usages:

https://github.com/apache/airflow/blob/dcd41f60f1c9b5583b49bfb49b6d85c640a2892c/airflow/models/taskinstance.py#L1362

https://github.com/apache/airflow/blob/dcd41f60f1c9b5583b49bfb49b6d85c640a2892c/providers/src/airflow/providers/smtp/notifications/templates/email.html#L28

https://github.com/apache/airflow/blob/dcd41f60f1c9b5583b49bfb49b6d85c640a2892c/docs/apache-airflow/howto/email-config.rst?plain=1#L76

So, to simplify what we need to pass from API server to the Task SDK, I want to simplify and remove things that aren't needed. In this case, this is good so we don't pass/expore secrets unnecesarily via `conf`
* conf

* conf check

* basic trigger api

* retrieve params

* refactor the api

* fix

* fix details page

* refactor the logic

* refactor

* reviews

* error on form

* data date validation

* remove empty runid and note
* Add support for XCom tab in Task Instance detail page.

* Add support for XCom page in browse section.

* Disable sorting since API doesn't support it.

* Fix run_id in xcom tests.

* Fix generated files.

* Refactor XCom page and XCom columns.
PR #44820 removed `conf` from task context, but these tests still expect
it.
Small improvements in airflow/api_fastapi/common/parameters.py:_transform_ti_states
@pierrejeambrun
Copy link
Member

I think some rebase went wrong. Can you fix the branch and resolve possible conflicts to facilitate the review please.

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Feb 22, 2025
@pierrejeambrun
Copy link
Member

@LefterisXefteris it's hard to provide guidance because I suspect a rebase went wrong and the change set contains hundred of other commits.

Can you clean the branch please ?

@github-actions github-actions bot removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Feb 25, 2025
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Apr 11, 2025
@pierrejeambrun
Copy link
Member

No updates, closing for now. Feel free to re-open if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AIP-38 | Add Pool Summary stats to Dashboard