Migrate Edge calls for Worker to FastAPI part 1 - Worker routes

[jscheffl]

To prepare EdgeWorker to be independent of AIP-44 Internal API, this PR is the second step in adding/migrating to FastAPI. The calls to "Worker" API to (1) register a worker and (2) set the state are now real REST API calls, not using internal API.

I would separate the other internal API calls to follow-up PRs as this is already quite large. Especially ecause for ongoing Airflow 2.10 Connexion API + Swagger manually need to be generated whereas the main workstram for Airflow 3 uses FastAPI.

[Copilot]

Copilot reviewed 5 out of 18 changed files in this pull request and generated 3 suggestions.

Files not reviewed (13)

• providers/src/airflow/providers/edge/CHANGELOG.rst: Language not supported
• providers/src/airflow/providers/edge/openapi/edge_worker_api_v1.yaml: Language not supported
• providers/src/airflow/providers/edge/cli/api_client.py: Evaluated as low risk
• providers/src/airflow/providers/edge/cli/edge_command.py: Evaluated as low risk
• providers/src/airflow/providers/edge/executors/edge_executor.py: Evaluated as low risk
• providers/src/airflow/providers/edge/provider.yaml: Evaluated as low risk
• providers/tests/edge/worker_api/routes/test_worker.py: Evaluated as low risk
• providers/src/airflow/providers/edge/models/edge_worker.py: Evaluated as low risk
• providers/src/airflow/providers/edge/worker_api/routes/rpc_api.py: Evaluated as low risk
• providers/tests/edge/worker_api/routes/test_rpc_api.py: Evaluated as low risk
• providers/src/airflow/providers/edge/worker_api/app.py: Evaluated as low risk
• providers/tests/edge/cli/test_edge_command.py: Evaluated as low risk
• providers/src/airflow/providers/edge/worker_api/routes/_v2_compat.py: Evaluated as low risk

Comments skipped due to low confidence (1)

providers/src/airflow/providers/edge/worker_api/datamodels.py:45

• The description for jobs_active should be updated to clarify that it is the number of active jobs the worker is running.

jobs_active: int = Field(0, description="Number of active jobs the worker is running.")

[potiuk]

One "bigger" thing about revealing too much for auth errors.

[jscheffl]

One "bigger" thing about revealing too much for auth errors.

Point taken. Thanks for the review. Actually it was a take-over from existing internal API. Would we need to harden this as well before we do a 2.10.4? See https://github.com/apache/airflow/blob/2.10.3/airflow/api_internal/endpoints/rpc_api_endpoint.py#L190 (now removed on main...)

[jscheffl]

@kaxil / @potiuk Thanks for the review! All things adjusted... but as AIP-44 needed to re-work a lot I needed to fully re-base and restore AIP-44 broken function. As Airflow 3 is now broken... after this PR v2.10 is working again.

Do you want to have a second round or good to merge as is? (And follow-ups will be taken care...)

[potiuk]

I had a quick look - I am good to go. I think we need to align on the near future strategy for breaking/non-breaking strategy for the edge worker - see #44494 (comment) but this one is good to go I think