[BEAM-11569] Enabling wheels to be built using Apache-based workflow

[pabloem]

Please add a meaningful description for your change here

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Dataflow	Flink	Samza	Spark	Twister2
Go		---		---		---
Java
Python				---		---
XLang				---		---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---		---	---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[pabloem]

r: @lostluck

Per @potiuk's comment on https://issues.apache.org/jira/browse/BEAM-11569, this should enable us to build the wheels for the release. Thoughts Rebo?

[lostluck]

LGTM

[potiuk]

Just one more update!

We found a better solution. Submodules. They seem to pass all the security requirements from infra, and they are very easy to add and maintain (and you avoid code duplication).

You can see the PR we run to have POC in Airflow: apache/airflow#13514

And disucssion at builds@a.o https://lists.apache.org/thread.html/rcf7f560dad70ed02d77ad131a670e24eb815e41f92a442a3153da98b%40%3Cbuilds.apache.org%3E

Jus to quote my words from the discussion:

This seems to works perfectly:

1. It always links to particular SHA commit not branch
2. No code duplication
3. GitHub Review nicely incorporates the change code from submodules
   whenever
   submodule is updated, so it fits naturally in the review workflow.
4. Seems that we can easily make it works with Github Actions (the
   submodule needst
   to be checked out in previous step of the job).
5. It's even easier to pull new versions.
6. It is equally easy to add any external action at any time
7. Passes all the INFRA requirements re: review + SHA - without any checks

We are going to migrate to it in Airflow today/tomorrow and we highly recommend this approach.

[potiuk]

BTW. I am going to delete or archive those repositories which I created so maybe even better if you switch to the submodules and use the original repo rather than airflow-'s copy :)

[pabloem]

ok thanks for mentioning that. Are you archiving them pretty soon?

[potiuk]

ok thanks for mentioning that. Are you archiving them pretty soon?

I might leave them lying for a while in archived status :).

But it should be super easy to switch. We are doing it in Airflow now: https://github.com/apache/airflow/pull/13514/files

[pabloem]

cool cool - will do. thanks.

[ashb]

We're hoping to delete these repos next week https://lists.apache.org/thread.html/r19149c3e8afc04c679ca4091c40447b6d8c7e1be7c9c8660d9975e90%40%3Cdev.airflow.apache.org%3E

[pabloem]

alright, I'll make the change today.