Add withResources to Java External transform

[ihji]

#21780

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Mention the appropriate issue in your description (for example: "addresses [BEAM-121] Add DisplayData for IO transforms #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment "fixes #" instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[ihji]

R: @chamikaramj

[robertwb]

I don't think this is the right direction either. If an external transform needs a certain set of files, that transform should be responsible for making the declaration, not the callee (and especially, it seems, the callee from another language).

I think the correct solution is to us resource hints (e.g. https://beam.apache.org/documentation/runtime/resource-hints/). There's also the issue of finding the resources at their new locations (on the worker). Presumably one would ask for a resource (e.g. file) to be staged against some kind of identifier, and on the worker one could use this identifier to look up the copied resource in its final destination.

[ihji]

I'm not sure resource hint is the right tool for this. From what I've read from the proto definition, resource hint is advisory (nice to have) and can be ignored by runners. However, the resource here is considered as one of the inputs of the expanded transform (must have) and should not be ignored (otherwise the transform will certainly fail). The staged resources are more like runtime dependencies not just hints.

I agree that it would be great if we could look up staged resources by identifiers.