[Feature Request]: Implement Native Java Remote Inference

[jrmccluskey]

What would you like to happen?

With the introduction of remote inference in the Python SDK (making requests to an external service for machine learning inference rather than performing that work on-worker) there has been an increasing interest in providing this in a native Java format. RunInference is currently enabled in Java via cross-language; however, because remote inference is not on-worker, there is no real requirement to have the pipeline code operate in Java. A native Java implementation of the RemoteModelHandler class, which accepts arbitrary clients and request code, would fulfill this need.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Ganeshsivakumar]

Hi @jrmccluskey , I'm interested to take this if no one else is working on it. Since it’s part of the beam 3.0, I thought I’d check here first.

[Ganeshsivakumar]

I've put together a short design doc for Native Java Remote Inference feature request. would like to kindly request feedback/suggestions on the approach.

Doc - https://docs.google.com/document/d/1b_JbiZoMJbjqi5ae0jnTaBr2gvvzbnWfxCRPvNdICVY/edit?usp=sharing

[jrmccluskey]

Sorry about the delay in responding, I seem to have missed the notification about your comments. I also, notably, missed the key step of assigning the issue to myself when I filed it (I have not actually started working on this yet, which is fine.) Let me look through the doc and give feedback.

[Ganeshsivakumar]

Hi Jake, that’s totally fine! I initially thought no one had picked it up, but I completely understand. Please go ahead with your planned work. I’d be very happy to collaborate or help with any specific part where I can contribute.

[ShreyPatel4]

Hi all, is this feature request still open ? - i would like to work on this .
@jrmccluskey @Ganeshsivakumar

[jrmccluskey]

It's currently open, Ganesh and I are collaborating on the effort. The design doc Ganesh wrote is at https://docs.google.com/document/d/1b_JbiZoMJbjqi5ae0jnTaBr2gvvzbnWfxCRPvNdICVY/edit?usp=sharing

[ShreyPatel4]

It's currently open, Ganesh and I are collaborating on the effort. The design doc Ganesh wrote is at https://docs.google.com/document/d/1b_JbiZoMJbjqi5ae0jnTaBr2gvvzbnWfxCRPvNdICVY/edit?usp=sharing

i am looking at it, will try to address the comments in doc