[Task]: Update RunInference Notebooks to use Custom Inference Functions

[jrmccluskey]

What needs to happen?

Once Beam 2.44 is released, the RunInference example notebooks should be updated to take advantage of the custom inference function capabilities added in #22572

Issue Priority

Priority: 2

Issue Component

Component: run-inference

[ziqi-ma]

Would it be better to just add one more notebook using Huggingface generate rather than modifying the existing examples (since the existing ones don't really require calling a non-default inference function)?

[smeet07]

@jrmccluskey @damccorm I would like to contribute to this issue , I have one question though. What is the name of the convenience function that was agreed upon in #24062

[damccorm]

It depends on the model handler, but for PyTorchTensor its make_tensor_model_fn -

beam/sdks/python/apache_beam/ml/inference/pytorch_inference.py

Line 154 in fbc7df4

def make_tensor_model_fn(model_fn: str) -> TensorInferenceFn:

Roughly they should all follow the make_<something>_fn pattern

[damccorm]

#27305 should address most of this, we still need to update https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb

[smeet07]

@damccorm what model should I use for the example?

[damccorm]

Hey @smeet07 we shouldn't need to use a new model, we just need to get rid of HuggingFaceStripBatchingWrapper from https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb and use a custom inference function instead