Generate: inner decoding methods are no longer public

[gante]

What does this PR do?

This PR sets the inner decoding methods (such as greedy_search) as private methods, with a deprecation warning.

This will enable us to complete the following projects with fewer speed bumps:

1. torch.compile: as seen in the PoC PR, a few interface pieces should be touched. E.g. Some special tokens can be a list (e.g. eos_token_id), which is not compile-friendly, and thus torch.tensor should be used instead;
2. Separate prefill function. This will significantly change the interface to the inner decoding methods;
3. More flexible, composable generate. Having multiple interfaces to maintain will constrain our ability to refactor the code.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Thanks, might be easier to have both contrastive_search and _ contrastive_search for overloading? Tho it gives freedom, up to you!

[ArthurZucker]

we could even leave it as is, this way custom self._contrastive_search are easily changeable for users? or something like that

[gante]

In the future, I don't want to have all these methods, as they are near copies of each other -- e.g. greedy_search and sample differ by <5 lines. As such, some of them will be deleted, so users won't be able to call e.g. _greedy_search :D