Require input_ids for repetition penalty

[ruben-aghayan]

What does this PR do?

This PR warns when using repetition penalty or ngram repetition penalty in decoder models on input_embed without input_ids args.

Previously, users were able to call repetition penalty on generate calls with input_embeds args. Since they don't actually have tokens, the repetition penalty was not applied to the input args, but only to the generated tokens.
An equivalent call (ie tokens corresponding to those embeddings) would behave differently by applying the repetition penalty to the input tokens.
This change makes it so that the repetition penalty is not applied and a warning is shown.

Testing

pytest tests/generation -vv -k 'not test_text_streamer_decode_kwargs'
test_text_streamer_decode_kwargs was giving an unrelated failure

Code Agent Policy

• I confirm that this is not a pure code agent PR.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@gante

[afurm]

Does prompt_input_ids.get() (or a follow-up check) need to handle the case where it's a list rather than a tensor? If input_ids is passed as a plain Python list, isinstance(..., torch.Tensor) would be False and this would raise the error even for valid input.

[ruben-aghayan]

Does prompt_input_ids.get() (or a follow-up check) need to handle the case where it's a list rather than a tensor? If input_ids is passed as a plain Python list, isinstance(..., torch.Tensor) would be False and this would raise the error even for valid input.

Thank you for your comment!

Is List considered valid input? Generate args are all tensors. Admittedly, input_ids goes in kwargs so is not explicitly typed. But event today, such an input would fail e.g.

  from transformers import AutoTokenizer, AutoModelForCausalLM

  tok = AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-gpt2")
  model = AutoModelForCausalLM.from_pretrained("hf-internal-testing/tiny-random-gpt2")

  ids = tok("Hello world", return_tensors="pt").input_ids[0].tolist()
  model.generate(input_ids=ids)

produces AttributeError: 'list' object has no attribute 'shape' since it's being treated as a tensor

[Rocketknight1]

@remi-or @McPatate for generation/CB, but feel free to pass it on to someone else if you're not comfortable reviewing it!

[remi-or]

I am unfamiliar with encoder and the inputs embeds, so I would prefer if it can be passed on. If no one picks this up I will when I have some room!

[Rocketknight1]

cc @Cyrilvallez for generation maybe, but if you're overloaded we might need to find someone to own generation code!

[Cyrilvallez]

Hey @ruben-aghayan! We can indeed raise in such cases, but this code should live inside _get_logits_processor!

[ruben-aghayan]

Hey @ruben-aghayan! We can indeed raise in such cases, but this code should live inside _get_logits_processor!

thanks + done

I noticed EncoderRepetitionPenaltyLogitsProcessor above just warns so I switched to warning (this would have been enough for my use case). Also extended it to NoRepeatNGramLogitsProcessor

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45389&sha=036192

[ruben-aghayan]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45389&sha=036192

looks unrelated?

[Cyrilvallez]

We don't want to skip, only warn that it will apply the repetition only on new tokens, vs applying it to the full sequence inclucing prompt