[torch.compile] Graph break+warning instead of error in _prepare_generation_config

[williamwen42]

I ran into this error when attempting to torch.compile miniCPM (https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5).

I tracked changes back to this PR #29443 and I think that we should be graph breaking (i.e. going ahead with copy.deepcopy) and issuing a warning in the case we need to, instead of crashing. But in the case where the copy doesn't need to be done, we can skip making the call in order to prevent the graph break.

cc @gante

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[gante]

Hi @williamwen42 👋 Thank you for opening this PR

The exception serves to nudge the users towards an equivalent solution that works, instead of enabling the code to run suboptimally (i.e. with a graph break).

Is there a use case that is prevented by the current exception that isn't solved by moving arguments to a GenerationConfig?

[williamwen42]

No use case off the top of my head, but I was trying to compile a publicly available model (not mine) and ran into the crash. I think warning + graph break is a better way to get things working, but also lets users know that they might want to consider changing code.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.