A question about the parameter `append_duration2caption`

[sen-mao]

Amazing work! InfinityStar can generate videos quickly and with great consistency!

It seems that during the inference phase, when generating a video, the parameter append_duration2caption must be set to 1. This means that the prefix "<<<t={mapped_duration}s>>>" must be added to the prompt, as shown in https://github.com/FoundationVision/InfinityStar/blob/0446330b6791c41d1ef5e6a41a2dfbcea92da279/tools/infer_video_480p.py#L115C20-L115C46. The result is as follows with prompt "<<<t=5s>>>A handsome smiling gardener inspecting plants, realistic cinematic lighting, detailed textures, ultra-realistic":

demo_i2v.mp4

When the parameter append_duration2caption is set to 0 (i.e., the prefix "<<<t={mapped_duration}s>>>" is not used), the result generated with the prompt "A handsome smiling gardener inspecting plants, realistic cinematic lighting, detailed textures, ultra-realistic" is as follows:

demo_i2v.mp4

It is evident that when the prefix is used, the video can be generated normally; whereas when the same prompt is used without the prefix, video generation fails. Given that both the parameter generation_duration and the variable scale_schedule are already employed to control the video duration, why is such a constraint imposed?

[happylicio]

@sen-mao To support autoregressive video training with different duration of clip，we add generation_duration into caption. Our release 720p model mainly train on 5s videos，you can try again by setting mapped_duration 4 or 6s, due to 0s is not a valid generation_duration in our training set