More references and suggestions regarding training

[zzk88862]

Hello, great work!

now, I'm looking to build a video autoqc system based on what you've done, aiming to spot artifacts in image2video-generated videos. The main focus is on detecting artifacts related to hand, face, and foot movements.
I have already annotated around 50,000 video-text pairs, where the video is the i2v generated video and the text is the corresponding artifact description.
However, I'm unsure how to construct the training data and train your model. Could you provide me with some better references or suggestions?

I will keep track of our work and provide feedback on my progress. Thank you!

[CodeGoat24]

Thanks for your support!

I suggest starting by designing the model input prompts based on your own data annotation schema to construct the training data.

You can refer to the input format used in our released training datasets.

Then, fine-tune our model using your data to observe the initial results.

We've just released UnifiedReward-qwen-7B, which is fully compatible with LLaMA-Factory, so you can fine-tune this model directly using that training framework.

[zzk88862]

ok, tks for your reply,

Once I have constructed the training data, can I execute bash dpo_video_understand_llava_video_7b.sh for fine-tuning? Are there any other parameter issues I need to pay attention to?

By the way, I noticed a potential issue with a parameter in dpo_video_understand_llava_video_7b.sh, should IMAGE_FOLDER be changed to VIDEO_FOLDER?