[inference]fix import bug and delete down useless init#4830

Xu-Kai merged 4 commits intohpcaitech:mainhpcaitech/ColossalAI:mainfrom

CjhHa1:fix_inferCjhHa1/ColossalAI:fix_inferCopy head branch name to clipboard