[inference] add reference and fix some bugs#4937

Xu-Kai merged 2 commits intohpcaitech:mainhpcaitech/ColossalAI:mainfrom

Xu-Kai:quant_refXu-Kai/ColossalAI:quant_refCopy head branch name to clipboard