The clamp in-place operation cannot modify the weight_scales tensor directly.

[StarrickLiu]

>>> import torch
>>> a = torch.tensor(1.5367e-08, dtype=torch.float16)
>>> a
tensor(0., dtype=torch.float16)
>>> a.to(float).clamp(min=1e-5)
tensor(1.0000e-05, dtype=torch.float64)
>>> a
tensor(0., dtype=torch.float16)
>>> a = a.to(float).clamp(min=1e-5)
>>> a
tensor(1.0000e-05, dtype=torch.float64)

As shown in the code.

This bug caused NaN values to appear in the checkpoint obtained through SmoothQuant.

[syuoni]

Thanks @StarrickLiu , I'm integrating this fix to our internal repo.

[syuoni]

The fix has been integrated to our internal repo. It will be available on next Github update.

Thanks again for the contribution!