Hyper parameter for cifar10-vggsmall

[BaptisteNguyen]

Hello,
What are the hyper parameter for the training of vggsmall on cifar 10?

[haibao-yu]

Hello,
What are the hyper parameter for the training of vggsmall on cifar 10?

Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop.

[haibao-yu]

Hello,
What are the hyper parameter for the training of vggsmall on cifar 10?

Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop.

For ResNet20 on Cifar10, I have gotten Top-1 Accuracy 90.2% with 2bit for weight and 2bit for activation, compared to 91.8% with full precision. I think the results are quite good, which will be better with further hyperparameters tuning. Following are the experiment details:

1. to train the full precision as a pre-trained model: I set the initial learning rate as 0.1 with Cosine schedule and 160 epochs, weight decay as 1e-4, batch size as 128
2. to initialize the step size: I set 1.0 for activation as the README
3. to train the quantized model: I set the initial learning rate as 0.2 with Cosine schedule and 90 epochs, weight decay as 1e-4, batch size as 512

[BaptisteNguyen]

Hello, Thank you for your answer. I will test this. Baptiste Nguyen

…

________________________________ De : walk2out [notifications@github.com] Envoyé : samedi 9 janvier 2021 12:58 À : hustzxd/LSQuantization Cc : BaptisteNguyen; Author Objet : Re: [hustzxd/LSQuantization] Hyper parameter for cifar10-vggsmall (#2) Hello, What are the hyper parameter for the training of vggsmall on cifar 10? Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop. For ResNet20 on Cifar10, I have gotten Top-1 Accuracy 90.2% with 2bit for weight and 2bit for activation, compared to 91.8% with full precision. I think the results are quite good, which will be better with further hyperparameters tuning. Following are the experiment details: 1. to train the full precision as a pre-trained model: I set the initial learning rate as 0.1 with Cosine schedule and 160 epochs, weight decay as 1e-4, batch size as 128 2. to initialize the step size: I set 1.0 for activation as the README 3. to train the quantized model: I set the initial learning rate as 0.2 with Cosine schedule and 90 epochs, weight decay as 1e-4, batch size as 512 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQEMV62KVRAL3FRVNVZMUATSZBAHNANCNFSM4TW3UMQQ>.