datasets train

[gaoyongyan]

Hello, I would like to ask you some questions about the KeyNet-pytorch project. We downloaded the ImageNet ILSVRC2012 data set, which contains the training set ILSVRC2012_img_train.tar, ILSVRC2012_img_train_t3.tar, the verification set ILSVRC2012_img_val.tar folder, we are in ILSVRC2012_img_train_t3 for the first time. Tar selected 9000 images, and selected 3000 images in ILSVRC2012_img_val.tar for training, but the training effect is not good, so I want to ask how your training data set is selected? Looking forward to your reply!

[bluedream1121]

It does not yield the same the tensorflow version of Key.Net, because of the differences by framework.(e.g. random seed,... )

Therefore, it would a little bit diffrenent performance compared to the Key.Net paper.

However, in my experience, If I use 4000 training set, 1500 validation set using ILSVRC2012, I obtained almost simliar scores of tensorflow version source code.

For training, I just use this command,

python train.py --data-dir ~/datasets/ImageNet2012/ILSVRC2012_img_val/

[gaoyongyan]

In this case, did you only use the ILSVRC2012_img_val validation set for training?

[bluedream1121]

Yes.

[gaoyongyan]

During the experiment, I would like to ask you the following questions and hope to give guidance:

1. The problem in the screenshot will appear in the data preprocessing stage. The reason I find is that there is insufficient memory, the memory is not released in time after data processing, and I can't find the relevant content of releasing memory in the program. Do you have this problem?
   
2. In addition, there is another detail that I don't quite understand. I'd like to ask you. There are 50000 pictures in the verification set. 9000 training sets and 3000 verification sets are selected. Why does the program select 4000 training and 1500 verification? And is the selection of data sets random or sequential?
3. Because of the above two problems, I selected 2500 for training and 500 for verification during training, and also experimented with other quantities, but the repeatability effect has not reached the effect of the original text, so I feel helpless that it is still the problem of data set selection. Thank you very much for your help!

[bluedream1121]

1. I did not experience that of problems. I use 32GB memory desktop to generate this synthetic data.

2. The official tensorflow implementation of Key.Net, they use 4000/1500 train/val dataset split. I follow that configuration.

3. Because of the difference of the random seed between pytorch and tensorflow, the repeatability score cannot be exactly same. Nevertheless, I achieve the repeatability score and matching score alomst same compared to the trained model using the official released code (https://github.com/axelBarroso/Key.Net).