CheckpointSaver: incorrect labelling of final model

[cemde]

Describe the bug
When using a CheckpointSaver among the validation handlers, the argument save_final=True results in every epoch being stored under net_final_iteration=1.pt when final_filename=None. I interpret the docs (default to checkpoint_final_iteration=N.pt) that the file will be renamed after each completed epoch.

To Reproduce
add save_final=True to the CheckpointHandler in the Lung Segmentation Tutorial

Expected behavior
net_final_iteration=1.pt -> net_final_iteration=2.pt -> ... -> net_final_iteration=MAX_EPOCHS.pt

Screenshots
The logging function also prints incorrect iteration numbers:

INFO:ignite.engine.engine.SupervisedEvaluator:Train completed, saved final checkpoint: net_final_iteration=1.pt
...
INFO:ignite.engine.engine.SupervisedEvaluator:Train completed, saved final checkpoint: net_final_iteration=1.pt
...
INFO:ignite.engine.engine.SupervisedEvaluator:Train completed, saved final checkpoint: net_final_iteration=1.pt

Environment

Ensuring you use the relevant python executable, please paste the output of:

MONAI version: 0.3.0+95.g535561e
Python version: 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 18:42:56)  [Clang 10.0.1 ]
OS version: Darwin (19.6.0)
Numpy version: 1.19.2
Pytorch version: 1.7.0
MONAI flags: HAS_EXT = False, USE_COMPILED = False

Optional dependencies:
Pytorch Ignite version: 0.4.2
Nibabel version: 3.2.0
scikit-image version: NOT INSTALLED or UNKNOWN VERSION.
Pillow version: 8.0.1
Tensorboard version: 2.4.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.8.1
ITK version: NOT INSTALLED or UNKNOWN VERSION.
tqdm version: 4.53.0
lmdb version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

Additional context
Using SupervisedEvaluator and SupervisedTrainer.

[Nic-Ma]

Hi @cemde ,

Thanks for your interest here.
Actually, save_final is used for trainer to save the last checkpoint when training completed or crashed, not for validator to save checkpoint in every validation interval.
For your use case, you can put below handler to the validation handlers:

CheckpointSaver(save_final=False, save_interval=1, epoch_level=True, n_saved=1)

Thanks.

[Nic-Ma]

Hi @cemde ,

I submitted PR #1341 to enhance the save_final in CheckpointSaver handler, when PR merged, you can also use:

CheckpointSaver(save_final=True, final_filename="val_final_model.pt")

to achieve your requirement.

Thanks.