implementation of masked autoencoder as a monai network

[Lucas-rbnt]

Description

Implementation of the Masked Autoencoder as described in the paper: Masked Autoencoders Are Scalable Vision Learners from Kaiming et al.

Masked autoencoders are a type of deep learning model that learn to reconstruct input data from partially masked or obscured versions of that data. They are structured to first "mask" or remove parts of the input image, and then attempt to reconstruct the missing pieces based solely on the available (unmasked) information.

This allows us to be much faster with fewer tokens being passed through the encoder. In addition, most representation learning methods are based on augmentations or different views of the same image and the quality of the representations depends heavily on the augmentations in question, which can be restrictive in the context of medical imaging. Masked autoencoders go some way towards overcoming this problem.
Its effectiveness has already been demonstrated in the literature for medical tasks in the paper Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation.

The PR concerns the implementation of this method and the associated tests (note currently the tests take 48seconds to pass that might be a bit long tell me).

So far 2D training on CIFAR data

went well

and 3d training on BraTS2021 data yields this

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[atbenmurray]

Taking a look

[atbenmurray]

@Lucas-rbnt

Please forgive the amount of time that it has taken to come back to you. We needed to review the situation with regards to licensing.

Unfortunately, we can't accept this PR in its present form. As the code has been adapted from github.com/facebookresearch/mae and that code is released under the CC BY-NC 4.0 license, it would cause significant complications for MONAI, which is released under Apache 2.0.

Although we appreciate the time and effort that has gone into this PR, any network implementation either needs to be licensed under a compatible license, or needs to be a clean-sheet implementation.

If you are still interested in implementing and submitting a masked autoencoder, either from clean-sheet or as an adaptation from a compatible license, please submit a new PR.