sincos Positional Embedding

[KumoLiu]

For example, this github has sincos positional embedding for Masked Autoencoder implementation for medical imaging. Adding 2D/3D positional information is most commonly used for ViT training. If you create an issue, I can try to contribute to it.

https://github.com/cvlab-stonybrook/SelfMedMAE/blob/68d191dfcc1c7d0145db93a6a570362de29e3b30/lib/models/mae3d.py#L15C5-L15C5

Originally posted by @NoTody in #6966 (reply in thread)