This repo contains a PyTorch implementation of the HALP optimizer from the paper High-Accuracy Low-Precision Training and a SVRG optimizer.
This repo is duplicated from the source code of PyTorch-based HALP repo from Megan Leszczynski under permission. This repo is only used for the Low-precision Random Fourier Feature project.
For HALP repo with up-to-date maintainance, please refer to HazyResearch/torchhalp.