This experiment was inspired by my (so far partial) lecture on Nested Learning: The Illusion of Deep Learning Architecture. The basic idea (at least what I understood) to motivate this experiment is that the Momentum optimizer mechanic is a long-term memory mechanism.