Skip to content

Conversation

@Advaitgaur004
Copy link
Contributor

Just added a few more optimisers to have some more powerful options for training! This PR brings in AdaGrad, RMSProp, and the classic Adam.

  • Added the new optimizers (AdaGrad, RMSProp, Adam) to the main optimizer file. I put them all in src/optimizer/sgd.c for now, but we can totally break them into separate files later if we want.
  • Updated cten.h with the new function declarations so they can be used across the project.
  • Kept SGD as is, so nothing breaks there.

@Advaitgaur004
Copy link
Contributor Author

to test the implemenation use this ==> https://gist.github.com/Advaitgaur004/e0ad0e99f1e4514bd6091f3cce10b23a

@Advaitgaur004
Copy link
Contributor Author

image

As expected, AdaGrad diminishes the learning rate, and after 300 iterations, it is about to reach the destination, but not entirely.

@Advaitgaur004
Copy link
Contributor Author

Choosing an optimizer can be tricky....many people default to Adam, but sometimes SGDM performs better.
Refer to this --> Here
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PrimedErwin PrimedErwin merged commit 00ef857 into pocketpy:test Aug 4, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants