Skip to content

Conversation

@Advaitgaur004
Copy link
Contributor

Added L2 regularisation (weight decay) to SGD, AdaGrad, RMSProp, and Adam optimisers. Each optimizer now takes an additional weight_decay parameter in its constructor.

  • Modified all optimiser constructors to accept a weight decay parameter
  • Gradient calculation now includes the weight_decay * param_value term

Quick Eg.

// Before: optim_sgd_new(n_params, params)
// After: optim_sgd_new(n_params, params, 0.01f)  // 0.01 weight decay

Should help keep those models from going crazy on the training data!!! 😮‍💨

@PrimedErwin PrimedErwin merged commit 889df03 into pocketpy:test Aug 11, 2025
5 checks passed
@Advaitgaur004 Advaitgaur004 deleted the weight-decay branch August 19, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants