Skip to content

Conversation

@Advaitgaur004
Copy link
Contributor

This PR introduces essential gradient clipping functionality to help stabilise training and prevent gradient explosion

New Functions:

  • cten_clip_grad_norm() - Clips gradients by global norm (L2)
// Clip gradients by their overall magnitude
void cten_clip_grad_norm(Tensor* params, int n_params, float max_norm);
  • cten_clip_grad_value() - Clips gradients to symmetric range [-max, max]
// Clip gradients to stay within [-max_value, max_value] 
void cten_clip_grad_value(Tensor* params, int n_params, float max_value);
  • cten_clip_grad_value_range() - Clips gradients to custom [min, max] range
// Clip gradients to custom range [min_value, max_value]
void cten_clip_grad_value_range(Tensor* params, int n_params, float min_value, float max_value);
  • cten_clip_grad_positive() - Clips only positive gradients above threshold
// Only clip gradients that are too positive (> max_value)
void cten_clip_grad_positive(Tensor* params, int n_params, float max_value);
  • cten_clip_grad_negative() - Clips only negative gradients below the threshold
// Only clip gradients that are too negative (< min_value)  
void cten_clip_grad_negative(Tensor* params, int n_params, float min_value);

@PrimedErwin PrimedErwin merged commit 6d8890f into pocketpy:test Aug 11, 2025
5 checks passed
@Advaitgaur004 Advaitgaur004 deleted the clipping branch August 19, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants