Skip to content

Add linear_ layer for neural networks#3074

Merged
davisking merged 3 commits intodavisking:masterfrom
Cydral:master
May 3, 2025
Merged

Add linear_ layer for neural networks#3074
davisking merged 3 commits intodavisking:masterfrom
Cydral:master

Conversation

@Cydral
Copy link
Contributor

@Cydral Cydral commented Apr 28, 2025

This PR adds a new linear_ layer that provides an optimized linear transformation for multi-dimensional inputs. The layer preserves the spatial dimensions of the input tensor while transforming the feature dimension.

Features

  • Supports both biased and unbiased linear transformations
  • Optimized for maintaining tensor dimensions (except for feature dimension)
  • Compatible with existing DNN API patterns and serialization

This implementation is more efficient than reshaping inputs to use fc_ layers when working with multi-dimensional data.
Similar functionality exists in other major frameworks like PyTorch (nn.Linear), which operates on the last dimension while preserving other dimensions.

Cydral added 2 commits April 28, 2025 22:10
…des an optimized linear transformation for multi-dimensional inputs.
@davisking davisking requested a review from Copilot May 3, 2025 14:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new linear_ layer that provides an optimized linear transformation for multi-dimensional inputs while preserving spatial dimensions except for the feature one. The implementation supports both biased and unbiased modes and is integrated into the DNN API with corresponding test cases added.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
dlib/test/dnn.cpp Added multiple tests exercising linear_ layer instantiations (e.g. linear_<1, LINEAR_NO_BIAS>) and a dedicated test_linear() function to validate outputs and gradients.
dlib/dnn/layers_abstract.h Introduced the abstract definition for the new linear_ layer and the associated linear_bias_mode enum with clear documentation.
dlib/dnn/layers.h Provided concrete implementations of the linear_ layer, including input/output mapping and wrapper aliases (linear and linear_no_bias).
Comments suppressed due to low confidence (1)

dlib/test/dnn.cpp:3561

  • [nitpick] Consider renaming 'xtmp' to a more descriptive name (like 'input_matrix') to improve code clarity in the test_linear() function.
matrix<float> xtmp(2, 4);

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@davisking
Copy link
Owner

Nice, thanks for the PR 😁

@davisking davisking merged commit 55134f1 into davisking:master May 3, 2025
9 of 10 checks passed
davisking pushed a commit to kSkip/dlib that referenced this pull request May 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants