Skip to content

rocAL Serialize PR 4 : Pipeline serialization support#410

Merged
rrawther merged 134 commits intoROCm:developfrom
fiona-gladwin:fg/ser2_pipe_serialize
Dec 16, 2025
Merged

rocAL Serialize PR 4 : Pipeline serialization support#410
rrawther merged 134 commits intoROCm:developfrom
fiona-gladwin:fg/ser2_pipe_serialize

Conversation

@fiona-gladwin
Copy link
Copy Markdown
Contributor

@fiona-gladwin fiona-gladwin commented Nov 5, 2025

Motivation

This PR implements pipeline serialization functionality for rocAL, enabling pipelines to be serialized to protobuf format for persistence and portability.

Technical Details

  • Introduces PipelineSerializer class to handle serialization of pipeline configuration, operators, and tensors
  • Adds new public APIs rocalSerialize() and rocalGetSerializedString() for serializing pipelines
  • Defines protobuf schema for representing pipeline structure

Test Plan

No new tests added. Must pass existing tests

NOTE : To be merged after PR #401

fiona-gladwin and others added 30 commits September 22, 2025 02:30
Remove the usage of API enums in internal node files
Add support to register all internal enums
Creates an argument instance for each arg passed in the Node
Handles different argument types i.e parameters, vectors, enums and predefined types
Add support to serialize the args and pipeline details
Add support to serialize the tensors.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Create single constructor with check for different data types in Argument class
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Comment thread rocAL/include/pipeline/node.h
Copy link
Copy Markdown
Collaborator

@rrawther rrawther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please address the review comments

Comment thread rocAL/include/pipeline/tensor.h Outdated
@LakshmiKumar23
Copy link
Copy Markdown
Contributor

@fiona-gladwin please resolve conflicts

@fiona-gladwin
Copy link
Copy Markdown
Contributor Author

@rrawther Addressed all comments. Please check and let me know if I need to make any further changes
@LakshmiKumar23 Resolved merge conflicts

Comment thread rocAL/proto/rocal.proto Outdated
LakshmiKumar23
LakshmiKumar23 previously approved these changes Dec 10, 2025
rrawther
rrawther previously approved these changes Dec 15, 2025
LakshmiKumar23
LakshmiKumar23 previously approved these changes Dec 15, 2025
Comment thread CHANGELOG.md Outdated
@LakshmiKumar23 LakshmiKumar23 self-requested a review December 15, 2025 23:07
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 16, 2025

Codecov Report

❌ Patch coverage is 1.45631% with 203 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rocAL/source/pipeline/pipeline_serializer.cpp 0.00% 136 Missing ⚠️
rocAL/source/api/rocal_api.cpp 0.00% 31 Missing ⚠️
rocAL/include/pipeline/pipeline_operator.h 0.00% 22 Missing ⚠️
rocAL/source/pipeline/master_graph.cpp 0.00% 11 Missing ⚠️
rocAL/include/pipeline/master_graph.h 0.00% 1 Missing ⚠️
rocAL/include/pipeline/node.h 50.00% 1 Missing ⚠️
rocAL/include/pipeline/tensor.h 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #410      +/-   ##
===========================================
- Coverage    80.64%   79.90%   -0.74%     
===========================================
  Files          279      281       +2     
  Lines        21288    21491     +203     
===========================================
+ Hits         17167    17172       +5     
- Misses        4121     4319     +198     
Files with missing lines Coverage Δ
rocAL/include/api/rocal_api.h 50.00% <ø> (ø)
rocAL/include/pipeline/pipeline_serializer.h 100.00% <100.00%> (ø)
rocAL/include/pipeline/master_graph.h 96.88% <0.00%> (-0.51%) ⬇️
rocAL/include/pipeline/node.h 81.25% <50.00%> (ø)
rocAL/include/pipeline/tensor.h 71.37% <0.00%> (ø)
rocAL/source/pipeline/master_graph.cpp 88.16% <0.00%> (-0.69%) ⬇️
rocAL/include/pipeline/pipeline_operator.h 7.41% <0.00%> (-32.59%) ⬇️
rocAL/source/api/rocal_api.cpp 46.24% <0.00%> (-23.12%) ⬇️
rocAL/source/pipeline/pipeline_serializer.cpp 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rrawther rrawther merged commit bf4a17c into ROCm:develop Dec 16, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants