Skip to content

refactor: Offload AES encryption/decryption to Web Worker#3377

Open
mondoreale wants to merge 9 commits intoworker-based-signingfrom
worker-based-encryption
Open

refactor: Offload AES encryption/decryption to Web Worker#3377
mondoreale wants to merge 9 commits intoworker-based-signingfrom
worker-based-encryption

Conversation

@mondoreale
Copy link
Contributor

@mondoreale mondoreale commented Jan 30, 2026

Summary

Offload CPU-intensive AES-256-CTR encryption/decryption to a Web Worker using Comlink, improving main thread responsiveness during stream message processing. This PR also fixes several Buffer vs Uint8Array serialization issues that caused test failures on Linux CI due to inconsistent binary type handling across the worker boundary.

Changes

@streamr/sdk

  • Add EncryptionService singleton with lazy worker initialization
  • Add EncryptionWorker with handlers for encrypt, decrypt, and group key operations
  • Extract low-level AES utilities to aesUtils.ts with concatBytes() helper to avoid Buffer.concat()
  • Add platform-specific createEncryptionWorker.ts for browser/Node.js/Jest/Karma environments
  • Update MessageFactory and decrypt.ts to use EncryptionService
  • Simplify EncryptionUtil.ts to delegate to shared aesUtils.ts
  • Fix StreamMessageTranslator.ts to convert protobuf binary fields (signature, content, newGroupKey.data) to plain Uint8Array after deserialization
  • Add createMockEncryptionService() test utility for synchronous test execution
  • Remove unused EncryptionService.decryptWithAES method

@streamr/utils

  • Fix EcdsaSecp256k1Evm.createSignature() to return plain Uint8Array instead of Buffer
  • Fix hexToBinary() to return plain Uint8Array with upfront hex validation

Watch Out

  • Test timeouts: Worker loading adds latency on first encryption/decryption call. Tests that previously had tight timeouts may need adjustment. The key request timeout was bumped in this PR to accommodate worker initialization time.
  • Binary type consistency: All public APIs now return Uint8Array as declared. Code that relied on Buffer-specific methods (like .toString('hex')) may need updates to use binaryToHex() or similar utilities.

Note

Medium Risk
Touches core message publish/decrypt paths and introduces worker-based crypto plus binary type conversions; failures could break encryption/decryption or cross-env serialization, though changes are mostly additive with good test coverage.

Overview
Moves AES-256-CTR stream message encryption/decryption (including group key wrapping/unwrapping) off the main thread by introducing EncryptionService + a Comlink-exposed EncryptionWorker, and rewiring publish (MessageFactory) and subscribe (decrypt, messagePipeline) flows to use it.

Extracts shared AES primitives into aesUtils.ts and introduces encryptionUtils.ts request/response shapes for worker communication, while removing AES/group-key helpers from EncryptionUtil/GroupKey and narrowing EncryptionUtil to asymmetric key-exchange + ML-KEM wrapping.

Hardens cross-environment binary handling by normalizing protobuf content/signature/newGroupKey.data to plain Uint8Array, returning Uint8Array from createSignature, and reimplementing hexToBinary to validate input and avoid Buffer.

Updates build/test plumbing to bundle/alias the new worker across Node/browser/Jest/Karma, adds a synchronous createMockEncryptionService() for tests, and adjusts a key-request timeout to account for worker startup latency.

Written by Cursor Bugbot for commit 3208dca. This will update automatically on new commits. Configure here.

@github-actions github-actions bot added the sdk label Jan 30, 2026
@mondoreale mondoreale force-pushed the worker-based-encryption branch from 49a0bc6 to be2c78a Compare February 2, 2026 12:07
…r boundary

Fix comlink serialization by returning Uint8Array from aesUtils

aesUtils now returns plain Uint8Array instead of Buffer, fixing
'Unserializable return value' errors on Linux CI where Buffer
objects don't serialize correctly through comlink's structured clone.
@mondoreale mondoreale force-pushed the worker-based-encryption branch from be2c78a to cf7ac0c Compare February 2, 2026 12:34
Matches the declared return type and ensures proper serialization
across environments (e.g., through comlink workers).
Ensures signature, content, and newGroupKey.data are plain Uint8Array
after fromBinary(), fixing Buffer serialization issues in tests.
@github-actions github-actions bot added the utils label Feb 2, 2026
Validates hex characters upfront with regex before parsing.
Now that both `createSignature` and `hexToBinary` return plain `Uint8Array`,
we can use strict equality comparison again.
@mondoreale mondoreale marked this pull request as ready for review February 2, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants