Skip to content

fix(dvcr): VMDK import error#2065

Merged
hardcoretime merged 4 commits into
mainfrom
fix/dvcr/vmdk-footer-err
Mar 16, 2026
Merged

fix(dvcr): VMDK import error#2065
hardcoretime merged 4 commits into
mainfrom
fix/dvcr/vmdk-footer-err

Conversation

@hardcoretime
Copy link
Copy Markdown
Contributor

@hardcoretime hardcoretime commented Mar 5, 2026

Description

This PR fixes the issue with importing VMDK disk images into DVCR (Deckhouse Virtualization Container Registry).

Technical changes:

  1. Early format detection: Modified getImageInfo() to detect VMDK format by reading the first 10MB and checking the magic number using image.Header.Match().

  2. VMDK-specific processing: Implemented getImageInfoVMDK() that creates a synthetic sparse VMDK file from:

    • Head buffer (10MB) - contains VMDK header and initial metadata
    • Tail buffer (50MB) - contains footer, Grain Directory, and Grain Tables
    • Sparse hole in the middle - actual disk data (not needed for qemu-img info)
  3. TailBuffer implementation: Added ring buffer that efficiently stores only the last N bytes from a stream without keeping the entire file in memory.

  4. Better error diagnostics: Changed cmd.Output() to cmd.CombinedOutput() to capture stderr from qemu-img for debugging.

  5. Separated processing paths:

    • getImageInfoVMDK() - for VMDK format using synthetic file approach
    • getImageInfoStandard() - for other formats (qcow2, iso, raw) using existing approach

Resource efficiency:

  • VMDK images: Fixed 60MB memory footprint (10MB head + 50MB tail ring buffer) regardless of source VMDK size. Synthetic sparse file uses ~60MB disk space.
  • Other formats: No changes, uses existing 64MB temp file approach.
  • Benefit: Can process multi-gigabyte VMDK images with constant low memory usage.

Why do we need it, and what problem does it solve?

Problem:

When importing VMDK disk images (especially streamOptimized format commonly used in OVF/OVA exports from VMware), the import failed with error:

qemu-img: Could not open '/tmp/tempfile': Invalid footer
exit status 1

What is the expected result?

VMDK images import successfully.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: core
type: fix
summary: "Fixed the creation of block devices from VMDK files (especially for VMDKs in the `streamOptimized` format used in exports from VMware)."

@hardcoretime hardcoretime added this to the v1.7.0 milestone Mar 5, 2026
@hardcoretime hardcoretime force-pushed the fix/dvcr/vmdk-footer-err branch 3 times, most recently from 5a2ea3f to cc947e1 Compare March 5, 2026 13:24
Signed-off-by: Roman Sysoev <roman.sysoev@flant.com>
@hardcoretime hardcoretime force-pushed the fix/dvcr/vmdk-footer-err branch from 9a8f547 to eff959a Compare March 10, 2026 10:51
Signed-off-by: Roman Sysoev <roman.sysoev@flant.com>
@hardcoretime hardcoretime force-pushed the fix/dvcr/vmdk-footer-err branch from eff959a to 4508e61 Compare March 10, 2026 12:25
@hardcoretime hardcoretime marked this pull request as ready for review March 10, 2026 13:19
@hardcoretime hardcoretime changed the title fix(dvcr): vmdk footer err fix(dvcr): vmdk import error Mar 10, 2026
@hardcoretime hardcoretime changed the title fix(dvcr): vmdk import error fix(dvcr): VMDK import error Mar 10, 2026
Signed-off-by: Roman Sysoev <roman.sysoev@flant.com>
@hardcoretime hardcoretime added e2e/run Run e2e test on cluster of PR author labels Mar 10, 2026
@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@hardcoretime hardcoretime added the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Mar 10, 2026

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: cancelled.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@hardcoretime hardcoretime added the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Mar 10, 2026

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: cancelled.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@hardcoretime hardcoretime added the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Mar 10, 2026

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: failure.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Mar 10, 2026
Comment thread images/dvcr-artifact/pkg/registry/registry.go Outdated
Comment thread images/dvcr-artifact/pkg/registry/registry.go Outdated
Comment thread images/dvcr-artifact/pkg/registry/registry.go Outdated
Comment thread images/dvcr-artifact/pkg/registry/registry.go Outdated
Comment thread images/dvcr-artifact/pkg/registry/registry.go Outdated
Signed-off-by: Roman Sysoev <roman.sysoev@flant.com>
@hardcoretime hardcoretime force-pushed the fix/dvcr/vmdk-footer-err branch from cd3460b to cd1fe8a Compare March 16, 2026 01:31
@hardcoretime hardcoretime requested a review from diafour March 16, 2026 08:04
@hardcoretime hardcoretime merged commit 181a0e3 into main Mar 16, 2026
27 of 28 checks passed
@hardcoretime hardcoretime deleted the fix/dvcr/vmdk-footer-err branch March 16, 2026 17:01
This was referenced Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants