Add dump-state command to write WinDbg-compatible .vmrs dump files#3523
Open
jstarks wants to merge 85 commits into
Open
Add dump-state command to write WinDbg-compatible .vmrs dump files#3523jstarks wants to merge 85 commits into
dump-state command to write WinDbg-compatible .vmrs dump files#3523jstarks wants to merge 85 commits into
Conversation
Add the foundation crates for VMRS dump file support: hvdef::save_restore - Hypervisor chunk definitions (VM_SAVE_CHUNK_ID, chunk header, VID_SAVED_STATE_DESCRIPTOR, prolog/epilog, VP indices, VP/VTL markers, OsId, and all x64/ARM64 register chunk structs for GP, control, debug, segment, table, and FP registers). These are zerocopy structs matching the canonical layout from the hypervisor's save/restore format. hvs_file - New crate implementing the HyperVStorage binary key-value file format used by .vmrs/.vmcx/.vsv files. Includes on-disk structure definitions (file header, object table, key table entries), a writer that builds files in a single sequential pass with typed values (Int, UInt, String, Array, Bool, Node) and file objects for large blobs, and a reader that parses the key hierarchy back. Round-trip tests verify all value types including file objects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Verified all struct layouts against the canonical source code in onecore/vm/common/hypervstorage/ and onecore/vm/tools/vm2dmp/dll/common/ hv/HvDuplicatedTypes.h. Fixed several discrepancies: VmSaveChunkHeader is DECLSPEC_ALIGN(16), making sizeof = 16 bytes (not 8). This was the most impactful fix as it affected alignment and padding in every containing chunk struct. All chunk structs now include explicit trailing padding to match the C sizeof with 16-byte alignment. OB_SAVE_CHUNK_VTL.Vtl is HV_VTL (UINT8), not UINT32. Changed both ObSaveChunkVtl and ObSaveChunkPartitionVtl to use u8. VSM_SAVE_CHUNK_VP_VTL_CONTROL_PAGE contains VpAssistPageVtlControlContents [24] + VtlIsRunnable (BOOLEAN), not a simple active_vtl u32. Replaced with correct struct VsmSaveChunkVpVtlControlPage. FP register structs no longer need explicit 8-byte padding after the header since the 16-byte header already provides natural alignment for the HV_UINT128 fields. Fixed key entry checksum offset from byte 10 to byte 12 in the writer (Type:1 + Flags:1 + Size:4 + ParentNodeTable:2 + ParentNodeOffset:4 = 12 bytes before the checksum field). Updated ROOT_NODE_ENTRY_SIZE from 32 to 33 (21-byte header + 12-byte NodeData). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a cross-validation test that writes a .vmrs file using hvs_file, then loads it with VmSavedStateDumpProvider.dll from the Windows SDK. The test uses pal's delayload mechanism and skips gracefully when the DLL isn't available. Fixed a critical bug where both header copies had sequence number 1. HyperVStorage treats identical sequence numbers as file corruption. Now copy 0 has sequence 1 (authoritative) and copy 1 has sequence 0. The DLL cross-validation currently reports E_INVALIDARG during file loading, indicating a remaining format mismatch in the key table structure. The test saves the file for manual inspection and returns without failing, as the scaffolding is in place for iterative debugging. Also added a diagnostic write_debug_vmrs test that dumps the full file structure (object table entries, key table headers, entry tree) to help debug format issues. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three fixes to the HVS file writer based on analysis of a real .vmrs file produced by Hyper-V: The root node is virtual — it lives in memory as m_RootNode, never stored in any key table. Parent reference (0, 0) is the sentinel meaning 'child of root'. Previously we were writing an explicit root node entry in the key table, which doesn't match the real format. Key table indices start at 1, not 0. Index 0 is reserved for the virtual root node's sentinel. Updated both the writer (table_index field in key table headers and parent references) and reader (node_path_map uses actual table index from header). Added OBJECT_ENTRY_FLAG_REQUIRED (0x01) to key table object table entries and proper checksums on empty chain-slot entries. Added a dll_binding_sanity_check test that loads the reference config VMRS through the Rust DLL bindings and confirms it gets the expected VM_SAVED_STATE_DUMP_E_PARTITION_STATE_NOT_FOUND error, proving the DLL binding works correctly. Our generated files still get E_INVALIDARG, indicating a remaining format difference to debug. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three fixes based on source code analysis and comparison with a real saved state VMRS file: Write an empty replay log region. HyperVStorage's InitializeForLoad unconditionally dereferences the replay log header buffer, causing UB when the header size is 0. Write a minimal valid replay log with signature 0x01110003, correct MaximumNumberOfEntries, and proper CRC. This fixed the E_INVALIDARG error. Free key table entries must have checksum 0. The source code's CalculateChecksum method returns 0 for Free entries (the CRC block is skipped). Previously we computed a CRC which failed validation. Fill the object table to full capacity (227 entries for 4096-byte alignment) to prevent the DLL from expanding it during load, which would require writing to a read-only file. Added tests: read_real_saved_state (verifies our reader parses a real VMRS), roundtrip_real_vmrs_through_writer (reads real partition state and rewrites via our writer), dll_loads_real_saved_state (confirms DLL loads the real VMRS successfully with correct VP count and arch). DLL status: progressed from E_INVALIDARG to ERROR_WRITE_PROTECT (0x80070013), indicating the file structure now loads but something triggers a write-back that fails on the read-only file handle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix three issues in the HVS file writer that prevented VmSavedStateDumpProvider.dll from loading files we produce: 1. NodeData.next_insertion_sequence was always 0. The DLL's CompleteMapUpdate checks this field and updates it if wrong, triggering DataChanged and a Commit() that fails on read-only files (ERROR_WRITE_PROTECT). Now set to the actual child count. 2. Insertion sequences were 0-based. The DLL's AddChild treats InsertionSequence==0 as uninitialized and reassigns it, again triggering DataChanged. Now 1-based to match production files. 3. Key table entries could leave a gap of 1-20 bytes at the end — too small for a Free entry (21 bytes minimum). The DLL's Verify requires entries to exactly fill the table (ERROR_FILE_CORRUPT). Now spills entries to the next table when the remainder would be too small. Also fix the cross_validate_with_dll test's RamMemoryBlock0 struct to use the correct 48-byte size (with alignment padding) instead of 40 bytes. Add a full round-trip test that reads all 1113 keys from the reference saved state VMRS, writes them through our writer, and verifies the DLL loads the result with correct VP count and architecture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the ROOT_NODE_ENTRY_SIZE constant — the root node is virtual (not stored on disk), so the constant was misleading and unused. Clean up cross_validate test: remove stale round-trip file probing, remove speculative DLL_LOCK (the DLL's global DumpProviderManager properly serializes with an SRW lock; file loading is concurrent-safe), and restore the panic on DLL error so failures are not silently hidden. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The DLL's key table verifier walks entries with a strict less-than loop: while (offset + sizeof(EntryHeader) < dataEnd). This means it cannot reach an entry starting at exactly dataEnd - 21. When the writer left a gap of exactly 21 bytes and filled it with a Free entry, the DLL's loop stopped early and the offset != dataEnd check failed, returning ERROR_FILE_CORRUPT. Fix the gap threshold from < to <= so that a remaining gap of exactly sizeof(EntryHeader) bytes is treated as too small, forcing the preceding entry to the next key table. The larger gap left behind gets a Free entry the DLL can actually reach. Add a unit test that generates 300 keys with varying data sizes to exercise many table-fill patterns and verifies no entry overflows its key table boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add /configuration/properties/version key to the synthetic test file so older DLL versions can find the VM version. Add type and VmwpVersion keys for completeness. Add an ignored dump_real_keys test for manual inspection of reference VMRS contents. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Make defs module pub(crate) — no ABI types leak to consumers - Remove public KeyType re-export; add KeyValue enum to reader as the public interface for reading typed values - Add read_value() that returns KeyValue, removing key_type() and is_file_object() from the public API - Add add_value() on the writer that accepts KeyValue - Move crc32 to lib.rs as pub(crate), remove from writer's public API - Make Value enum private to the writer - add_array() now auto-promotes to file object at >= 2048 bytes, matching Hyper-V's ShouldUseFileObject behavior - Add missing docs on all public types and remove expect(missing_docs) - Fix all warnings Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove KeyValue, read_value, and add_value. Replace with: - ValueType enum (Int/UInt/String/Array/Bool) for type inspection - value_type() method on reader - Typed read_* methods remain the primary read interface - read_array transparently handles file objects (no separate read_file_object) - add_file_object is now private (add_array auto-promotes) The writer and reader are now fully decoupled — no shared types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…prefixes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace HashMap with BTreeMap for the reader's key store. keys() now returns keys in sorted order without the caller needing to sort. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use FromZeros::new_zeroed() + as_mut_bytes() to read file headers, object table headers, and object table entries directly into typed structs on the stack instead of bouncing through byte arrays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…g None Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Store ValueType and is_file_object in KeyEntry instead of the on-disk KeyType and raw flags byte. Unknown types are rejected during parsing, not deferred to value_type(). WrongKeyType error now reports ValueType instead of a raw u8 discriminant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use crc32fast's streaming API to hash around the checksum field instead of zeroing and restoring it. Eliminates temporary Vec allocations at every checksum call site. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move struct_checksum to lib.rs as pub(crate). Reader's verify_header_checksum now uses it directly instead of allocating a Vec to zero the checksum field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… sorted order Remove the insertion_sequences HashMap field. Pending keys are sorted by path in finish(), so children of the same parent are contiguous. Insertion sequences are assigned by counting children per parent in a single pass over the sorted entries. Node locations use a BTreeMap instead of HashMap. The overall structure is now: collect node paths (BTreeSet), build entries, count children, assign sequences, layout into key tables — all without HashMap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of pre-collecting all node path prefixes into a BTreeSet, emit node entries during the single pass over sorted pending keys. When the ancestor path for a key hasn't been seen yet, emit its node entry inline. The emitted_nodes set only tracks which nodes have been created (for dedup), not all possible prefixes upfront. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the emitted_nodes BTreeSet with a stack that tracks the current position in the tree. Pop on divergence, push on descent — node entries are emitted exactly once as the stack grows. Remove the redundant parent_path field from EntryData; parent is computed from path by trimming the last segment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Encode UTF-16LE directly into the output buffer with a placeholder length prefix, then patch it. One allocation instead of two. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Iterate pending_keys by value instead of by reference, moving path and value out of each PendingKey instead of cloning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Children of the same parent are contiguous after sorting, so a single counter that resets on parent change assigns insertion sequences without any map. Node next_insertion_sequence is set by scanning forward from each node to count direct children. The only remaining BTreeMap is node_locations, which is needed for parent pointer fixup during key table layout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assign insertion sequences during the key table layout loop instead of a separate pass. Reuse the computed parent path for node_locations lookup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Borrow the rest slice separately to avoid needing to clone the node path for the inner comparison loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move insertion sequence assignment into the first pass where entries are created. The layout pass now only handles key table placement, parent pointer fixup, and checksums. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Parent pointers are resolved from a stack of ancestor (path, table, offset) tuples. Since nodes always precede their children in the sorted entry list, the parent is always on the stack. No collections remain in the writer — zero HashMap, BTreeMap, or BTreeSet. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add proper zerocopy structs for the replay log header and entry header in defs.rs. Rewrite the replay log emission in the writer to use the typed struct instead of manual byte offset manipulation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
smalis-msft
reviewed
May 19, 2026
smalis-msft
reviewed
May 19, 2026
Sort workspace dependency entries by top-level folder. Simplify DumpState closure and key-table parent stack-pop condition. Add assert for vp_stop_count invariant in resume_vps.
mattkur
reviewed
May 19, 2026
mattkur
reviewed
May 19, 2026
mattkur
reviewed
May 19, 2026
mattkur
reviewed
May 19, 2026
mattkur
reviewed
May 19, 2026
mattkur
reviewed
May 19, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 26 out of 27 changed files in this pull request and generated 8 comments.
Comments suppressed due to low confidence (2)
vmm_core/hyperv_dump/src/vmrs_writer.rs:116
RamBlock*values are written withblock_len = min(1MiB, remaining), so the final block of a range can be < 1MiB. Persupport/hvs_file/FORMAT.md, sizes < 1MiB are interpreted as XPRESS-compressed, so emitting a short uncompressed tail block will likely be misread by consumers. Consider always writing exactly 1 MiB perRamBlock(zero-fill the remainder whenremaining < 1MiB) or implement XPRESS compression for short blocks.
while gpa < gpa_end {
let block_len = DATA_BLOCK_SIZE.min((gpa_end - gpa) as usize);
let buf = &mut block_buf[..block_len];
reader.read_gpa(gpa, buf)?;
key_buf.clear();
write!(key_buf, "/savedstate/RamBlock{data_block_idx}").unwrap();
self.hvs.add_array(&key_buf, buf)?;
data_block_idx += 1;
vmm_core/hyperv_dump/src/vmrs_writer.rs:90
finish()can produce a.vmrswith zeroRamMemoryBlock*entries when no memory ranges are added. The format doc in this PR (support/hvs_file/FORMAT.md) states/savedstate/RamMemoryBlock0is a minimum required key for the dump provider to open the file. Either enforce at least one memory range (returnInvalidInput) or write the required minimal memory metadata so the writer can’t generate an unloadable file.
/// Writes the complete `.vmrs` file, reading guest memory on demand.
///
/// `partition_state` is the blob from [`crate::PartitionStateBuilder::finish`].
/// Memory is streamed through a reusable 1 MiB buffer — at no point
/// is the entire guest address space materialized in memory.
pub fn finish(
mut self,
partition_state: &[u8],
reader: &mut dyn GuestMemoryReader,
) -> io::Result<W> {
// VM version
self.hvs.add_int("/savedstate/VmVersion", VM_VERSION);
self.hvs
.add_int("/configuration/properties/version", VM_VERSION);
// Partition state
self.hvs
.add_array("/savedstate/savedVM/partition_state", partition_state)?;
// Memory layout: split ranges into 1 MiB blocks, streaming each
// block through a reusable buffer.
let mut data_block_idx = 0u64;
let mut block_buf = vec![0u8; DATA_BLOCK_SIZE];
let mut key_buf = String::new();
for (i, range) in self.ranges.iter().enumerate() {
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds the ability to dump a running VM's processor state and memory to a
.vmrsfile that can be opened with WinDbg viaVmSavedStateDumpProvider.dll. This is useful for offline debugging of guest crashes without needing a live debugger attached.The implementation is split across three layers:
hvs_fileis a new standalone crate that implements the HyperV Storage binary key-value file format used by.vmrs,.vmcx, and.vsvfiles. It provides both a reader and writer, with no VMM dependencies. The format is documented in FORMAT.md.hyperv_dumpbuilds onhvs_fileto assemble complete.vmrsdump files. It constructs the hypervisor save/restore chunk stream fromvirtregister types then wraps it with memory block metadata and streams guest RAM through a caller-providedGuestMemoryReadertrait. Tests validate that this crate produces files that are compatible with the Hyper-V dump provider DLL.vmm_core / openvmm integration exposes dump generation through
VmRpc::DumpStateand adump-stateREPL command. The partition unit stops VPs and builds the partition state blob; the worker streams guest memory to disk. Thehyperv_dumpdependency in vmm_core is gated behind adumpcargo feature, enabled for OpenVMM but not OpenHCL.New hypervisor save/restore chunk definitions are added to
hvdef::save_restorefor the on-disk structures (chunk headers, register chunks for x64 and ARM64, VTL markers, etc.).