🐛 Bug: GovTool Metadata Hash Validation Ignores extra byte in the Manually Created Files

[bosko-m]

Area

Other

Domain

gov.tools

Which wallet were you using?

No response

Context

GovTool allows users to draft a proposal, collaborate with the community, and ultimately submit it on-chain as a Governance Action. Each Governance Action requires CIP-108-compliant metadata, which GovTool provides as a downloadable .jsonld file during submission.

Users can store this file on GitHub, typically via:

• Option A: Add file → Upload files (file remains unchanged; validation passes)
• Option B: Add file → Create new file (manual paste from clipboard; GitHub appends a byte 0x0A)

Currently, if the user chooses Option B, the resulting file has a 1-byte difference due to the extra byte 0x0A. In this case, GovTool is expected to fail metadata validation, as the hash should no longer match.

However, due to the JSON parsing library used by GovTool, the extra byte appears to be ignored when parsing the content and generating the hash. As a result, GovTool incorrectly validates the modified file as correct, allowing submission of a Governance Action with a hash that should be invalid.

Steps to reproduce

1. Create a proposal and proceed to Governance Action submission.
2. Download the metadata .jsonld file generated by GovTool.
3. Open GitHub repository and use Add file → Create new file.
4. Paste the content from the .jsonld file into the editor (GitHub appends extra byte 0x0A).
5. Submit the Governance Action using the URL of this manually created file.

Actual behavior

• GovTool accepts the metadata file with the extra byte 0x0A.
• The resulting hash matches the original, despite the file being technically different.
• Governance Action is successfully submitted with incorrect file integrity.

Expected behavior

• GovTool should perform exact byte-level hashing of the metadata file.
• Metadata with any deviation (e.g., trailing whitespace) should cause validation to fail.
• Governance Action submission should be blocked if the metadata hash does not match exactly.

[gitmachtl]

The issue with the governance action gov_action1fpqwxp2kxvnntr8hpkh9q9djm78ccdww7qlhg5safugh4stmcwzqql5lauu on mainnet is a cutoff of the last byte in the file provided via the URL link. In this case this is a 0x0a or LineFeed. For some reason, gov.tools calculate the hash without that last byte.

[brouwerQ]

When I download a gov metadata file from gov tools, there doesn't seem to be a newline in it, so the newline would be introduced afterwards.
The validation code in https://github.com/IntersectMBO/govtool/blob/develop/govtool/metadata-validation/src/app.service.ts seems to do a hash calculation on raw data first and when no match, it falls back to parsing the JSON, converting it back to a string in a canonical form and comparing the hash of this.
Because the original file will also be generated using this same canonical form, this matches.

Just skipping the fallbacks in case of generating txs (or maybe just in all cases), will fix this I think.

[bosko-m]

The issue with the governance action gov_action1fpqwxp2kxvnntr8hpkh9q9djm78ccdww7qlhg5safugh4stmcwzqql5lauu on mainnet is a cutoff of the last byte in the file provided via the URL link. In this case this is a 0x0a or LineFeed. For some reason, gov.tools calculate the hash without that last byte.

@gitmachtl Thx Martin, edited to be more precise.

@brouwerQ Also Tom, yes, exactly the place we were looking at. Thank you.

[gitmachtl]

ok, so the issue is actually the "fallback" to use the canonized hash as verification.

govtool/govtool/metadata-validation/src/app.service.ts

Lines 81 to 117 in f282593

	const hashedMetadata = blake.blake2bHex(rawData, undefined, 32);
	if (hashedMetadata !== hash) {
	// Optionally validate on a parsed metadata
	const hashedParsedMetadata = blake.blake2bHex(
	JSON.stringify(parsedData, null, 2),
	undefined,
	32,
	);
	if (hashedParsedMetadata !== hash) {
	// Optional support for the canonized data hash
	// Validate canonized data hash
	const canonizedMetadata = await jsonld.canonize(JSON.parse(rawData), {
	safe: false,
	});
	const hashedCanonizedMetadata = blake.blake2bHex(
	canonizedMetadata,
	undefined,
	32,
	);
	if (hashedCanonizedMetadata !== hash) {
	throw MetadataValidationStatus.INVALID_HASH;
	}
	}
	}
	} catch (error) {
	Logger.error(LoggerMessage.METADATA_VALIDATION_ERROR, error);
	if (Object.values(MetadataValidationStatus).includes(error)) {
	status = error;
	}
	}
	return { status, valid: !Boolean(status), metadata };
	}
	}

once the check on the rawData hash failed, it compares the canonized hash. which is a wrong usecase in this scenario. because canonization was especially designed to deliver the same result every time and eliminate the issues with linebreaks/linefeeds or even a different sorting of keys in a json.

thats the reason this did not trigger, because the canonized hash would be ok. but not the raw file hash.

so the most simple solution is to get rid of the extra step to compare the canonized hash and error out if just the rawData hash does not match.

EDIT: just saw @brouwerQ posted the same above - lol