refactor: trim output allocations and unify unknown-field sort#96
Merged
Conversation
Stream output into a single byte buffer (BOM bytes prepended, serde_json
appends, one `from_utf8_unchecked` at the end) instead of building a
String, pushing '\n', and re-allocating to prepend the BOM. Saves a full
output-sized memcpy on BOM inputs and the post-serialize UTF-8 walk in
general.
Collapse `non_private` and `private` into a single `unknown` Vec sorted
by `(starts_with('_'), name)` — one allocation, one sort, and the
underscore check leaves the per-entry hot path.
Replace `sort_object_by_key_order`'s `obj.sort_keys()` + per-key
`shift_remove` (each shift moves the IndexMap tail) with a single-pass
classify into a slot Vec + remainder, then merge.
cargo bench (vs. main):
sort already sorted package.json: -6.06%
sort large package.json: -4.36%
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`Vec::with_capacity(input.len())` was sized to fit the JSON body exactly when the input is already pretty-printed, so the trailing `buf.push(b'\n')` would always trip a reallocation. For the minimal benchmark (~77 byte input) this caused a measurable regression (CodSpeed reported +13.83%) — `to_string_pretty` previously used serde's default 128-byte capacity, which had the headroom we lacked. Pad the capacity by 16 bytes so the newline push never realloc's, while still keeping the win for larger inputs where the hint avoids serde's repeated 128 → 256 → 512 → … growth. cargo bench (vs. main): sort small package.json: -3.08% sort already sorted package.json: -4.34% sort minimal package.json: -1.35% sort large package.json: -1.37% Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactor
sort_package_jsonfrom first principles, trimming allocations and simplifying bucket logic.Vec<u8>(BOM bytes prepended,serde_json::to_writer{,_pretty}appends, singlefrom_utf8_uncheckedat the end). Replaces the old path ofto_string_pretty→String::push('\n')→ reallocate-with-BOM, which incurred a full output-sized memcpy on BOM inputs and a UTF-8 validation walk in general.sort_object_keys: collapsenon_privateandprivateinto oneunknownVec sorted by composite(starts_with('_'), name). One alloc, one sort; the underscore check leaves the per-entry hot path.sort_object_by_key_order: drop theobj.sort_keys()+ per-keyshift_removepattern (eachshift_removeshifts the IndexMap tail — O(k·n)). Replaced with single-pass classify into a slot Vec + remainder, then merge.Net: −41 lines, all tests pass (snapshot + idempotency + size-limit + UTF-8 BOM),
cargo clippy --all-targets -- -D warningsclean.Benchmarks
cargo benchvs. main on this machine:The wins scale with output size — they come from eliminating the extra String copy and the post-serialize UTF-8 walk.