Skip to content

test: cover frontmatter with pbt#1166

Closed
dubzzz wants to merge 3 commits intonpmx-dev:mainfrom
dubzzz:dubzzz/frontmatter
Closed

test: cover frontmatter with pbt#1166
dubzzz wants to merge 3 commits intonpmx-dev:mainfrom
dubzzz:dubzzz/frontmatter

Conversation

@dubzzz
Copy link
Copy Markdown
Contributor

@dubzzz dubzzz commented Feb 7, 2026

I'm continuing my work on extending the usage of property based testing via fast-check to other parts of the codebase. Lately I wanted to play with parseBasicFrontmatter. Given it was covered by classical tests I believed it was something useful to cover and make fully reliable.

To date, I started with a very first version of a property based test. But it found some behaviors I was not expecting to encountered so I filtered some. Before continuing and cleaning up this PBT test that is now more and more complex to handle some edge-cases that might be bug I want to know if these edge-cases are expected or not.

In this PR, all the tests added without PBT are responsible to capture on of the edge-cases. The tests using it are green, the ones with it.fails are red if we drop the .fails. Are these results expected?

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
npmx.dev Ready Ready Preview, Comment Feb 7, 2026 6:22pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs.npmx.dev Ignored Ignored Preview Feb 7, 2026 6:22pm
npmx-lunaria Ignored Ignored Feb 7, 2026 6:22pm

Request Review

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 7, 2026

@dubzzz is attempting to deploy a commit to the serhalp's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Feb 7, 2026

📝 Walkthrough

Walkthrough

This PR enhances test coverage for the parseBasicFrontmatter function by introducing property-based testing using fast-check and adding comprehensive test cases. The changes cover edge cases including escaped quotes, scientific notation handling, whitespace trimming, array values, and quoted strings with special characters. A new property-based test validates round-trip parsing with randomly generated frontmatter containing varied data types (booleans, numbers, strings, arrays). No public APIs or exported entities are modified, with 106 lines of test code added.

Possibly related PRs

Suggested reviewers

  • danielroe
🚥 Pre-merge checks | ✅ 1
✅ Passed checks (1 passed)
Check name Status Explanation
Description check ✅ Passed The PR description clearly explains the author's work on extending property-based testing to parseBasicFrontmatter, detailing the discovery of unexpected edge-case behaviours and the purpose of the added tests.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov bot commented Feb 7, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
1508 1 1507 6
View the top 1 failed test(s) by shortest run time
test/unit/shared/utils/parse-basic-frontmatter.spec.ts > parseBasicFrontmatter > should parse back every key-value pair from generated frontmatter
Stack Traces | 0.000599s run time
Error: Unable to use "stringMatching" against a regex using the flag i
 ❯ Module.stringMatching node_modules/.pnpm/fast-check@4.5..../lib/arbitrary/stringMatching.js:149:19
 ❯ .../shared/utils/parse-basic-frontmatter.spec.ts:153:23
 ❯ node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@voidzero-dev/vite-plus-test/dist/@vitest/runner/index.js:145:14
 ❯ node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@voidzero-dev/vite-plus-test/dist/@vitest/runner/index.js:915:28
 ❯ node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@voidzero-dev/vite-plus-test/dist/@vitest/runner/index.js:1243:24
 ❯ runWithTimeout node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@voidzero-dev/vite-plus-test/dist/@vitest/runner/index.js:1209:12
 ❯ node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@voidzero-dev/vite-plus-test/dist/@vitest/runner/index.js:1653:42
 ❯ Traces.$ node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@.../dist/chunks/traces.CCmnQaNT.js:142:29
 ❯ trace node_modules/.pnpm/@voidzero-dev+vite-plus-test@0.0.0-833c515fa25cef20905a7f9affb156dfa6f151ab_@types+node_4157347c09905ce729a9a0faf64ab7ea/node_modules/@.../dist/chunks/test.B8ej_ZHS.js:239:23

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
test/unit/shared/utils/parse-basic-frontmatter.spec.ts (2)

152-186: Extract the PBT builders to keep the test callback under ~50 lines.

This it block is well over the guideline size; moving the arbitraries into helpers or module-scope constants would keep the test focused and easier to scan.

As per coding guidelines "Keep functions focused and manageable (generally under 50 lines)".


187-212: Configure numRuns and size limits to keep CI time predictable.

10,000 runs with unconstrained dictionary/array sizes can be expensive. Fast-check's recommended approach is to set a global time limit (interruptAfterTimeLimit) in your test setup and make numRuns configurable via environment variable for separate PR and nightly jobs. Additionally, the fc.dictionary and fc.array calls here lack explicit size bounds—consider adding maxLength constraints:

Suggested changes

Set up global timeout in your test configuration (e.g. vitest.config.ts):

const fc = require('fast-check');
fc.configureGlobal({
  numRuns: Number(process.env.FC_NUM_RUNS ?? 100),
  interruptAfterTimeLimit: process.env.CI ? 5_000 : undefined, // 5s in CI
});

Then update the arbitraries to cap sizes:

- const valueArb = fc.oneof(
+ const valueArb = fc.oneof(
    ...singleValueArbs(false),
    fc
-     .array(fc.oneof(...singleValueArbs(true)), { minLength: 1 })
+     .array(fc.oneof(...singleValueArbs(true)), { minLength: 1, maxLength: 5 })
      .map(arr => ({
        raw: `[${arr.map(v => v.raw).join(', ')}]`,
        expected: arr.map(v => v.expected),
      })),
  )
- const frontmatterContentArb = fc.dictionary(keyArb, valueArb).map(dict => {
+ const frontmatterContentArb = fc.dictionary(keyArb, valueArb, { maxKeys: 10 }).map(dict => {

@@ -1,4 +1,5 @@
import { describe, expect, it } from 'vitest'
import fc from 'fast-check'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check TS module interop flags
find . -name 'tsconfig*.json' -type f | head -5 | xargs -I {} sh -c 'echo "=== {} ===" && cat {}'

Repository: npmx-dev/npmx.dev

Length of output: 769


🏁 Script executed:

# Check existing fast-check imports
rg "from ['\"]fast-check['\"]" --type=ts --type=tsx -n

Repository: npmx-dev/npmx.dev

Length of output: 88


🏁 Script executed:

# Search for existing fast-check imports without file type restriction
rg "from ['\"]fast-check['\"]" -n

Repository: npmx-dev/npmx.dev

Length of output: 293


🏁 Script executed:

# Check if there's a test-specific tsconfig
find . -path '*/test/**' -name 'tsconfig*.json' -o -path '*/tests/**' -name 'tsconfig*.json'

Repository: npmx-dev/npmx.dev

Length of output: 43


🏁 Script executed:

# Look for vitest config and test setup
find . -name 'vitest.config.*' -o -name 'vite.config.*' | head -5

Repository: npmx-dev/npmx.dev

Length of output: 79


Use namespace import to match existing test suite convention.

The codebase consistently uses import * as fc from 'fast-check' in other test files. Align with this established pattern.

Import update
-import fc from 'fast-check'
+import * as fc from 'fast-check'
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import fc from 'fast-check'
import * as fc from 'fast-check'

Copy link
Copy Markdown
Member

@danielroe danielroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we only need this to handle frontmatter we ourselves create

I would likely migrate us to use gray-matter in future, so it's probably not worth investing too much time fixing any issues in the function (which this almost certainly did find!)

@dubzzz
Copy link
Copy Markdown
Contributor Author

dubzzz commented Feb 7, 2026

Thanks @danielroe, I'll probably close this one then. No need to try to find bugs in it for now

@dubzzz dubzzz closed this Feb 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants