POC: Transition Ubuntu vulnerabilities from OVAL to OSV feeds

[getvictor]

Goal

User story
As an engineer working on vulnerability detection,
I want to build a proof of concept that validates OSV feeds as a replacement for OVAL on Ubuntu
so that we can confirm OSV produces more accurate results before committing to a full migration.

This is a POC for #39900.

Changes

Product

• UI changes: No changes
• CLI (fleetctl) usage changes: No changes
• YAML changes: No changes
• REST API changes: No changes
• Fleet's agent (fleetd) changes: No changes
• Fleet server configuration changes: No changes
• Exposed, public API endpoint changes: No changes
• fleetdm.com changes: No changes
• GitOps mode UI changes: No changes
• GitOps generation changes: No changes
• Activity changes: No changes
• Permissions changes: No changes
• Changes to paid features or tiers: No changes
• My device and fleetdm.com/better changes: No changes
• Usage statistics: No changes
• Other reference documentation changes: No changes
• First draft of test plan added
• Once shipped, requester has been notified
• Once shipped, dogfooding issue has been filed

Engineering

This POC should cover the following areas:

1. Add OSV feed

• Download Ubuntu OSV data from the osv.dev GCS bucket (https://storage.googleapis.com/osv-vulnerabilities/Ubuntu/all.zip) or the official https://github.com/canonical/ubuntu-security-notices
• Figure out how to incrementally process the newest updates instead of processing everything every time. (This is possible with the Google feed)

2. Check the results against OVAL using osquery-perf

• Run both OSV and OVAL vulnerability scans in parallel against a representative set of Ubuntu hosts using osquery-perf.
• Compare the results to identify:
  • False positives eliminated by OSV (e.g., CVE-2024-30205 on emacs-common per #39370).
  • Any false negatives introduced by OSV (vulnerabilities caught by OVAL but missed by OSV).
  • Overall difference in CVE counts per host across Ubuntu 20.04, 22.04, and 24.04.

3. Have a fallback to OVAL for 1 or more releases

• Implement a fallback mechanism so that if OSV data is unavailable or incomplete for a specific Ubuntu release, the system falls back to existing OVAL-based scanning.
• This ensures no regression in coverage during the transition period.

4. Recommended: get more customer data

• Collect vulnerability scan results from customer environments (with appropriate permissions) to validate OSV accuracy against real-world host configurations.

• Compare OSV vs OVAL results on customer data to quantify the improvement in false positive rates.

• Test plan is finalized

• Contributor API changes: No changes

• Feature guide changes: No changes

• Database schema migrations: No changes

• Load testing: Compare OSV scan performance against current OVAL scan across Ubuntu hosts using osquery-perf

• Pre-QA load test: Yes — run comparative scan with osquery-perf before QA

• Load testing/osquery-perf improvements: No changes

• This is a premium only feature: No

ℹ️ Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Risk level: Low
• Risk description: This is a POC that runs in parallel with existing OVAL scanning. No production vulnerability data is modified.

Test plan

• Verify OSV data can be downloaded and parsed for Ubuntu 20.04, 22.04, and 24.04.
• Verify CVE-2024-30205 is NOT flagged for emacs-common on Ubuntu 24.04 when using OSV (the known false positive from #39370).
• Verify CVE-2024-39331 IS still flagged for emacs-common on Ubuntu 24.04 when using OSV (legitimate vulnerability).
• Run osquery-perf comparison between OVAL and OSV results and document differences.
• Verify fallback to OVAL works when OSV data is missing for a release.

Testing notes

This is a POC. Testing focuses on validating accuracy of OSV data compared to OVAL, not on production readiness.

Confirmation

1. Engineer: Added comment to user story confirming successful completion of test plan.
2. QA: Added comment to user story confirming successful completion of test plan.

[getvictor]

Timebox to 8