ME-6452 triangle picking

[mbalajee]

https://fieldwire.atlassian.net/browse/ME-6452

This PR introduces CPU-side triangle picking for glTF meshes. The picker computes a ray from screen coordinates, intersects it against mesh geometry on the CPU using a BVH (tinybvh), and returns the closest hit.

In addition, the BVH intersection is extended to support skipping hidden triangles, so geometry that has been hidden at runtime is not pickable.

Motivation

We need reliable triangle-level picking results on the CPU:

• Works independently of GPU readbacks
• Enables accurate selection against mesh triangles
• Supports runtime visibility changes (e.g., user hides objects / parts of meshes) without rebuilding heavy data every pick

What’s included

• Screen → world ray construction (ray computed from screen coordinates and camera parameters)
• CPU intersection pipeline:
  • Uses mesh vertex/index data from the glTF source
  • Builds/uses a BVH via tinybvh for fast ray traversal
  • Computes closest hit triangle and intersection details
• Hidden triangle filtering:
  • When a user hides a subset of triangles for an entity, Android stores that hidden subset
  • Subsequent pick calls provide the hidden subset to the picker
  • tinybvh is patched so the traversal/intersection can skip triangles that are marked hidden

Why patch tinybvh?

To support runtime filtering (“don’t pick hidden triangles”) efficiently while still using an indexed + BVH-based approach.

Options considered:

1. Rebuild BVH / BVH vertices on every pick

   • Correct but too expensive to do per interaction

2. Expand vertices to explicit triangles + custom intersection

   • Makes filtering easy but duplicates vertex data heavily and requires maintaining our own intersection path. Consumes 3-4x times more memory than option 3.

3. Indexed geometry + BVH traversal (chosen)

   • Best memory characteristics
   • Requires a tinybvh extension to allow per-triangle skipping during intersection

Verdict: use indexed geometry for memory efficiency and extend tinybvh to filter hidden triangles.

Memory metrics

Representative measurements comparing expanded triangles vs indexed route:

160K

• Total registered meshes: 13
  • positions: 99,656.3 KB
  • indices: 187,590 KB
  • bvhTriangles: 750,360 KB (option 2)
  • bvhVertices: 132,875 KB (option 3)

60K

• Total registered meshes: 28
  • positions: 34,653.8 KB
  • indices: 67,554.9 KB
  • bvhTriangles: 270,219 KB (option 2)
  • bvhVertices: 46,205 KB (option 3)

Behavior notes

• If no hidden triangle subset is provided, picking considers the full mesh.
• If hidden triangles are provided for the entity, intersections against those triangles are ignored and traversal continues to find the next valid closest hit.

Testing

• Verify a ray pick returns the expected triangle hit on visible geometry.
• Hide an entity / subset and confirm subsequent picks do not return hits on hidden triangles.
• Ensure performance is acceptable for repeated picks (e.g., continuous selection / hover scenarios).

Before hiding	After hiding (0 to 107 - front wall)
after.mov	before.mov

[github-actions]

Please add a release note line to NEW_RELEASE_NOTES.md. If this PR does not warrant a release note, add the 'internal' label to this PR.

[Copilot]

Pull request overview

Adds CPU-side triangle picking for glTF meshes by registering mesh geometry during asset loading, building a BVH (tinybvh), and enabling optional runtime filtering to skip hidden triangles during intersection.

Changes:

• Introduces PickingRegistry to extract mesh positions/indices, build BVHs, and perform screen-to-world ray picks.
• Defers mesh registration until buffers are loaded, then registers meshes in ResourceLoader.
• Patches tinybvh traversal to optionally skip triangles based on caller-provided ranges.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
third_party/tiny-bvh/tiny_bvh.h	Adds optional pick-time triangle filtering hook during BVH intersection.
samples/gltf_viewer.cpp	Updates sample click handling to demonstrate CPU picking / registry usage.
libs/gltfio/src/ResourceLoader.cpp	Registers deferred meshes into the picking registry after buffer loading.
libs/gltfio/src/PickingRegistry.cpp	Implements mesh extraction, BVH build, screen-ray computation, and picking.
libs/gltfio/src/FilamentAsset.cpp	Clears picking registry on asset destruction; exposes getPickingRegistry().
libs/gltfio/src/FFilamentAsset.h	Stores pending mesh registrations and embeds PickingRegistry.
libs/gltfio/src/AssetLoader.cpp	Defers mesh registration until cgltf buffers are loaded.
libs/gltfio/include/gltfio/PickingRegistry.h	Declares public picking registry API and related data structures.
libs/gltfio/include/gltfio/FilamentAsset.h	Adds public FilamentAsset::getPickingRegistry() API.
libs/gltfio/CMakeLists.txt	Adds new header/source to the gltfio build.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.