Rigged hand visualizer #7993
Rigged hand visualizer #7993
Conversation
CDiaz-MS
added
the
Platform - Leap Motion
CDiaz-MS
left a comment
CDiaz-MS
left a comment
There was a problem hiding this comment.
@rolandsmeenk Thanks for contributing!
While testing the example scene with leap motion hands, I noticed the rigged hands visual is not visible. The tracking is working because the controllers with the associated pointers are there, but I cannot see the rigged hands in editor.
Use discard in MRTK access, add missing readonly modifier, add "hand tracking" devices to XR SDK
Move some editor-only MigrationHandler code into editor-only assemblies
* Improved Keyboard introducing: - TextMeshPro, MipMapped icons, texture atlas
Move assemblyinfo files to fix the build
|
@CDiaz-MS I cleaned up a little too much which caused the meshes to be empty. I reimported the models that are used by the prefabs and restored the connections. It should work now. |
There was a problem hiding this comment.
@rolandsmeenk I still cannot see the hands, but I am getting an error with the .blend files because I do not have Blender installed. Maybe try a .fbx export for the rigs?
|
@CDiaz-MS Ok, replaced with FBX versions. Hopefully you will some hands now. :) |
|
The .blend -> .fbx worked. I am seeing the hands! |
|
@rolandsmeenk Added some gifs to the PR description so people can see it in action |
CDiaz-MS
left a comment
CDiaz-MS
left a comment
There was a problem hiding this comment.
Some initial code style comments to align with our code guidelines
This PR will also need tests, here is the docs for writing tests
This feature is super cool, thanks so much for contributing!
|
|
||
| public GameObject GameObjectProxy => gameObject; | ||
|
|
||
| public IMixedRealityController Controller { get; set; } | ||
|
|
||
| public Transform Wrist; | ||
| public Transform Palm; |
There was a problem hiding this comment.
I have highlighted a couple here but doc comments are useful for public properties, mrtk coding guidelines
| [Tooltip("Hands are typically rigged in 3D packages with the palm transform near the wrist. Uncheck this if your model's palm transform is at the center of the palm similar to Leap API hands.")] | ||
| public bool modelPalmAtLeapWrist = true; | ||
|
|
||
| [Tooltip("Allows the mesh to be stretched to align with finger joint positions. Only set to true when mesh is not visible.")] | ||
| public bool deformPosition = true; | ||
|
|
There was a problem hiding this comment.
Public properties should be capitalized, also applies to the properties below, naming guidelines mrtk
|
|
||
| // Precalculated values for LeapMotion testhand | ||
| FingerTipLengths[TrackedHandJoint.ThumbTip] = 0.02167f; | ||
| FingerTipLengths[TrackedHandJoint.IndexTip] = 0.01582f; | ||
| FingerTipLengths[TrackedHandJoint.MiddleTip] = 0.0174f; | ||
| FingerTipLengths[TrackedHandJoint.RingTip] = 0.0173f; | ||
| FingerTipLengths[TrackedHandJoint.PinkyTip] = 0.01596f; |
There was a problem hiding this comment.
It's best to avoid seemingly magic numbers, it's helpful to store the pre-calculated values in another var with a descriptive name.
|
Would be nice to have a way to easily tie this into the HandMesh bool found in the HandTracking profile. Also, I'd love to see the distance between index and thumb used to create a visual feedback in the material to signal pinch strength. Otherwise nice work! This is a great contribution! |
…ng-rendering-2019 Fix controller mapping profile rendering in Unity 2019.3
Migrating PressableButton and PressableButtonHololens2 to use local positioning for movingButtonVisuals and iconText objects.
Do you mean the HandMesh Prefab? I looked at adding this into the BaseHandVisualizer, but chose to not add more functionality to that and implement a separate visualizer for now. Maybe there can be a composite hand visualizer where a user can mix joint prefabs, hand mesh and a skinnedmesh to their liking.
Pinch strength visualization would be indeed be a nice feature, but I would prefer to keep that outside of this PR. I can imagine this would also be useful on the dynamic handmesh. |
…dHandVisualizer.md Co-authored-by: CDiaz-MS <53493796+CDiaz-MS@users.noreply.github.com>
…dHandVisualizer.md Co-authored-by: CDiaz-MS <53493796+CDiaz-MS@users.noreply.github.com>
…dHandVisualizer.md Co-authored-by: CDiaz-MS <53493796+CDiaz-MS@users.noreply.github.com>
Retrieval of finger child joints is now done with safety checks Added options to mark the root finger joint as metacarpal to accommodate for more diverse handrigs
…k/MixedRealityToolkit-Unity into RiggedHandVisualizer
|
@CDiaz-MS Mmm, from what you describe I would either expect that the "Global Left Hand Visualizer" in the controller mapping was not set or that the prefab would not have "Update when offscreen" enabled on the SkinnedMeshRenderer. Both are not the case. I made a fresh clone, but both hands show up with LeapMotion in the editor. Did a rebase of the branch, but I did not see any issues there. Can you see the RiggedHandLeft and RiggedHandRight instances appear in the MixedRealityPlaySpace when a hand is detected by LeapMotion? In my experience I sometimes need to reenter the LeapMotion FOV to get both of my hands recognized (LM in desktop setup). Otherwise I don't know what could cause this. |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
@rolandsmeenk Tested again and both of the hands are showing up for me. I also added some updated gifs to the description with the new hand models. |
12 participants
Overview
Implemented a RiggedHandVisualizer that can be used to visualize a skinned mesh for hands.
Implementation is based on and has visual parity with the LeapMotion RiggedHand/RiggedFinger combi. Fixed an issue in the LeapMotionArticulatedHand where the wrong thumb metacarpal index was copied from the leapmotion data. The handmodels are a customization of the LeapMotion handmodels with a bit rounder shapes and a different material.
Cleaner handmodels may be needed, but my rigging skills are lacking.
Tested in editor with Leapmotion and on HoloLens 2.
Gifs
Changes
Related to #4257