Skip to content

Experimentation to try to improve the Dementhon planar pose estimation#1940

Draft
s-trinh wants to merge 2 commits intolagadic:masterfrom
s-trinh:test_improve_Dementhon_pose_flipping_2
Draft

Experimentation to try to improve the Dementhon planar pose estimation#1940
s-trinh wants to merge 2 commits intolagadic:masterfrom
s-trinh:test_improve_Dementhon_pose_flipping_2

Conversation

@s-trinh
Copy link
Copy Markdown
Contributor

@s-trinh s-trinh commented Apr 5, 2026

Code for experimentation.


Original issue:

  • when using Dementhon to compute the planar pose for AprilTag detection, in some configurations it is easy to observe "pose flipping" issue

When adding VVS after each estimated pose inside the Dementhon pipeline, it greatly improves the pose estimation:

  • no more "pose flipping" has been observed from the quick tests
  • to test, the easiest thing is to add in "tutorial/detection/tag/tutorial-apriltag-detector-live.cpp" a second vpDetectorAprilTag detector2(opt_tag_family); and to duplicate everything to have side-by-side the two displays (e.g. Dementhon vs Lagrange or Dementhon vs Homography VVS, etc.)

For Lagrange:

  • "pose flipping" is less frequent compared to Dementhon
  • but I have managed to trigger it when aligning the tag frame such as having the tag x-axis pointing downward, the y-axis pointing to the right and to rotate around the x-axis
  • and when the tag is small and at the border of the image, e.g. bottom right
  • but I think it is more how I grasp the tag with my hand (which impacts the corner locations extraction) rather then a specific configuration compared to the Dementhon issue which is easy to trigger with my tests

Currently, distortion is not handled when using the AprilTag pose estimation methods.
But I think the code modification to support it is quite easy, such as in the PR code.


if (fallback_2nd_solution) {
if (cMo2) {
// Fallback: set default cMo2 to identity and set error to an invalid value
cMo2->eye();
if (projErrors2) {
*projErrors2 = -1.0;
}
vpTRACE("Second solution is only computed for HOMOGRAPHY_ORTHOGONAL_ITERATION");
}
}

Currently, the second pose is set to -1 when it is not possible to compute it, but maybe it should be set to INF/HUGE_VAL.
I don't know if it is possible to have a second reprojection error inferior to the first one. But this would avoid issue if the poses are sorted and consistent with the AprilTag library (the second pose cannot always be computed for instance):

estimate_pose_for_tag_homography(info, solution1);
*err1 = orthogonal_iteration(v, p, &solution1->t, &solution1->R, 4, nIters);
solution2->R = fix_pose_ambiguities(v, p, solution1->t, solution1->R, 4);
if (solution2->R) {
solution2->t = matd_create(3, 1);
*err2 = orthogonal_iteration(v, p, &solution2->t, &solution2->R, 4, nIters);
} else {
*err2 = HUGE_VAL;
}

s-trinh added 2 commits April 5, 2026 11:13
…reduce "pose flipping" issue. Quick code to test supporting distortion coefficients when using pose estimation from the AprilTag library.
Start test considering planar case with 4 points...
===================================================

Reference pose
 tx  = -0.01
 ty  = -0.02
 tz  = 0.3
 tux = vpMath::rad(20)
 tuy = vpMath::rad(-20)
 tuz = vpMath::rad(10)

-------------------------------------------------

Pose estimated by Lagrange
 tx  = -0.01
 ty  = -0.02
 tz  = 0.3
 tux = vpMath::rad(20)
 tuy = vpMath::rad(-20)
 tuz = vpMath::rad(10)

Based on 3D parameters pose by Lagrange is well estimated
Based on 2D residual (4.79627e-15) pose by Lagrange is well estimated
Based on pixel residual (2.86994e-12) pose by Lagrange is well estimated
--------------------------------------------------

Pose estimated by Dementhon
 tx  = 0.00517802
 ty  = -0.00184481
 tz  = 0.255955
 tux = vpMath::rad(20)
 tuy = vpMath::rad(-20)
 tuz = vpMath::rad(10)

ref[0] - est[0] = -0.015178 > 0.001
ref[1] - est[1] = -0.0181552 > 0.001
ref[2] - est[2] = 0.0440448 > 0.001
Based on 3D parameters pose by Dementhon is badly estimated
Based on 2D residual (0.0611488) pose by Dementhon is badly estimated
Based on pixel residual (36.6893) pose by Dementhon is badly estimated

Start test considering planar case with 4 points and noise on the projection...
===================================================
P[0]:
	u_theo = 99.1025	u_noisy = 99.0916
	v_theo = 4.99783	v_noisy = 5.00482
	x_theo = -0.154829	y_noisy = -0.154847
	y_theo = -0.23167	x_noisy = -0.231659
P[1]:
	u_theo = 218.235	u_noisy = 218.219
	v_theo = 28.2036	v_noisy = 28.0766
	x_theo = 0.0437243	y_noisy = 0.0436986
	y_theo = -0.192994	x_noisy = -0.193206
P[2]:
	u_theo = 190.856	u_noisy = 190.767
	v_theo = 140.716	v_noisy = 140.684
	x_theo = -0.00190676	y_noisy = -0.00205435
	y_theo = -0.00547405	x_noisy = -0.00552594
P[3]:
	u_theo = 77.0917	u_noisy = 77.1407
	v_theo = 127.315	v_noisy = 127.41
	x_theo = -0.191514	y_noisy = -0.191432
	y_theo = -0.027809	x_noisy = -0.0276495

Reference pose
 tx  = -0.01
 ty  = -0.02
 tz  = 0.3
 tux = vpMath::rad(20)
 tuy = vpMath::rad(-20)
 tuz = vpMath::rad(10)

-------------------------------------------------

Pose estimated by Lagrange
 tx  = -0.010082
 ty  = -0.0197436
 tz  = 0.299904
 tux = vpMath::rad(20.2926)
 tuy = vpMath::rad(-19.9402)
 tuz = vpMath::rad(9.99346)

Based on 3D parameters pose by Lagrange is well estimated
Based on 2D residual (0.000208394) pose by Lagrange is well estimated
Based on pixel residual (0.125036) pose by Lagrange is well estimated
--------------------------------------------------

Pose estimated by Dementhon
 tx  = 0.00556462
 ty  = -0.00176678
 tz  = 0.25591
 tux = vpMath::rad(20.0353)
 tuy = vpMath::rad(-20.2263)
 tuz = vpMath::rad(9.95496)

ref[0] - est[0] = -0.0155646 > 0.005
ref[1] - est[1] = -0.0182332 > 0.005
ref[2] - est[2] = 0.0440904 > 0.005
Based on 3D parameters pose by Dementhon is badly estimated
Based on 2D residual (0.0616508) pose by Dementhon is badly estimated
Based on pixel residual (36.9905) pose by Dementhon is badly estimated
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 5, 2026

Codecov Report

❌ Patch coverage is 84.84848% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.76%. Comparing base (10ba152) to head (bddbb1f).
⚠️ Report is 26 commits behind head on master.

Files with missing lines Patch % Lines
modules/detection/src/tag/vpDetectorAprilTag.cpp 66.66% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1940      +/-   ##
==========================================
+ Coverage   48.68%   48.76%   +0.07%     
==========================================
  Files         532      532              
  Lines       69311    69342      +31     
  Branches    32414    32438      +24     
==========================================
+ Hits        33746    33812      +66     
+ Misses      24974    24965       -9     
+ Partials    10591    10565      -26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

fspindle added a commit to fspindle/visp that referenced this pull request Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant