雅可比矩阵咨询

[chnhs]

您好,
非常感谢您开源 libcppt 项目, 我从中学会了很多. 最近正在按照您的思路实现相关内容, 但是在调试过程中出现了一些问题, 特向您咨询一下

在上述求解残差对线特征雅可比矩阵时, 获取到非常大的雅可比矩阵
​

想咨询一下这种情况属于正常情况吗? 冒昧的打扰, 请您谅解.

[cart0909]

@tjuhuangshuai 您好
請問您跑的datasets是哪個呢，今天我這邊測出來值大概是兩百多而已。

[chnhs]

@cart0909 这个是我拿小觅设备采集的图像, 没有使用公开的 Euroc 等数据集. 您的结果是直接用 libcppt 跑 euroc 数据集跑出来的吗?

[cart0909]

@tjuhuangshuai 是的，線的效果如下圖

我今天測了一下在LineProjectionFactor上面印出殘差對線的Jacobian值如下

對應的代碼位置

我也試過TUM VIO datasets效果大概像這樣

[chnhs]

谢谢您特地做了实验, 我又检查了一下代码, 没有发现问题. 想咨询一下您这种表示方式适合于大尺度场景吗? 比如下面这个图.同时看到您的上面对垂足有个约束, (p1_star - q1_star).norm() > 5 . 这个对线段长度大于 5 米的可以使用吗?

[chnhs]

@cart0909 您好, 我思考了一下. 假设我们我们一直沿着一条路前行, 由于线坐标是表示在世界坐标系下, 那么我们线坐标在世界坐标系下的正交表示一个很小的变化, 就会引起线坐标在相机坐标系下很大的变化.这么解释这种现象是否合理呢?

[chnhs]

@cart0909 hello, 有一个大型的室外数据集https://daniilidis-group.github.io/penncosyvio/ , 您也可以测试一下,格式和 euroc 一样

[cart0909]

@tjuhuangshuai 您好
第一個問題，目前我對於線的outlier的確沒有好的處理方式，一些paper上對於outlier的處理我可能尚未吸收理解，所以才會假設了一個長度的限制，這的確不是個好限制。
第二個問題，我是認為用世界描述是沒什麼問題，畢竟點也是可以用世界描述下去做BA，可以在討論看看？
第三這個datasets還沒有試過，我之前也有找到一些整理很好的datasets這邊也給您參考
https://sites.google.com/view/awesome-slam-datasets/

PS. pl-vio的工作目前是被我暫停中，原因是得花太多力氣，剛開始先rectify影像，再來每偵都得抽FLD or LSD，匹配的方法也沒有太多研究，不像光流比較魯棒，後端優化收歛性也不像以往只有點的殘差好，也是覺得遺憾，但是能幫助到您也是很開心的。

[chnhs]

@cart0909 嘿嘿, 谢谢您的帮助, 收获的确挺大的.
对于点的世界坐标表示, 我认为由于点线坐标的表示不同, 会造成很大的影响. 毕竟线在优化过程中是用正交表示的, 其四个量表示的是旋转, 而线的正交表示在世界坐标系变化一些, 会造成在 C 系下观测的线很大变化, 进而影响着残差的计算. 这样可能会造成 残差对线的雅可比矩阵 很大.您那有长一些的数据集吗? 可以跑一下看一下雅可比矩阵的大小. 我这面也在下, 然后咱们可以对比一下.
同时, 我在线的优化过程中发现每次迭代线的变化量很大, 可能对应着您说的后端收敛性.

[chnhs]

@cart0909 hello, 现在每次运行都出现错误, 您帮忙看一下, 麻烦了
roslaunch cppt_vio tum.launch
... logging to /home/.ros/log/451a3362-7a3c-11e9-b764-37157c0df6c6/roslaunch-ThinkPad-X1-Carbon-3rd-27576.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ThinkPad-X1-Carbon-3rd:43541/

SUMMARY

PARAMETERS

• /cppt_vio/config_file: /home/poi...
• /cppt_vio/pl_system: False
• /rosdistro: kinetic
• /rosversion: 1.12.14

NODES
/
cppt_vio (cppt_vio/cppt_vio)
rvizvisualisation (rviz/rviz)

auto-starting new master
process[master]: started with pid [27586]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 451a3362-7a3c-11e9-b764-37157c0df6c6
process[rosout-1]: started with pid [27599]
started core service [/rosout]
process[cppt_vio-2]: started with pid [27606]
process[rvizvisualisation-3]: started with pid [27617]
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/point_line/devel_isolated/cppt_vio/lib/cppt_vio/cppt_vio...(no debugging symbols found)...done.
Starting program: /home/point_line/devel_isolated/cppt_vio/lib/cppt_vio/cppt_vio /camera/left/image_raw:=/cam0/image_raw /camera/right/image_raw:=/cam1/image_raw /imu/data_raw:=/imu0 __name:=cppt_vio __log:=/home/.ros/log/451a3362-7a3c-11e9-b764-37157c0df6c6/cppt_vio-2.log
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x1835a20 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x18380f0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1835540 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1831530 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
[ERROR] [1558273489.856574488]: PluginlibFactory: The plugin for class 'voxblox_rviz_plugin/VoxbloxMesh' failed to load. Error: According to the loaded plugin descriptions the class voxblox_rviz_plugin/VoxbloxMesh with base class type rviz::Display does not exist. Declared types are rviz/Axes rviz/Camera rviz/DepthCloud rviz/Effort rviz/FluidPressure rviz/Grid rviz/GridCells rviz/Illuminance rviz/Image rviz/InteractiveMarkers rviz/LaserScan rviz/Map rviz/Marker rviz/MarkerArray rviz/Odometry rviz/Path rviz/PointCloud rviz/PointCloud2 rviz/PointStamped rviz/Polygon rviz/Pose rviz/PoseArray rviz/PoseWithCovariance rviz/Range rviz/RelativeHumidity rviz/RobotModel rviz/TF rviz/Temperature rviz/WrenchStamped rviz_plugin_tutorials/Imu
0x1a9e240 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1c9bd40 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1b22160 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1b219b0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1c9b650 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1a9e240 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1ca12f0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1d02390 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1cff6e0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1cffce0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1d01fc0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x1ca12f0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd479f0) ): Attempt to set a screen on a child window.
0x18380f0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1831530 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1835540 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1a11130 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1c9b650 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1a9e240 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1c9bd40 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1d01fc0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1ca12f0 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1d02390 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1e92f60 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1c9bb20 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
0x1a71370 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xd43820) ): Attempt to set a screen on a child window.
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
error opening trace file: Permission denied (13)
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[New Thread 0x7fffe8922700 (LWP 27750)]
[New Thread 0x7fffe3fff700 (LWP 27751)]
[New Thread 0x7fffe37fe700 (LWP 27752)]
[New Thread 0x7fffe2ffd700 (LWP 27757)]
[ INFO] [1558273490.476476331]: Loaded config_file: /home/point_line/src/examples/vio/../../config/calib/tum.yaml
[ INFO] [1558273490.477162168]: Loaded pl_system: 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0519 21:44:50.477572 27620 config_loader.cpp:101] number of camera: 1
I0519 21:44:50.477649 27620 config_loader.cpp:103] cam0 image size: [512 x 512]
I0519 21:44:50.477668 27620 config_loader.cpp:104] cam0 model type: FISHEYE
I0519 21:44:50.477679 27620 config_loader.cpp:115] cam0 intrinsic: 190.978 190.973 254.932 256.897
I0519 21:44:50.477705 27620 config_loader.cpp:123] cam0 distortion: 0.00348239 0.000715035 -0.00205324 0.000202937
I0519 21:44:50.477720 27620 config_loader.cpp:129] cam0 p_bc: 0.0455748 -0.0711618 -0.0446812
I0519 21:44:50.477733 27620 config_loader.cpp:132] cam0 q_bc: 0.00768851 -0.0133554 -0.694797 0.719041
I0519 21:44:50.477747 27620 config_loader.cpp:150] cam0r intrinsic: 190.442 190.434 252.599 254.917
I0519 21:44:50.477761 27620 config_loader.cpp:157] cam0r distortion: 0.00340032 0.00176628 -0.00266313 0.000329952
I0519 21:44:50.477774 27620 config_loader.cpp:163] cam0r p_rl: -0.101061 -0.00197646 -0.00117564
I0519 21:44:50.477787 27620 config_loader.cpp:166] cam0r q_rl: -0.999725 -0.0234543 0.000337641 -0.000403876
I0519 21:44:50.477799 27620 config_loader.cpp:173] acc_noise: 0.04
I0519 21:44:50.477809 27620 config_loader.cpp:174] gyr_noise: 0.004
I0519 21:44:50.477818 27620 config_loader.cpp:175] acc_bias_noise: 0.0004
I0519 21:44:50.477828 27620 config_loader.cpp:176] gyr_bias_noise: 2e-05
I0519 21:44:50.477838 27620 config_loader.cpp:177] gravity_magnitude: 9.81007
I0519 21:44:50.477849 27620 config_loader.cpp:178] voc_path: /home/point_line/src/config/voc/brief_k10L6.bin
I0519 21:44:50.477856 27620 config_loader.cpp:179] brief_pattern_path: /home/point_line/src/config/brief/brief_pattern.yml
loop start load vocabulary
[New Thread 0x7fffe1170700 (LWP 27760)]
[New Thread 0x7fffe096f700 (LWP 27761)]
[New Thread 0x7fffda9a1700 (LWP 27762)]
[New Thread 0x7fffda1a0700 (LWP 27763)]
[ INFO] [1558273490.966519254]: Player is ready.
I0519 21:44:56.423516 27763 backend.cpp:67] this frame -----------------------------MARGIN_OLD
I0519 21:44:56.423789 27763 backend.cpp:77] 1111111111111111111111111
I0519 21:44:56.473680 27763 backend.cpp:67] this frame -----------------------------MARGIN_OLD

Thread 9 "cppt_vio" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffda1a0700 (LWP 27763)]
0x00007fffeeab1986 in ceres::internal::ProgramEvaluator<ceres::internal::BlockEvaluatePreparer, ceres::internal::BlockJacobianWriter, ceres::internal::NullJacob---Type to continue, or q to quit---
ianFinalizer>::Evaluate(ceres::internal::Evaluator::EvaluateOptions const&, double const*, double*, double*, double*, ceres::internal::SparseMatrix*) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
(gdb) bt
#0 0x00007fffeeab1986 in ceres::internal::ProgramEvaluator<ceres::internal::BlockEvaluatePreparer, ceres::internal::BlockJacobianWriter, ceres::internal::NullJacobianFinalizer>::Evaluate(ceres::internal::Evaluator::EvaluateOptions const&, double const*, double*, double*, double*, ceres::internal::SparseMatrix*) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#1 0x00007fffeeb4673d in ceres::internal::TrustRegionMinimizer::EvaluateGradientAndJacobian(bool) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#2 0x00007fffeeb4710b in ceres::internal::TrustRegionMinimizer::IterationZero() ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#3 0x00007fffeeb4b7d6 in ceres::internal::TrustRegionMinimizer::Minimize(ceres::internal::Minimizer::Options const&, double*, ceres::Solver::Summary*) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#4 0x00007fffeeb3dd05 in ceres::Solver::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#5 0x00007fffeeb3e5b9 in ceres::Solve(ceres::Solver::Options const&, ceres::Pro---Type to continue, or q to quit---
blem*, ceres::Solver::Summary*) ()
from /home/point_line/src/src/../3rdParty/ceres-solver-1.14.0/build/lib/libceres.so.1
#6 0x00007ffff6513310 in BackEnd::SolveBA() ()
from /home/point_line/devel_isolated/cppt/lib/libcppt.so
#7 0x00007ffff651ee19 in BackEnd::ProcessFrame(std::shared_ptrBackEnd::Frame) () from /home/point_line/devel_isolated/cppt/lib/libcppt.so
#8 0x00007ffff64ff147 in System::BackEndProcess() ()
from /home/point_line/devel_isolated/cppt/lib/libcppt.so
#9 0x00007ffff53a3c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff58786ba in start_thread (arg=0x7fffda1a0700)
at pthread_create.c:333
#11 0x00007ffff4b0941d in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)

[cart0909]

請問這是ceres內部的問題，還是call到我這邊function的問題呢?
看起來backtrace最終是指到ceres內部function?
有沒有debug的backtrace呢?

[chnhs]

@cart0909 这个没有调试过, 我可以尝试一下. 您也可以试一下, 我跑的是 dataset-outdoors4_512_16.bag 这个 tum 的数据集.

[chnhs]

@cart0909 hello,这个是 Debug 版本的调试信息, 您看一下

Thread 9 "cppt_vio" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd21a0700 (LWP 2871)]
0x00007fffeeab1896 in ceres::internal::ProgramEvaluator<ceres::internal::BlockEvaluatePreparer, ceres::internal::BlockJacobianWriter, ceres::internal::NullJacob---Type to continue, or q to quit---
ianFinalizer>::Evaluate (this=0x7fffb807f8a0, evaluate_options=...,
state=0x7fffb8083e00, cost=0x7fffb8083da0, residuals=0x7fffb8088a00,
gradient=0x7fffb8084620, jacobian=0x7fffb8083720)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/program_evaluator.h:163
163 options_.evaluation_callback->PrepareForEvaluation(
(gdb) bt
#0 0x00007fffeeab1896 in ceres::internal::ProgramEvaluator<ceres::internal::BlockEvaluatePreparer, ceres::internal::BlockJacobianWriter, ceres::internal::NullJacobianFinalizer>::Evaluate (this=0x7fffb807f8a0, evaluate_options=...,
state=0x7fffb8083e00, cost=0x7fffb8083da0, residuals=0x7fffb8088a00,
gradient=0x7fffb8084620, jacobian=0x7fffb8083720)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/program_evaluator.h:163
#1 0x00007fffeeb4661d in ceres::internal::TrustRegionMinimizer::EvaluateGradientAndJacobian (this=this@entry=0x7fffb8083ad0,
new_evaluation_point=new_evaluation_point@entry=true)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/trust_region_minimizer.cc:230
#2 0x00007fffeeb46feb in ceres::internal::TrustRegionMinimizer::IterationZero
(this=this@entry=0x7fffb8083ad0)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/trust_region_minimizer.cc:204
#3 0x00007fffeeb4b6b6 in ceres::internal::TrustRegionMinimizer::Minimize (
this=0x7fffb8083ad0, options=..., parameters=0x7fffb8080d40,
solver_summary=0x7fffd21982b0)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/trust_region_minimizer.cc:72
#4 0x00007fffeeb3dbe5 in ceres::(anonymous namespace)::Minimize (
summary=0x7fffd21982b0, pp=0x7fffd2197be0)
---Type to continue, or q to quit---
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/solver.cc:466
#5 ceres::Solver::Solve (this=this@entry=0x7fffd2198040, options=...,
problem=problem@entry=0x7fffd21980d0, summary=summary@entry=0x7fffd21982b0)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/solver.cc:602
#6 0x00007fffeeb3e499 in ceres::Solve (options=...,
problem=problem@entry=0x7fffd21980d0, summary=summary@entry=0x7fffd21982b0)
at /home/point_line/src/3rdParty/ceres-solver-1.14.0/internal/ceres/solver.cc:635
#7 0x00007ffff6513360 in BackEnd::SolveBA (this=0x692b20)
at /home/point_line/src/src/backend.cpp:606
#8 0x00007ffff651ee49 in BackEnd::ProcessFrame (this=0x1400000000,
frame=std::shared_ptr (count 3, weak 0) 0x7fffb800b720)
at /home/point_line/src/src/backend.cpp:100
#9 0x00007ffff64ff1d7 in System::BackEndProcess (this=0x690a20)
at /home/point_line/src/src/system.cpp:202
#10 0x00007ffff53a3c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007ffff58786ba in start_thread (arg=0x7fffd21a0700)
at pthread_create.c:333
#12 0x00007ffff4b0941d in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

[chnhs]

@cart0909 hello, 终于可以跑通了, 我将 ceres 从1.14.0 替换为 2.0.0, 并且从新安装了 Eigen 库. 下面截图是Jline 的结果, 也发现了较大的线特征雅可比.