OpenCV projectPoints (): inconsistent covariance / standard deviation of estimated positions

I use OpenCV in the pose estimation algorithm, where I am also trying to get an estimate of the uncertainty of the evaluation pose. My posture assessment is done through 3D 2D matching and PNP algorithm. To estimate the uncertainty, I try to use the projectPoints () function to replay 3D points on the image plane and use the internally calculated Jacobi matrix to obtain covariance.

My map function contains a planar set of three-dimensional points. My camera path is simple: I start the proximity to the functions, move back and thereby far from the functions, and then closer until I reach the starting point. I use a 3D simulator (Unreal Engine) to get these images, and I have the correct camera inside (and it has zero distortion). The problem I see is that the covariance and subsequent standard deviation (for translation) evaluated using projectPoints () do not correlate with the actual position error. As I move on, as expected, the grade worsens and the postures are unstable, but the standard deviation decreases. I attach a graph of the error of the distance of the estimated pose and, accordingly, the calculated standard deviation, and they have opposite profiles.The code where I am trying to calculate this standard deviation is as follows.

cv::Mat rvec, J;
cv::eigen2cv(pose.rotation(), rvec);
cv::Mat tvec = cv::Mat(1, 3, CV_32FC1, cv::Scalar::all(0));

tvec.at<float>(0, 0) = pose.center()[0];
tvec.at<float>(0, 1) = pose.center()[1];
tvec.at<float>(0, 2) = pose.center()[2];

cv::Mat p;
cv::projectPoints(objectPoints, rvec, tvec, K, dist, p, J);

std::vector <cv::Point2f> reprojected;

for (int i = 0; i < p.rows; i++) {
    reprojected.push_back(cv::Point2f(p.at<double>(i, 0), p.at<double>(i, 1)));
}
cv::Mat Sigma = cv::Mat(J.t() * J, cv::Rect(0, 0, 6, 6)).inv();
cv::Mat std_dev;
sqrt(Sigma.diag(), std_dev);

; :

? , (, Z ), ?

+1

All Articles