opencv - How do I do an hand-in-eye calibration with cv2.calibrateHandEye()? - Stack Overflow

admin管理员组

文章数量:1391968

I am trying to do a hand-in-eye calibration.
OpenCV Hand-Eye Calibration

In my case, I am looking for the relative transformation between a MoCap point mounted on my robot arm, usually this would be the gripper, and my camera, which is at the tip of the robot arm. I am using a MoCap because I do not trust the kinematics of my robot.

To estimate the pose of my camera, I use a chessboard pattern and cv2.solvePnP() with the intrinsic parameters that I obtained from camera calibration.

I now have a number of poses (6DOF) of the camera given in world coordinates and a number of poses of the MoCap (also 6DOF) and also given in world coordinates. So, if the MoCap point would be exactly in the optical center of the camera, the two poses would be the same.

Since this is not the case, I would like to know the respective transformation between the camera and the MoCap point. For this, if I understand correctly, I need Hand-Eye-Calibration.

My understanding of cv2.calibrateHandEye() was that I input the previously named sets of homogeneous matrices split up in their rotation and translation parts, which outputs the optimized relative transformation.

At the moment, to test my process, I am using Blender to generate the chessboard images and to have the correct ground truth positions of the camera there.

Therefore, I now have two sets of matrices: the ground truth poses of the camera in Blender. And the poses of the camera estimated with cv2.solvePnp(). They are quite similar, but I would like to know the difference between them in order to evaluate the accuracy of my camera calibration and pose estimation. That's why I want the optimal relative transformation between these sets of matrices. I expected that the cv2.calibrateHandEye() would return something close to the identity matrix as the poses are, as already said, very similar. When I try this, the result seems very random with a large translation part as well.

In order to understand the function better, I tried the following:

random_camera_matrices = sl.load_matrices_from_json("path")
random_relative_transformation = np.array([
    [0.861, 0.0228, 0.507, 0.0399],
    [0.352, 0.692, -0.629, 0.0151],
    [-0.365, 0.721, 0.588, -0.926],
    [0, 0, 0, 1]
])
poses = random_camera_matrices.copy()
corrected_poses = [random_relative_transformation @ m for m in poses]
T_correction = calibrate_hand_eye(corrected_poses, random_camera_matrices)

I had the expectation, and hope, that it would return the random_relative_transformation as output, as this is the relative transformation between the sets of matrices. Instead it returns:

[[-0.96767031 -0.21716481  0.12827163  0.19322249]
 [ 0.17215769 -0.9403822  -0.29333095 -0.26266732]
 [ 0.18432552 -0.2617647   0.94736653 -0.18585191]
 [ 0.          0.          0.          1.        ]]

I am wondering if I completely misunderstand the function or the concepts behind it. Also, how can I interpret and use the output of the function?

本文标签： opencvHow do I do an handineye calibration with cv2calibrateHandEye()Stack Overflow