With my eyes outside my hand, when I can see a point P through the lens of the depth camera, I would like to know what the coordinates of this point are in the robot coordinates? Usually a hand-eye calibration is required.
But this method is more troublesome. Because you need a calibration version. placed on the operator's hand of the robot. This information is then obtained through a series of photos. Related technical methods can be found on the Internet a lot. This time I hope to try another method of calibration.
The first step is to solve the forward kinematics of the robot. That is to say, after the joints of my robot are determined, I can determine the position of the end. This needs to be done first. Describe and then find a series of transformation matrices. Finally, we can learn justice and solve it.
The second step is to calibrate the internal parameters of the camera. Remove distortion. This is also a relatively common method. No longer. After calibrating the internal parameters of the camera, we can project each two-dimensional point into a three-dimensional space under the camera's coordinate system.
The third step is to place a small object that can be recognized at the end of the robot. For example, a ping pong ball. On that picture red or green. Easy to divide.
The fourth step is to let the robot's arm move freely in space. Every time you move to a position, record what the camera captures. Color Avatar and Depth Avatar. In this way, we get the coordinates of the point under the robot coordinates from the forward kinematics of the robot. The camera coordinates of the point are obtained from the photo taken by the camera.
The last fifth step, a point to help us. After both coordinate systems have their own coordinates, they can be obtained through the least squares method. Transformation matrix between coordinates.
The algorithm of the fifth step is shown in the code below.