OpenCV每日函数 相机校准calibrateCamera函数

一、什么是相机校准

1、概述

        估计相机参数的过程称为相机校准。

        这意味着我们拥有相机所需的所有信息（参数或系数），以确定现实世界中的3D 点与其在该校准相机捕获的图像中对应的 2D 投影（像素）之间的准确关系。

        通常这意味着恢复两种参数

        内部参数：例如相机/镜头系统的镜头的焦距、光学中心和径向畸变系数。

        外部参数：这是指相机相对于某个世界坐标系的方向（旋转和平移）。

2、calibrateCamera函数

        该函数估计每个视图的内在相机参数和外在参数。 必须指定每个视图中 3D 对象点的坐标及其对应的 2D 投影。 这可以通过使用具有已知几何形状和易于检测的特征点的对象来实现。 这样的对象称为校准装置或校准模式，OpenCV 内置支持将棋盘作为校准装置。 目前，内在参数的初始化（当CALIB_USE_INTRINSIC_GUESS 未设置时）仅针对平面校准模式（其中对象点的 Z 坐标必须全为零）实现。 只要提供初始 cameraMatrix，也可以使用 3D 校准装置。

        下面是OpenCV官方文档

OpenCV: Camera Calibration and 3D Reconstructionhttps://docs.opencv.org/4.6.0/d9/d0c/group__calib3d.html#ga93efa9b0aa890de240ca32b11253dd4a

二、不同类型的相机校准方法

        以下是主要类型的相机校准方法：

        校准图案：当我们完全控制成像过程时，执行校准的最佳方法是从不同的视点捕获物体或已知尺寸图案的多个图像。下面使用的基于棋盘格的方法属于这一类。我们也可以使用已知尺寸的圆形图案代替棋盘图案。

        几何线索：有时我们在场景中还有其他几何线索，例如可用于校准的直线和消失点。

        基于深度学习：当我们对成像设置几乎没有控制权时（例如，我们有一个场景的单个图像），仍然可以使用基于深度学习的方法获得相机的校准信息。

三、使用棋盘校准

        棋盘图案在图像中是独特且易于检测的。不仅如此，棋盘上正方形的角非常适合定位它们，因为它们在两个方向上都有明显的梯度。此外，这些角还与它们位于棋盘线的交点有关。所有这些事实都用于在棋盘图案中稳健地定位正方形的角。

 1、从不同的角度捕捉棋盘的多张图像

2、查找棋盘格的 2D 坐标

        OpenCV 提供了一个名为的内置函数 findChessboardCorners ，用于查找棋盘并返回角的坐标。

        c++函数原型

bool findChessboardCorners(InputArray image, Size patternSize, OutputArray corners, int flags = CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_NORMALIZE_IMAGE )

        python函数原型

retval, corners = cv2.findChessboardCorners(image, patternSize, flags)

         OpenCV 的函数cornerSubPix 接收原始图像和角点位置，并在原始位置的一个小邻域内寻找最佳角点位置。该算法本质上是迭代的，因此我们需要指定终止标准（例如迭代次数和/或准确性）。

        c++函数原型

void cornerSubPix(InputArray image, InputOutputArray corners, Size winSize, Size zeroZone, TermCriteria criteria)

        python函数原型

cv2.cornerSubPix(image, corners, winSize, zeroZone, criteria)

3、校准相机

        校准的最后一步是将世界坐标中的 3D 点及其在所有图像中的 2D 位置传递给 OpenCV 的calibrateCamera方法。该实现基于Zhengyou Zhang 的一篇论文。需要了解线性代数相关知识。

        c++函数原型

double calibrateCamera(InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, Size imageSize, InputOutputArray cameraMatrix, InputOutputArray distCoeffs, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs)

        python函数原型

retval, cameraMatrix, distCoeffs, rvecs, tvecs = cv2.calibrateCamera(objectPoints, imagePoints, imageSize)

四、参考代码

1、c++参考

#include <opencv2/opencv.hpp>
#include <opencv2/calib3d/calib3d.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <stdio.h>
#include <iostream>
 
// Defining the dimensions of checkerboard
int CHECKERBOARD[2]{6,9}; 
 
int main()
{
  // Creating vector to store vectors of 3D points for each checkerboard image
  std::vector<std::vector<cv::Point3f> > objpoints;
 
  // Creating vector to store vectors of 2D points for each checkerboard image
  std::vector<std::vector<cv::Point2f> > imgpoints;
 
  // Defining the world coordinates for 3D points
  std::vector<cv::Point3f> objp;
  for(int i{0}; i<CHECKERBOARD[1]; i++)
  {
    for(int j{0}; j<CHECKERBOARD[0]; j++)
      objp.push_back(cv::Point3f(j,i,0));
  }
 
 
  // Extracting path of individual image stored in a given directory
  std::vector<cv::String> images;
  // Path of the folder containing checkerboard images
  std::string path = "./images/*.jpg";
 
  cv::glob(path, images);
 
  cv::Mat frame, gray;
  // vector to store the pixel coordinates of detected checker board corners 
  std::vector<cv::Point2f> corner_pts;
  bool success;
 
  // Looping over all the images in the directory
  for(int i{0}; i<images.size(); i++)
  {
    frame = cv::imread(images[i]);
    cv::cvtColor(frame,gray,cv::COLOR_BGR2GRAY);
 
    // Finding checker board corners
    // If desired number of corners are found in the image then success = true  
    success = cv::findChessboardCorners(gray, cv::Size(CHECKERBOARD[0], CHECKERBOARD[1]), corner_pts, CV_CALIB_CB_ADAPTIVE_THRESH | CV_CALIB_CB_FAST_CHECK | CV_CALIB_CB_NORMALIZE_IMAGE);
     
    /* 
     * If desired number of corner are detected,
     * we refine the pixel coordinates and display 
     * them on the images of checker board
    */
    if(success)
    {
      cv::TermCriteria criteria(CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 30, 0.001);
       
      // refining pixel coordinates for given 2d points.
      cv::cornerSubPix(gray,corner_pts,cv::Size(11,11), cv::Size(-1,-1),criteria);
       
      // Displaying the detected corner points on the checker board
      cv::drawChessboardCorners(frame, cv::Size(CHECKERBOARD[0], CHECKERBOARD[1]), corner_pts, success);
       
      objpoints.push_back(objp);
      imgpoints.push_back(corner_pts);
    }
 
    cv::imshow("Image",frame);
    cv::waitKey(0);
  }
 
  cv::destroyAllWindows();
 
  cv::Mat cameraMatrix,distCoeffs,R,T;
 
  /*
   * Performing camera calibration by 
   * passing the value of known 3D points (objpoints)
   * and corresponding pixel coordinates of the 
   * detected corners (imgpoints)
  */
  cv::calibrateCamera(objpoints, imgpoints, cv::Size(gray.rows,gray.cols), cameraMatrix, distCoeffs, R, T);
 
  std::cout << "cameraMatrix : " << cameraMatrix << std::endl;
  std::cout << "distCoeffs : " << distCoeffs << std::endl;
  std::cout << "Rotation vector : " << R << std::endl;
  std::cout << "Translation vector : " << T << std::endl;
 
  return 0;
}

2、python参考

import cv2
import numpy as np
import os
import glob
 
# Defining the dimensions of checkerboard
CHECKERBOARD = (6,9)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
 
# Creating vector to store vectors of 3D points for each checkerboard image
objpoints = []
# Creating vector to store vectors of 2D points for each checkerboard image
imgpoints = [] 
 
 
# Defining the world coordinates for 3D points
objp = np.zeros((1, CHECKERBOARD[0] * CHECKERBOARD[1], 3), np.float32)
objp[0,:,:2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
prev_img_shape = None
 
# Extracting path of individual image stored in a given directory
images = glob.glob('./images/*.jpg')
for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    # Find the chess board corners
    # If desired number of corners are found in the image then ret = true
    ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD, cv2.CALIB_CB_ADAPTIVE_THRESH + cv2.CALIB_CB_FAST_CHECK + cv2.CALIB_CB_NORMALIZE_IMAGE)
     
    """
    If desired number of corner are detected,
    we refine the pixel coordinates and display 
    them on the images of checker board
    """
    if ret == True:
        objpoints.append(objp)
        # refining pixel coordinates for given 2d points.
        corners2 = cv2.cornerSubPix(gray, corners, (11,11),(-1,-1), criteria)
         
        imgpoints.append(corners2)
 
        # Draw and display the corners
        img = cv2.drawChessboardCorners(img, CHECKERBOARD, corners2, ret)
     
    cv2.imshow('img',img)
    cv2.waitKey(0)
 
cv2.destroyAllWindows()
 
h,w = img.shape[:2]
 
"""
Performing camera calibration by 
passing the value of known 3D points (objpoints)
and corresponding pixel coordinates of the 
detected corners (imgpoints)
"""
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
 
print("Camera matrix : \n")
print(mtx)
print("dist : \n")
print(dist)
print("rvecs : \n")
print(rvecs)
print("tvecs : \n")
print(tvecs)