Part11. The meaning of digital images
Images in OpenCV actually refer to digital images. Before introducing the concept of images, let’s introduce a few basic concepts:
Pixel is the basic unit or element of an image, or the smallest unit of an image. Pixels in an image contain different pixel values. For gray and white images, the pixel value is a value between 0-255; for a color image with RGB 3 channels, the pixel value of each channel is 0-255; for a two-dimensional black and white image, These pixels form a two-dimensional matrix; for two-dimensional color pictures, these pixels are a multi-dimensional matrix.
Digital Image Processing (Digital Image Processing), it is called computer image processing technology.
After briefly introducing these basic concepts, the Mat class in OpenCV will be introduced next. Starting from this article, the main language used to use OpenCV is C++. Of course, as long as you have some programming foundation in common high-level languages, it is not difficult to understand.
Part22. Basic structure of Mat
Since OpenCV 2.x, Mat is the most basic and important class of OpenCV . Mat is the abbreviation of Matrix, which means matrix. It is a container for images, which is a two-dimensional vector. Mat is a class used to represent images in OpenCV. For the C++ version, Mat supports automatic memory management without the need to apply for and release memory, so it is very friendly to programmers. (Of course, you can also create it manually)
Mat can store not only images but also matrices.
Mat consists of two data parts: matrix header and pointer.
class CV_EXPORTS Mat{
public:
// ... a lot of methods ...
...
/*! includes several bit-fields:
- the magic signature
- continuity flag
- depth
- number of channels
*/
int flags;
//! the array dimensionality, >= 2
int dims;
//! the number of rows and columns or (-1, -1) when the array has more than 2 dimensions
int rows, cols;
//! pointer to the data
uchar* data;
//! pointer to the reference counter;
// when array points to user-allocated data, the pointer is NULL
int* refcount;
// other members
...
};Matrix header: Contains matrix size, storage method, storage address and other information. Corresponds to flags, dims, rows, cols, data pointer, and refcount pointer in the Mat class. So the size of the matrix header is constant .
Pointer: Points to a matrix containing pixel values (data can be stored in any dimension according to different storage methods selected), which is the space pointed by the data pointer. Typically, the matrix is several orders of magnitude larger than the matrix header.
In image processing, the simplest operations we use daily are to create, transfer, and copy Mat objects.
During the image creation process, the large overhead mainly comes from the matrix. If you want to copy and assign Mat objects and use deep copy, the efficiency will be greatly reduced. It happens that OpenCV itself uses a reference counting mechanism (you can see the refcount pointer in the Mat class), that is, the matrix pointer of the Mat object points to the same address. This mechanism allows us to use shallow copies.
The constructor and assignment operator of Mat only copy the matrix header and matrix pointer , but not the matrix itself.
Part33. Reading and creation of Mat
Next, let's learn how to use the Mat object and use it to read and create images.
13.1 Reading of images
Before image creation, let's first introduce image reading. We can read Mat objects from pictures, videos, etc. For example, the following code reads a Mat object from an image file.
String fileName = "/Users/tony/images/test.jpg";
Mat srcImage;
srcImage = imread(fileName);
//判断图像是否加载成功
if (srcImage.empty()){
cout << "图像加载失败" << endl;
return -1;
}The cv::imread() function is used to read Mat objects from files. Among them, the first parameter fileName is the file name containing the absolute path of the file.
CV_EXPORTS_W Mat imread( const String& filename, int flags = IMREAD_COLOR );Its second parameter indicates the image reading mode, the default is IMREAD_COLOR , which indicates converting the image into a three-channel BGR color image. The following are the flags that can be used by the imread function:
enum ImreadModes {
IMREAD_UNCHANGED = -1, //按原样返回加载的图像（会带上alpha通道）。忽略EXIF方向。
IMREAD_GRAYSCALE = 0, //将图像转为单通道灰度图
IMREAD_COLOR = 1, //将图像转为BGR三通道彩色图像
IMREAD_ANYDEPTH = 2, //如果图像深度为16-bit/32-bit则会返回该深度图像，否则返回8-bit图像
IMREAD_ANYCOLOR = 4, //按照任意颜色图像格式读取
IMREAD_LOAD_GDAL = 8, //使用gdal驱动程序加载图像
IMREAD_REDUCED_GRAYSCALE_2 = 16, //将图像转为单通道灰度图且图像尺寸变为1/2
IMREAD_REDUCED_COLOR_2 = 17, //将图像转为BGR三通道彩色图像且图像尺寸变为1/2
IMREAD_REDUCED_GRAYSCALE_4 = 32, //将图像转为单通道灰度图且图像尺寸变为1/4
IMREAD_REDUCED_COLOR_4 = 33, //将图像转为BGR三通道彩色图像且图像尺寸变为1/4
IMREAD_REDUCED_GRAYSCALE_8 = 64, //将图像转为单通道灰度图且图像尺寸变为1/8
IMREAD_REDUCED_COLOR_8 = 65, //将图像转为BGR三通道彩色图像且图像尺寸变为1/8
IMREAD_IGNORE_ORIENTATION = 128 //忽略EXIF中的方向标识，不旋转图像
};If we want to read the image in its original type, we can select IMREAD_UNCHANGED , so that the original type of the image will be consistent with the type read in.
There is an imreadmulti function which is similar to the imread function and is used to read multiple images from a file. For example, read multiple Mat objects from a tiff file.
CV_EXPORTS_W bool imreadmulti(const String& filename, CV_OUT std::vector<Mat>& mats, int flags = IMREAD_ANYCOLOR);23.2 Creation of Mat
The Mat class has many constructors and itself has many functions that can be used to create Mat objects. This article and the following content do not intend to explain each function in detail, so here we only list common scenarios in actual use.
3.2.1 Create using constructor
Create a 3*3 3-channel matrix, with the value of each pixel being (0,0,255).
cv::Mat src(3, 3, CV_8UC3, cv::Scalar(0, 0, 255));
std::cout << "src = " << std::endl << src << std::endl;Output result:
src =
[ 0, 0, 255, 0, 0, 255, 0, 0, 255;
0, 0, 255, 0, 0, 255, 0, 0, 255;
0, 0, 255, 0, 0, 255, 0, 0, 255]Here, CV_8UC3 is expressed as a matrix in 3-channel Unsigned 8bits format , that is, BGR 3-channel.
A little tidying up, matrix data type: CV_<bit_depth>(S|U|F)C<number_of_channels>
bit_depth, number of bits, such as 8 bits, 16 bits, 32 bits, 64 bits
S|U|FS: signed int, signed integer U: unsigned int, unsigned integer F: float, single-precision floating point type
C<number_of_channels> represents the number of channels of an image, for example: 1: single-channel image, indicating grayscale image. 3: 3-channel image, representing RGB color image. 4: 4-channel image, representing an RGB image with an Alpha (transparency) channel.
In OpenCV, similar matrix data types include CV_16SC3, CV_32FC3, CV_64FC3 and so on. In the next article, we will introduce the matrix data type in detail.
As another example, use the constructor to create a Mat object of a specified size (400*400) and specify the color value (0,0,255) of each pixel. Actually, it will show a red picture.
cv::Mat src(cv::Size(400,400), CV_8UC3, cv::Scalar(0,0,255));
imshow("red",src);Output result:
Scalar literally means scalar, which is a template class of 4 vector elements derived from Vec. The Scalar type is widely used in OpenCV to pass pixel values.
The common constructors of Scalar are
Scalar_();
Scalar_(_Tp v0, _Tp v1, _Tp v2=0, _Tp v3=0);
Scalar_(_Tp v0);
Scalar_(const Scalar_& s);
Scalar_(Scalar_&& s) CV_NOEXCEPT;When Scalar represents color, the single-channel image is represented by the subscript [0], and the three-channel image is represented by the subscripts [0], [1], [2] to represent the B, G, and R channels. Therefore, cv::Scalar(0,0,255) corresponds to red.
3.2.2 Create using array
Similarly, use the constructor to create a 3*3 matrix.
int array[2] = { 3, 3 };
cv::Mat src(2, array, CV_8UC1, cv::Scalar::all(0));
std::cout << "src = " << std::endl << src << std::endl;Output result:
src =
[ 0, 0, 0;
0, 0, 0;
0, 0, 0]The first parameter of the constructor represents the dimension of the matrix, and the second parameter represents an integer array specifying the shape of the n-dimensional array. Therefore, the array array here represents the number of each dimension.
3.2.3 Create using create function
Use the create() function to create a 3*3 two-dimensional single-channel matrix.
cv::Mat src;
src.create(3, 3, CV_8UC1);
std::cout << "src = " << std::endl << src << std::endl;Output result:
src =
[ 0, 0, 51;
2, 0, 96;
0, 0, 64]The create() function can only create a matrix of a specified size and a specified matrix data type, and cannot set an initial value for the matrix. It reallocates memory for the matrix data when changing the matrix size, so each data in the matrix it creates is a random value .
3.2.4 Special matrix creation
OpenCV has functions similar to MATLAB that can quickly assign values and create matrices, generating all-0 matrices, identity matrices, and diagonal matrices.
cv::Mat mat_zeros = cv::Mat::zeros(3,3,CV_8UC1); // 全 0 矩阵
cout<<"mat_zeros="<<endl<<mat_zeros<<endl;
cv::Mat mat_ones = cv::Mat::ones(3,3,CV_8UC1); // 单位矩阵
cout<<"mat_ones="<<endl<<mat_ones<<endl;
cv::Mat mat_eye = cv::Mat::eye(3,3,CV_8UC1); // 对角矩阵
cout<<"mat_eye="<<endl<<mat_eye<<endl;Output result:
mat_zeros=
[ 0, 0, 0;
0, 0, 0;
0, 0, 0]
mat_ones=
[ 1, 1, 1;
1, 1, 1;
1, 1, 1]
mat_eye=
[ 1, 0, 0;
0, 1, 0;
0, 0, 1]3.2.5 Create using custom matrix Mat
We can also define some matrices with relatively small data size ourselves, for example:
cv::Mat src = (cv::Mat_<double>(3, 3) << 1, 0, 0, 0, 1, 0, 0, 0, 1);
std::cout << "src = " << std::endl << src << std::endl;Output result:
src =
[1, 0, 0;
0, 1, 0;
0, 0, 1]3.2.6 Extract the ROI of a Mat object
ROI (Region Of Interest) represents the region of interest. Often, extracting the ROI facilitates further analysis of the image.
Commonly used methods for extracting ROI areas include:
Use cv::Rect to specify the coordinates of the upper left corner of the rectangle , as well as its width and height, to extract the ROI area.
Mat src = imread("/Users/tony/images/test.jpg");
Mat roi = src(Rect(300, 400, 200, 300));//Rect 四个形参分别表示 x 坐标，y 坐标，宽，高Use cv::Range to specify the range of rows or columns of interest and extract the ROI area.
Mat src = imread("/Users/tony/images/test.jpg");
Mat roi = src(Range(150, 150 + 100), Range(250, 250 + 100));//Range两个形参分别是：起始行或列，起始行或列+偏移量As another example, read a picture and display the image. Then extract the ROI of the image and display that ROI.
cv::Mat src = imread("...");
imshow("src",src);
Mat roi = src(Rect(1200, 800, 1500, 2500));
imshow("roi",roi);Output result:
Part44. Assignment of Mat
34.1 Shallow copy and deep copy
Shallow copy: Only the pointer to an object is copied, but the object itself is not copied. The old and new objects will share the same memory, and modifications to either will affect the other. Deep copy: Create an identical object. The old and new objects do not share memory, and modifications to either party will not affect the other.
As mentioned before, Mat's copy constructor and assignment operator are both shallow copies, and the extraction of ROI is also a shallow copy . For example:
Mat a = imread("/Users/tony/images/test.jpg");
Mat b(a); // 拷贝构造函数
Mat c = a; // 赋值运算符When any object in a, b, c is deleted, the remaining two objects will not point to empty data. The matrix data will only be released when all three objects are deleted. As mentioned earlier, this relies on the reference counting mechanism (refcount pointer) of the Mat object. Its essence is used for memory management of objects of different types pointing to the same data address. This will only happen when the number of matrix data references is 0. Release matrix data.
44.2 clone() and copyTo()
Sometimes, we still need to create a new Mat object and copy the matrix data itself. Then we can use the clone() and copyTo() functions to implement deep copy .
For example:
Mat a;
Mat b = a.clone();// 对 a 进行克隆
Mat c;
a.copyTo(c);// 将 a 拷贝到 c 对象When any one of the objects a, b, c is changed, the other two objects will not be affected.
The copyTo() function has two forms:
srcImage.copyTo(dstImage): Copy the contents of srcImage to dstImage;
srcImage.copyTo(dstImage, mask): mask is a mask . When srcImage and mask are operated, the result is copied to dstImage. Among them, mask must be of CV_8U type, and the size must be consistent with srcImage and dstImage.
Mask operation rules:
At any position (x,y) in the image, if the pixel value of mask is equal to 1, then dstImage(x,y) = srcImage(x,y). If the pixel value of mask is equal to 0, then dstImage(x,y) = 0
Therefore, when using the copyTo() function, copy the pixels corresponding to the original srcImage that is not 0 on the mask. The copy result is copied to the target object dstImage.
Take an example of copyTo() function and using mask:
cv::Mat a = (cv::Mat_<double>(3, 3) << 0, 0, 0, 0, 0, 0, 0, 240, 0);
std::cout << "a = " << std::endl << a << std::endl;
Mat mask = Mat::eye(3,3,CV_8UC1);
std::cout << "mask = " << std::endl << mask << std::endl;
Mat roi;
a.copyTo(roi,mask);
std::cout << "roi = " << std::endl << roi << std::endl;Output result:
a =
[0, 0, 0;
0, 0, 0;
0, 240, 0]
mask =
[ 1, 0, 0;
0, 1, 0;
0, 0, 1]
roi =
[0, 0, 0;
0, 0, 0;
0, 0, 0]Part55. Summary
As a preparation for getting started, this article briefly introduces the basic knowledge related to images, the basic structure of Mat, and the creation/reading/assignment of Mat. On this basis, a lot of knowledge is also derived, such as matrix data types, masks, etc., these contents are very important. Therefore, they will be used in the subsequent content, so they will be introduced in further detail.
[ Java and Android Technology Stack] Public Account
Pay attention to Java/Kotlin server, desktop, Android, machine learning, and client-side intelligence
For more exciting content, please pay attention to: