One: Introduction to V4L2
1. Concept
V4L2 is the abbreviation of Video for linux2, which is the kernel driver of video equipment in Linux, and provides a set of unified interface specifications for drivers and applications. In Linux, a video device is a device file, which can be read and written like a normal file.
It can support multiple devices, and it can have the following interfaces:
(1) Video capture interface: The equipment for this application can be a high-frequency head or a camera. The original design of V4L2 is for this function.
(2) Video output interface (video output interface): can drive the computer's peripheral video image equipment-like equipment that can output TV signal format.
(3) Direct transmission video interface (video overlay interface): Its main job is to directly output the signal collected from the video capture device to the output device without passing through the system's CPU.
(4) Video interval blanking signal interface (VBI interface): It allows applications to access the video signal in the transmission blanking period.
(5) Radio interface: It can be used to process the audio stream received from AM or FM tuner equipment.
The structure diagram of V4L2 in linux system:
2. Programming mode
v4L2 is a programming framework for uvc drive-free usb devices, mainly used to collect usb cameras, etc. The programming mode is as follows:
(1) Turn on the device, set the initial parameters, and set the video image collection window, the size and format of the collected dot matrix through the V4L2 interface;
(2) Apply for image frame buffers, and perform memory mapping, and map these frame buffers from kernel space to user space, so that applications can read and process image data;
(3) Enqueue the frame buffer and start video capture;
(4) The driver starts the video data collection. The application program takes out the frame buffer from the video capture output queue, and after processing, puts the frame buffer into the video capture input queue again, and collects continuous video data cyclically;
(5) Release resources and stop collecting work.
Two: V4L2 instructions and workflow
It can be seen from the structure diagram that V4L2 is a character device, and most of the functions of V4L2 are exported through the ioctl of the device file.
1. Classification of common ioctl commands:
- Query Capability: Query the functions supported by the device, only VIDIOC_QUERYCAP.
- Priority related: including VIDIOC_G_PRIORITY, VIDIOC_S_PRIORITY, set priority.
- Capture related: video capture related Ioctl.
ID description VIDIOC_ENUM_FMT Enumerate all data formats supported by the device VIDIOC_S_FMT Set data format VIDIOC_G_FMT Get data format VIDIOC_TRY_FMT Same as VIDIOC_S_FMT, but does not change the state of the device VIDIOC_REQBUFS Request a video buffer from the device, that is, initialize the video buffer VIDIOC_QUERYBUF Query the status of the buffer VIDIOC_QBUF Obtain a frame of video data from the device VIDIOC_DQBUF Return the video buffer to the device, VIDIOC_OVERLAY Start or stop overlay VIDIOC_G_FBUF Get the framebuffer parameters of the video overlay device or OSD device VIDIOC_S_FBUF Set framebuffer parameters VIDIOC_STREAMON Start streaming I/O operation, capture or output device VIDIOC_STREAMOFF Close stream I/O operations - TV video standard:
ID description VIDIOC_ENUMSTD Enumerate all standards supported by the device VIDIOC_G_STD Get the standard currently in use VIDIOC_S_STD Set video standards VIDIOC_QUERYSTD Some devices support automatic detection of the video standard of the input source. At this time, use this ioctl to query the detected video standard
2. Specific flow chart
Three: software framework
1. Open the device to view properties
Use the VIDIOC_QUERYCAP command to view the structure used in the device properties. The prototype of tV4l2Cap is v4l2_capability to describe the driver information of the video capture device. According to the capabilities, it can be judged whether the current device is a capture device, and check whether to use memory mapping or direct reading to obtain image data .
struct v4l2_capability {
__u8 driver[16]; // 驱动名字
__u8 card[32]; // 设备名字
__u8 bus_info[32]; // 设备在系统中的位置
__u32 version; // 驱动版本号
__u32 capabilities; // 设备支持的操作
__u32 device_caps;
__u32 reserved[3]; // 保留字段
};
struct v4l2_capability tV4l2Cap;
iFd = open(strDevName, O_RDWR);
if (iFd < 0)
{
DBG_PRINTF("can not open %s\n", strDevName);
return -1;
}
ptVideoDevice->iFd = iFd;
iError = ioctl(iFd, VIDIOC_QUERYCAP, &tV4l2Cap);
memset(&tV4l2Cap, 0, sizeof(struct v4l2_capability));
iError = ioctl(iFd, VIDIOC_QUERYCAP, &tV4l2Cap);
if (iError) {
DBG_PRINTF("Error opening device %s: unable to query device.\n", strDevName);
goto err_exit;
}
if (!(tV4l2Cap.capabilities & V4L2_CAP_VIDEO_CAPTURE))
{
DBG_PRINTF("%s is not a video capture device\n", strDevName);
goto err_exit;
}
if (tV4l2Cap.capabilities & V4L2_CAP_STREAMING) {
DBG_PRINTF("%s supports streaming i/o\n", strDevName);
}
if (tV4l2Cap.capabilities & V4L2_CAP_READWRITE) {
DBG_PRINTF("%s supports read i/o\n", strDevName);
}
2. Display all supported formats
The v4l2_fmtdesc structure describes the format information supported by the current camera. Use the command VIDIOC_ENUM_FMT to query. The index in the v4l2_fmtdesc structure should be set, starting from 0; the v4l2_buf_type type should also be set. If the camera device is used, the v4l2_buf_TYPE type should be set to V4L2_BUF_CAPTURE, because camera is a CAPTURE device. The other content in the structure is filled by the driver.
struct v4l2_fmtdesc {
__u32 index; // 要查询的格式序号，应用程序设置
__u32 type; // 帧类型，应用程序设置
__u32 flags; // 是否为压缩格式
__u8 description[32]; // 格式名称
__u32 pixelformat; //所支持的格式
__u32 reserved[4];
};
struct v4l2_fmtdesc tFmtDesc;
memset(&tFmtDesc, 0, sizeof(tFmtDesc));
tFmtDesc.index = 0;
tFmtDesc.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
while ((iError = ioctl(iFd, VIDIOC_ENUM_FMT, &tFmtDesc)) == 0) {
if (isSupportThisFormat(tFmtDesc.pixelformat))
{
ptVideoDevice->iPixelFormat = tFmtDesc.pixelformat;
break;
}
tFmtDesc.index++;
}3. Set the image frame format
The struct v4l2_format structure needs to set enum v4l2_buf_type type and struct v4l2_pix_format pix in union fmt. Enum v4l2_buf_type type is set to V4L2_BUF_TYPE_VIDEO_CAPTURE because it uses a camera device, which is a CAPTURE device. struct v4l2_pix_format pix sets the length, width and format of a frame of image. Since it needs to adapt to the LCD output, the length and width are set to the length and width supported by the LCD.
struct v4l2_format {
__u32 type;
union {
struct v4l2_pix_format pix; /* V4L2_BUF_TYPE_VIDEO_CAPTURE */
struct v4l2_pix_format_mplane pix_mp; /* V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE */
struct v4l2_window win; /* V4L2_BUF_TYPE_VIDEO_OVERLAY */
struct v4l2_vbi_format vbi; /* V4L2_BUF_TYPE_VBI_CAPTURE */
struct v4l2_sliced_vbi_format sliced; /* V4L2_BUF_TYPE_SLICED_VBI_CAPTURE */
struct v4l2_sdr_format sdr; /* V4L2_BUF_TYPE_SDR_CAPTURE */
struct v4l2_meta_format meta; /* V4L2_BUF_TYPE_META_CAPTURE */
__u8 raw_data[200]; /* user-defined */
} fmt;
};
struct v4l2_format tV4l2Fmt;
/* set format in */
GetDispResolution(&iLcdWidth, &iLcdHeigt, &iLcdBpp);
memset(&tV4l2Fmt, 0, sizeof(struct v4l2_format));
tV4l2Fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
tV4l2Fmt.fmt.pix.pixelformat = ptVideoDevice->iPixelFormat;
tV4l2Fmt.fmt.pix.width = iLcdWidth;
tV4l2Fmt.fmt.pix.height = iLcdHeigt;
tV4l2Fmt.fmt.pix.field = V4L2_FIELD_ANY;
iError = ioctl(iFd, VIDIOC_S_FMT, &tV4l2Fmt);
4. Apply for buffer zone
Apply for a buffer with NB_BUFFER buffered frames, __u32 count is the number of buffered frames; enum v4l2_buf_type type is the same as before, set to V4L2_BUF_TYPE_VIDEO_CAPTURE; enum v4l2_memorymemory is used to distinguish between memory mapping and user pointer, we use memory mapping, So set to V4L2_MEMORY_MMAP.
struct v4l2_requestbuffers {
__u32 count;
__u32 type; /* enum v4l2_buf_type */
__u32 memory; /* enum v4l2_memory */
__u32 reserved[2];
};
struct v4l2_requestbuffers tV4l2ReqBuffs;
/* request buffers */
memset(&tV4l2ReqBuffs, 0, sizeof(struct v4l2_requestbuffers));
tV4l2ReqBuffs.count = NB_BUFFER;
tV4l2ReqBuffs.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
tV4l2ReqBuffs.memory = V4L2_MEMORY_MMAP;
iError = ioctl(iFd, VIDIOC_REQBUFS, &tV4l2ReqBuffs);
if (iError)
{
DBG_PRINTF("Unable to allocate buffers.\n");
goto err_exit;
}
5. Map the requested buffer frame from kernel space to user space and put the buffer frame in a queue
struct v4l2_buffer {
__u32 index;
__u32 type;
__u32 bytesused;
__u32 flags;
__u32 field;
struct timeval timestamp;
struct v4l2_timecode timecode;
__u32 sequence;
/* memory location */
__u32 memory;
union {
__u32 offset;
unsigned long userptr;
struct v4l2_plane *planes;
__s32 fd;
} m;
__u32 length;
__u32 reserved2;
__u32 reserved;
};
struct v4l2_buffer tV4l2Buf;
if (tV4l2Cap.capabilities & V4L2_CAP_STREAMING)
{
/* map the buffers */
for (i = 0; i < ptVideoDevice->iVideoBufCnt; i++)
{
memset(&tV4l2Buf, 0, sizeof(struct v4l2_buffer));
tV4l2Buf.index = i;
tV4l2Buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
tV4l2Buf.memory = V4L2_MEMORY_MMAP;
iError = ioctl(iFd, VIDIOC_QUERYBUF, &tV4l2Buf);
if (iError)
{
DBG_PRINTF("Unable to query buffer.\n");
goto err_exit;
}
ptVideoDevice->iVideoBufMaxLen = tV4l2Buf.length;
ptVideoDevice->pucVideBuf[i] = mmap(0 /* start anywhere */ ,
tV4l2Buf.length, PROT_READ, MAP_SHARED, iFd,
tV4l2Buf.m.offset);
if (ptVideoDevice->pucVideBuf[i] == MAP_FAILED)
{
DBG_PRINTF("Unable to map buffer\n");
goto err_exit;
}
}
/* Queue the buffers. */
for (i = 0; i < ptVideoDevice->iVideoBufCnt; i++)
{
memset(&tV4l2Buf, 0, sizeof(struct v4l2_buffer));
tV4l2Buf.index = i;
tV4l2Buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
tV4l2Buf.memory = V4L2_MEMORY_MMAP;
iError = ioctl(iFd, VIDIOC_QBUF, &tV4l2Buf);
if (iError)
{
DBG_PRINTF("Unable to queue buffer.\n");
goto err_exit;
}
}
}