One, file operation
1. Static files and dynamic files
Files are usually stored in the file system in the block device. We call this kind of files a static file.
When we go to open to operate this file, the kernel will create an open file data structure in the process to record the information about our operation of this file (the kernel will also apply for a piece of memory in the memory to remove the memory of this static file from the block device The specific address read in the memory is used for management, this time it is called a dynamic file)
Why is it designed like this? Because the read operation of the block device is not flexible, and the memory can be read in bytes, the establishment of a dynamic file mechanism is easy to manage.
2. File descriptor
The file descriptor is essentially a number. This number is linked to the previous (the kernel will create an open file data structure in the process to record the information we operate on this file).
The function is to distinguish multiple files opened by a program.
3. API usage for reading and writing files
3.1, open
int open(const char *pathname, int flags); the return value is the file descriptor
Note: The fd program must be recorded, and all future operations to this file must rely on this fd to correspond to this file, and finally when closing the file, fd is also required to specify to close the file. If the fd is lost before we close the file, it will be miserable. The file cannot be closed and cannot be read or written.
The legal range of file descriptor fd in Linux is 0 or a positive number, and it cannot be a negative number. You can use this to determine whether the open file operation is successful
3.2、read
size_t read(int fd, void *buf. size_t count)
3.3、write
size_t write(int fd, const void *buf, size_t count)
3,4, the flags parameter in open
Read and write permissions: O_RDONLY O_WRONLY O_RDWR
When opening a file that exists and has content: O_APPEND, O_TRUNC
O_APPEND at the bottom layer is to do one more synchronization operation on the file pointer. When there are multiple fd executing the same file, if it is opened in O_APPEND mode, then the movement of one file pointer will also be synchronized with the other, so as to achieve the effect of appending. The operation of O_APPEND is atomic and cannot be interrupted.
When opening a file that does not exist: O_CREAT, O_EXCL
We open a file by default in blocking mode. If you want to open the file in a non-blocking way, add the O_NONBLOCK flag to the flag. (Only used for device files, not for ordinary files)
O_SYNC write blocks waiting for the bottom layer to finish writing before returning to the application layer, writing to the hardware and then returning instead of the buffer area.
When there is no O_SYNC, write just writes the content into the bottom buffer to return, and then the bottom layer (the code responsible for implementing open and write operations in the operating system, as well as the code for reading and writing hard disks and other low-level hardware in the OS) at the appropriate time The content in buf will be synchronized to the hard disk at one time. This design is to improve the performance and sales of the hardware operation, and to increase the life of the hardware; but sometimes we hope that the hardware is not easy to wait, and write our content directly to the hard disk. At this time, we can use the O_SYNC flag.
3,5, lseek
off_t lseek (int fd, off_t offset, int whence);
file pointer; indicates that we are currently operating the position of the file stream, because this pointer cannot be directly accessed, thus providing the lseek api to access.
The return value of off_t is the offset of the beginning of the file in the file pointer, so you can use lseek to get the file size
ret = lseek(fd, 0, SEEK_END); the
offset parameter is the offset for the whyce flag
lseek constructs empty files to facilitate multi-threaded operation
3.6. The fcntl function
int fcntl(int fd, int cmd, / args /)
uses certain commands to operate fd.
For example, the cmd of F_DUPFD is to copy the file descriptor
4. How to manage files in linux system
4.1.
How does the inode (i-node) operating system in the static file access the hard disk to get the file;
read the hard disk content management table —>
find the sector-level information of the file we want to access —>
use this information to query Go to the area where the content is actually stored ->
get the file we want; (note that the operating system initially got the file name)
In the hard disk management, the file is used as the unit, and each file has a corresponding inode data structure node to store the information of the file.
4.2. The vnode (v node) in the dynamic file. The
operation of opening the file in the process is actually a process. However, each process has a data structure to store all the information in the process. This is called the process information table.
In the table, there will be a pointer to a file management table, and the file management table is to record all the files and related information opened by the current process, that is, the fd file descriptor.
What we finally find through the fd file descriptor is the management structure vnode of a file that has been opened.
5. Realization of file sharing
The principle of automatic allocation of fd, (0, 1, 2 are already occupied)
this is file sharing, multiple fd, that is, the file table finally points to the same
file. The core of V node file sharing is how to get multiple fd points a file with
the same process many times to open the same file
in different processes are open the same file
using linux system provides two API dup and dup2 complete copy of the file descriptor
int dup (int oldfd) cannot point to the new fd, and
int dup2 (int oldfd, int newfd) can be assigned a new fd automatically according to the operating system allocation principle
You can use dup2 to complete output relocation.
In the linux command line,> relocation is actually open to open a file, close to close the standard output stdout, and then use dup2 to associate the fd of the opened file with the standard output with fd of 1.
6. Standard IO library
Compared with file IO, the standard IO library has an extra layer of encapsulation to support different operating systems, and an extra layer of buffering mechanism provides better performance. The rest of the functions are similar
7. File attribute and its attribute acquisition
7.1. File type
Ordinary file (- regular file)
directory file (d directory)
character device file (c character)
pipe file (p pipe)
socket file (s socket)
symbolic link file (l link)
7.2. Get common file attributes
7.2.1, stat, fstat, lstat functions
int stat(const char *path, struct stat *buf)
int fstat(int fb, struct stat *buf)
int lstat(const char *path, struct stat *buf) The difference between this and stat is in the symbolic link file, lstat refers to the attributes of the symbolic link file itself, and stat is the file attribute pointed to by the symbolic link file
struct
stat is a structure defined by the kernel, declared in <sys/stat.h>, so we can use it. All the elements in this structure add up to our file attribute information.
7.2.2 File attributes and file permissions Obtain
the file type flag in the file attribute In the mode_t st_mode element of the struct stat structure, this element is actually a bit flag defined by bits (somewhat similar to the CPSR register of ARM CPU Mode bit definition). This thing is composed of many flag bits and records a lot of information. If you want to search by bit & operation, you will know the result, but because these bit definitions are not easy to remember, the linux system defines a lot of macros for everyone in advance. Proceed accordingly.
In addition to the file type, st_mode also records an important information: file permissions.
The access function checks the permission setting,
that is, you can judge whether you have permission to do this operation before operating a certain file
int access(const char*pathname, int mode); return 0 to execute successfully, return -1 to fail without permission
mode; F_OK to test the file Does it exist, R_OK, W_OK, X_OK
chmod/fchmode permission modification setting
int chmod(const char*pathname, int mode); return 0 to execute successfully, return -1 to fail without permission
chmod(argv[1], S_IRUSR|S_IWUSR|S_IXUSR).
umask and file permission mask.
File mask is a global variable setting maintained in the linux system. The function of umask is to set the default permissions of newly created files in our system.
umask 0022 corresponds to file permissions rwxr-xr-x
7.2.3, read the directory file
opendir readdir function
dirent structure
7.2.4. Non-reentrant functions and reentrant functions
(usually non-reentrant version function name _r).
Reentrant means that the memory returned by the function can be operated again, and non-reentrant means that it cannot be modified and can only be read.
2. Advanced IO
1. Blocking and non-blocking
Blocking; the current process will always block and wait here until the conditions are met.
Non-blocking; will not wait, execute directly, so the execution result may fail
How to achieve non-blocking IO access; add O_NONBLOCK flag when opening, or use fcntl to set file descriptor attributes.
2. Solve the problem of concurrent IO
2.1, polling
It is to use the while loop to process, but it has been occupying the cpu to affect the efficiency
2, 2, IO multiplexing
It is to add an API to monitor and improve efficiency based on the low efficiency of while(1) polling. Actually come while1 in this api and set sleep, not the operation of while1 between us
select and poll
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/select.h>
#include <sys/time.h>
int main(void)
{
// 读取鼠标
int fd = -1, ret = -1;
char buf[200];
fd_set myset;
struct timeval tm;
fd = open("/dev/input/mouse1", O_RDONLY);
if (fd < 0)
{
perror("open:");
return -1;
}
// 当前有2个fd，一共是fd一个是0
// 处理myset
FD_ZERO(&myset);//将集合情况
FD_SET(fd, &myset);//将mouse1设备的fd设置到set中
FD_SET(0, &myset);//将键盘fd0标准输入设备fd设置进入
tm.tv_sec = 10;//设置超时时间，超过还未反应则提示失败
tm.tv_usec = 0;
//select内部循环遍历监督
ret = select(fd+1, &myset, NULL, NULL, &tm);
if (ret < 0)
{
perror("select: ");
return -1;
}
else if (ret == 0)
{
printf("超时了\n");
}
else
{
// 等到了一路IO，然后去监测到底是哪个IO到了，处理之
if (FD_ISSET(0, &myset))
{
// 这里处理键盘
memset(buf, 0, sizeof(buf));
read(0, buf, 5);
printf("键盘读出的内容是：[%s].\n", buf);
}
if (FD_ISSET(fd, &myset))
{
// 这里处理鼠标
memset(buf, 0, sizeof(buf));
read(fd, buf, 50);
printf("鼠标读出的内容是：[%s].\n", buf);
}
}
return 0;
}
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <poll.h>
int main(void)
{
// 读取鼠标
int fd = -1, ret = -1;
char buf[200];
struct pollfd myfds[2] = {
0};
fd = open("/dev/input/mouse1", O_RDONLY);
if (fd < 0)
{
perror("open:");
return -1;
}
// 初始化我们的pollfd
myfds[0].fd = 0; // 键盘
myfds[0].events = POLLIN; // 等待读操作
myfds[1].fd = fd; // 鼠标
myfds[1].events = POLLIN; // 等待读操作
ret = poll(myfds, fd+1, 10000);
if (ret < 0)
{
perror("poll: ");
return -1;
}
else if (ret == 0)
{
printf("超时了\n");
}
else
{
// 等到了一路IO，然后去监测到底是哪个IO到了，处理之
if (myfds[0].events == myfds[0].revents)
{
// 这里处理键盘
memset(buf, 0, sizeof(buf));
read(0, buf, 5);
printf("键盘读出的内容是：[%s].\n", buf);
}
if (myfds[1].events == myfds[1].revents)
{
// 这里处理鼠标
memset(buf, 0, sizeof(buf));
read(fd, buf, 50);
printf("鼠标读出的内容是：[%s].\n", buf);
}
}
return 0;
}
2.3, asynchronous IO
The working method of the asynchronous IO of the interrupt response system implemented by the operating system with software is: our current process registers an asynchronous IO event (using signal to register a signal SIGIO processing function), and then the current process can process its own affairs normally. After the asynchronous event occurs, the current process will receive a SIGIO signal to execute the bound processing function to process the asynchronous event.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <signal.h>
int mousefd = -1;
// 绑定到SIGIO信号，在函数内处理异步通知事件
void func(int sig)
{
char buf[200] = {
0};
if (sig != SIGIO)
return;
read(mousefd, buf, 50);
printf("鼠标读出的内容是：[%s].\n", buf);
}
int main(void)
{
// 读取鼠标
char buf[200];
int flag = -1;
mousefd = open("/dev/input/mouse1", O_RDONLY);
if (mousefd < 0)
{
perror("open:");
return -1;
}
// 把鼠标的文件描述符设置为可以接受异步IO
flag = fcntl(mousefd, F_GETFL);
flag |= O_ASYNC;
fcntl(mousefd, F_SETFL, flag);
// 把异步IO事件的接收进程设置为当前进程
fcntl(mousefd, F_SETOWN, getpid());
// 注册当前进程的SIGIO信号捕获函数
signal(SIGIO, func);
// 读键盘
while (1)
{
memset(buf, 0, sizeof(buf));
//printf("before 键盘 read.\n");
read(0, buf, 5);
printf("键盘读出的内容是：[%s].\n", buf);
}
return 0;
}
2.4, storage mapping IO
Use mmpp function to complete memory mapping, pay attention to sharing rather than copying, the
case LCD picture display, IPC shared memory
Multithreading can also solve the problem of concurrent IO