Recently, I have been reading articles about mmap writing files. In order to consolidate the learning results, there are many open source libraries that use mmap instead of sharepreference on the Internet, but there is no open source library that uses mmap to realize log printing, so I implemented an Android system. The tool log_kid that uses mmap to print logs.
Source code sharing: https://gitee.com/gggl/log_kid
Principles of MMAP:
There are many explanations on the principle of MMAP on the Internet. In fact, there are three main points to remember:
Traditional IO needs to be copied twice from user space to kernel space when writing.
There is only one memory copy for MMAP to read and write files, which is to read directly from the disk to the user space when a page fault occurs.
Using MMAP to write files can be imagined as a memory operation.
You can refer to: https://bbs.huaweicloud.com/blogs/291892
There are many on this part of the web.
log_kid implementation principle:
The implementation of log_kid refers to the implementation of mmkv.
1. log_kid is written into the log file, and the first 4 bytes of the file are the content size of the file. Since mmap needs to map the memory of an integer multiple of the page each time (the system stipulates that the reading and writing of mmap is realized by the page fault interrupt: https://juejin.cn/post/6956031662916534279 ), so there will be some 0s at the end of the file Fill data, and control the specific writing position of the file by writing 4 bytes in the file header.
2. Write the log through write_log(tag, message).
The first 4 bytes of each log are the size of the tag, followed by the content of the tag, the size of the message of 4 bytes, and the content of the message.
file structure:
Source code process:
MemoryFile corresponds to the log file in the hard disk.
initialization file
open to create a file
fstat gets the file size, if the file size is not an integer multiple of the page, expand to an integer multiple of the page
if (fileSize < pageSize || fileSize % pageSize != 0)
truncate函数对文件进行扩容，但是truncate扩容的文件属于稀疏文件，当系统出现内存或者硬盘空间不足的时候可能出现崩溃的问题，所以扩容后用0填充扩容的内容。
mmap映射文件，mmap具体参数含义网络上的讲解也比较多。
void * mmap(void *start, size_t length, int prot , int flags, int fd, off_t offset)获取文件大小，读取ptr的前4个字节，如果是新文件则为一个页的大小。
memcpy(&actualSize , ptr , 4) ;
void MemoryFile::reloadFromFile() {
int lastSlash = logFilePath.find_last_of('/') + 1 ;
std::string path = logFilePath.substr(0 , lastSlash) ;
mkPath(path) ;
fd = open(logFilePath.c_str() , O_RDWR | O_CREAT | O_CLOEXEC, S_IRWXU) ;
if (fd < 0) {
LOGD("faild to open %s" , logFilePath.c_str()) ;
} else {
struct stat st = {} ;
if (fstat(fd , &st) != -1) {
fileSize = (int)st.st_size ;
}
bool newFile = fileSize == 0 ;
if (fileSize < pageSize || fileSize % pageSize != 0) {
int roundSize = ((fileSize / pageSize) + 1 ) * pageSize ;
if (truncate(roundSize)) {
fileSize = roundSize ;
}
}
smmap() ;
if (newFile) {
actualSize = 0 ;
memcpy(ptr , &actualSize , 4) ;
} else {
memcpy(&actualSize , ptr , 4) ;
}
actualSize = actualSize + 4 ;
}
}日志写入：
计算本次写入的数据总大小：logSize，如果总大小加上actualSize大于文件大小，则文件进行扩容，grow，每次扩容为上次文件的一倍大小。
writeInt和writeStr为真正写入数据的方法。
bool MemoryFile::writeLog(const char *tag, const char *msg) {
int logSize = 0 ;
int tagSize = strlen(tag) ;
int msgSize = strlen(msg) ;
int intSize = 4 ;
logSize = tagSize + msgSize + intSize * 2 ;
if (actualSize + logSize > fileSize) {
grow() ;
}
writeInt(tagSize) ;
writeStr(tag , tagSize) ;
writeInt(msgSize) ;
writeStr(msg , msgSize) ;
memcpy(ptr , &actualSize , 4) ;
return true ;
}
void MemoryFile::writeInt(int &value) {
memcpy(ptr + actualSize , &value , 4) ;
actualSize += 4 ;
}
void MemoryFile::writeStr(const char *value , int &size) {
memcpy(ptr + actualSize , value , size) ;
actualSize += size ;
}
总结：
整个项目实现简单轻便，易于理解，易于上手，可能还达不到商业级组件的高度，还需慢慢打磨，也希望有兴趣的同学可以一起来维护这套项目。
实测表现：
使用mmap写入日志速度大概快于传统IO的4-5倍以上，目前项目并没有进一步优化，优化后的速度应该还可以进一步提升。
源码分享：https://gitee.com/gggl/log_kid
