MaixDock implements face detection
Article directory
Preparation part:
- Maix series development board
I use MaixDock
to connect hardware here:
2. MaixPy_IDE
download link: MaixPy_IDE
select exe, after downloading, click to install directly, after opening it looks like this, those who have used openmv should be very familiar with this interface.
If the frame buffer on the right is not displayed after opening, click a button on the right and drag the frame buffer out.
Run the first program, light up the lcd
Connect the development board to the computer. If you have not used the CH340 before, you can check in the device manager to see if the device is read. If not, you can download this driver: USB to serial port Windows driver. Open the IDE
, You can select the development board in the toolbar. But at present, it seems that whichever you choose is the same, whether it is the routine or the github library pointed to, there is
a routine in the IDE, and the meaning of each sentence is as follows
import sensor, image, time, lcd #导入摄像头，图像，时钟，lcd屏幕库
lcd.init(freq=15000000) #设置lcd的频率，如果屏幕体质比较差就减小点，体质比较好就增大点
sensor.reset() # 摄像头初始化
sensor.set_pixformat(sensor.RGB565) # 选择图像格式，可以选择RGB565或者灰度
sensor.set_framesize(sensor.QVGA) # 设置图像分辨率
sensor.skip_frames(time = 2000) # 刚初始化图像不稳定，跳过前面的一段图像
clock = time.clock() # Create a clock object to track the FPS.
while(True):
clock.tick() # 跟踪运行的时间
img = sensor.snapshot() # 读取摄像头的一帧图像并放到img里
lcd.display(img) # 将img,也就是那帧图像用lcd显示
print(clock.fps()) # 输出当前帧率，由于传输图像到IDE是需要消耗性能的，所以裸机跑是要比在IDE上帧率高
To know the specific usage of these functions, you can refer to the API manual
of this document . Then click the link below
to select the current corresponding serial port. If you don’t know, you can check it in the device manager. After
clicking and running
for a few seconds, you can see the IDE and LCD screen Real-time display of camera images
Implement face detection
Before implementing face detection, let's understand how to complete the firmware update, because some boards may not have the latest firmware, which may cause some problems.
First download a tool to write firmware and models
Download link: kflash_gui
Select the corresponding platform, here is windows, after downloading, unzip, find kflash_gui.exe and execute it.
Then download the latest firmware, click on the latest firmware download address , and then there will be a bunch of different types of firmware, we only look at the files ending in .bin. This is the official description, but now the nomenclature has changed slightly. What I use here is: After downloading, use the tool just now to burn the firmware into it. Click to open the file, select the firmware you just downloaded (ends with .bin), the initial address is 0x00000 by default, then select the development board, and then choose to download to Flash or SRAM, after downloading to Flash, power off and on again, it is still valid , but if it is downloaded to SRAM, it will be invalid after power off, so it is recommended to download it to Flash. For the port, choose the port of CH340 on the device manager in your computer. The baud rate is the default, and the speed mode selects high speed. If the download fails , you can try to select low-speed mode, and then try to lower the baud rate. After burning the firmware, connect to MaixPy IDE again. If there is no problem, you can go to the next step and download the model. The download link of the face detection model: Official model Select this: After decompression, don’t look at its suffix, which seems to be unknown, but it is actually similar to .zip. After decompression, you can see a .kmodel file and a json file. Open the json file, and you can see the address 0x300000 where we should download this firmware. Now open the software Kflash_gui that just updated the firmware. The starting address is 0x300000. Note that there is no missing 0 here , 0x300000 is a little more than 3M, It will not affect the firmware just written. If a 0 is missing, the firmware just written will not be used, and you will find that you can’t connect when you open the IDE. The rest are the same as before, just select and download. Open the IDE again, and after the connection is successful, it means that the above is all right.
Then the code for face detection is here: demo_find_face.py
github now needs to go online scientifically, if not, there is also a mirror image of this on gitee: demo_find_face.py
After opening, copy the code to the IDE, pay attention to whether the address here is consistent with the
above If there is no problem, there may be this error:
if there is no problem, after connecting, click Run to see the effect:
It can be seen that yolo2 only needs 24ms to recognize a frame, which is still very fast. But the recognition rate here is actually not high. It seems that the recognition rate is higher when the image is relatively small. I will study it further.
In addition, the model is written into the flash here, and we can also write it into the SD card.
Below are the annotations I wrote
import sensor, image, lcd, time #导入模块
import KPU as kpu #导入KPU模块
import gc, sys #导入模块，gc是内存回收相关模块，sys是系统的一些内置的模块
# 定义一个函数，用来将错误打印到lcd屏幕上，（这些错误也同时会答应到IDE的串行终端上）
def lcd_show_except(e):
import uio #uio输入/输出流
err_str = uio.StringIO() #创建一个用于文本形式输入/输出的内存文件类对象
sys.print_exception(e, err_str) #打印异常
err_str = err_str.getvalue() #获取保存数据的底层缓冲区的内容。
img = image.Image(size=(224,224)) #创建图像，并将异常信息写入
img.draw_string(0, 10, err_str, scale=1, color=(0xff,0x00,0x00))
lcd.display(img) #打印到LCD上
def main(model_addr=0x300000, lcd_rotation=0, sensor_hmirror=False, sensor_vflip=False): #设置了模块的起始地址，lcd上图像的旋转角度，图像的水平以及垂直方向是否翻转
try:
sensor.reset() #重启摄像头
except Exception as e:
raise Exception("sensor reset fail, please check hardware connection, or hardware damaged! err: {}".format(e))
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_hmirror(sensor_hmirror)
sensor.set_vflip(sensor_vflip) #摄像头的色彩，帧大小，翻转设置
sensor.run(1) #1表示开始抓取图像
lcd.init(type=1) #lcd初始化
lcd.rotation(lcd_rotation) #lcd旋转角度
lcd.clear(lcd.WHITE) #清屏
anchors = (1.889, 2.5245, 2.9465, 3.94056, 3.99987, 5.3658, 5.155437, 6.92275, 6.718375, 9.01025) #模型的锚点，不能改动
try:
task = None
task = kpu.load(model_addr) #从flash或者文件系统中加载模型，不需要关键词，传入参数即可，比如0x300000；"/sd/m.kmodel"
print(task) #{"model_addr": 3145728, "model_size": 388776, "model_path": "(null)", "net_args": "(null)"}
#yolo2初始化 参数分别为：
#模型对象，
#概率阈值， 只有是这个物体的概率大于这个值才会输出结果
#box_iou 门限, 为了防止同一个物体被框出多个框，当在同一个物体上框出了两个框，这两个框的交叉区域占两个框总占用面积的比例 如果小于这个值时， 就取其中概率最大的一个框
#anchor 的锚点数， 这里固定为 len(anchors)//2
#锚点
kpu.init_yolo2(task, 0.5, 0.3, 5, anchors) # threshold:[0,1], nms_value: [0, 1]
while(True):
img = sensor.snapshot() #读取一帧图像
t = time.ticks_ms() #记录当前时间
objects = kpu.run_yolo2(task, img) #运行yolo2， 返回一个kpu_yolo2_find 列表
t = time.ticks_ms() - t #记录当前时间，与上一次的做差，得到运行一帧需要的时间
if objects: #
for obj in objects: #列表不为空的话，遍历并画出个矩形框
img.draw_rectangle(obj.rect())
img.draw_string(0, 200, "t:%dms" %(t), scale=2) #在图像上显示时间
lcd.display(img) #将图像打印到lcd上
except Exception as e:
raise e
finally:
if not task is None:
kpu.deinit(task) #释放模型占用的内存， 立即释放， 但是变量还在
if __name__ == "__main__":
try:
main( model_addr=0x300000, lcd_rotation=0, sensor_hmirror=False, sensor_vflip=False) #运行main函数
# main(model_addr="/sd/m.kmodel")
except Exception as e:
sys.print_exception(e)
lcd_show_except(e)
finally:
gc.collect() #运行垃圾回收。