With the advent of the AI era, how to save energy and improve efficiency in cloud data centers is becoming a hot topic. Among them, SR-IOV technology is one of the important solutions for building a green and low-carbon cloud data center due to its high flexibility and the ability to save computing power for virtualized services.
1. What is SR-IOV technology?
SR-IOV is an extended specification of the PCIe specification defined by the international organization PCI-SIG. It aims to eliminate VMM's intervention in virtualized I/O operations, improve data transmission performance, and provide independent memory space for VM (virtual machine). Terminal, DMA data stream. I/O devices with SR-IOV functions can manage and create multiple VFs (virtual functions) based on the PCIe specification. PCIe PF (physical function) is the main entity on the PCIe bus. A PCIe device has one or more PFs. PF sets the number of VFs and starts or stops VFs globally. VFs can complete data access and transfer without VMM intervention.
Figure 1: Comparison between traditional virtual machines and SR-IOV virtual machines
Under the SR-IOV function, interrupts originally borne by the VMM are directly handled by the virtual machine, which improves the device I/O performance. At the same time, the virtual machine can directly interact with the PCIe device, greatly reducing the burden on the physical host CPU, making it possible Take on more virtual machine devices. In addition, through the SR-IOV function, the demand for PCIe devices can be reduced, PCIe slots can be saved, and hardware resources can be used more fully.
Based on the above advantages, many SSD manufacturers have begun to deploy SR-IOV. Currently, Yilian has taken the lead in successfully implementing SR-IOV function applications on enterprise-level UH8 series and data center-level UH7 series products, and has put them into use in multiple cloud data centers.
2. Verification of Yilian SR-IOV solution
1 Verification environment
1.1 This hardware configuration verification
·CPU：2* Intel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz 24cores/48threads
·Memory: 16*32GB 3200MT/S DDR4
·System disk: 1*960G 12Gbps
·SSD: 1*UH8 series 3.84T
·HBA card: 1*MegaRAID 9440-8i
·NIC：2* SC332
1.2 This verification software configuration
·Operating system: CentOS Linux release 8.3.2011
·Kernel: Linux 4.18.0-240.el8.x86_64
·Testing tools: version 2.6 and above
·QEMU-KVM：2.12
·NVMe open source driver: 1.11.1
·umtool tool: 1.0.1.5
2 Verification methods
In this verification, in the scenario of virtualized partitioning, a PCIe 4.0 3.84TB SSD (bound to 16 CPU cores) is configured into two VFs with a capacity of 1.92TB (bound to 8 CPUs) through SR-IOV technology. 4 960G VFs (bound with 4 cores), 8 480G VFs (bound with 2 cores), as shown in the diagram below.
Figure 2: Virtualization partitioning scenario
The basic model of testing IO is as follows:
·Read and write bandwidth (bs=128k, job=1, iodepth=128, read/write)
·Read IOPS (bs=4k, job=16, iodepth=128, randread)
·写 IOPS（bs=4k, job=8, iodepth=64, randwrite）
In the scenario where the disk is divided into 2/4/8 VFs, the SR-IOV solution is verified under the conventional IO test model.
3 Verification results
3.1 Full use of SSD performance to effectively improve utilization
The comparison chart between the total performance and the performance of the entire disk in different VF scenarios under the SR-IOV function is as follows (the following data is the verification data for this time and is for reference only).
*Note: Total bandwidth/total IOPS = average performance × number of VFs
Figure 3: Comparison of disk performance before and after using SR-IOV
As can be seen from Figure 3, under this SR-IOV solution verification, the fluctuation of the sum of PF performance and VF performance is lower than that of the original disk, basically within 5%, achieving full performance for the Yilian UH series SSD. use. In cloud scenarios, it can reduce the number of PCIe devices required by users, effectively improve the utilization rate of tenants, and realize cost reduction and efficiency enhancement of cloud virtualization services.
3.2 VF performance is stable in each scenario, with high scalability and flexibility
In order to verify the stable performance of the VF, tests were carried out in the scenarios of dividing 2/4/8 VFs, and the results are shown in the figure below.
Note: Deviation value = (VF mean value - VF performance) / VF mean value * 100%
Figure 4: Deviation diagram between each VF and the mean under the SR-IOV function
The data shows that the basic deviation between each VF test value and the VF average is less than 1%. The performance distribution among VFs is relatively stable and has high scalability. The number of VFs can be set and managed flexibly according to user needs.
3.3 SR-IOV with QoS performance isolation is obvious
After splitting the SSD into multiple virtual disks, in order to avoid the situation where multiple virtual machines are running at the same time and the same virtual machine is loading and unloading services on the disk at the same time, which will interfere with each other in performance, Yilian has made specific performance adjustments for the SR-IOV feature. Isolated design - SR-IOV with QoS can control the flow of each VF through the SSD chip to achieve balanced performance, ensure the large bandwidth, low latency and other characteristics of the virtual SSD disk, and meet the requirements of high-performance data storage in cloud data centers needs.
Figure 5: Performance of VF under specific business scenarios
3. The evolution direction of Yilian SR-IOV technology
1. The performance isolation function of SR-IOV is further optimized
Currently, Yilian SR-IOV uses NS to logically isolate VF data. However, because all VFs use the common Nand Flash space, there will inevitably be a performance impact between each VF, although it is controlled to a certain range under the scheduling of the QoS function. However, it is still unavoidable in some more complex business scenarios. In order to solve this problem, Memorylink will continue to deepen the application of SR-IOV on SSD, further optimize the performance isolation algorithm, and achieve better isolation effect.
2. Data security of SR-IOV
Virtual machine data security is very important. The current generation of virtual machine data can be cleared through trim. However, there are still shortcomings for customers with high data security requirements such as finance and government. For example, when the SSD leaves the data center cabinet, the data will be deleted. Possibility of theft. Yilian's research direction on virtual machine data security has begun to bear fruit. In the future, when the virtual machine is released, the physical data corresponding to the SSD will be completely destroyed at the same time, providing guarantee for maintaining data security.
3. Intelligence of SR-IOV
The current generation of Yilian's SR-IOV has the performance statistics capability of each VF. The new generation SSD supports intelligent analysis of virtual machine performance, performance configuration feedback based on performance status, and intelligent diagnosis and remote repair of virtual machine status.
To sum up, the SR-IOV technology of Yilian UH8 series and UH7 series SSDs is a sustainable storage solution that can truly reduce costs and increase efficiency, and is green and low-carbon. It can effectively improve disk utilization, reduce the waste of storage space, and achieve energy conservation and emission reduction. At the same time, the Yilian SR-IOV solution shows significant isolation, which can better satisfy users' requirements for AI and distributed applications in cloud environments. Databases, high-performance enterprise applications and other needs.
Over the years, Yilian has continued to increase its investment in R&D and innovation of SSD products, combining its multiple advantages in the field of flash memory, and combining hardware technology with user application scenarios to continuously iterate more advanced SSD products to help the stable operation of enterprise business systems. Yilian's SR-IOV solution is the best verification of the company's R&D and innovation capabilities. In the future, Yilian will further consolidate its product innovation capabilities and contribute to the high-quality and sustainable development of the digital economy.