First understand the concept of virtualization
Virtualization refers to a process of logical abstraction, isolation, redistribution, and management of resources. Usually, there are broad and narrow definitions of virtualization. In a broad sense, it includes platform virtualization, application virtualization, storage virtualization, network virtualization, device virtualization, and so on. In a narrow sense, virtualization specifically refers to simulating and running multiple operating system platforms on a computer.
The purpose of virtualization is to achieve efficient resource utilization, flexibility, isolation and security through the abstraction and management of hardware resources, thereby providing a more efficient, reliable and scalable computing environment.
The relationship between virtualization technology and cloud computing:
- Wikipedia cloud computing definition: Cloud computing is a computing model that provides dynamically scalable virtualized resources as a service through the Internet.
- Cloud computing is based on virtualization technology and provides on-demand computing resources and services through the Internet. Cloud computing provides higher-level abstraction and services of virtualization technologies, such as virtual machine instances, container services, and function computing. It provides features such as multi-tenant environment, automatic management, and elastic expansion, so that users can quickly acquire and release computing resources according to actual needs.
- Virtualization technology provides basic resource pool and resource management capabilities for cloud computing, enabling cloud computing to realize resource sharing and dynamic allocation. Cloud computing provides virtualization technology with broader application scenarios and services, enabling it to be better applied to actual business and service delivery.
KVM virtualization technology
The full name of KVM is Kernel-based Virtual Machine, that is, a kernel-based virtual machine , which is a full virtualization solution using hardware-assisted virtualization technology. For I/O devices (such as hard disks, network cards, etc.), KVM supports full virtualization of QEMU emulation and half virtualization of virtio mode. Since its birth, KVM has been positioned as an all-virtual implementation based on hardware virtualization support . Since it was integrated after version 2.6 of the Linux kernel, the Linux kernel becomes a de facto Hypervisor (virtual machine manager, also called VMM (Virtual Machine Monitor)) , but the hardware management is still done by the Linux Kernel.
Hypervisors and VMMs:
Hypervisor and VMM (virtual machine monitor) can be understood as the same thing, they are both used to monitor and control the virtual machine operating system. In the process of creating a virtual machine, a whole set of hardware resources will be virtualized, such as hard disk, memory, CPU, network equipment, etc. These tasks are all taken care of by the hypervisor. In addition, it is also responsible for the resources of the virtual system running process Allocation and lifecycle management of virtual machines, etc.
A KVM client corresponds to a Linux process, each vCPU corresponds to a thread under this process, and there are threads that process I/O independently, belonging to the same process group. Therefore, the Linux Kernel on the host machine can schedule KVM virtual machines just like ordinary Linux processes. This mechanism enables Linux Kernel's process optimization and scheduling function optimization strategies to be used for KVM virtual machines. For example, the permission and priority of the KVM client can be limited through the process permission limitation function.
Since KVM is embedded in the Linux kernel, except hardware-assisted virtualization (such as VT-d) hardware devices can be seen by the virtual machine, other I/O devices are simulated by QEMU, so QEMU is a natural friend of KVM .
It can be said that KVM is a VMM built on top of hardware-assisted virtualization technology. But not all hardware virtualization technologies are required to support KVM virtualization. KVM’s minimum dependence on hardware is the hardware virtualization support of the CPU (for example: Intel’s VT-x technology and AMD’s AMD-V technology), while other Hardware virtualization support for memory and I/O will improve the performance of the entire KVM virtualization. Therefore, when deploying the KVM function on a virtual machine, the first thing to do is to check the host Host (that is, the virtual machine on the host, here is to enable the CPU virtualization function on the virtual machine of VMware Workstations.
Then in the Linux system, you can confirm that the VT-x function is supported by the following command:
If there is no output, it means that your system does not support virtualization processing and KVM cannot be used.
"vmx" is the virtualization technology identifier for Intel CPUs, and "svm" is the virtualization technology identifier for AMD CPUs.
Install KVM and related tools:
sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager- qemu-kvm: Basic package for KVM virtualization
- libvirt-daemon-system: libvirt daemon, provides API and manages virtualization functions
- libvirt-clients: Contains client utilities for communicating with the libvirt daemon
- bridge-utils: Toolset for creating and managing network bridges
- virt-manager: Graphical tool for managing virtual machines
Confirm that the libvirtd service is up and running:
sudo systemctl start libvirtd
sudo systemctl status libvirtd
#确保在系统启动时自动启动
sudo systemctl enable libvirtdCreate or manage virtual machines through virt-manger (graphical tool)
Or use virsh (command line tool) to create or manage virtual machines
Here are some commonly used virsh commands:
virsh list: list currently running virtual machinesvirsh start <domain>: Start the virtual machine with the specified namevirsh shutdown <domain>: Shut down the virtual machine with the specified namevirsh reboot <domain>: Restart the virtual machine with the specified namevirsh create <xmlfile>: Create a virtual machine according to the specified XML filevirsh destroy <domain>: Forcibly stop the virtual machine with the specified namevirsh undefine <domain>: Delete the virtual machine configuration with the specified namevirsh console <domain>: Enter the virtual machine console with the specified namevirsh dominfo <domain>: Display the virtual machine information of the specified namevirsh domstate <domain>: Display the status of the virtual machine with the specified name
Next, analyze the Makefile of kvm:
It can be seen that the Makefile of kvm mainly generates three modules, kvm.o and kvm-intel.o, kvm-amd.o.
- kvm.o is the core module of kvm. kvm basically only implements the relevant parts of hardware-assisted virtualization, and uses Qemu to simulate and realize the unsupported ones.
- kvm-intel.o is the intel platform architecture virtualization module, platform-related
- kvm-amd.o is the amd architecture virtualization module, platform-dependent
Information through train: Linux kernel source code technology learning route + video tutorial kernel source code
Learning through train: Linux kernel source code memory tuning file system process management device driver/network protocol stack
KVM architecture
KVM is a full virtualization technology supported by hardware virtualization, so it can run almost all operating systems on corresponding hardware. The core of KVM virtualization is mainly composed of the following two modules:
1) The KVM kernel module, which is part of the standard Linux kernel, is a module that provides virtualization functions and is mainly responsible for virtualization of CPU and memory, including: creation of clients, allocation of virtual memory, and switching of CPU execution modes , vCPU register access, vCPU execution.
2) QEMU user mode tool, which is an ordinary Linux process, provides device simulation functions for the client, including simulating BIOS, PCI/PCIE bus, disk, network card, graphics card, sound card, keyboard, mouse, etc. At the same time, it interacts with the KVM module in the kernel state through the ioctl system call.
As shown in the figure above, under the KVM virtualization architecture, each client is a QEMU process , and there are as many QEMU processes as there are virtual machines on a host. Each virtual CPU in the client corresponds to an execution thread in the QEMU process, and there is only one KVM kernel module in a host Host, and all virtual machines interact with this kernel module.
KVM kernel module
The VM kernel module is the core module of KVM virtualization. It consists of two parts in the kernel: one is the processor architecture-independent part, which can be seen in the lsmod command, called the kvm module ; the other is the processor architecture-related part. Part, on the Intel platform is the kvm_intel kernel module. As shown in the figure below, the main function of KVM is to initialize the CPU hardware, turn on the virtualization mode, and then run the virtual machine in the virtual environment, and provide certain support for the operation of the virtual machine.
Taking an Intel CPU architecture server as an example, the process of KVM opening and initializing the hardware to support the running of the virtual machine is as follows:
Step1: When loaded by the kernel, the KVM module will first initialize the internal data structure.
Step2: After getting ready, the KVM module detects the current CPU of the system, then turns on the virtualization mode switch in the CPU control register CR4, and puts the host operating system ( including the KVM module itself) in the CPU virtualization mode by executing the VMXON instruction The root mode of root operation .
Step3: The KVM module creates a special device file /dev/kvm and waits for commands from user space.
Step4: The creation and operation of the subsequent virtual machine is essentially a process in which the QEMU in the user space and the KVM module in the kernel space cooperate with each other.
The /dev/kvm device here is more critical. It can be used as a standard character device to cache the context of switching between user space and kernel space, that is, the ioctl call context, which is the communication between the KVM module and the user space QEMU interface. :
QEMU user mode device emulation
QEMU was originally a well-known open source virtual machine software project. It is not only a full-featured virtual machine monitor, but also undertakes the work of device simulation in the QEMU-KVM software stack.
The virtual machine originally implemented by QEMU is a pure software implementation, that is, we often say that the CPU instruction simulation of the virtual machine is realized through binary translation, so the performance is relatively low. However, its advantage is that it is cross-platform, and it can even support that the client computer and the host computer are not of the same architecture, such as running the ARM client computer on the x86 platform. At the same time, QEMU can perfectly fit with mainstream Hypervisors, including: Xen, KVM, Hyper-v, and various VMware Hypervisors, etc., providing virtualized I/O devices for these Hypervisors.
The reason why QEMU and KVM are inseparable is the QEMU-KVM software protocol stack we often say. During the running of the virtual machine, QEMU will enter the kernel through the system call ioctl provided by the KVM kernel module, and the KVM kernel module is responsible for placing the virtual machine in a special mode of the processor to run. Once encountering a virtual machine for I/O operations, the KVM kernel module will return to QEMU from the exit of the last system call ioctl, and QEMU will be responsible for parsing and simulating these devices. In addition, the configuration and creation of virtual machines, the virtual devices that virtual machines rely on, the user operating environment and interaction when virtual machines are running, and some special technologies for virtual machines, such as live migration, are all implemented by QEMU itself of.
In short, KVM itself does not perform any simulation, and requires user space applications to set the address space of a client virtual server QEMU through the interface and provide it with simulated I/O. The KVM module implements processor virtualization and memory virtualization. /dev/kvmWith the support of hardware virtualization technology, the KVM module of the kernel and the device simulation of QEMU work together to form a virtualized computer software and hardware system that is completely consistent with the physical computer system.
Monitor KVM events
An example is provided in the bcc program to monitor kvm related events
source code:
kvm_hypercall.py https://github.com/iovisor/bcc/blob/master/examples/tracing/kvm_hypercall.py
The result of the operation is as follows:
This program uses the following static tracepoints related to kvm:
kvm_entry: When the vCPU (virtual central processing unit) of the virtual machine switches from the host CPU to the virtual machine CPU and starts executing virtual instructions, the kvm_entry trace point is triggered, and vcpu_id is used to identify a specific virtual CPU.
kvm_exit: When the vCPU of the virtual machine executes a virtual instruction and switches from the virtual machine CPU back to the host CPU, the kvm_exit tracking point is triggered, and exit_reason indicates the specific reason why the virtual CPU is launched to the host CPU.
kvm_hypercall: Used to track hypercalls (hypercalls) in virtual machines. When the Guest OS of the virtual machine needs to perform some higher-privileged operations (such as: update the page table, access to physical resources, etc.), because it cannot complete these operations in the non-privileged domain, it will be handed over to the Hypervisor through a super call To complete these operations, where nr represents the hypercall number.
exit_reason in kvm_exit:
12, 18, and 32 in the above figure represent access to the task priority register (TP), operating system-specific events, and the processor is in the AP reset hold state.
hypercall_nr (hypercall number):
The No. 5 hypercall indicates that the hypervisor program is requested to wake up the vCPU in the dormant state.
