1. Introduction

This article is mainly based on the source code of Linux Kernel Version 4.13.0-36-generic to conduct an in-depth analysis of its process model. The specific contents are as follows:

1. How the operating system organizes processes

2. How the process state transitions

3. How processes are scheduled

4. Your own view of the operating system process model

(Note: The link address of Linux Kernel Version 4.13.0-36-generic source code: https://elixir.bootlin.com/linux/v4.13/source/kernel )

2. Process

2.1 Understanding of the process

1). A process is an abstraction of a running program. A process is an instance of an executing program, including the current values of the program counter, registers, and variables.

2). Narrow definition: A process is an instance of a running program (an instance of a computer program that is being executed).

3). Broad definition: A process is a running activity of a program with a certain independent function about a certain data set. It is the basic unit of the dynamic execution of the operating system . In the traditional operating system , the process is not only the basic allocation unit , but also the basic execution unit.

2.2 Approaching the process

Rather than struggling with the definition of a process, practice it yourself.

On the Windows operating system, we can view various processes by opening the Task Manager.

On the Linux operating system, we can enter ps aux at the terminal command line and press Enter to view all processes.

Explanation of each field:

(PID: process ID; %CPU: CPU occupancy; %MEM: physical memory; VSZ: virtual memory; RSS: actual physical memory; STAT: process status; START: startup time; COMMAND: process name)

3. How the operating system organizes processes

　　task_struct is a data structure of the Linux kernel. It saves process information and can be found in include/linux/sched.h. All system processes exist in the kernel in the form of a task_struct linked list.

Now let 's introduce some of the main contents contained in task_struct :

PID: the identity of the process

Processor: Identifies the CPU the user is using

State: identifies the state of the process (there are six, see later)

Prority: The priority of the real-time process, invalid for ordinary processes

Policy: Indicates the process scheduling policy

A. _ Process identifier PID

Process identifier PID is a unique identifier that describes the process to distinguish other processes. It is defined in task_struct as follows:

For the value range of PID, in include/linux/threads.h, there are the following macro definitions:

When CONFIG_BASE_SMALL is set to 0, the value range of PID is 0~32767, and the maximum number of processes in all systems is 32768.

B. _ state of the process

Defined in task_struct as follows:

　　The possible values of the State member are as follows:

　　Now let's introduce several states of the process in Linux:

　　1. TASK-RUNNING: Indicates that the process is either executing or is about to execute.

　　2. Interruptible blocking state (TASK-UBERRUPTIBLE): Indicates that the process is blocked until a certain condition is true. If the condition is met, the state becomes runnable.

　　3. Uninterruptible blocking state (TASK-UN INTERRUPTIBLE): cannot be woken up by accepting a signal.

　　4. TASK-ZOMBLE: Indicates that the execution of the process is terminated, but the parent process has not used wait() and other system calls to know its termination information.

　　5. TASK-STOPPED: Indicates that the process is stopped from executing

The basic state of the process is as follows:

C. _ process priority

　　Priority is defined in task_struct as follows:

Fourth, how the process is scheduled

4.1 Understanding of scheduling

　　In the operating system, this part of the selection work is called the scheduler, and the algorithm used by this program is called the scheduling algorithm. Scheduling algorithms can be divided into two categories according to how clock interrupts are handled: non-preemptive and preemptive. Different environments require different scheduling algorithms, in other words, in different systems, the optimization of the scheduler is different, so it is necessary to divide three environments here: batch, interactive and real-time.

Typical scheduling algorithms are:

(1) First come first serve

(2) Short assignments are preferred

(3) Priority

(4) High response ratio is preferred

(5) Time slice rotation

　　Here, I would like to introduce the CFS (Completely Fair Scheduler) scheduler, that is, the Completely Fair Scheduler.

4.2 CFS Scheduling Algorithm

CFS was adopted after Linux Kernel 2.6.23, using red- black tree to implement, the algorithm efficiency is O(log(n)).

Scheduling Algorithms The two core points of the scheduling algorithm are which process is scheduled to execute and how long the scheduled process takes to execute. The former is called the scheduling policy , and the latter is the execution time .

4.2.1、Scheduling strategy

　　CFS assigns a virtual clock vruntime to each process in cfs_rp (CFS's run queue). The scheduler always selects the process with the lowest vruntime value for execution. It records the running time of the process, and its size has a quantitative calculation relationship with the weight and running time of the process.

　　vruntime = actual running time * 1024 / process weight

　　All processes use a weight of 1024 with a nice value of 0 as a benchmark to calculate their own vruntime increase speed. Some conversion relationships are given below:

　　Time allocated to a process = scheduling period * process weight / sum of all process weights
　　vruntime = actual running time * 1024 / process weight
　　vruntime = (scheduling period * process weight / sum of all process weights) * 1024 / process weight
　　vruntime = ( Scheduling period/sum of all process weights) * 1024

　　Although processes have different weights, their vruntime growth rate should be the same regardless of weight. A process with a smaller vruntime value means that it used the CPU for a short time and was treated unfairly, so it is selected as the next process to run.

4.2.2、Execution time

CFS uses the proportion of the priorities of all schedulable processes in the current system to determine the time slice for each process to execute, namely:

Time allocated to a process = scheduling period * process weight / sum of all processes.

4.2.3、 Kernel implementation of CFS scheduling algorithm

A. _ Red-Black Tree - Skeleton

1. The red-black tree is self-balancing, and no path is more than twice as long as any other path.

2. Running on the tree occurs in O(log n) time (n is the number of nodes in the tree), which can quickly and efficiently insert or delete tasks.

A wave of stolen pictures... Then continue to steal pictures, please see the red-black tree data structure

(The above content and related structures can be found in include/linux/sched.h.)

Next, let's introduce the structure of CFS in detail. The scheduling entity sched_entity, which represents the scheduling unit to be given, can be equated with a process when the group scheduling is closed. Each task_struct has a sched_entity process vruntime and weight are stored in this structure.

The sched_entity is organized together by a red-black tree. All sched_entities are inserted into the red-black tree with vruntime as the key. At the same time, the leftmost node of the tree is cached, that is, the node with the smallest vruntime, so that the process with the smallest vruntime can be quickly selected.

B. _ two important structures

Completely fair queue cfs_rq : Describes various running information of a common process running on a cpu in the TASK_RUNNING state.

struct cfs_rq {
       struct load_weight load;   // Total process weight of run queue 
      unsigned int nr_running, h_nr_running; /Number of processes
 
      u64 exec_clock;   // The running clock 
      u64 min_vruntime; / The vruntime advance value of the cpu running queue is generally the smallest in the red-black tree
#ifndef CONFIG_64BIT
      u64 min_vruntime_copy;
#endif
 
      struct rb_root tasks_timeline; // The root node of the red-black tree 
      struct rb_node *rb_leftmost; // points to the node with the smallest vruntime value
 
      struct sched_entity *curr, *next, *last, *skip;
 
#ifdef CONFIG_SCHED_DEBUG
      unsigned int nr_spread_over;
#endif
 
#ifdef CONFIG_SMP
      
      struct sched_avg avg;
      u64 runnable_load_sum;
      unsigned long runnable_load_avg;
#ifdef CONFIG_FAIR_GROUP_SCHED
      unsigned long tg_load_avg_contrib;
      unsigned long propagate_avg;
#endif
      atomic_long_t removed_load_avg, removed_util_avg;
#ifndef CONFIG_64BIT
      u64 load_last_update_time_copy;
#endif
 
#ifdef CONFIG_FAIR_GROUP_SCHED
      
      unsigned long h_load;
      u64 last_h_load_update;
      struct sched_entity *h_load_next;
#endif /* CONFIG_FAIR_GROUP_SCHED */
#endif /* CONFIG_SMP */
 
#ifdef CONFIG_FAIR_GROUP_SCHED
      struct rq *rq;             // There are run queues for ordinary processes and run queues for real-time processes in the system. These queues are all included in the rp run queue
 
      
      int on_list;
      struct list_head leaf_cfs_rq_list;
      struct task_group *tg;    /* group that "owns" this runqueue */
 
#ifdef CONFIG_CFS_BANDWIDTH
      int runtime_enabled;
      u64 runtime_expires;
      s64 runtime_remaining;
 
      u64 throttled_clock, throttled_clock_task;
      u64 throttled_clock_task_time;
      int throttled, throttle_count;
      struct list_head throttled_list;
#endif /* CONFIG_CFS_BANDWIDTH */
#endif /* CONFIG_FAIR_GROUP_SCHED */
};

Scheduling entity sched_entity: record the running status information of a process

struct sched_entity {
    /* For load-balancing: */ 
   struct load_weight load;   // The weight of the process 
   struct rb_node run_node; // The red-black tree node in the run queue 
   struct list_head group_node; // Related to group scheduling 
   unsigned int          on_rq; / / Whether the process is in the TASK_RUNNING state now 
 
   u64 exec_start; // The start time of a scheduling tick 
   u64 sum_exec_runtime; // The actual time that the process has been running 
   u64 vruntime; // The virtual running time 
   u64 prev_sum_exec_runtime; // Before this scheduling, the process has been running time
 
   u64 nr_migrations;
 
   struct sched_statistics     statistics;
 
#ifdef CONFIG_FAIR_GROUP_SCHED
   int           depth;
    struct sched_entity *parent; // parent process in group scheduling 
   /* rq on which this entity is (to be) queued: */ 
   struct cfs_rq *cfs_rq; // Which run queue the process is in at this time 
   /* rq "owned" by this entity/group: */ 
   struct cfs_rq * my_q;
 #endif
 
   struct sched_avg     avg ____cacheline_aligned_in_smp;
};

V. Views on the Operating System Process Model

　　Operating systems provide a conceptual model of executing serial processes in parallel, processes can be created and terminated dynamically, and each process has its own address space. Information is exchanged between processes through inter-process communication primitives, and a process can be running, managing, or blocking. For process scheduling, the CFS scheduler was introduced in the Linux kernel version 2.6.24, while the O(1) scheduler was used before. CFS is responsible for allocating CPU resources to executing processes, with the goal of maximizing program interaction performance and minimizing overall CPU usage. It is implemented using a red-black tree .