Original Song Baohua Linux Reading Field September 4
The younger brother has been promoting the flame map before, and it turns out that many children's shoes use the flame map in everything. To be honest, the flame diagram is particularly suitable for analyzing runtime hot spots (whether it is on-cpu, off-cpu, or memory, etc., the imagination of the flame diagram can be infinitely enlarged), but you have to analyze one if it is a timing problem, such as a system Slow startup, slow startup of a software, using flame diagrams may help a little, but the help is certainly subtle.
Because this problem of a certain process being slow is a timing problem. It is not a runtime hot issue, so the most important thing for you is to draw a sequence diagram of your process. There may be I/O constraints, or it may be the CPU madness, and maybe someone is sleeping stupidly.
There is a killer tool in the Linux industry to analyze the slow startup of Linux itself, called bootchart, which actually describes the IO and CPU usage of the process during the startup process. Note that this type of graph has a common feature, the horizontal axis is time, and the vertical axis is the state of CPU, thread, etc. (running, sleeping, IO, etc.).
Bootchart is really good for analyzing the boot process, but you usually start a certain software slowly? Or more broadly, is a particular process extremely slow? Or more broadly, how does the program I write run in the system, and how do several threads run? We have to describe its timing diagram.
At this time, we can use perf's timechart. For example, let’s write a very simple code that contains 2 thread cycles to do things and sleep: 
we run the above a.out, and we use perf to record the sched status of the system. :
1 ~$ sudo perf sched record -a
2 ^C[ perf record: Woken up 1 times to write data ]
3 [ perf record: Captured and wrote 1.909 MB perf.data (9039 samples) ]Next, timechart is generated:
1 ~$ sudo perf timechart
2 Written 6.4 seconds of trace to output.svg.We use firefox to open this sequence diagram:
On the timing chart, we see the running status of my 8 CPUs, and the status of 2 threads in a.out that are blue (running) and gray (sleeping) for a while. We can see that the system is almost using CPU2 and CPU7 to run the 2 threads in our a.out that take up more CPU.
However, perf timechart is not the finale in this type of tool. To be honest, it's a bit too rough! Let's take a look at the famous kernel shark-kernelshark.
Next, we use trace-cmd to record trace points related to sched:
1 ~$ sudo trace-cmd record -e 'sched_wakeup*' -e sched_switch -e 'sched_migrate*'
2 Hit Ctrl^C to stop recording
3 ^CCPU0 data recorded at offset=0x60e000
4 61440 bytes in size
5 CPU1 data recorded at offset=0x61d000
6 184320 bytes in size
7 CPU2 data recorded at offset=0x64a000
8 24576 bytes in size
9 CPU3 data recorded at offset=0x650000
10 12288 bytes in size
11 CPU4 data recorded at offset=0x653000
12 12288 bytes in size
13 CPU5 data recorded at offset=0x656000
14 86016 bytes in size
15 CPU6 data recorded at offset=0x66b000
16 172032 bytes in size
17 CPU7 data recorded at offset=0x695000
18 28672 bytes in size
Use kernelshark to open the recorded point:
1 ~$ kernelshark trace.dat
2 Loading "trace.dat"See the following graphical interface: 
We can perform various operations on the UI. For example, if we pay attention to a.out, we choose task: 
We get this view: For 
example, at the point drawn on my picture, the color of a.out occurs The following text expresses that the reason for the change is scheduling. The goal of scheduling is swapper. This scheduling point occurs on CPU4:
If you expect a horizontal axis is time, and a vertical axis is a graph of what each thread and CPU are doing in a certain period of time, kernelshark, of course you deserve it.