One Linux command a day (41): mpstat command

This article is an original article of Joshua 317. Please note: reprinted from Joshua 317 blog https://www.joshua317.com/article/196

1, Introduction

mpstat (full English Name: Multiprocessor Statistics) command under Linux is a real-time system monitoring tool, which can display CPU data changing with time to find out the relationship between CPU utilization and time. This command is included in the tool set sysstat. You need to install sysstat before using it.

This information is stored in the / proc/stat file. In a multi CPUs system, it can view not only the average status information of all CPUs, but also the information of specific CPUs. mpstat has the following features: you can view the statistics of each computing core in a multi-core CPU; A similar tool vmstat can only view the overall CPU of the system.

If the system is not installed, you can install it from sudo install sysstat.

2, Format description

mpstat [ options ] [ <interval> [ <count> ] ]
mpstat [parameter]

Usage: mpstat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -A ] [ -u ] [ -V ] [ -I { SUM | CPU | SCPU | ALL } ]
[ -P { <cpu> [,...] | ON | ALL } ]

3, Option description

-A  Equivalent to-u -I ALL -P ALL
-I  Can specify SUM CPU SCPU ALL Four parameters,SUM Represents the total number of interrupts per processor, CPU Represents the number of interrupts per second per core, SCPU Indicates the number of soft interrupts per second for each core. The kernel version is 2.6.31 After that.
-P  Statistical CPU Number, general ALL
-u  Output column information
-V  View tool version number

delay  Sampling interval
count  Sampling times

4, Command function

Display CPU data that changes over time to find out the relationship between CPU utilization and time

5, Common usage

5.1 execution monitoring first CPU

# mpstat -P 1

Linux 3.10.0-1127.19.1.el7.x86_64 (service-01)  11/15/2021      _x86_64_        (4 CPU)

03:00:24 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
03:00:24 PM    1    0.29    0.00    0.28    0.03    0.00    0.00    0.00    0.00    0.00   99.41

CPU	    processor ID
%user   stay internal In the time period, user status CPU time(%),Not included nice Process with negative value  (usr/total)*100
%nice   stay internal During the time period, nice Process with negative value CPU time(%)   (nice/total)*100
%sys    stay internal Time period, kernel time(%)       (system/total)*100
%iowait stay internal During the time period, the hard disk IO waiting time(%) (iowait/total)*100
%irq    stay internal Hard interrupt time(%)     (irq/total)*100
%soft   stay internal Soft interrupt time in time period(%)     (softirq/total)*100
%steal	Displays the virtual machine manager when serving another virtual processor CPU Percentage of time spent in involuntary waiting	steal/total*100
%guest	Displays when the virtual processor is running CPU Percentage of time spent	guest/total*100
%gnice  gnice/total*100
%idle   stay internal During the time period, CPU Remove waiting disk IO Idle time for any reason other than operation idle time(%) (idle/total)*100

CPU The total working time is calculated as follows:
total_cur = user + system + nice + idle + iowait + irq + softirq
total_pre = pre_user + pre_system + pre_nice + pre_idle + pre_iowait + pre_irq + pre_softirq
user = user_cur – user_pre
total = total_cur - total_pre

5.2 execute and monitor all CPU s

# mpstat -P ALL

Linux 3.10.0-1127.19.1.el7.x86_64 (service-01)  11/15/2021      _x86_64_        (4 CPU)

03:00:49 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
03:00:49 PM  all    0.28    0.00    0.28    0.03    0.00    0.00    0.00    0.00    0.00   99.40
03:00:49 PM    0    0.29    0.00    0.28    0.08    0.00    0.00    0.00    0.00    0.00   99.35
03:00:49 PM    1    0.29    0.00    0.28    0.03    0.00    0.00    0.00    0.00    0.00   99.41
03:00:49 PM    2    0.29    0.00    0.28    0.01    0.00    0.00    0.00    0.00    0.00   99.42
03:00:49 PM    3    0.27    0.00    0.27    0.01    0.00    0.00    0.00    0.00    0.00   99.44

5.3 make statistics 5 times every 1 time unit

# mpstat 1 5

Linux 3.10.0-1127.19.1.el7.x86_64 (service-01)  11/15/2021      _x86_64_        (4 CPU)

03:01:26 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
03:01:27 PM  all    0.25    0.00    0.50    0.00    0.00    0.00    0.00    0.00    0.00   99.25
03:01:28 PM  all    0.00    0.00    0.25    0.00    0.00    0.00    0.00    0.00    0.00   99.75
03:01:29 PM  all    0.50    0.00    0.50    0.00    0.00    0.00    0.00    0.00    0.00   99.00
03:01:30 PM  all    0.25    0.00    0.50    0.50    0.00    0.00    0.00    0.00    0.00   98.75
03:01:31 PM  all    0.25    0.00    0.25    0.00    0.00    0.00    0.00    0.00    0.00   99.50
Average:     all    0.25    0.00    0.40    0.10    0.00    0.00    0.00    0.00    0.00   99.25

5.4 viewing soft interrupt statistics

#Once every 1 second, 2 times in total
# mpstat -I SCPU 1 2

Linux 3.10.0-1127.19.1.el7.x86_64 (service-01)  11/15/2021      _x86_64_        (4 CPU)

03:14:43 PM  CPU       HI/s    TIMER/s   NET_TX/s   NET_RX/s    BLOCK/s BLOCK_IOPOLL/s  TASKLET/s    SCHED/s  HRTIMER/s      RCU/s
03:14:44 PM    0       0.00      71.00       0.00      20.00       1.00       0.00       0.00      30.00       0.00      41.00
03:14:44 PM    1       0.00      64.00       0.00      12.00       0.00       0.00       0.00      13.00       0.00      42.00
03:14:44 PM    2       0.00      71.00       0.00       2.00       0.00       0.00       0.00      16.00       0.00      53.00
03:14:44 PM    3       0.00     115.00       0.00       0.00       0.00       0.00       0.00      25.00       0.00      66.00
03:14:45 PM    0       0.00      55.00       0.00       6.00       5.00       0.00       0.00      26.00       0.00      35.00
03:14:45 PM    1       0.00      14.00       0.00       2.00       0.00       0.00       0.00       3.00       0.00       9.00
03:14:45 PM    2       0.00      36.00       0.00       2.00       0.00       0.00       0.00      10.00       0.00      21.00
03:14:45 PM    3       0.00      81.00       0.00       0.00       0.00       0.00       0.00      18.00       0.00      45.00
Average:       0       0.00      63.00       0.00      13.00       3.00       0.00       0.00      28.00       0.00      38.00
Average:       1       0.00      39.00       0.00       7.00       0.00       0.00       0.00       8.00       0.00      25.50
Average:       2       0.00      53.50       0.00       2.00       0.00       0.00       0.00      13.00       0.00      37.00
Average:       3       0.00      98.00       0.00       0.00       0.00       0.00       0.00      21.50       0.00      55.50

6, Expand the difference between vmstat and mpstat

1. The difference between vmstat and mpstat commands: mpstat can display the statistics of each processor, while vmstat displays the statistics of all processors. Therefore, poorly written applications that do not use a multithreaded architecture may run on a multiprocessor machine without using all processors. This causes one CPU to overload while the other CPUs are idle. These types of problems can be easily diagnosed through mpstat.

2. All CPU summaries in vmstat are suitable for mpstat. When the% idle number is low, it generally indicates that there is a problem of insufficient CPU. When the% iowait number is high, there are some problems in the I/O subsystem under the current load.

This article is an original article of Joshua 317. Please note: reprinted from Joshua 317 blog https://www.joshua317.com/article/196

Posted on Fri, 19 Nov 2021 09:25:12 -0500 by Rippie