mpstat Command in Linux with Examples
mpstat is a powerful and versatile command-line tool within a Linux system that allows detailed reporting of various processor-related statistics. Indeed, part of the sysstat package, mpstat delivers comprehensive CPU utilization and performance data, thus becoming an essential utility for both system administrators and performance analysts as well as developers aiming to optimize system resources.
mpstat is most useful with multi-processor environments, and statistics can be gathered for each individual CPU core. It sets up the first processor to be CPU 0, the second to be CPU 1, and so forth. Therefore, by employing mpstat, you can analyze systems with multiple cores in great detail in search of bottlenecks, balance the workloads, or fine-tune the system performance.
Installation of mpstat
Before you get going with the usage of mpstat, verify it is installed on your system. Installation instructions depend on your Linux distribution:
In Red Hat based Linux
sudo yum install sysstatIn ubuntu based Linux
sudo apt install sysstatImmediately after installation, you may need to activate the sysstat service in order for it to collect the data properly:
sudo systemctl enable sysstat && sudo systemctl start sysstatSyntax
mpstat [options] [interval [count]]where,
- options: Numerous flags to customize the output
- interval: Secs between reports
- count: Number of reports
Basic Example
mpstat
This command prints an aggregate view of CPU-usage statistics since system startup. There is much useful metrics like:
- %usr: Percentage of CPU spent at user-level
- %sys: Percentage of CPU spent at kernel level
- %iowait: Percent of CPU spent idle due to wait of input/output requests
- %idle: CPU idle time, not including wait time for I/O operations
Common options for mpstat Command
Option | Description |
|---|---|
-P | print CPU utilization of specified processor(s) |
-A | Display all CPU statistics and report percpu statistics |
-u | print CPU utilization |
-I | print interrupt statistics |
-n | print network statistics |
--dec= | Print in specified number of decimal places, from 0 to 2. |
All CPUs, 2 second intervals for 5 reports:
mpstat -P ALL 2 5
This will print out an overview of the details of the performance for each CPU, on a 10-second period, updating every 2 seconds. It is also very useful in detecting imbalances of workload distribution among cores.
Detailed interrupt statistics:
mpstat -I ALL
This command would print out some statistics detailing interrupts, from both hardware and software points of view. It is particularly very helpful in tracking interrupt-related performance problems.
Print statistics with higher resolution:
mpstat -P ALL --dec=2 1 2
This command will print statistics of all CPUs to two decimal places of precision, updating every one second for 3 reports. Increased precision improves the analysis.
Understanding mpstat
To properly analyze your system, you need to be familiar with how to read the output of mpstat. Here's a more detailed explanation of the most important metrics.
Metric | Description | Interpretation |
|---|---|---|
%usr | User-level CPU usage | High values indicate CPU-bound application workloads |
%sys | System CPU usage percentage | This could imply excessive number of system calls or some driver problem |
%iowait | CPU idled while waiting for I/O completion | High %iowait can imply some form of I/O bottleneck (disks or networks) |
%idle | CPU idled time, excluding iowait | Low values indicate high %cpu utilization |
%irq | CPU usage for servicing hardware interrupts | High values may indicate problems due to hardware or drivers |
%soft | percentage of time spent servicing software interrupts | high values may indicate that network or I/O processing is going on |
Real-World Scenarios and Troubleshooting
Let's look at a few real world scenarios where mpstat can be quite helpful:
Scenario 1: Find the CPU-bound processes
mpstat -P ALL 1 10 | grep -v CPU | awk '$12 < 10 {print $3, $12}'
This command identifies the CPUs which are idling less than 10%, thus possibly CPU-bound processes. Combine it with 'top' or 'ps' to figure which ones are really eating high CPU usage time
Scenario 2: Identify I/O bottlenecks
mpstat 1 60 | awk '$6 > 10 {print "High I/O wait at " strftime("%H:%M:%S"), "iowait: " $6}'This command will monitor I/O wait times for 60 seconds, alerting when they exceed 10%. High I/O waits might be indicative of a disk or network bottleneck.
Scenario 3: Interrupt Handling Analysis
mpstat -I SUM -P ALL 1 5
This command can be used to analyze how interrupts are being distributed across the CPUs. A very uneven or skewed distribution might suggest that some interrupt balancing or CPU affinity adjustments need to be done.
Conclusion
This is one powerful tool the Linux system administrator will want to have in his bag of tricks. It provides detailed, real-time information on CPU performance and can be quite useful for performance tuning, capacity planning, and troubleshooting. Once you master mpstat along with other system monitoring tools, you are virtually at the helm of running your system at its best.
With the advent of multi-core processors and highly virtualized environments, mpstat becomes even more relevant, as with regular monitoring and analysis via mpstat, performance issues can be detected before they start to hurt the users, ensuring a smooth and efficient function of your Linux systems.
