SAR command in Linux to monitor system performance

sar (System Activity Report) It can be used to monitor Linux system's resources like CPU usage, Memory utilization, I/O devices consumption, Network monitoring, Disk usage, process and thread allocation, battery performance, Plug and play devices, Processor performance, file system and more. Linux system Monitoring and analyzing aids in understanding system resource usage which can help to improve system performance to handle more requests. 

By default, the SAR command displays the result on the output screen, in addition, result can also be stored in the file specified by the -o filename option. 
Any user can collect information about system performance using system activity flags. The SAR command will show only CPU monitoring activity if any flag is not specified by the user. 

Why Use SAR Instead of Other Monitoring Tools?

Linux offers different system monitoring commands including top, htop, vmstat, iostat, and free, but SAR (System Activity Report) possesses certain merits that have resulted in it becoming an option for monitoring performance as well as to troubleshoot.

• Historical Data Collection: Unlike to top and htop, which give real-time statistics, SAR takes snapshots for future analysis.
• Extensive Reports: SAR monitors CPU, memory, disk, network, and I/O activity, while vmstat is focused on processes and memory.
• Lightweight & Automated: In comparison to GUI tools, SAR has low overhead and may run in the background without affecting system performance.
• Enhanced for Troubleshooting & Trend Analysis: Unlike free, which shows only the actual memory usage, SAR preserves logs and therefore is useful in solving performance issues over time.

Installation of `sar` in Linux

`sar` may not be installed by default. We need to install `sysstat` before using it 

To install `sar` in Ubuntu

sudo apt install sysstat

To install `sar` in RedHat Linux (9)

sudo dnf install sysstat

Syntax of `sar` command in Linux

sar -[ options ] time_interval number_of_tines_to_display

How to Display and Analyze Collected SAR Data

SAR saves system activity logs that can be analyzed later using different options.

To View the Latest Logs Stored Automatically

This command displays all system activity reports, including CPU, memory, I/O, and network statistics.

sar -A

To Read Past System Logs (Yesterday’s Data)

This retrieves yesterday's recorded system data, useful for troubleshooting past performance issues.

sar -f /var/log/sysstat/sa$(date --date='yesterday' +%d)

To Extract CPU Usage from Past Data

This command fetches CPU stats recorded on the 10th day of the month.

sar -u -f /var/log/sysstat/sa10

To View Network Usage Over Time

This helps track network bandwidth and detect unusual traffic spikes.

sar -n DEV 2 5

To Export Data for Further Analysis

This stores SAR logs in a file, which can be imported into Excel, Grafana, or custom scripts for better visualization.

sar -A -o output_file

Examples and Options

To start `sar` Service 

Syntax:

systemctl start sysstat.service

`--help` option in `sar` command

Syntax:

sar --help

We need to enter the root password for authentication.

To verify the `sar` version

Syntax:

sar -V

CPU Usage Details

To report CPU details a total of 5 times with the interval of 2 seconds. If the interval command is set to zero, average statistics from the time system started are presented. If the count is not provided and the interval is given, statistics are provided continuously after every interval. 

Syntax:

sar -u 2 5

Memory Usage Details

To report about the amount of memory used, amount of memory free, available cache, available buffers total 3 times with the interval of 1 second. 

Syntax:

sar -r 1 3

File System Details

To report about file systems mounted on the device total 5 times with the interval of 2 seconds. 

Syntax:

sar -F 2 5

Block Device Details

To report about block devices details total 3 times with the interval of 1 second. 

Syntax:

sar -d 1 3

Run Queue Length and Load Average

To report run queue length, number of processes and load average 

Syntax:

sar -q 2 5

CPU Usage for a Specific Core

To report cpu usage for given core

sar -P 1 1 3

Network Interface Details

To report about network interface, network speed, IPV4, TCPV4, ICMPV4 network traffic and errors

Syntax:

 sar -n DEV 1 3 | egrep -v lo

Process, Kernel Thread, I-node, and File Table Details

To report details about the process, kernel thread, i-node, and the file tables 

Syntax:

sar -v 1 3

How to Automate SAR for System Monitoring

Instead of running SAR manually, you can schedule it to log system performance automatically

Enable the SAR Data Collector on Startup

sudo systemctl enable sysstat

Verify Data Collection is Enabled

cat /etc/default/sysstat       # Make sure ENABLED="true" is set.

Than restart the service

sudo systemctl restart sysstat

Modify the Data Collection Interval

sudo nano /etc/cron.d/sysstat

Update the recording frequency from every 10 minutes (default) to every 5 minutes

*/5 * * * * root /usr/lib/sysstat/sa1 1 1

Schedule a Daily SAR Summary Report

@daily root /usr/lib/sysstat/sar -A > /var/log/sysstat/summary_report_$(date +\%F).log

Messages, Semaphores, and Process Details

To report messages, semaphores and processes details for all processors and system-wide. 

sar -mu -P ALL

Note: The above command might not work on older systems or older sysstat versions. The correct approach is to split the commands or use compatible flags. Like Some users may be using Ubuntu 20.04 or earlier, which comes with older versions of sysstat. This might cause incompatibilities with certain flags.

Swapping Statistics

To report statistics about swapping 

Syntax:

sar -S 1 3

I/O Operation Details

To report details about I/O operations like transaction per second, read per second, write per second.

syntax:

sar -b 1 3

Context Switching, Process Creation, and Swap Details

To report statistics about context switching, number of processes created per second, number of swap per second .

sar -w 1 3

Paging Statistics

To report paging statistics (KBs paged-in/sec, KBs paged-out/sec, pagefault/sec etc.) 

sar -B 2 5

Real-World Scenarios Where SAR is Useful

1. Finding CPU Bottlenecks in Web Servers

If your server slows down regularly, you can use below command which identifies CPU spikes and shows if background processes or high application loads are at fault.

sar -u 1 5

2. Tracking Memory Usage During High-Traffic Events

If your online store crashes during sales time, execute the following command to monitor memory usage in real-time, which can help detect memory leaks or memory-hungry processes.

sar -r 5 10

3. Detecting Disk I/O Performance Issues

If your database query is taking longer than normal, inspect disk activity

sar -d 2 5

4. Analyzing Network Traffic Spikes

If your server is slow, but CPU and memory are okay, inspect network usage

sar -n DEV 1 5

Conclusion

In this article we discussed The SAR (System Activity Report) command in Linux is a powerful tool for monitoring and analyzing system resources. It provides detailed information on CPU usage, memory utilization, I/O devices, network monitoring, disk usage, and more. SAR helps system administrators and developers understand system performance, identify bottlenecks, and optimize resource allocation for improved efficiency and stability. It is an essential tool for managing and optimizing Linux systems.