Monitor disk input/output

Author: Ivan Voras IvanVoras FreeBSD

Reviewer: name contact BSD flavour

Reviewer: name contact BSD flavour


Concept

A system's disk input/output can have a dramatic impact on performance(((performance))). Know how to use the utilities available on BSD systems to monitor disk I/O and interpret their results.

Introduction

Monitoring disk I/O can be crucial for troubleshooting a machine. There are a few common symptoms that may indicate an overtasked disk I/O systems: slow or delayed starting of applications and shells, slow remote logins, or slowness in the specific main task the machine is doing (e.g. e-mail server, database server, etc.). Thus it's important to reliably monitor access rates and throughput.

iostat(((iostat)))

There are several ways this can be done, but the most common one is iostat(8). When started without arguments it will display one or more header lines listing devices and a single statistics line that represents the current I/O performance of those devices. This single snapshot is often not a reliable indicator of true I/O performance and it's more useful to specify the -w N argument to iostat which tells it to display statistics in a loop, every N seconds. On a big machine, there may be more devices than fit the screen so iostat will by default display only 5 devices. The portable way to override this, usable on all BSD's is to specify device names on the command line, but FreeBSD has extended iostat with -n N argument whichtells it to display at most N devices.

vmstat(((vmstat)))

The vmstat utility displays low-level information from the kernel. When started without arguments it will display a snapshot of statistics, but if called with -w N argument it will loop and display a line of statistics every N seconds, similar to iostat. The specific information iostat displays differs among the systems but it usually includes the amount of free memory, number of page faults, memory paging activity (swap), and CPU stats. vmstat is important as it's a quick way to find out if the system's high I/O rates are due to memory swapping.

systat(((systat)))

The systat utility is more complex than those already mentioned, as it's a full-screen utility that is usually used for long-term performance tracking (for example: started on a spare console in the system room that is overseen by administrators). It has several display modes, which differ among BSD systems, but the common ones are iostat, vmstat, netstat, mbufs, swap and pigs. The display mode is specified directly on the command line (but prefixed with a - on FreeBSD).

  • iostat mode shows I/O statistics similar to the iostat utility
  • vmstat mode shows kernel statistics similar to the vmstat utility
  • netstat mode shows network I/O statistics similar to the netstat utility
  • mbufs mode shows network buffers statistics
  • swap mode shows swap usage
  • pigs mode shows processes with highest CPU usage

Different BSD systems have some useful additions to the list of display modes, for example FreeBSD has ifstat mode for per-network-interface statistics, and NetBSD has a ps mode that displays a list of processes. See specific man pages for more information.

nfsstat(((nfsstat)))

The nfsstat utility shows NFS statistics. When started without arguments it will display a screenfull of information about NFS, but if called with -w N it will display a two line statics every N seconds (about client and server NFS usage).

gstat(((gstat)))

The gstat utility is specific to FreeBSD. It's a full-screen utility requiring root privileges that shows I/O statistics for all GEOM devices, including virtual devices. With gstat, I/O can be monitored for individual disk partitions, virtual devices such as RAID geoms, memory drives and all other GEOM devices.

Examples

The following will continously monitor I/O statistics for first two SCSI drives on FreeBSD:

> iostat -w 1 da0 da1

To see an overview of NFS performance, use:

> nfsstat

Practice Exercises

  1. Start a "fork bomb" program (usually one can be found in ports/packages/pkgsrc of the system) and monitor how the system behaves with each of above utilities.

More information

iostat(8), gstat(8) on FreeBSD, systat(1), vmstat(1), nfsstat(1)