Benchmarking

C H A P T E R 17

Benchmarking

The benchmarking process involves identifying the highest standard of excellence and performance, learning and understanding these standards, and finally adapting and applying them to improve the performance. Benchmarks are most often used to provide an idea of how fast any software or hardware runs.

Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The variety of workloads that these systems experience also adds in to this difficulty. One of the most widely researched areas in storage subsystem is file system design, implementation, and performance.

This chapter describes benchmark suites to test Lustre and includes the following sections:

Bonnie++ Benchmark

IOR Benchmark

IOzone Benchmark

17.1 Bonnie++ Benchmark

Bonnie++ is a benchmark suite that having aim of performing a number of simple tests of hard drive and file system performance. Then you can decide which test is important and decide how to compare different systems after running it. Each Bonnie++ test gives a result of the amount of work done per second and the percentage of CPU time utilized.

There are two sections to the program's operations. The first is to test the I/O throughput in a fashion that is designed to simulate some types of database applications. The second is to test creation, reading, and deleting many small files in a fashion similar to the usage patterns.

Bonnie++ is a benchmark tool that test hard drive and file system performance by sequential I/O and random seeks. Bonnie++ tests file system activity that has been known to cause bottlenecks in I/O-intensive applications.

To install and run the Bonnie++ benchmark:

1. Download the most recent version of the Bonnie++ software:

http://www.coker.com.au/bonnie++/

2. Install and run the Bonnie++ software (per the ReadMe file accompanying the software).

Sample output:

Version  1.03 	--Sequential Output-- --Sequential Input- --Random--
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
mds	2G			38118	22		21245		10		51967		10	90.0		0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
16   510   0 +++++ +++   283   1   465   0 +++++ +++   291   1
mds,2G,,,38118,22,21245,10,,,51967,10,90.0,0,16,510,0,+++++,+++,283,1,465,0,+++++,+++,291,1
Version  1.03 	--Sequential Output-- --Sequential Input- --Random--
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
mds              2G 27460  92 41450  25 21474  10 19673  60 52871  10  88.0   0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
 
files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
16 29681  99 +++++ +++ 30412  90 29568  99 +++++ +++ 28077  82
mds,2G,27460,92,41450,25,21474,10,19673,60,52871,10,88.0,0,16,29681,99,+++++,+++,30412,90,29568,99,+++++,+++,28077,82

17.2 IOR Benchmark

The IOR_survey script tests the performance of the Lustre file system. It uses IOR (Interleaved or Random), a script used for testing performance of parallel file systems using various interfaces and access patterns. IOR uses MPI for process synchronization.

Under the control of compile-time defined constants (and, to a lesser extent, environment variables), I/O is done via MPI-IO. The data are written and read using independent parallel transfers of equal-sized blocks of contiguous bytes that cover the file with no gaps and that do not overlap each other. The test consists of creating a new file, writing it with data, then reading the data back.

The IOR benchmark, developed by LLNL, tests system performance by focusing on parallel/sequential read/write operations that are typical of scientific applications.

To install and run the IOR benchmark:

1. Satisfy the prerequisites to run IOR.

a. Download lam 7.0.6 (local area multi-computer):

http://www.lam-mpi.org/7.0/download.php

b. Obtain a Fortran compiler for the Fedora Core 4 operating system.

c. Download the most recent version of the IOR software:

http://sourceforge.net/projects/ior-sio

2. Install the IOR software (per the ReadMe file and User Guide accompanying the software).

3. Run the IOR software. In user mode, use the lamboot command to start the lam service and use appropriate Lustre-specific commands to run IOR (described in the IOR User Guide).

Sample Output:

IOR-2.9.0: MPI Coordinated Test of Parallel I/O
Run began: Fri Sep 29 11:43:56 2006
Command line used: ./IOR -w -r -k -O lustrestripecount 10 -o test
Machine: Linux mds
Summary:
api			= POSIX
test filename		= test
access			= single-shared-file
clients			= 1 (1 per node)
repetitions		= 1
xfersize		= 262144 bytes
blocksize		= 1 MiB
aggregate filesize	= 1 MiB
access		bw(MiB/s)	block(KiB)	xfer(KiB)	open(s)		wr/rd(s)	close(s)	iter
------		---------	---------	--------	------		--------	--------	----
write		173.89		1024.00		256.00		0.000030	0.005701	0.000016	0   
read		278.49		1024.00		256.00		0.000009	0.003566	0.000012	0   
 
Max Write:	173.89 MiB/sec (182.33 MB/sec)
Max Read:	278.49 MiB/sec (292.02 MB/sec)
 
Run finished: Fri Sep 29 11:43:56 2006

17.3 IOzone Benchmark

IOZone is a file system benchmark tool which generates and measures a variety of file operations. Iozone has been ported to many machines and runs under many operating systems. Iozone is useful to perform a broad file system analysis of a vendor’s computer platform. The benchmark tests file I/O performance for the operations like read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read/write, pread/pwrite variants, aio_read, aio_write, mm, etc.

The IOzone benchmark tests file I/O performance for the following operations: read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read/write, pread/pwrite variants, aio_read, aio_write, and mmap.

To install and run the IOzone benchmark:

1. Download the most recent version of the IOZone software from this location:

http://www.iozone.org

2. Install the IOZone software (per the ReadMe file accompanying the IOZone software).

3. Run the IOZone software (per the ReadMe file accompanied with the IOZone software).

Sample Output
Iozone:	Performance Test of File I/O
	Version $Revision: 3.263 $
Compiled for 32 bit mode.
Build: linux 
Contributors:William Norcott, Don Capps, Isom Crawford, 
	Kirby Collins, Al Slater, Scott Rhine, Mike Wisner,
	Ken Goss, Steve Landherr, Brad Smith, Mark Kelly, 
	Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million,
	Jean-Marc Zucconi, Jeff Blomberg, Erik Habbinga, 
	Kris Strecker, Walter Wong.
Run began: Fri Sep 29 15:37:07 2006
Network distribution mode enabled.
Command line used: ./iozone -+m test.txt
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random		random		bkwd		record		stride		KB		reclen			write rewrite	read			reread		read		write		read		rewrite			read   fwrite	frewrite			fread		freread
 
512		4		194309		406651		728276		792701		715002			498592  638351	700365			587235		190554		378448		686267		765201
 
iozone test complete.