VDBench - Revision history

Elliswilson at 16:26, 10 February 2025

2025-02-10T16:26:25Z

← Older revision		Revision as of 09:26, 10 February 2025
Line 20:		Line 20:
	<li>Download VDBench. The current official version is available from the Oracle Technology Network (OTN):		<li>Download VDBench. The current official version is available from the Oracle Technology Network (OTN):
	<blockquote>		<blockquote>
	~~http~~://www.oracle.com/~~technetwork~~/server-storage/vdbench-downloads~~-1901681~~.html		https://www.oracle.com/downloads/server-storage/vdbench-downloads.html
	~~</blockquote>~~
	~~The download is free, but Oracle requires that users register an account. An older version remains on SourceForge:~~
	~~<blockquote>~~
	~~https://sourceforge.net/projects/vdbench/~~
	</blockquote>		</blockquote>
			The download is free, but Oracle requires that users register an account.
	<li>Unzip the VDBench archive:		<li>Unzip the VDBench archive:
	<pre style="overflow-x:auto">		<pre style="overflow-x:auto">
	mkdir $HOME/vdbench		mkdir $HOME/vdbench
	cd $HOME/vdbench $$ unzip $HOME/~~vdbench50406~~.zip		cd $HOME/vdbench $$ unzip $HOME/vdbench50407.zip
	</pre>		</pre>
	</li>		</li>
Line 70:		Line 67:
	</pre>		</pre>

	Initially, structure the read tests to run first. This gives the best opportunity to discover and fix any potential error in the benchmark configuration before a write test is run that destroys the data. Adjust the input files according to the results obtained when looking at ~~optimisation~~; for example, if write performance is poor, one may wish to disable the read tests altogether while adjusting parameters that affect write performance.		Initially, structure the read tests to run first. This gives the best opportunity to discover and fix any potential error in the benchmark configuration before a write test is run that destroys the data. Adjust the input files according to the results obtained when looking at optimization; for example, if write performance is poor, one may wish to disable the read tests altogether while adjusting parameters that affect write performance.
	<br/><br/>		<br/><br/>
	Create one test profile for each device under test. Templates containing multiple storage definitions will be evaluated for future use once issues relating to CPU utilisation have been resolved (see notes).		Create one test profile for each device under test. Templates containing multiple storage definitions will be evaluated for future use once issues relating to CPU utilisation have been resolved (see notes).
Line 79:		Line 76:
	done		done
	</pre>		</pre>
	<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to ~~visualise~~ the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy drives performing within +/-5% of one another. Faulty drives normally stand out quite clearly.		<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to visualize the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy drives performing within +/-5% of one another. Faulty drives normally stand out quite clearly.
	<li>Replace any bad drives and re-run VDBench against those targets.		<li>Replace any bad drives and re-run VDBench against those targets.
	</ol>		</ol>
Line 110:		Line 107:
	done		done
	</pre>		</pre>
	<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to ~~visualise~~ the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy volumes performing within +/-5% of one another.		<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to visualize the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy volumes performing within +/-5% of one another.
	<li>If any exceptions are discovered, examine the affected LUN to identify the root cause. If a hardware fault has been identified, replace the affected component.		<li>If any exceptions are discovered, examine the affected LUN to identify the root cause. If a hardware fault has been identified, replace the affected component.
	<li>If one or more disk drives have been replaced, re-run vdbench against the replacement device(s). Note that it may be necessary to destroy the RAID volume in order to re-run the vdbench test case for individual drives. When individual testing is complete, re-assemble the RAID volume and re-run the benchmark for RAID LUNs.		<li>If one or more disk drives have been replaced, re-run vdbench against the replacement device(s). Note that it may be necessary to destroy the RAID volume in order to re-run the vdbench test case for individual drives. When individual testing is complete, re-assemble the RAID volume and re-run the benchmark for RAID LUNs.
	<li>~~Finalise~~ the results and record in the spreadsheet.		<li>Finalize the results and record in the spreadsheet.
	</ol>		</ol>

Line 205:		Line 202:

	<code>rd=run2</code>		<code>rd=run2</code>
	\| Marks the start of a run definition. Must appear after the storage definitions and workload definitions. <code>run1</code> is an arbitrary label, unique within the definition file. It is recommended that the run definition name reflect the test characteristics, e.g.: <code>sdX_run_r_seq_iomax</code> (<code>sdX</code> is the device label, <code>run</code> ~~menas~~ run definition, <code>r</code> for read workload, <code>seq</code> for sequential workload, <code>iomax</code> means maximum I/O rate).		\| Marks the start of a run definition. Must appear after the storage definitions and workload definitions. <code>run1</code> is an arbitrary label, unique within the definition file. It is recommended that the run definition name reflect the test characteristics, e.g.: <code>sdX_run_r_seq_iomax</code> (<code>sdX</code> is the device label, <code>run</code> means run definition, <code>r</code> for read workload, <code>seq</code> for sequential workload, <code>iomax</code> means maximum I/O rate).
	\|- style="text-align: left; vertical-align: top;"		\|- style="text-align: left; vertical-align: top;"
	! scope="row"\| <code>wd=wd1</code>		! scope="row"\| <code>wd=wd1</code>
Line 259:		Line 256:
	== References ==		== References ==

	* ~~http~~://www.oracle.com/~~technetwork~~/server-storage/vdbench-downloads~~-1901681~~.html		* https://www.oracle.com/downloads/server-storage/vdbench-downloads.html
	* https://sourceforge.net/projects/vdbench

	[[Category:Benchmarking]]		[[Category:Benchmarking]]

Malcolm: Created page with "== Description == VDBench is an I/O workload generator for measuring storage performance and verifying the data integrity of direct-attached and network connected storage. Th..."

2017-06-29T11:01:25Z

Created page with "== Description == VDBench is an I/O workload generator for measuring storage performance and verifying the data integrity of direct-attached and network connected storage. Th..."

New page

== Description ==

VDBench is an I/O workload generator for measuring storage performance and verifying the data integrity of direct-attached and network connected storage.
The software is known to run on several operating platforms.

== Purpose ==

VDBench is typically used to establish baseline performance characteristics of block storage, both for individual disk drives and for RAID LUNs. It can also be used at the file system level to simulate application IO. The processes in this document will use the benchmark to verify the performance of individual drives and LUNs. VDBench will destroy content when running write workloads against raw devices. Do not use on raw devices containing production data.

== Preparation ==

<ol>
<li>Install the Java Run-time environment, if not already present on the target machine. Use the official JRE from http://java.com. The complete list of supported run-times is available at: https://www.java.com/en/download/manual.jsp.
<ul><li>Unless Java is a permanent fixture of the platform run-time, download the 64-bit tarball rather than the RPM package. This will allow the Java software to be installed in an arbitrary, isolated directory structure and easily deleted when the benchmark is concluded. For example:
<pre style="overflow-x:auto">
cd $HOME
tar zxf $HOME/jre-8u131-linux-x64.tar.gz
</pre>
</ul>
<li>Download VDBench. The current official version is available from the Oracle Technology Network (OTN):
<blockquote>
http://www.oracle.com/technetwork/server-storage/vdbench-downloads-1901681.html
</blockquote>
The download is free, but Oracle requires that users register an account. An older version remains on SourceForge:
<blockquote>
https://sourceforge.net/projects/vdbench/
</blockquote>
<li>Unzip the VDBench archive:
<pre style="overflow-x:auto">
mkdir $HOME/vdbench
cd $HOME/vdbench $$ unzip $HOME/vdbench50406.zip
</pre>
</li>
<li>Update the <code>vdbench</code> wrapper script contained within the VDBench distribution to point to the JRE location. e.g.:
<pre style="overflow-x:auto">
cd $HOME/vdbench
sed -i.inst 's/^$java$=.*$/\1=$HOME\/jre1.8.0_131\/bin\/java/' vdbench
</pre>
</li>
<li>Run a quick test to ensure that vdbench can run on the target system:
<pre style="overflow-x:auto">
./vdbench -t
</pre>
</li>
</ol>

Older versions of VDBench require CSH or TCSH, so we can be thankful for small mercies.

== Benchmark Execution ==

=== Establish Baseline Performance of Individual Drives ===

<ol>
<li>Ensure that all individual disks are presented to the operating platform for testing. For some storage arrays, one must create separate LUNS for each target device.
<li>Create a test profile for each target designed to run read only and write only tests. Use the following to create a template:

<pre style="overflow-x:auto">
for i in b c d e f; do
sed 's/sdX/sd'${i}'/g' > input_sd${i}_rw_test <<__EOF
# SD -- Storage Definition
sd=sdX,lun=/dev/sdX,openflags=o_direct
# WD -- Workload Definition
wd=sdX_wd_r_seq,sd=sd*,xfersize=1024k,rdpct=100,seekpct=sequential
wd=sdX_wd_w_seq,sd=sd*,xfersize=1024k,rdpct=0,seekpct=sequential
# RD -- Run Definition
rd=sdX_run_r_seq_iomax,wd=sdX_wd_r_seq,iorate=max,elapsed=100,interval=10,forthreads=(1-1024,d),warmup=20
rd=sdX_run_w_seq_iomax,wd=sdX_wd_w_seq,iorate=max,elapsed=100,interval=10,forthreads=(1-1024,d),warmup=20
__EOF
done
</pre>

Initially, structure the read tests to run first. This gives the best opportunity to discover and fix any potential error in the benchmark configuration before a write test is run that destroys the data. Adjust the input files according to the results obtained when looking at optimisation; for example, if write performance is poor, one may wish to disable the read tests altogether while adjusting parameters that affect write performance.
<br/><br/>
Create one test profile for each device under test. Templates containing multiple storage definitions will be evaluated for future use once issues relating to CPU utilisation have been resolved (see notes).
<li>Run each <code>vdbench</code> test case in sequence:
<pre style="overflow-x:auto">
for i in b c d e f; do
./vdbench -f input_sd${i}_rw_test -o o_sd${i}_rw_test.tod
done
</pre>
<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to visualise the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy drives performing within +/-5% of one another. Faulty drives normally stand out quite clearly.
<li>Replace any bad drives and re-run VDBench against those targets.
</ol>

=== Establish Baseline Performance of RAID LUNs ===

<ol>
<li>Create the RAID LUNs that will be used to establish the file system storage volumes for Lustre.
<li>Repeat the VDBench benchmark using the same test profile as for individual storage devices:
<pre style="overflow-x:auto">
for i in b c d e f; do
sed 's/sdX/sd'${i}'/g' > input_sd${i}_raid_rw_test <<__EOF
# VDBench baseline performance test for RAID Volumes
# SD -- Storage Definition
sd=sdX,lun=/dev/sdX,openflags=o_direct
# WD -- Workload Definition
wd=sdX_raid_wd_r_seq,sd=sd*,xfersize=1024k,rdpct=100,seekpct=sequential
wd=sdX_raid_wd_w_seq,sd=sd*,xfersize=1024k,rdpct=0,seekpct=sequential
# RD -- Run Definition
rd=sdX_raid_run_r_seq_iomax,wd=wd_r_seq,iorate=max,elapsed=100,interval=10,forthreads=(1-1024,d),warmup=20
rd=sdX_raid_run_w_seq_iomax,wd=wd_w_seq,iorate=max,elapsed=100,interval=10,forthreads=(1-1024,d),warmup=20
__EOF
done
</pre>
Note that the device names may be different for assembled LUNs, depending on the driver used and/or vendor-supplied software. e.g. MD RAID devices are typically /dev/mdX and kernel multipath devices are typically /dev/dm-XX. This can vary by Linux distribution as well as by storage vendor.
<li>Run each vdbench test case in sequence:
<pre style="overflow-x:auto">
for i in b c d e f; do
./vdbench -f input_sd${i}_raid_rw_test -o o_sd${i}_raid_rw_test.tod
done
</pre>
<li>Tabulate the results in a spreadsheet such as Excel and generate graphs to visualise the data. Establish the performance trend and look for any exceptions. One should normally expect to see healthy volumes performing within +/-5% of one another.
<li>If any exceptions are discovered, examine the affected LUN to identify the root cause. If a hardware fault has been identified, replace the affected component.
<li>If one or more disk drives have been replaced, re-run vdbench against the replacement device(s). Note that it may be necessary to destroy the RAID volume in order to re-run the vdbench test case for individual drives. When individual testing is complete, re-assemble the RAID volume and re-run the benchmark for RAID LUNs.
<li>Finalise the results and record in the spreadsheet.
</ol>

== Notes ==

=== VDBench Test Definition Files ===
In a vdbench input template, there are 3 main sections that are of importance:
* SD: storage definition
* WD: workload definition
* RD: run definition

Definitions must be recorded in the template in the specific order listed above, i.e. SD, then WD and finally RD.

Each definition is contained on a single line. Continuation over multiple lines can be managed by using the standard shell continuation character '\' (backslash) at the end of the line. The continuation character must be immediately preceded by whitespace and must be the last character on the line.

==== SD: Storage Definition ====

SD, the storage definition, is used to define the characteristics of the disk or LUN to be tested, e.g.:

<pre style="overflow-x:auto">
sd=sdb,lun=/dev/sdb,openflags=o_direct
</pre>

{| border=1 cellpadding=5 style="border-collapse: collapse; border: solid 1px #ccc;"
|- style="text-align: left; vertical-align: bottom; background-color: #eee;"
! scope="col"| Parameter
! scope="col" style="width:50%" | Description
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>sd=sdb</code>
| Marks the start of a storage definition. <code>sdb</code> is an arbitrary label and must be unique within the definition file. Using the device name is recommended – the WWID or some other unique identifier that can be uniquely associated with the device under test are also suitable. Multiple storage definitions can be listed, one per line.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>lun=/dev/sdb</code>
| The path to the storage target.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>openflags=o_direct</code>
| Additional controls or options for opening or closing LUNs or files.
Note: On Linux, one must specify <code>openflags=o_direct</code> when referencing a device file, e.g. <code>/dev/sdX</code>
|-
|}

==== WD: Workload Definition ====

WD, the workload definition describes the test characteristics for a given storage definition:

<pre style="overflow-x:auto;">
wd=wd1,sd=sd*,xfersize=1024k,rdpct=100,seekpct=sequential
wd=wd2,sd=sd*,xfersize=1024k,rdpct=0,seekpct=sequential
</pre>

In the above examples, <code>wd1</code> represents a 100% sequential read workload and <code>wd2</code> represents a 100% sequential write workload.

{| border=1 cellpadding=5 style="border-collapse: collapse; border: solid 1px #ccc;"
|- style="text-align: left; vertical-align: bottom; background-color: #eee;"
! scope="col"| Parameter
! scope="col" style="width:50%" | Description
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>wd=wd1</code>
| Marks the start of a workload definition. Must appear after the storage definitions and before any run definitions. <code>wd1</code> is an arbitrary label and must be unique within the definition file. It is recommended that the workload definition name reflect the type of test, e.g.: <code>sdX_wd_r_seq</code> (where <code>sdX</code> is the device label, <code>wd</code>stands for workload definition, <code>r</code> is read workload, <code>seq</code> means sequential workload).
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>sd=sd*</code>
| The name of the storage definitions to use. There can be more than one, e.g.: <code>sd=(sd1,sd2)</code>. An asterisk (<code>*</code>) indicates all storage definitions listed in the template. In this example, <code>sd*</code> refers to all storage definitions with a label beginning with <code>sd</code>.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>xfersize=1024k</code>
| The data transfer size distribution. Normally use <code>1024k</code> for Lustre workloads.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rdpct=100</code>

<code>rdpct=0</code>
| The percentage number of read transactions in the workload. <code>0</code> indicates zero reads (in other words, a 100% write workload). <code>100</code> indicates a 100% read workload.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>seekpct=sequential</code>
| The percentage of random seeks in the workload. <code>0</code> or <code>sequential</code> indicates zero random seeks. <code>100</code> or <code>random</code> means every I/O goes to a random seek address.
|-
|}

==== RD: Run Definition ====

RD, the run definition, specifies the workload definition to run, the IO rates to generate and how long to run for.

<pre style="overflow-x:auto">
rd=run1,wd=wd1,iorate=max,elapsed=100,interval=10,warmup=20,forthreads=(1-1024,d)
rd=run2,wd=wd2,iorate=max,elapsed=100,interval=10,warmup=20,forthreads=(1-1024,d)
</pre>

{| border=1 cellpadding=5 style="border-collapse: collapse; border: solid 1px #ccc;"
|- style="text-align: left; vertical-align: bottom; background-color: #eee;"
! scope="col"| Parameter
! scope="col" style="width:50%" | Description
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rd=run1</code>

<code>rd=run2</code>
| Marks the start of a run definition. Must appear after the storage definitions and workload definitions. <code>run1</code> is an arbitrary label, unique within the definition file. It is recommended that the run definition name reflect the test characteristics, e.g.: <code>sdX_run_r_seq_iomax</code> (<code>sdX</code> is the device label, <code>run</code> menas run definition, <code>r</code> for read workload, <code>seq</code> for sequential workload, <code>iomax</code> means maximum I/O rate).
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>wd=wd1</code>

<code>wd=wd2</code>
| The workload definition(s) to use. Normally just select one.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>elapsed=100</code>
| The time, in seconds, for each run. Must be at least 2x the reporting interval. Does not include any warmup time, if specified (total run time will be elapsed time plus warm-up).
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>interval=10</code>
| The reporting interval, i.e. the number of seconds between each report update.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>warmup=20</code>
| The time to wait before recording results in the run total. Must be a multiple of the reporting interval. In the above example, the first 2 reports will not be recorded in the overall results. The result of the warmup runs will still be reported in the output but will not form part of the overall result.
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>forthreads=(1-1024,d)</code>
| Create a loop for generating results for different thread counts. <code>(1-1024,d)</code> represents a range from 1 to 1024 threads, (d)oubling the thread count on each iteration (i.e. 1, 2, 4, 8, 16, ... 1024).
|-
|}

=== Running VDBench ===

When executing the <code>vdbench</code> command, use the <code>-f</code> flag to refer to the input test definition and <code>-o</code> to refer to the output directory that will contain the results. e.g.:

<pre style="overflow-x:auto">
./vdbench -f sdb_read_write -o o_sdb_read_write
</pre>

The input definition file name should conform to the following format:

<pre style="overflow-x:auto">
input_<device>_<test type>_test
</pre>
e.g.:
<pre style="overflow-x:auto">
input_sdb_rw_test
</pre>

The output directory name should conform to the following format:

<pre style="overflow-x:auto">
o_<device>_<test type>_test.tod
</pre>

e.g.:
<pre style="overflow-x:auto">
o_sdb_rw_test.tod
</pre>

The suffix, <code>.tod</code>, instructs VDBench to add the date and time of day as the suffix to the output directory. This helps to prevent test results data being overwritten on repeat test runs.

== References ==

* http://www.oracle.com/technetwork/server-storage/vdbench-downloads-1901681.html
* https://sourceforge.net/projects/vdbench

[[Category:Benchmarking]]