Elliswilson at 15:56, 2 December 2024

2024-12-02T15:56:58Z

← Older revision		Revision as of 08:56, 2 December 2024
Line 185:		Line 185:
	raw /dev/raw/raw1 0 0		raw /dev/raw/raw1 0 0
	</pre>		</pre>

			== References ==
			* [https://doc.lustre.org/lustre_manual.xhtml#benchmark.sgpdd-survey Lustre Manual: Testing I/O Performance of Raw Hardware (sgpdd-survey)]

	[[Category:Benchmarking]]		[[Category:Benchmarking]]

Malcolm: /* SGPDD-Survey Input Parameters */

2017-06-26T06:14:27Z

SGPDD-Survey Input Parameters

← Older revision		Revision as of 23:14, 25 June 2017
Line 146:		Line 146:
	! scope="row"\| <code>rslt_loc</code>		! scope="row"\| <code>rslt_loc</code>
	\| Directory to contain results. Must exist before benchmark is run.		\| Directory to contain results. Must exist before benchmark is run.
	\| <code>rslt_loc=/var/tmp/sgpdd-~~survey/~~</code>		\| <code>rslt_loc=/var/tmp/sgpdd-survey_out</code>
	\|- style="text-align: left; vertical-align: top;"		\|- style="text-align: left; vertical-align: top;"
	! scope="row"\| <code>scsidevs</code>		! scope="row"\| <code>scsidevs</code>

Malcolm: /* Benchmark Execution */

2017-06-26T06:13:45Z

Benchmark Execution

← Older revision		Revision as of 23:13, 25 June 2017
Line 55:		Line 55:
	! scope="row"\| <code>rslt_loc</code>		! scope="row"\| <code>rslt_loc</code>
	\| Directory to contain results. Must exist before benchmark is run		\| Directory to contain results. Must exist before benchmark is run
	\| <code>/var/tmp/sgpdd-~~survey/~~</code>		\| <code>/var/tmp/sgpdd-survey_out</code>
	\|- style="text-align: left; vertical-align: top;"		\|- style="text-align: left; vertical-align: top;"
	! scope="row"\| <code>scsidevs</code>		! scope="row"\| <code>scsidevs</code>
Line 74:		Line 74:

	<pre style="overflow-x:auto;">		<pre style="overflow-x:auto;">
	mkdir -p /var/tmp/~~sgpdd_survey_out~~		mkdir -p /var/tmp/sgpdd-survey_out

	crglo=1 crghi=256 \		crglo=1 crghi=256 \
	thrlo=1 thrhi=4096 \		thrlo=1 thrhi=4096 \
	size=51200 \		size=51200 \
	rslt_loc=/var/tmp/~~sgpdd_survey_out~~ \		rslt_loc=/var/tmp/sgpdd-survey_out \
	scsidevs="ct7-oss1:/dev/sdb ct7-oss1:/dev/sdc ct7-oss1:/dev/sdd" \		scsidevs="ct7-oss1:/dev/sdb ct7-oss1:/dev/sdc ct7-oss1:/dev/sdd" \
	sgpdd-survey		sgpdd-survey

Malcolm: Created page with "== Description == `sgpdd-survey` is an IO workload generator for benchmarking the performance of disk storage, generating large sequential IO workload on the targe..."

2017-06-26T05:20:19Z

Created page with "== Description == <code>sgpdd-survey</code> is an IO workload generator for benchmarking the performance of disk storage, generating large sequential IO workload on the targe..."

New page

== Description ==

<code>sgpdd-survey</code> is an IO workload generator for benchmarking the performance of disk storage, generating large sequential IO workload on the target storage devices. It is a wrapper for the <code>sgp_dd</code> (SCSI Generic Parallel DD) command found in the SCSI device utilities (<code>sg3_utils</code>) package.

<code>sgp_dd</code> is a scalable version of the “<code>dd</code>” command with options for multi-threaded IO with different blocksizes and regionsizes. From the <code>sgp_dd(8)</code> man page:

<blockquote style="font-style: italic; border-left: solid #ccc 2px; padding: 2px 7px 2px 7px;">
"[<code>sgp_dd</code> is] specialised for "files" that are Linux SCSI generic (sg) and raw devices. Similar syntax and semantics to dd(1) but does not perform any conversions. Uses POSIX threads to increase the amount of parallelism. This improves speed in some cases."
</blockquote>

== Purpose ==

SGPDD-survey is intended to evaluate the raw performance of all LUNs in the storage arrays attached to a server. The benchmark is useful in testing the overall throughput of a storage controller and is typically run after establishing the baseline performance of the individual drives and/or LUNs attached through the controller.

The logic behind this process flow is to determine whether or not the storage controller or HBA or some other higher-level component within the server platform introduces a performance bottleneck, at least for bandwidth. One can normally derive the theoretical bandwidth of the data path from the hardware specifications. Low-level specifications for the disk drives and the I/O paths will determine the theoretical maximum bandwidth of the storage subsystem. One must consider the bandwidth of the PCIe bus, HBA, storage controller and the individual storage devices (disk drives and SSDs). The slowest component in the IO path will determine the maximum throughput that can be obtained by the system overall. The <code>sgpdd-survey</code> benchmark can help to determine how close to the theoretical bandwidth the system is operating and highlight any significant deviation from the ideal.

<code>sgpdd-survey</code> is typically run within an individual host across all LUNs presented to that host. In high availability (HA) server configurations, where the storage will be presented in an active-passive failover configuration, the benchmark is normally run against the "primary" storage targets for each server only. That is, the LUNs are assumed to be mapped to their preferred servers.

== Preparation ==

The <code>sgpdd-survey</code> script is distributed in the <code>lustre-iokit</code> package, and requires the sgp_dd program from the <code>sg3_utils</code> package. Install the <code>lustre-iokit</code> package on each of the machines that will be evaluated by the benchmark. On RHEL / CentOS systems, <code>yum</code> will automatically resolve additional dependencies.

Configure all storage volumes into their production configuration and present the LUNs for use on the target host.

== Benchmark Execution ==

The <code>sgpdd-survey</code> script takes its parameters from environment variables established at run time. The parameters that are of most interest are as follows (refer to the section [[#SGPDD-Survey_Input_Parameters|SGPDD-Survey Input Parameters]] for a detailed breakdown of the parameters and how to calculate suitable values):

{| border=1 cellpadding=5 style="border-collapse: collapse; border: solid 1px #ccc;"
|- style="text-align: left; vertical-align: bottom; background-color: #eee;"
! scope="col"| Name
! scope="col" style="width:50%" | Description
! scope="col"| Typical Values
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>crglo</code>
| Initial number of concurrent regions
| <code>1</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>crghi</code>
| Maximum number of concurrent regions
| <code>256</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>thrlo</code>
| Initial number of threads
| <code>1</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>thrhi</code>
| Maximum number of threads
| <code>4096</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>size</code>
| Data set size in MiB per storage device (LUN)
| <code>2.5 * size(RAM) / count(LUNs)</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rslt_loc</code>
| Directory to contain results. Must exist before benchmark is run
| <code>/var/tmp/sgpdd-survey/</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>scsidevs</code>
| List of storage targets to test. Space separated list enclosed in quotes; each target has the format <code><hostname>:<device path></code>. Must choose one of <code>scsidevs</code> or <code>rawdevs</code> but not both
| Example:<br/>
<code>"oss01:/dev/sdb oss01:/dev/sdc"</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rawdevs</code>
| List of raw devices to test. Space separated list enclosed in quotes; each target has the format <code><hostname>:<device path></code>. Must choose one of <code>scsidevs</code> or <code>rawdevs</code> but not both
| Example:<br/>
<code>"oss01:/dev/raw/raw1 oss01:/dev/raw/raw2"</code>
|-
|}

The <code>scsidevs</code> option is typically chosen for hardware RAID storage systems, whereas the <code>rawdevs</code> option is typically required for software RAID devices.

The following is an example command line for executing <code>sgpdd-survey</code>:

<pre style="overflow-x:auto;">
mkdir -p /var/tmp/sgpdd_survey_out

crglo=1 crghi=256 \
thrlo=1 thrhi=4096 \
size=51200 \
rslt_loc=/var/tmp/sgpdd_survey_out \
scsidevs="ct7-oss1:/dev/sdb ct7-oss1:/dev/sdc ct7-oss1:/dev/sdd" \
sgpdd-survey
</pre>

== SGPDD-Survey Input Parameters ==

{| border=1 cellpadding=5 style="border-collapse: collapse; border: solid 1px #ccc;"
|- style="text-align: left; vertical-align: bottom; background-color: #eee;"
! scope="col"| Name
! scope="col" style="width:50%" | Description
! scope="col"| Typical Values
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>crglo</code>

<code>crghi</code>
| The range of concurrent regions to exercise on each iteration, per device. Starting at <code>crglo</code>, the number of concurrent regions is doubled on each iteration until <code>crghi</code> is reached.

The <code>crglo</code> and <code>crghi</code> parameters control how many independent regions on the storage device will be read or written simultaneously. This is intended to simulate multiple Lustre clients accessing each OST.

More regions mean less performance, as this will increase the amount of seeking the storage devices need to do.
| <code>crglo=1</code>

<code>crghi=256</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>thrlo</code>

<code>thrhi</code>
| The range of threads to iterate over. For each value of the concurrent region range, run <code>sgp_dd</code> with the number of threads starting at <code>thrlo</code> until <code>thrhi</code> is reached. The thread count is doubled on every iteration of the thread count loop. The starting thread count is either <code>thrlo</code> or <code>crglo</code>, whichever is greater.

The <code>thrlo</code> and <code>thrhi</code> parameters are used to direct the number of worker threads running in parallel. This is intended to simulate the Lustre OSS threads.
| <code>thrlo=1</code>

<code>thrhi=4096</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>size</code>
| The data set size in MiB per storage device (LUN). The total dataset size for the entire benchmark is calculated as the LUN dataset size multiplied by the number of LUNs in the benchmark:

<pre style="overflow-x:auto;">
ds_total = size * count(LUNs)
</pre>

Set <code>size</code> to a small value (e.g. 100MB) to quickly test the configuration for correctness.

For a full benchmarking run, set <code>ds_total</code> to a value greater than or equal to twice the target system's RAM. This circumvents any cacheing that may occur within the target system. From this, calculate the value of the <code>size</code> input parameter for <code>sgpdd-survey</code> as follows:

<pre style="overflow-x:auto;">
size = ds_total / count(LUNs)
</pre>
where <code>ds_total = 2.5 * RAM</code>

As an example, for a server with 32GB RAM and 4 LUNs, the input size is calculated as follows:

<pre style="overflow-x:auto;">
size = (32GB * 2.5) / 4 LUNs
= (32768MiB * 2.5) / 4
= 81920 / 4
= 20480
</pre>
| <code>2.5 * sizeof(RAM) / count(LUNs), in MB</code>

e.g.:

<code>size=20480</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rslt_loc</code>
| Directory to contain results. Must exist before benchmark is run.
| <code>rslt_loc=/var/tmp/sgpdd-survey/</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>scsidevs</code>
| List of storage targets to test. Space separated list enclosed in quotes; each target has the format <code><hostname>:<device path></code>. The host component is optional and can be used when <code>sgpdd-survey</code> is scaled across multiple servers (ref: LU-2043). Normally, sgpdd-survey benchmarking tasks are contained within individual hosts.

Must choose one of <code>scsidevs</code> or <code>rawdevs</code> but not both. Under normal circumstances, use <code>scsidevs</code> unless working with MD RAID, in which case use <code>rawdevs</code>.
| Example:

<code>"scsidevs=oss01:/dev/sdb oss01:/dev/sdc"</code>
|- style="text-align: left; vertical-align: top;"
! scope="row"| <code>rawdevs</code>
| List of raw devices to test. Space separated list enclosed in quotes; each target has the format <code><hostname>:<device path></code>. When benchmarking software RAID, use the <code>raw</code> command to map the MD RAID device path to a raw device. See the [[#MD_RAID_and_Raw_Devices| MD RAID and raw devices]] section for more detailed information.

Must choose one of <code>scsidevs</code> or <code>rawdevs</code> but not both. <code>rawdevs</code> is normally only used when benchmarking MD RAID.
| Example:

<code>rawdevs="oss01:/dev/raw/raw1 oss01:/dev/raw/raw2"</code>
|-
|}

== MD RAID and Raw Devices ==

Software RAID volumes created with MD RAID should be submitted to the benchmark using the <code>rawdevs</code> input parameter, not <code>scsidevs</code>. For this to work, bind the MD RAID block device to a raw device using the <code>raw</code> command. e.g.:

<pre style="overflow-x:auto;">
raw /dev/raw/raw1 /dev/md0
</pre>

This will create a raw binding for the MD RAID device <code>/dev/md0</code> to <code>/dev/raw/raw1</code>. The raw device name is arbitrary but take care not to overwrite an existing binding as the binding will be replaced without issuing a warning. One can also specify the major, minor numbers for the block device to be mapped. To list the existing raw bindings:

<pre style="overflow-x:auto;">
raw -qa
</pre>

To remove a binding, map the raw device to major and minor device numbers 0 0, e.g.:

<pre style="overflow-x:auto;">
raw /dev/raw/raw1 0 0
</pre>

[[Category:Benchmarking]]

SGPDD Survey - Revision history

Elliswilson at 15:56, 2 December 2024

Malcolm: /* SGPDD-Survey Input Parameters */

Malcolm: /* Benchmark Execution */

Malcolm: Created page with "== Description == sgpdd-survey is an IO workload generator for benchmarking the performance of disk storage, generating large sequential IO workload on the targe..."

Malcolm: Created page with "== Description == `sgpdd-survey` is an IO workload generator for benchmarking the performance of disk storage, generating large sequential IO workload on the targe..."