ZFS Compression - Revision history

Snehring: Merge into ZFS, pending deletion

2026-06-05T19:21:23Z

Merge into ZFS, pending deletion

← Older revision		Revision as of 12:21, 5 June 2026
Line 1:		Line 1:
	ZFS supports transparent compression of data at the block level for datasets. Enabling compression can improve storage efficiency by decreasing the amount of data written to disk and improve read performance for large files, as less data needs to be read from disk. The tradeoff is that compression consumes additional CPU resources on the Lustre server that imports a zfs dataset with compression enabled.		see
			[[ZFS#compression]]
	Compression is typically most beneficial for OSTs since this is where most of the filesystem data is stored. Enabling compression for MDTs is safe, but there may not be as much benefit since most of the data on an MDT is file metadata which ZFS often compresses by default. If you are using Data on MDT (DoM), then there may be a benefit to using compression since some file data is stored on the MDT. The decision to enable or disable compression on OSTs will need to be evaluated by each site depending on their specific needs and performance requirements. Since compression can be enabled/disabled without affecting exising data, it is easy to test both scenarios without the risk of data loss.

	~~== Check ZFS compression ==~~

	~~To check if compression is enabled for an OST, run:~~

	~~<pre>zfs get compression <OST_dataset></pre>~~

	~~Example output will look something like this:~~

	[~~root@oss1]~~[~~~]# zfs get compression lustre-OST0000~~
	~~NAME PROPERTY VALUE SOURCE~~
	~~lustre-OST0000 compression off local~~

	~~== Set ZFS compression ==~~

	~~To enable compression, run:~~

	~~<pre>zfs set compression=on <OST_dataset></pre>~~

	~~This turns on compression with the default algorithm (lz4 since~~ ZFS version 0.6.5). Other compression algorithms are available, and a list of supported algorithms can be found by looking at the '''zfsprops''' man page. To use a different algorithm, replace the '''compression=on''' option with '''compression=<algorithm>'''. The '''lz4''' algorithm is likely the best choice for now. It has been tested extensively, and provides very good compression balanced with performance. It also implements an early abort feature<ref>https://github.com/openzfs/zfs/blob/e0bf43d64ed01285321bf6c3a308f699c5483efc/module/zfs/lz4_zfs.c#~~L520</ref>, meaning it stops trying to compress if initial~~ compression ~~results do not meet a threshold, so performance with incompressible data isn't degraded.~~

	~~== Check Compression Ratio ==~~

	~~To check compression performance, the compression ratio can be queried on the Lustre server:~~

	~~<pre>zfs get compressratio <OST_dataset></pre>~~

	~~From a user's perspective, the logical size and on-disk size of a file can be determined by running the following commands from a Lustre client:~~

	~~# Check logical usage~~
	~~du --apparent-size -h <file>~~

	~~# Check on-disk usage~~
	~~du -h <file>~~

	~~[[Category:ZFS]][[Category:Howto~~]]

Rfmohr at 18:08, 30 April 2026

2026-04-30T18:08:10Z

← Older revision		Revision as of 11:08, 30 April 2026
Line 1:		Line 1:
	ZFS supports transparent compression of data at the block level for datasets. Enabling compression can improve storage efficiency by decreasing the amount of data written to disk and improve read performance for large files, as less data needs to be read from disk.		ZFS supports transparent compression of data at the block level for datasets. Enabling compression can improve storage efficiency by decreasing the amount of data written to disk and improve read performance for large files, as less data needs to be read from disk. The tradeoff is that compression consumes additional CPU resources on the Lustre server that imports a zfs dataset with compression enabled.

	If ~~desired~~, compression will need to be enabled ~~individually for all relevant OSTs~~.		Compression is typically most beneficial for OSTs since this is where most of the filesystem data is stored. Enabling compression for MDTs is safe, but there may not be as much benefit since most of the data on an MDT is file metadata which ZFS often compresses by default. If you are using Data on MDT (DoM), then there may be a benefit to using compression since some file data is stored on the MDT. The decision to enable or disable compression on OSTs will need to be evaluated by each site depending on their specific needs and performance requirements. Since compression can be enabled/disabled without affecting exising data, it is easy to test both scenarios without the risk of data loss.

			== Check ZFS compression ==

	~~== Check ZFS~~ compression ==		To check if compression is enabled for an OST, run:

	<pre>zfs get compression <~~OST/dataset/here~~></pre>		<pre>zfs get compression <OST_dataset></pre>

			Example output will look something like this:

			[root@oss1][~]# zfs get compression lustre-OST0000
			NAME PROPERTY VALUE SOURCE
			lustre-OST0000 compression off local

	== Set ZFS compression ==		== Set ZFS compression ==

	<pre>zfs set compression=on <~~OST/dataset/here~~></pre>		To enable compression, run:

			<pre>zfs set compression=on <OST_dataset></pre>

	turns on compression with the default algorithm (lz4 since ZFS version 0.6.5). ~~You~~ can ~~chose values other than "~~on~~" for the~~ compression algorithm~~, see~~ '~~man zfsprops~~' for ~~details~~.		This turns on compression with the default algorithm (lz4 since ZFS version 0.6.5). Other compression algorithms are available, and a list of supported algorithms can be found by looking at the '''zfsprops''' man page. To use a different algorithm, replace the '''compression=on''' option with '''compression=<algorithm>'''. The '''lz4''' algorithm is likely the best choice for now. It has been tested extensively, and provides very good compression balanced with performance. It also implements an early abort feature<ref>https://github.com/openzfs/zfs/blob/e0bf43d64ed01285321bf6c3a308f699c5483efc/module/zfs/lz4_zfs.c#L520</ref>, meaning it stops trying to compress if initial compression results do not meet a threshold, so performance with incompressible data isn't degraded.

	To ~~use a different algorithm~~, ~~use for example "~~compression~~=zstd" instead of "~~on".		== Check Compression Ratio ==

			To check compression performance, the compression ratio can be queried on the Lustre server:

	~~'''lz4''' is likely the best choice now. It has been tested extensively, and provides very good compression balanced with performance. It implements an early abort feature~~<~~ref~~>~~https://github.com/openzfs/~~zfs~~/blob/e0bf43d64ed01285321bf6c3a308f699c5483efc/module/zfs/lz4_zfs.c#L520~~</~~ref~~>~~, meaning it stops trying to compress if initial compression results do not meet a threshold, so performance with incompressible data isn't degraded.~~		<pre>zfs get compressratio <OST_dataset></pre>

	~~== Check compression ratio ==~~

	~~On OSS~~:		From a user's perspective, the logical size and on-disk size of a file can be determined by running the following commands from a Lustre client:
	~~<pre>zfs get compressratio <OST/dataset/here></pre>~~

	~~On client machines:~~		# Check logical usage
	~~<pre>~~du --apparent-size -h <file~~/dir~~>~~</pre> will show logical~~ usage ~~of a file (in human readable units), whereas <pre>~~du -h <file>~~</pre> will show actual usage on disk (in human readable units).~~		du --apparent-size -h <file>

			# Check on-disk usage
			du -h <file>

	[[Category:ZFS]][[Category:Howto]]		[[Category:ZFS]][[Category:Howto]]

Snehring: sentence needed a 'the'

2025-10-22T21:08:27Z

sentence needed a 'the'

← Older revision		Revision as of 14:08, 22 October 2025
Line 12:		Line 12:
	<pre>zfs set compression=on <OST/dataset/here></pre>		<pre>zfs set compression=on <OST/dataset/here></pre>

	turns on compression with default algorithm (lz4 since ZFS version 0.6.5). You can chose values other than "on" for the compression algorithm, see 'man zfsprops' for details.		turns on compression with the default algorithm (lz4 since ZFS version 0.6.5). You can chose values other than "on" for the compression algorithm, see 'man zfsprops' for details.

	To use a different algorithm, use for example "compression=zstd" instead of "on".		To use a different algorithm, use for example "compression=zstd" instead of "on".

Snehring: Reworked a bit to be less 'notes about compression'. Updated some information.

2024-10-15T20:31:52Z

Reworked a bit to be less 'notes about compression'. Updated some information.

← Older revision		Revision as of 13:31, 15 October 2024
Line 1:		Line 1:
	~~Notes on enabling and working with~~ ZFS compression. ~~This has shown at LLNL~~ to ~~compress their HPC dataset size by about 50%~~ and ~~also~~ improve read performance for large files, ~~while not hurting write performance (~~as ~~long as the OSS nodes have sufficiently fast CPUs)~~.		ZFS supports transparent compression of data at the block level for datasets. Enabling compression can improve storage efficiency by decreasing the amount of data written to disk and improve read performance for large files, as less data needs to be read from disk.

	~~We assume this is for ZFS on lustre~~, ~~but~~ will ~~work the same~~ for standalone ZFS. To do this on a Lustre/ZFS system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata, since most metadata is already compressed by default in ZFS.		If desired, compression will need to be enabled individually for all relevant OSTs.

	~~'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.~~

	~~<STORAGE> = the pool, filesystem, or directory you choose.~~

	== Check ZFS compression ==		== Check ZFS compression ==

	<pre>zfs get compression <~~STORAGE~~></pre>		<pre>zfs get compression <OST/dataset/here></pre>

	== Set ZFS compression ==		== Set ZFS compression ==

	<pre>zfs set compression=on <~~STORAGE~~></pre>		<pre>zfs set compression=on <OST/dataset/here></pre>

	turns on compression with default algorithm (~~lzjb~~). You can chose values other than "on" for the compression algorithm, see 'man ~~zfs~~' for details.		turns on compression with default algorithm (lz4 since ZFS version 0.6.5). You can chose values other than "on" for the compression algorithm, see 'man zfsprops' for details.

	To use a different algorithm, use for example "compression=~~lz4~~" instead of "on".		To use a different algorithm, use for example "compression=zstd" instead of "on".

	'''lz4''' is likely the best choice now. It has been tested extensively, and provides very good compression balanced with performance. ~~Basically~~, it stops trying to compress ~~after compressing some~~ initial ~~part of the~~ data ~~and getting poor results. ''Details from experts on this topic is needed'~~'		'''lz4''' is likely the best choice now. It has been tested extensively, and provides very good compression balanced with performance. It implements an early abort feature<ref>https://github.com/openzfs/zfs/blob/e0bf43d64ed01285321bf6c3a308f699c5483efc/module/zfs/lz4_zfs.c#L520</ref>, meaning it stops trying to compress if initial compression results do not meet a threshold, so performance with incompressible data isn't degraded.

	~~I think you might only need to set this on pools, and filesystems should inherit, but to be safe, you can apply it to everything~~.

	== Check compression ratio ==		== Check compression ratio ==

	On OSS:		On OSS:
	<pre>zfs get compressratio <~~STORAGE~~></pre>		<pre>zfs get compressratio <OST/dataset/here></pre>

	On client machines:		On client machines:
	~~Keep in mind that fastidious users will likely notice their data apparently shrank if they move things around to a compressed filesystem. "~~du --~~aparent~~-size" on ~~some files or directories can help show what is happening~~.		<pre>du --apparent-size -h <file/dir></pre> will show logical usage of a file (in human readable units), whereas <pre>du -h <file></pre> will show actual usage on disk (in human readable units).

	[[Category:ZFS]][[Category:Howto]]		[[Category:ZFS]][[Category:Howto]]

Adilger at 23:11, 7 August 2015

2015-08-07T23:11:27Z

← Older revision		Revision as of 16:11, 7 August 2015
Line 1:		Line 1:
	Notes on enabling and working with ~~zfs~~ compression		Notes on enabling and working with ZFS compression. This has shown at LLNL to compress their HPC dataset size by about 50% and also improve read performance for large files, while not hurting write performance (as long as the OSS nodes have sufficiently fast CPUs).

	We assume this is for ~~zfs~~ on lustre, but will work the same for standalone ~~zfs~~. To do this on a ~~lustre~~/~~zfs~~ system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata, since most metadata is already compressed by default in ZFS.		We assume this is for ZFS on lustre, but will work the same for standalone ZFS. To do this on a Lustre/ZFS system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata, since most metadata is already compressed by default in ZFS.

	'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.		'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.

Adilger: minor comment about ZFS metadata compression

2015-08-07T23:07:50Z

minor comment about ZFS metadata compression

← Older revision		Revision as of 16:07, 7 August 2015
Line 1:		Line 1:
	Notes on enabling and working with zfs compression		Notes on enabling and working with zfs compression

	We assume this is for zfs on lustre, but will work the same for standalone zfs. To do this on a lustre/zfs system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata.		We assume this is for zfs on lustre, but will work the same for standalone zfs. To do this on a lustre/zfs system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata, since most metadata is already compressed by default in ZFS.

	'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.		'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.

Sknolin at 02:09, 1 May 2015

2015-05-01T02:09:00Z

← Older revision		Revision as of 19:09, 30 April 2015
Line 30:		Line 30:
	On client machines:		On client machines:
	Keep in mind that fastidious users will likely notice their data apparently shrank if they move things around to a compressed filesystem. "du --aparent-size" on some files or directories can help show what is happening.		Keep in mind that fastidious users will likely notice their data apparently shrank if they move things around to a compressed filesystem. "du --aparent-size" on some files or directories can help show what is happening.

			[[Category:ZFS]][[Category:Howto]]

Sknolin at 02:08, 1 May 2015

2015-05-01T02:08:26Z

← Older revision		Revision as of 19:08, 30 April 2015
Line 24:		Line 24:

	== Check compression ratio ==		== Check compression ratio ==

			On OSS:
	<pre>zfs get compressratio <STORAGE></pre>		<pre>zfs get compressratio <STORAGE></pre>

			On client machines:
			Keep in mind that fastidious users will likely notice their data apparently shrank if they move things around to a compressed filesystem. "du --aparent-size" on some files or directories can help show what is happening.

Sknolin at 02:05, 1 May 2015

2015-05-01T02:05:13Z

← Older revision		Revision as of 19:05, 30 April 2015
Line 7:		Line 7:
	<STORAGE> = the pool, filesystem, or directory you choose.		<STORAGE> = the pool, filesystem, or directory you choose.

	# Check ZFS compression		== Check ZFS compression ==

	<pre>zfs get compression <STORAGE></pre>		<pre>zfs get compression <STORAGE></pre>

	# Set ZFS compression		== Set ZFS compression ==

	<pre>zfs set compression=on <STORAGE></pre>		<pre>zfs set compression=on <STORAGE></pre>
Line 23:		Line 23:
	I think you might only need to set this on pools, and filesystems should inherit, but to be safe, you can apply it to everything.		I think you might only need to set this on pools, and filesystems should inherit, but to be safe, you can apply it to everything.

	# Check compression ratio		== Check compression ratio ==

	<pre>zfs get compressratio <STORAGE></pre>		<pre>zfs get compressratio <STORAGE></pre>

Sknolin: Created page with "Notes on enabling and working with zfs compression We assume this is for zfs on lustre, but will work the same for standalone zfs. To do this on a lustre/zfs system, you have..."

2015-05-01T02:04:39Z

Created page with "Notes on enabling and working with zfs compression We assume this is for zfs on lustre, but will work the same for standalone zfs. To do this on a lustre/zfs system, you have..."

New page

Notes on enabling and working with zfs compression

We assume this is for zfs on lustre, but will work the same for standalone zfs. To do this on a lustre/zfs system, you have to perform these commands on every OST in the system. I would definitely not bother doing this on metadata.

'''NOTE'' - here I'll assume you always work on a whole pool or filesystem, but these commands should work on any arbitrary directory too.

<STORAGE> = the pool, filesystem, or directory you choose.

# Check ZFS compression

<pre>zfs get compression <STORAGE></pre>

# Set ZFS compression

<pre>zfs set compression=on <STORAGE></pre>

turns on compression with default algorithm (lzjb). You can chose values other than "on" for the compression algorithm, see 'man zfs' for details.

To use a different algorithm, use for example "compression=lz4" instead of "on".

'''lz4''' is likely the best choice now. It has been tested extensively, and provides very good compression balanced with performance. Basically, it stops trying to compress after compressing some initial part of the data and getting poor results. ''Details from experts on this topic is needed''

I think you might only need to set this on pools, and filesystems should inherit, but to be safe, you can apply it to everything.

# Check compression ratio

<pre>zfs get compressratio <STORAGE></pre>