ZFS Compression: Difference between revisions

From Lustre Wiki
Jump to navigation Jump to search
mNo edit summary
(Merge into ZFS, pending deletion)
Tag: Replaced
 
Line 1: Line 1:
ZFS supports transparent compression of data at the block level for datasets. Enabling compression can improve storage efficiency by decreasing the amount of data written to disk and improve read performance for large files, as less data needs to be read from disk.  The tradeoff is that compression consumes additional CPU resources on the Lustre server that imports a zfs dataset with compression enabled.
see
 
[[ZFS#compression]]
Compression is typically most beneficial for OSTs since this is where most of the filesystem data is stored.  Enabling compression for MDTs is safe, but there may not be as much benefit since most of the data on an MDT is file metadata which ZFS often compresses by default.  If you are using Data on MDT (DoM), then there may be a benefit to using compression since some file data is stored on the MDT.  The decision to enable or disable compression on OSTs will need to be evaluated by each site depending on their specific needs and performance requirements.  Since compression can be enabled/disabled without affecting exising data, it is easy to test both scenarios without the risk of data loss.
== Check ZFS compression ==
 
To check if compression is enabled for an OST, run:
<pre>zfs get compression <OST_dataset></pre>
 
Example output will look something like this:
 
[root@oss1][~]# zfs get compression lustre-OST0000
NAME          PROPERTY    VALUE          SOURCE
lustre-OST0000  compression  off            local
== Set ZFS compression ==
 
To enable compression, run:
 
<pre>zfs set compression=on <OST_dataset></pre>
This turns on compression with the default algorithm (lz4 since ZFS version 0.6.5). Other compression algorithms are available, and a list of supported algorithms can be found by looking at the '''zfsprops''' man page.  To use a different algorithm, replace the '''compression=on''' option with '''compression=<algorithm>'''.  The '''lz4''' algorithm is likely the best choice for now. It has been tested extensively, and provides very good compression balanced with performance. It also implements an early abort feature<ref>https://github.com/openzfs/zfs/blob/e0bf43d64ed01285321bf6c3a308f699c5483efc/module/zfs/lz4_zfs.c#L520</ref>, meaning it stops trying to compress if initial compression results do not meet a threshold, so performance with incompressible data isn't degraded.
== Check Compression Ratio ==
 
To check compression performance, the compression ratio can be queried on the Lustre server:
 
<pre>zfs get compressratio <OST_dataset></pre>
 
From a user's perspective, the logical size and on-disk size of a file can be determined by running the following commands from a Lustre client:
 
# Check logical usage
du --apparent-size -h <file>
# Check on-disk usage
du -h <file>
 
[[Category:ZFS]][[Category:Howto]]

Latest revision as of 12:21, 5 June 2026

see ZFS#compression

Retrieved from "http://wiki.lustre.org/index.php?title=ZFS_Compression&oldid=5562"