Backing Up a Lustre File System

Lustre allows backups at several levels. Generally, filesystem-level backups are recommended over device-level backups, in order to allow backing up and restoring individual files/directories on demand. However, it is also recommended that regular MDT device-level backups are also done (e.g. weekly), since restoring from a device-level MDT backup (at most a few TB) is much faster than restoring all of the data in a filesystem (which may be PB of data).

Lustre Filesystem-level Backups

A Lustre filesystem-level backup gives you full control over the files to back up and allows restoration of individual files as needed. Filesystem-level backups are also easier to integrate into existing backup solutions.

Lustre filesystem-level backups are performed from a Lustre client (or many clients working parallel in different directories) rather than on individual server nodes, similar to backing up any other file system. However, due to the large size of most Lustre file systems, it is not always possible to complete a full backup in a timely manner. We recommend that you back up subsets of the file system, such as user or project sub-directories, filesets for a single user, and/or incrementally by date.

Device-level Image Backups

Device-level image backups use dd, e2image, or zfs send to backup individual MDT or OST devices. These are mostly useful when replacing the hardware of a single target device. However, having MDT device-level backups can be very useful if there is severe corruption of the MDT device. Even if the backup is several days or a week old, restoring this backup would be preferable to total filesystem loss. Device-level image backups can be considerably faster than Target-level Filesystem backups since they can stream at full device speeds, rather than handling millions of individual files

For information about device-level image backups or using the Linux Logical Volume Manager (LVM) snapshot feature to make incremental backups of a Lustre file system, see Backing Up and Restoring an MDT or OST (ldiskfs Device Level) in the Lustre Operations Manual.

Target-level Filesystem Backups

Target-level filesystem backups use tar or rsync to back up the files and (critically) xattrs from the underlying MDT or OST filesystem. This is useful when replacing the hardware of a single target device, but where the formatting parameters of the filesystem are being changed (e.g. ldiskfs inode ratio, filesystem features, etc). These will likely use less space than a Device-level Image backup, since they do not backup "empty space", but will likely take much longer to process due to handling millions or billions of individual files on the filesystem in a random order.

For information about target-level filesystem backups, see Backing Up an OST or MDT (Backend File System Level) in the Lustre Operations Manual.