Upgrading to a New Version of Lustre

Upgrading to a New Version of Lustre

This page covers upgrade procedures for Lustre 2.x filesystems. For the authoritative reference, see the Upgrading a Lustre File System chapter of the Lustre Operations Manual.

Interoperability Rules

Upgrade Type	Servers	Clients
Major (2.x → 2.y)	All servers must be upgraded together	Clients may be upgraded independently
Minor (2.x.y → 2.x.z)	All servers upgraded together, or rolling upgrade	Clients may be upgraded independently

Key rule: All servers (MGS, MDS, OSS) must run the same major Lustre version. Clients have more flexibility — older clients can generally connect to newer servers (check Release Notes for specific interoperability).

Major Release Upgrade (2.x → 2.y)

This procedure requires stopping the entire filesystem. Plan for downtime.

Pre-Upgrade Checklist

• [ ] Back up the filesystem. At minimum, create a device-level backup of ALL MDTs — the MDT is the most critical component.
• [ ] Review the Release Notes for the target version. Note any required kernel versions or distribution changes.
• [ ] Document all persistent parameter settings (lctl get_param -R) — a writeconf during upgrade would erase them.
• [ ] Plan the OS upgrade path if the new Lustre version requires a newer kernel.

Upgrade Procedure

Step 1: Back up the filesystem.

Create a complete, restorable backup. If a full backup is not practical, a device-level backup of each MDT is strongly recommended.

Step 2: Shut down the filesystem.

Unmount in order: clients → MDTs → OSTs → MGT.

# On all clients:
umount /mnt/lustre

# On MDS:
umount /mnt/mdt

# On each OSS:
umount /mnt/ost*

# On MGS (if separate):
umount /mnt/mgt

Step 3: Upgrade the OS on servers.

Install a Linux distribution and kernel version compatible with the target Lustre release (see Release Notes). Reboot all servers.

Step 4: Upgrade the OS on clients.

Install a compatible Linux distribution and kernel. Reboot all clients.

Step 5: Install Lustre server packages on all servers.

# On each server (MGS, MDS, OSS):
yum --nogpgcheck install lustre-*.rpm kmod-lustre-*.rpm \
    lustre-osd-ldiskfs-mount-*.rpm kmod-lustre-osd-ldiskfs-*.rpm \
    e2fsprogs-*.rpm

# Verify:
rpm -qa | egrep "lustre|e2fsprogs"

For ZFS backends, install the ZFS OSD packages instead of ldiskfs.

Step 6: Install Lustre client packages on all clients.

# On each client:
yum --nogpgcheck install lustre-client-*.rpm kmod-lustre-client-*.rpm

# Or for DKMS:
yum --nogpgcheck install lustre-client-*.rpm lustre-client-dkms-*.rpm

# Verify:
rpm -qa | egrep "lustre|kernel"

Step 7: (Optional) Enable new features while targets are unmounted.

See #Optional Feature Enablement below.

Step 8: Start the filesystem.

Mount in order: MGT → MDT(s) → OST(s) → clients.

# On MGS (or combined MGS/MDS):
mount -t lustre /dev/mgt_device /mnt/mgt

# On each MDS:
mount -t lustre /dev/mdt_device /mnt/mdt

# On each OSS:
mount -t lustre /dev/ost_device /mnt/ost0

# On each client:
mount -t lustre mgsnode@tcp:/fsname /mnt/lustre

Note: After a major upgrade, the mount order for the first start is MGT → MDT → OST → clients. For normal subsequent starts, the order is MGT → OST → MDT → clients.

Step 9: (Optional) Enable post-mount features.

Some features can only be enabled after targets are mounted. See #Optional Feature Enablement below.

Step 10: (Optional) Attach new MDTs into the namespace.

If you added new MDTs in Step 7, create directories on them:

lfs mkdir -i <mdt_index> /mnt/lustre/new_directory

Step 11: Verify the upgrade.

# Check filesystem health:
lfs df /mnt/lustre
lctl get_param health_check

# Check version:
lctl get_param version

Minor Release Upgrade (2.x.y → 2.x.z)

Minor releases support rolling upgrades — you can upgrade servers and clients one at a time without shutting down the filesystem.

Rolling Upgrade Procedure

Step 1: Back up the filesystem (device-level MDT backup recommended).

Step 2: Upgrade servers one at a time.

For each server (start with OSSs, then MDS, then MGS):

# If using HA: fail over the targets to the partner server
# Unmount the Lustre targets on this server
umount /mnt/ost*

# Install new packages
yum --nogpgcheck install lustre-*.rpm kmod-lustre-*.rpm ...

# Remount (or fail back)
mount -t lustre /dev/ost_device /mnt/ost0

Step 3: Upgrade clients one at a time.

For each client:

umount /mnt/lustre
yum --nogpgcheck install lustre-client-*.rpm kmod-lustre-client-*.rpm
mount -t lustre mgsnode@tcp:/fsname /mnt/lustre

Optional Feature Enablement

These features can be enabled during an upgrade. Each is optional and some are irreversible (prevent downgrade to older versions).

Feature	Since Version	When to Enable	Command	Reversible?
Project quotas	2.10	While target is unmounted (Step 7)	tune2fs -O project /dev/target_device	No — prevents use by older ldiskfs
Wide striping (ea_inode)	2.13	While MDT is unmounted (Step 7)	tune2fs -O ea_inode /dev/mdt_device	Yes
index_in_idif	2.7	After OST is mounted (Step 9)	lctl set_param osd-ldiskfs.*.osd_index_in_idif=1	No — prevents use by older Lustre
DNE (additional MDTs)	2.4	While new MDT device is unmounted (Step 7)	mkfs.lustre --reformat --fsname=FS --mdt --mgsnode=MGS --index=N /dev/device	N/A
Striped directories	2.8	After MDTs are mounted (Step 10)	lfs mkdir -c 2 /mnt/lustre/striped_dir	N/A
Auto-distribute subdirs	2.13	After MDTs are mounted (Step 10)	lfs setdirstripe -D -c 1 -i -1 /mnt/lustre/dir	N/A

Warning: Enabling project quotas or index_in_idif is a one-way operation. Once enabled, the target cannot be used by older Lustre versions. Only enable these if you are certain you will not need to downgrade.

Version-Specific Notes

• Lustre 2.15: If no default directory layout is set on the root directory, the MDS automatically configures round-robin distribution across all MDTs for new subdirectories.
• Lustre 2.13+: lfs setdirstripe -D can automatically distribute new subdirectories to less-full MDTs.
• Pre-2.4 clients: Cannot access directories on MDTs other than MDT0. Accessing such directories returns an I/O error.
• Client kernel version: The kernel running on the client must match the version expected by the lustre-client-modules package.

See Also

• Lustre Release Information — Current and past releases
• Release Terminology — Version numbering scheme
• Simplified Interoperability — Interoperability design details
• Lustre Common Mistakes — Pitfalls to avoid during upgrades