Lustre Configuration Log Management

This page covers three powerful but potentially destructive commands for managing Lustre configuration logs: writeconf, replace_nids, and clear_conf. Understand the implications of each before using them.

Background

Lustre stores persistent configuration in configuration logs on the MGT. These logs tell servers and clients how to find each other and record persistent parameter settings (those set with lctl conf_param).

When you change server parameters, add targets, or reconfigure networking, these logs may need to be updated. The three commands below provide different levels of log manipulation.

writeconf — Regenerate Configuration Logs

writeconf erases the existing configuration log and regenerates it from scratch using the information stored on each target device.

When to Use

The configuration log is corrupted and the filesystem cannot start.
You are changing server NIDs and replace_nids is not available or not sufficient.

What It Destroys

Warning: writeconf erases:

All OST pool definitions
All persistent parameters set with lctl conf_param

Parameters set with lctl set_param -P are not erased (they are stored differently on the MGS).

Before running writeconf, save the current configuration:

mgs# lctl --device MGS llog_print testfs-client
mgs# lctl --device MGS llog_print testfs-MDT0000
mgs# lctl --device MGS llog_print testfs-OST0000

Keep this output so you can recreate pool definitions and conf_param settings after writeconf.

Procedure

Step 1: Shut down the filesystem completely.

# Unmount clients
client# umount /mnt/lustre

# Unmount MDTs
mds# umount /mnt/mdt

# Unmount OSTs
oss# umount /mnt/ost*

# MGS can remain mounted if it is separate from the MDT

Step 2: Run writeconf on all MDTs first, then all OSTs.

On each MDS:

tunefs.lustre --writeconf /dev/mdt_device

On each OSS:

tunefs.lustre --writeconf /dev/ost_device

Step 3: Restart the filesystem in order.

# Mount MGT (if separate and not already mounted)
# Mount MDT(s), starting with MDT0000
# Mount OSTs, starting with OST0000
# Mount clients

Step 4: Recreate pool definitions and conf_param settings.

Use the saved output from the pre-writeconf backup to restore your settings.

replace_nids — Change Server Network Addresses

replace_nids updates the NID (network address) of a server in the configuration log without destroying other configuration. This is the preferred method for NID changes.

When to Use

A server is moving to a new machine or IP address.
A new network card is installed.
IP addresses are being reassigned.

Procedure

Step 1: Update LNet configuration on the affected server(s) so the new NID is correct. Verify with:

lctl list_nids

Step 2: Shut down the filesystem.

client# umount /mnt/lustre
mds# umount /mnt/mdt
oss# umount /mnt/ost*

Step 3: Start the MGS only (if MGS shares a device with MDT):

mount -t lustre /dev/mdt_device -o nosvc /mnt/mgt

Step 4: Run replace_nids.

lctl replace_nids <device_name> <new_nid>[,<nid2>,<nid3>...]

Example:

lctl replace_nids testfs-OST0013 192.168.1.100@tcp
lctl replace_nids testfs-MDT0000 10.0.0.5@tcp,10.0.0.5@o2ib

Step 5: Stop the MGS (if started with nosvc):

umount /mnt/mgt

Step 6: Restart the filesystem normally.

Advantages Over writeconf

Does not erase pool definitions.
Does not erase conf_param settings.
Backs up the previous config log with .bak suffix.
Also cleans old, invalidated records from the log while preserving current settings.

clear_conf — Clean SKIP Records

clear_conf removes records marked as SKIP from the configuration log. These accumulate over time as parameters are changed, and cleaning them reduces log size and startup time.

When to Use

Configuration logs have grown large due to many parameter changes.
You want to clean up obsolete SKIP entries without a full writeconf.

Procedure

Step 1: Shut down the filesystem.

Step 2: Start the MGS only (if shared with MDT):

mount -t lustre /dev/mdt_device -o nosvc /mnt/mgt

Step 3: Run clear_conf.

# Clear a specific device's config:
lctl clear_conf testfs-MDT0000

# Or clear all configs for a filesystem:
lctl clear_conf testfs

Step 4: Stop the MGS and restart normally.

The previous config log is backed up with a timestamp suffix (e.g., Lustre-MDT0000-1476454535.bak).

Comparison

	writeconf	replace_nids	clear_conf
Purpose	Full config rebuild	Change server NID	Remove SKIP records
Destroys pools?	Yes	No	No
Destroys conf_param?	Yes	No	No
Destroys set_param -P?	No	No	No
Requires full shutdown?	Yes	Yes	Yes
Backs up old config?	No	Yes (.bak)	Yes (timestamp.bak)
Risk level	High	Low	Low

Lustre Configuration Log Management

Contents