Lustre Quota Administration
Overview
Lustre quotas restrict how much disk space (blocks) and how many files (inodes) a user, group, or project can consume. The quota subsystem consists of:
- QMT (Quota Master Target) — runs on the MDT, holds the global quota limits.
- QSD (Quota Slave Device) — runs on each OST and MDT, enforces limits locally.
- qunit — the allocation granularity the QMT grants to each QSD. The QMT dynamically adjusts qunit size based on remaining quota and the number of slaves.
Space accounting is enabled by default on ldiskfs. Quota enforcement must be explicitly enabled.
Enabling Quota Enforcement
Enable enforcement for user, group, and project quotas on all targets:
# Enable user, group, and project quota enforcement lctl set_param -P osd-*.fsname*.quota_slave_dt.enabled=ugp lctl set_param -P osd-*.fsname*.quota_slave_md.enabled=ugp
To verify:
lctl get_param osd-*.*.quota_slave_dt.enabled lctl get_param osd-*.*.quota_slave_md.enabled
Valid values: u (user), g (group), p (project), or any combination (e.g. ugp). Use none to disable.
Setting User and Group Quotas
Set block and inode limits for a user:
# -b soft block limit, -B hard block limit (in KiB by default) # -i soft inode limit, -I hard inode limit lfs setquota -u bob -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs
Set limits for a group:
lfs setquota -g engineers -b 10G -B 12G -i 500000 -I 600000 /mnt/testfs
Suffixes K, M, G, T, P are accepted for block limits.
Viewing Quotas
# Show quota for a specific user lfs quota -u bob /mnt/testfs # Show quota for a specific group lfs quota -g engineers /mnt/testfs # Show quota usage per OST for a user lfs quota -u bob -v /mnt/testfs # Show quotas for all users lfs quota -u -a /mnt/testfs
Default Quotas
Default quotas apply to all users (or groups) that do not have explicit limits set:
# Set default user quota lfs setquota -U -b 1G -B 2G -i 100000 -I 200000 /mnt/testfs # Set default group quota lfs setquota -G -b 5G -B 10G -i 500000 -I 1000000 /mnt/testfs # View default quotas lfs quota -U /mnt/testfs lfs quota -G /mnt/testfs
Default quotas use -U, -G, or -P (uppercase) instead of -u, -g, -p.
Grace Periods
When a user exceeds the soft limit but remains below the hard limit, a grace period timer starts. If usage is not reduced below the soft limit before the grace period expires, the soft limit is enforced as a hard limit.
The default grace period is 1 week.
Set grace periods:
# Set block and inode grace period for users (format: XXwXXdXXhXXmXXs) lfs setquota -u -t -b 86400 -i 604800 /mnt/testfs # Set group grace period to 3 days for blocks, 1 week for inodes lfs setquota -g -t -b 3d -i 1w /mnt/testfs
Clearing Quotas
To remove quota limits for a user, set all limits to zero:
lfs setquota -u bob -b 0 -B 0 -i 0 -I 0 /mnt/testfs
Architecture Notes
The QMT on the MDT is the single authority for quota limits. When a QSD on an OST needs to allocate space for a user, it requests a qunit allocation from the QMT. The qunit size is dynamic — it shrinks as quota usage approaches the limit (to reduce over-allocation) and grows when usage is low. This means:
- Quota enforcement is approximate until the user is near the limit. A user may slightly exceed a limit before enforcement kicks in.
- After changing a quota limit, it may take a short time for all QSDs to receive the updated allocation.