Lustre Quota Troubleshooting
Introduction
This document is based on and tested with Lustre 2.15 and LDISKFS. If your Lustre version is older or you are using ZFS, the content may not be applicable.
Space accounting and project ID tracking are enabled by default on a LDISKFS file system. However, quota restrictions are NOT enabled by default. User, Group and Project quota restrictions must be manually enabled on all targets, as well as configuring data/block and/or metadata/inode limits.
Adding to This Guide
If you have improvements, corrections, or more information to share on this topic please contribute to this page.
Status Verification
To verify the status of space accounting and quota, run the following on every Lustre node that serves an MDT or OST and has the target mounted
lctl get_param osd-*.fsname-*.quota_slave_*.info | sed "/^osd/c\----------------------------------------"
Closely inspect the values for quota enabled: and space acct:. The values can be none or any combination of the letters u, g and p, representing user, group and project. The default value for quota enabled: is none and the default value for space acct: is ugp
Repairing wrong Quotas
Lustre quotas are recomputed if you turn them off and on. However, if UIDs and GIDs on the ldiskfs level are wrong (see above) this does not repair wrong quotas. Since manual repair is hardly possible, the only viable solution is to wait until Lustre version 2.6 or higher can be installed and use the online file system check (option layout of LFSCK) to repair wrong quotas. Note that we have not yet tried this solution.
References and Links
- Roland Laifer, "Lustre tools for ldiskfs investigation and lightweight I/O statistics", LAD2015. http://www.eofs.eu/fileadmin/lad2015/slides/13_Roland_Laifer_kit_20150922.pdf