Lustre Quota Troubleshooting

Adding to This Guide

If you have improvements, corrections, or more information to share on this topic please contribute to this page.

Introduction

This document is based on and tested with Lustre 2.15 and LDISKFS. If your Lustre version is older or you are using ZFS, the content may not be applicable.

Space accounting and project ID tracking are enabled by default on a LDISKFS file system. However, quota restrictions for user, group and project are NOT enabled by default. User, Group and Project quota restrictions must be manually enabled on all targets, as well as configuring data/block and/or metadata/inode limits.

Status Verification

SPACE ACCOUNTING

To verify the status of space accounting, run the following on every Lustre node that serves an MDT or OST and has the target mounted.

lctl get_param osd-*.*.quota_slave_*.info

If the value for space acct: is anything other than ugp, likely the quota and/or project file system feature is not enabled. Inspect the superblock information to see if the quota and project features are enabled.

tune2fs -l /dev/sdb | grep features

If quota and project do not appear in the list of Filesystem features:, they will need to be added for space accounting to function

QUOTA

To verify the quota configuration, run the following on every Lustre node that serves an MDT or OST and has the target mounted.

lctl get_param osd-*.*.quota_slave_*.enabled 

A MDT will display two different output stanza per target: dt for data/block and md for metadata/inode.

osd-ldiskfs.testfs-MDT0000.quota_slave_dt.enabled=none
osd-ldiskfs.testfs-MDT0000.quota_slave_md.enabled=none
osd-ldiskfs.testfs-MDT0001.quota_slave_dt.enabled=none
osd-ldiskfs.testfs-MDT0001.quota_slave_md.enabled=none

An OST will display only one output stanza per target: dt for data/block.

osd-ldiskfs.testfs-OST0000.quota_slave_dt.enabled=none
osd-ldiskfs.testfs-OST0001.quota_slave_dt.enabled=none

The values for enabled= can be none or any combination of the letters u (for user), g (for group) and p (for project). The default value is none indicating NO QUOTA ENFORCEMENT IS ENABLED

Multiple examples in the Lustre Operations Manual outline how to configure quota.

Enable missing LDISKFS Features

To enable the quota and project LDISKFS file system features, stop Lustre, unmount all targets and run the following:

tune2fs -O quota, project /dev/sdb

Your path to the appropriate block device (/dev/sdb) may be different. Exercise caution when using tune2fs

Multiple File Systems

The MGS can store configuration data about multiple file system. IF a "wild card" lctl command is used to set quota parameters (no file system name specified), it will apply to any/all file systems that are not specifically configured by a higher index set_param entry. See the example below:

lctl set_param -P osd-*.*.quota_slave_dt.enabled=none
lctl set_param -P osd-*.testfs*.quota_slave_dt.enabled=ugp

lctl --device MGS llog_print params
- { index: 2, event: set_param, device: general, parameter: osd-*.*.quota_slave_dt.enabled, value: none }
- { index: 5, event: set_param, device: testfs, parameter: osd-*.testfs*.quota_slave_dt.enabled, value: ugp }

A file system named "testlab" will match index 2. The enabled value will be none

A file system named "testfs" will match index 2, and subsequently match index 5. The enabled value will be ugp