Lustre Quota Troubleshooting

Introduction

This document is based on and tested with Lustre 2.15 and LDISKFS. If your Lustre version is older or you are using ZFS, the content may not be applicable.

Space accounting and project ID tracking are enabled by default on a LDISKFS file system. However, quota restrictions are NOT enabled by default. User, Group and Project quota restrictions must be manually enabled on all targets, as well as configuring data/block and/or metadata/inode limits.

Adding to This Guide

If you have improvements, corrections, or more information to share on this topic please contribute to this page.

Status Verification

To verify the status of space accounting and quota, run the following on every Lustre node that serves an MDT or OST and has the target mounted

lctl get_param osd-*.fsname-*.quota_slave_*.info | sed "/^osd/c\----------------------------------------" 

1. A MDT will display two different output stanza per target: dt for data/block and md for metadata/inode.

[root@testhost-mds01 ~]# lctl get_param osd-*.testfs-*.quota_slave_*.info | sed "/^osd/c\----------------------------------------"|less
----------------------------------------
target name:    testfs-MDT0000
pool ID:        0
type:           dt
quota enabled:  none
conn to master: setup
space acct:     ugp
user uptodate:  glb[0],slv[0],reint[0]
group uptodate: glb[0],slv[0],reint[0]
project uptodate: glb[0],slv[0],reint[0]
----------------------------------------
target name:    testfs-MDT0000
pool ID:        0
type:           md
quota enabled:  u
conn to master: setup
space acct:     ugp
user uptodate:  glb[1],slv[1],reint[0]
group uptodate: glb[0],slv[0],reint[0]
project uptodate: glb[0],slv[0],reint[0]

1. 1. An OST will display only one output stanza per target: dt for data/block

Closely inspect the values for quota enabled: and space acct:. The values can be none or any combination of the letters u, g and p, representing user, group and project. The default value for quota enabled: is none and the default value for space acct: is ugp

Quota Modification

User, group and project quota can be enabled or disabled on a per target basis. Typically, it is desirable to have all target's quota configuration set the same.

References and Links

• Roland Laifer, "Lustre tools for ldiskfs investigation and lightweight I/O statistics", LAD2015. http://www.eofs.eu/fileadmin/lad2015/slides/13_Roland_Laifer_kit_20150922.pdf