OST Pool Quotas

From Lustre Wiki
Jump to navigation Jump to search

Description

OST Pool Quotas feature gives an ability to limit user's(group's/project's) disk usage at OST pool.

Each Quota Pool(QP) duplicates content of OST pool. Thus QP could be tuned with standard lctl pool_new/add/remove/erase commands. All QPs are subset of a global pool that includes all OSTs and MDTs(DOM case).

See more details in OST Pool Quotas HLD.

DOM and MDT pools

From Quota Master point of view "data" MDTs are regular members together with OSTs. But until we haven't a mechanism to group MDTs in pools, Quota Pools support only OSTs.

New options at lfs quota/setquota

2 new options were introduced to support QP at lfs quota and setquota:

  • --pool "pool_name" is new option to be used with lfs setquota to set quota for user/qroup/project per quota pool.
    Example: lfs setquota -u quota_usr --pool "qpool1" -B10M /mnt/lustre
  • --pool "pool_name" is an option to show quota for user/group/project per quota pool.
    Example: lfs quota -u quota_usr --pool "qpool1" /mnt/lustre

How To setup QP

Hard limit example

Let's imagine you haven't any OST pool(if you want to setup already existed pool, go to 3):

  1. lctl pool_new qpool1 /mnt/lustre
  2. lctl pool_add qpool1 OST000 /mnt/lustre
  3. lfs setquota -u quota_usr -B100T /mnt/lustre /* it is a limit for global pool. QPs don't work properly without that */
  4. lfs setquota -u quota_usr --pool "qpool1" -B100M /mnt/lustre

Note, you need to setup wide-system hard limit before setting QP limit. If you don't need to limit user at all OSTs and MDTs at system, only per pool, I would recommend to set some unrealistic big hard limit. Without that OSTs don't know that quota for quota_usr is enabled.

Soft limit example

  1. lctl pool_new qpool1 /mnt/lustre
  2. lctl pool_add qpool1 OST000 /mnt/lustre
  3. lfs setquota -u quota_usr -B100T /mnt/lustre /* notify OSTs that to enforce quota for quota_usr */
  4. lfs setquota -t -u --block-grace 30 --pool qpool1 /mnt/lustre /* set block grace 30 s for all users at qpool1 */
  5. lfs setquota -u quota_usr -b10M --pool qpool1 /mnt/lustre /* soft limit 10M for quota_usr */
  6. /runas -u quota_usr dd if=/dev/urandom of=/mnt/lustre/dir/f1 bs=1M count=15
  7. [root@dhcppc3 tests]# lfs quota -u quota_usr --pool qpool1 /mnt/lustre
Disk quotas for usr quota_usr (uid 60000):
    Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
   /mnt/lustre   14336*  10240 107374182400     26s       1       0       0       - 

Links

Docs

OST Pool Quotas HLD

OST Pool Quota Test Plan

OST Pool Quotas Test Report

Introducing Pool Quotas

Tickets

LU-11023

Patches

Name Landed
LU-11023 quota: remove quota pool ID yes
LU-11023 quota: quota pools for OSTs yes