Lustre OST Pool Management

From Lustre Wiki
Jump to navigation Jump to search

Overview

OST pools group a subset of OSTs under a named label, allowing administrators to control file placement. Pools are used at file creation time — once a file is created, changing pool membership does not move existing data.

Rules:

  • An OST can belong to multiple pools.
  • No ordering of OSTs within a pool is defined.
  • Stripe allocation within a pool follows normal stripe allocation rules.
  • Pool membership can change at any time; existing files are unaffected.

Creating and Managing Pools

Create a Pool

lctl pool_new fsname.poolname

Example:

lctl pool_new testfs.flash_pool

Add OSTs to a Pool

# Add individual OSTs
lctl pool_add testfs.flash_pool OST[0-3]

# Add a range of OSTs
lctl pool_add testfs.flash_pool OST[0000-0007]

# Add a single OST
lctl pool_add testfs.flash_pool OST0010

Remove OSTs from a Pool

lctl pool_remove testfs.flash_pool OST0003
lctl pool_del testfs.flash_pool OST[0004-0007]

Note: pool_remove and pool_del are equivalent for removing OSTs from a pool.

Destroy a Pool

# Remove all OSTs first, then destroy
lctl pool_destroy testfs.flash_pool

The pool must be empty (all OSTs removed) before it can be destroyed.

Listing Pools and Pool Membership

# List all pools in a filesystem
lctl pool_list testfs
lfs pool_list testfs

# List OSTs in a specific pool
lctl pool_list testfs.flash_pool
lfs pool_list testfs.flash_pool

# Also visible via proc
lctl get_param lov.testfs-*.pools.flash_pool

Using Pools for File Striping

Set a Directory to Use a Pool

# New files in /mnt/testfs/fast will be striped across flash_pool OSTs
lfs setstripe -p flash_pool /mnt/testfs/fast

# Combine pool with stripe count
lfs setstripe -p flash_pool -c 4 /mnt/testfs/fast

Create a Single File in a Pool

lfs setstripe -p flash_pool /mnt/testfs/myfile

Check Pool Assignment

lfs getstripe -p /mnt/testfs/fast
lfs getstripe /mnt/testfs/myfile

Pool Quotas

Since Lustre 2.14, quotas can be set per OST pool, limiting how much space a user/group/project can consume on a specific pool.

Important: A global quota limit must be set before pool quota limits will work.

# Step 1: Set a global limit (required)
lfs setquota -u alice -B 100T /mnt/testfs

# Step 2: Set a pool-specific limit
lfs setquota -u alice --pool flash_pool -B 1T /mnt/testfs

# View pool quota
lfs quota -u alice --pool flash_pool /mnt/testfs

Pool quotas also work with group (-g) and project (-p) quotas.

See OST Pool Quotas for full details.

Use Cases

Tiered Storage

Group fast NVMe OSTs into a flash_pool and slower HDD OSTs into an archive_pool. Direct hot data to the flash pool and cold data to the archive pool:

lctl pool_new testfs.flash_pool
lctl pool_add testfs.flash_pool OST[0000-0003]  # NVMe OSTs

lctl pool_new testfs.archive_pool
lctl pool_add testfs.archive_pool OST[0004-0015]  # HDD OSTs

lfs setstripe -p flash_pool /mnt/testfs/scratch
lfs setstripe -p archive_pool /mnt/testfs/archive

Rack-Aware Placement

Group OSTs by physical rack to keep a job's I/O local:

lctl pool_new testfs.rack1
lctl pool_add testfs.rack1 OST[0000-0007]

lctl pool_new testfs.rack2
lctl pool_add testfs.rack2 OST[0008-0015]

FLR Fault Domains

With File Level Redundancy (FLR/mirroring), place each mirror on a different pool to ensure hardware fault isolation:

lfs mirror create -N -p pool_a -N -p pool_b /mnt/testfs/critical_file

See Also