Lustre OST Pool Management
Overview
OST pools group a subset of OSTs under a named label, allowing administrators to control file placement. Pools are used at file creation time — once a file is created, changing pool membership does not move existing data.
Rules:
- An OST can belong to multiple pools.
- No ordering of OSTs within a pool is defined.
- Stripe allocation within a pool follows normal stripe allocation rules.
- Pool membership can change at any time; existing files are unaffected.
Creating and Managing Pools
Create a Pool
lctl pool_new fsname.poolname
Example:
lctl pool_new testfs.flash_pool
Add OSTs to a Pool
# Add individual OSTs lctl pool_add testfs.flash_pool OST[0-3] # Add a range of OSTs lctl pool_add testfs.flash_pool OST[0000-0007] # Add a single OST lctl pool_add testfs.flash_pool OST0010
Remove OSTs from a Pool
lctl pool_remove testfs.flash_pool OST0003 lctl pool_del testfs.flash_pool OST[0004-0007]
Note: pool_remove and pool_del are equivalent for removing OSTs from a pool.
Destroy a Pool
# Remove all OSTs first, then destroy lctl pool_destroy testfs.flash_pool
The pool must be empty (all OSTs removed) before it can be destroyed.
Listing Pools and Pool Membership
# List all pools in a filesystem lctl pool_list testfs lfs pool_list testfs # List OSTs in a specific pool lctl pool_list testfs.flash_pool lfs pool_list testfs.flash_pool # Also visible via proc lctl get_param lov.testfs-*.pools.flash_pool
Using Pools for File Striping
Set a Directory to Use a Pool
# New files in /mnt/testfs/fast will be striped across flash_pool OSTs lfs setstripe -p flash_pool /mnt/testfs/fast # Combine pool with stripe count lfs setstripe -p flash_pool -c 4 /mnt/testfs/fast
Create a Single File in a Pool
lfs setstripe -p flash_pool /mnt/testfs/myfile
Check Pool Assignment
lfs getstripe -p /mnt/testfs/fast lfs getstripe /mnt/testfs/myfile
Pool Quotas
Since Lustre 2.14, quotas can be set per OST pool, limiting how much space a user/group/project can consume on a specific pool.
Important: A global quota limit must be set before pool quota limits will work.
# Step 1: Set a global limit (required) lfs setquota -u alice -B 100T /mnt/testfs # Step 2: Set a pool-specific limit lfs setquota -u alice --pool flash_pool -B 1T /mnt/testfs # View pool quota lfs quota -u alice --pool flash_pool /mnt/testfs
Pool quotas also work with group (-g) and project (-p) quotas.
See OST Pool Quotas for full details.
Use Cases
Tiered Storage
Group fast NVMe OSTs into a flash_pool and slower HDD OSTs into an archive_pool. Direct hot data to the flash pool and cold data to the archive pool:
lctl pool_new testfs.flash_pool lctl pool_add testfs.flash_pool OST[0000-0003] # NVMe OSTs lctl pool_new testfs.archive_pool lctl pool_add testfs.archive_pool OST[0004-0015] # HDD OSTs lfs setstripe -p flash_pool /mnt/testfs/scratch lfs setstripe -p archive_pool /mnt/testfs/archive
Rack-Aware Placement
Group OSTs by physical rack to keep a job's I/O local:
lctl pool_new testfs.rack1 lctl pool_add testfs.rack1 OST[0000-0007] lctl pool_new testfs.rack2 lctl pool_add testfs.rack2 OST[0008-0015]
FLR Fault Domains
With File Level Redundancy (FLR/mirroring), place each mirror on a different pool to ensure hardware fault isolation:
lfs mirror create -N -p pool_a -N -p pool_b /mnt/testfs/critical_file