Data on MDT
Lustre file system performance is currently optimized for large files. This results in additional RPC round-trips to the OSTs, which hurt small file performance. This project aims to correct this deficiency by allowing the data for small files to be placed on the MDT so that these additional RPCs can be eliminated and performance correspondingly improved. Used in conjunction with DNE, this will preserve efficiency without sacrificing horizontal scale.
Users or system administrators will set a layout policy that locates small files to be stored on an MDT. Files that grow beyond this size will use Progressive File Layouts to extend larger files onto OST objects and leave the start of the file on the MDT.
- Implement a new layout for files with data on MDT.
- Client IO stack must allow IO requests to an MDT device.
- Design of a tunable for specifying size limit for files on a MDT.
- Integration of DoM with PFL to automatically extend files from the MDT to OSTs.
- Explicitly allocating small files on the MDT by default directory striping.
- A discussion of interoperability (or lack thereof) of old clients and servers with Data on MDT.
Out of Scope
- Automatically locating small files on the MDT after the file has started to be written.