Lustre 101

These slides were originally hosted by ORNL in 2016 as part of a self-paced web-based course series designed to provide an introduction to Lustre. The course was targeted at experienced system administrators who were relatively new to Lustre. Although the content may be old, some of information/concepts presented in this course are still relevant to newer versions of Lustre.

• Introduction to Lustre - This presentation provides a general overview of the Lustre file system for anyone wanting to learn more about basic Lustre functionality, features, and architecture. The basic components of Lustre are discussed, including the LNet transport layer. Information about Lustre file striping is also included.
• Hardware Selection and Benchmarking for Lustre - This presentation covers topics relevant to the process of hardware selection for a Lustre file system. Recommendations for servers, clients, and networking are provided. In addition, general concepts for benchmarking a Lustre file system are discussed, and a list of some useful tools is provided.
• Basic Lustre Installation and Setup from Stock RPMs - This presentation illustrates how to setup a simple Lustre file system using the stock Lustre RPM packages. The process for installing the RPM packages will be covered, along with a description of the configuration files that are needed by Lustre. Options for formatting and mounting the Lustre storage are also covered.
• Creating a Lustre Test System from Source with Virtual Machines - This presentation describes how to build a small Lustre file system using virtual machines that can be used as a test platform for anyone wanting to experiment with Lustre. Rather than use the stock Lustre RPM packages, the process of building Lustre from source code will be demonstrated.
• Lustre Tuning and Advanced LNet Configuration
• File System Administration and Monitoring
• Analysis of Crash Dumps and Log Files
• Evaluating the Functionality, Performance, and Reliability of Lustre
• LNet Bonding
• Lustre and Memory
• Lustre Over Long-Haul Connections Using LNet Routers
• Working with Problematic Nodes
• Recovery and Eviction

The following tutorials were presented at the International Workshop on the Lustre Ecosystem held in 2015 and 2016:

2015

• Network Contention and Congestion Control: Lustre Fine-Grained Routing
• Improve Large-Scale Storage System Performance via Topology-aware and Balanced Data Placement
• LNET Configuration
• Robust Health Monitoring
• Failure Handling
• An Introduction to fileUtils

2016

• A 12 Step Program for Lustre Filesystem Addiction
• Lustre Job Stats Metric Aggregation at OLCF
• Managing Lustre on a Budget
• Securing Lustre with Nodemap and Shared Key
• How to Install, Tune, and Monitor a ZFS Based Lustre File System