Lustre Internals Documentation Update

Organization

Goals

• Incremental updating of Lustre internals documentation
• Near term focus is material in support of upcoming Introduction To Lustre Development webinar
• The available time for those with Lustre internals knowledge is limited, maximizing the efficiency of their efforts is a priority.

General

• Updates are moving to Category:Internals and pages with the Internals category in order to better facilitate linking and incremental updating.
• For the near future full documents (ULFI, LIAGI) are being moved over to a single wiki page. Adding subpage support to MediaWiki is a topic for the next LOWG call and may provide some flexibility there moving forward.
• Lustre internals is a regular topic on the Lustre Working Group calls, and the Lustre Internals Documentation Update project is discussed regularly on the Lustre.org Working Group calls. Please join us.

Acronyms

• LIAGI (Lustre Internals: A Gentle Introduction)
• ULFI (Understanding Lustre Filesystem Internals)

Todo

High

• Migrate ULFI from shared document to wiki to facilitate targeted linking and updating. (in-progress)
• Migrate LIAGI from web to wiki
• Update Subsystem Map for current subsystems
• Add Layered Object wiki page, reference source-tree clio.txt and LIAGI resources
• Create OSD internals page and reference source-tree osd-api.txt

Medium

• Work with developers to identify areas that need more work.
• Update wiki PtlRPC page and reference PtlRPC ULFI chapter as well as Lustre Protocol Document
• Generate Doxygen comments from current source tree and put on web
• Review Doxygen comments in code base and find methods to reference them from wiki
• Keep Lustre_Internals wiki page?
• Migrate Lustre Protocol Document to wiki?

Low

• Automate generation/uploading of Doxygen comments to the web
• Investigate lustre.org mirror of Lustre git source tree and resources available to facilitate web presentation of the source (e.g. http://elixir.free-electrons.com/linux/latest/source )

Feedback

Feedback from discussions with community members on updating Lustre internals documentation in general:

• Top-down versus bottom-up approach
  • Differing viewpoints on this. Possible to find way to address both?

• Separate documents (e.g. Understanding Lustre Filesystem Internals document) versus documentation in the code (e.g. Doxygen)
  • Possible to incrementally expand and update both, balance what's in them, and have them mutually reference?

Resources

Lustre Protocol Documentation

• http://wiki.opensfs.org/Contract_SFS-DEV-005
• https://jira.whamcloud.com/browse/LUDOC-280
• https://build.whamcloud.com/job/lustre-protocol-reviews/lastSuccessfulBuild/artifact/protocol.html

Presentations

Sequoia and the ZFS OSD (LUG2013)

• http://cdn.opensfs.org/wp-content/uploads/2013/04/Morrone_Sequoia_LUG2013.pdf

Intel® Lustre* File Level Replication (LUG2014)

• http://wiki.lustre.org/images/a/ad/LUG2014-File_Level_Replication-Xiong.pdf

Distributed Name Space Phase I (LUG2013)

• http://cdn.opensfs.org/wp-content/uploads/2013/04/LUG-2013_DNE.pdf

Lustre & Kerberos: in theory and in practice (LUG2015)

• http://wiki.lustre.org/images/e/ec/Lustre-and-Kerberos_Buisson.pdf

Documentation in Lustre Tree

Doxygen Code Documentation

• Old: http://wiki.old.lustre.org/lid/doxygen.api/modules.html
• TODO: Generate new version from codebase and make available on web.

Overview of the Lustre Client I/O (CLIO) subsystem

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/clio.txt

LFSCK

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/lfsck.txt

• LFSCK master slave design
• Object traversal design reference

Lock Ordering

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/lock-ordering

/* dot input file for lock-ordering diagram */

Overview of the Lustre Object Storage Device API

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/osd-api.txt

Overview of Dynamic LNet Configuration

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/dlc.txt

Lustre versioning

http://git.whamcloud.com/fs/lustre-release.git/blob_plain/HEAD:/Documentation/versioning.txt

Old Wiki Architectural Documents

Page	Comments
Architecture - Backup
Architecture - CROW
Architecture - CTDB with Lustre
Architecture - Caching OSS
Architecture - Changelogs
Architecture - Changelogs 1.6
Architecture - Client Cleanup
Architecture - Clustered Metadata
Architecture - Commit on Share
Architecture - Cuts
Architecture - DMU OSD
Architecture - DMU Zerocopy
Architecture - End-to-end Checksumming
Architecture - Epochs
Architecture - External File Locking
Architecture - FIDs on OST
Architecture - Feature FS Replication
Architecture - Fileset
Architecture - Flash Cache
Architecture - Free Space Management
Architecture - GNS
Architecture - HSM	No useful information
Architecture - HSM Migration
Architecture - HSM and Cache	No useful information
Architecture - IO system
Architecture - Interoperability 1.6 1.8 2.0
Architecture - Interoperability fids zfs
Architecture - LRE Images
Architecture - Libcfs
Architecture - Llog over OSD
Architecture - Lustre DLDs
Architecture - Lustre HLDs
Architecture - Lustre Logging API
Architecture - MDS-on-DMU
Architecture - MDS striping format
Architecture - MPI IO and NetCDF
Architecture - MPI LND
Architecture - Metadata API
Architecture - Migration (1)
Architecture - Migration (2)
Architecture - Multiple Interfaces For LNET
Architecture - Network Request Scheduler
Architecture - New Metadata API
Architecture - OSS-on-DMU
Architecture - Open by fid
Architecture - PAG
Architecture - Pools of targets
Architecture - Profiling Tools for IO
Architecture - Proxy Cache
Architecture - Punch and Extent Migration
Architecture - Punch and Extent Migration Requirements
Architecture - Recovery Failures
Architecture - Request Redirection
Architecture - Scalable Pinger
Architecture - Security
Architecture - Server Network Striping
Architecture - Simple Space Balance Migration
Architecture - Simplified Interoperation
Architecture - Space Manager
Architecture - Sub Tree Locks
Architecture - User Level Access
Architecture - User Level OSS
Architecture - Userspace Servers
Architecture - Version Based Recovery
Architecture - Wide Striping
Architecture - Wire Level Protocol
Architecture - Write Back Cache
Architecture - Writing Architecture Documents
Architecture - ZFS TinyZAP
Architecture - ZFS for Lustre
Architecture - ZFS large dnodes
Architecture Descriptions

Old Site Lustre Internals Documentation Area

http://wiki.old.lustre.org/lid/

Glossary

http://wiki.old.lustre.org/lid/glossary/glossary.html

Brief descriptions of Lustre concepts, objects and major components indexed in various ways.

Lustre Internals: A Gentle Introduction

http://wiki.old.lustre.org/lid/agi/agi.html

Subsystem Map

TODO: Generate new version of

• http://wiki.old.lustre.org/lid/subsystem-map-old/subsystem-map.html