Lustre Internals Documentation Update
- 1 Organization
- 2 Todo
- 3 Feedback
- 4 Resources
- 4.1 Lustre Protocol Documentation
- 4.2 Presentations
- 4.3 Documentation in Lustre Tree
- 4.4 Old Wiki Architectural Documents
- 4.5 Old Site Lustre Internals Documentation Area
- Incremental updating of Lustre internals documentation
- Near term focus is material in support of upcoming Introduction To Lustre Development webinar
- The available time for those with Lustre internals knowledge is limited, maximizing the efficiency of their efforts is a priority.
- Updates are moving to Category:Internals and pages with the Internals category in order to better facilitate linking and incremental updating.
- For the near future full documents (ULFI, LIAGI) are being moved over to a single wiki page. Adding subpage support to MediaWiki is a topic for the next LOWG call and may provide some flexibility there moving forward.
- Lustre internals is a regular topic on the Lustre Working Group calls, and the Lustre Internals Documentation Update project is discussed regularly on the Lustre.org Working Group calls. Please join us.
- LIAGI (Lustre Internals: A Gentle Introduction)
- ULFI (Understanding Lustre Filesystem Internals)
- Migrate ULFI from shared document to wiki to facilitate targeted linking and updating. (in-progress)
- Migrate LIAGI from web to wiki
- Update Subsystem Map for current subsystems
- Add Layered Object wiki page, reference source-tree clio.txt and LIAGI resources
- Create OSD internals page and reference source-tree osd-api.txt
- Work with developers to identify areas that need more work.
- Update wiki PtlRPC page and reference PtlRPC ULFI chapter as well as Lustre Protocol Document
- Generate Doxygen comments from current source tree and put on web
- Review Doxygen comments in code base and find methods to reference them from wiki
- Keep Lustre_Internals wiki page?
- Migrate Lustre Protocol Document to wiki?
- Automate generation/uploading of Doxygen comments to the web
- Investigate lustre.org mirror of Lustre git source tree and resources available to facilitate web presentation of the source (e.g. http://elixir.free-electrons.com/linux/latest/source )
Feedback from discussions with community members on updating Lustre internals documentation in general:
- Top-down versus bottom-up approach
- Differing viewpoints on this. Possible to find way to address both?
- Separate documents (e.g. Understanding Lustre Filesystem Internals document) versus documentation in the code (e.g. Doxygen)
- Possible to incrementally expand and update both, balance what's in them, and have them mutually reference?
Lustre Protocol Documentation
Sequoia and the ZFS OSD (LUG2013)
Intel® Lustre* File Level Replication (LUG2014)
Distributed Name Space Phase I (LUG2013)
Lustre & Kerberos: in theory and in practice (LUG2015)
Documentation in Lustre Tree
Doxygen Code Documentation
- Old: http://wiki.old.lustre.org/lid/doxygen.api/modules.html
- TODO: Generate new version from codebase and make available on web.
Overview of the Lustre Client I/O (CLIO) subsystem
- LFSCK master slave design
- Object traversal design reference
/* dot input file for lock-ordering diagram */
Overview of the Lustre Object Storage Device API
Overview of Dynamic LNet Configuration
Old Wiki Architectural Documents
Old Site Lustre Internals Documentation Area
Brief descriptions of Lustre concepts, objects and major components indexed in various ways.
Lustre Internals: A Gentle Introduction
TODO: Generate new version of