Development: Difference between revisions

From Lustre Wiki
Jump to navigation Jump to search
(→‎Updating Documentation: update hpdd.intel.com URLs to whamcloud.com)
(update for more recent usage)
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Lustre Development Activities ==
== Lustre Development Activities ==
The next release under development (accepting new features)
The next release under development (accepting new features)
* [[Release 2.14.0]]
* [[Release 2.16.0]]


If you are looking for small development projects to add minor improvements to Lustre and/or become more familiar with the code and development process, there is a keyword '''easy''' that is added to tickets that are suitable:
If you are looking for small development projects to add minor improvements to Lustre and/or become more familiar with the code and development process, there is a keyword '''easy''' that is added to tickets that are suitable:
Line 14: Line 14:
The [[Code Reviewers]] page describes how to find developers willing to perform code reviews on particular sections of the Lustre code.
The [[Code Reviewers]] page describes how to find developers willing to perform code reviews on particular sections of the Lustre code.


== Upstream contributing ==
== Upstream Linux Contributing ==


The [[Upstream contributing]] page covers how to contribute to the Linux kernel Lustre client.
The Lustre client code is being reworked for upstream kernel submission as part of the 'master' Lustre development branch.  Submitted patches should follow the normal Lustre patch submission process. Once all issues that are known to be blocking upstream submission have been addressed, then the Lustre client code will be resubmitted to the upstream kernel.


== Updating Documentation ==
== Updating Documentation ==
Line 25: Line 25:
== Developer Meetings ==
== Developer Meetings ==


* [[Developer Day 2023-05-01]]
* [[Lustre_User_Group_2022#Monday,_May_9_%E2%80%93_Tutorial_Day|Tutorial Day 2022-05-09]]
* [[Lustre_User_Group_2020#Wednesday,_August_5th_Virtual_Developer_Day|Developer Day 2020-08-05]]
* [[Developer Day 2019-05-14]]
* [[Developer Day 2019-05-14]]
* [[Developer Day 2018-04-23]]
* [[Developer Day 2018-04-23]]
Line 35: Line 38:
== Feature Development ==
== Feature Development ==


All known development efforts are tracked on the [[Projects]] page.  The [[:Category:Architecture|Architecture]] and [[:Category:Design|Design]] documents of various projects are available for review and future reference.
Major development efforts are tracked on the [[Projects]] page.  The [[:Category:Architecture|Architecture]] and [[:Category:Design|Design]] documents of various projects are available for review and future reference.


Please see [[Lustre Coding Style Guidelines]] and [[Lustre Style Guide Includes]] for information about writing Lustre code, and [[Submitting Changes]] for details on how to submit patches.  There is also [[Lustre Script Coding Style]] for writing test scripts.
Please see [[Lustre Coding Style Guidelines]] and [[Lustre Style Guide Includes]] for information about writing Lustre code, and [[Submitting Changes]] for details on how to submit patches.  There is also [[Lustre Script Coding Style]] for writing test scripts.
* [[Media:LUG2023-Lustre-Autotest-Maloo-Gerrit.pdf|Tips to effectively use Lustre Autotest, Maloo, Git, and Gerrit]]


Patches and landing status can be tracked via some Gerrit searches courtesy of John Hammond:
Patches and landing status can be tracked via some Gerrit searches courtesy of John Hammond:
Line 49: Line 54:
As Lustre runs in the Linux kernel it can be sensitive to changes between Linux kernel version. While Lustre is used in production on a wide variety of Linux distributions, testing efforts are concentrated on a limited number of primary Linux distributions. There are usually some secondary Linux distributions that either have been demonstrated to work to some reasonable degree or are work-in-progress towards becoming a primary distribution in the future. The primary Linux distributions are what is called out in the release page and the Lustre support matrix, but both the primary and secondary Linux distributions will be mentioned in the Lustre change log.
As Lustre runs in the Linux kernel it can be sensitive to changes between Linux kernel version. While Lustre is used in production on a wide variety of Linux distributions, testing efforts are concentrated on a limited number of primary Linux distributions. There are usually some secondary Linux distributions that either have been demonstrated to work to some reasonable degree or are work-in-progress towards becoming a primary distribution in the future. The primary Linux distributions are what is called out in the release page and the Lustre support matrix, but both the primary and secondary Linux distributions will be mentioned in the Lustre change log.


At the time of writing, the primary Linux distribution for Lustre servers is the latest Red Hat 7.7 kernel and the primary Linux distributions for Lustre clients are the latest RHEL 7.7, RHEL 8.0, Ubuntu 18.04 and SLES 12 SP4. Please consult lustre/Changelog in the Lustre tree for the latest details and known secondary Linux distributions.  
At the time of writing, the primary Linux distribution for Lustre servers is the latest Red Hat 9.3 kernel and the primary Linux distributions for Lustre clients are the latest RHEL 8.9, RHEL 9.3, Ubuntu 22.04 and SLES 15 SP5. Please consult lustre/Changelog in the Lustre tree for the latest details and known secondary Linux distributions.  


Please note that wherever Red Hat or RHEL is mentioned that it includes RHEL-like Linux distributions like CentOS and Scientific Linux.
Please note that wherever Red Hat or RHEL is mentioned that it includes RHEL-like Linux distributions like CentOS and Rocky Linux.


== Protocol Documentation ==
== Lustre Internals Documentation ==


There is some work-in-progress [https://build.hpdd.intel.com/job/lustre-protocol-reviews/lastSuccessfulBuild/artifact/protocol.html Lustre Protocol Documentation] available ([https://build.hpdd.intel.com/job/lustre-protocol-reviews/lastSuccessfulBuild/artifact/protocol.pdf PDF format]).  While the documentation is not complete, it may still prove to be a valuable resource for developers, and contributions and improvements to the [https://review.whamcloud.com/#/q/project:doc/protocol protocol documentation project] are welcome.
To gain a better understanding of the Lustre code and how it works, a good resource is
[https://info.ornl.gov/sites/publications/Files/Pub166872.pdf Understanding Lustre Internals].


[[Category: Development]]
[[Category: Development]]

Latest revision as of 16:18, 24 April 2024

Lustre Development Activities

The next release under development (accepting new features)

If you are looking for small development projects to add minor improvements to Lustre and/or become more familiar with the code and development process, there is a keyword easy that is added to tickets that are suitable: easier development items

Submitting Changes

The Submitting Changes page covers steps you'll need to follow to contribute code changes.

Code Reviews

The Code Reviewers page describes how to find developers willing to perform code reviews on particular sections of the Lustre code.

Upstream Linux Contributing

The Lustre client code is being reworked for upstream kernel submission as part of the 'master' Lustre development branch. Submitted patches should follow the normal Lustre patch submission process. Once all issues that are known to be blocking upstream submission have been addressed, then the Lustre client code will be resubmitted to the upstream kernel.

Updating Documentation

Updating the Lustre User Manual is a very useful way for non-coders to contribute to Lustre. See Lustre Manual Changes for more information on how to contribute changes, and the list of open documentation tickets.

Developer Meetings

Feature Development

Major development efforts are tracked on the Projects page. The Architecture and Design documents of various projects are available for review and future reference.

Please see Lustre Coding Style Guidelines and Lustre Style Guide Includes for information about writing Lustre code, and Submitting Changes for details on how to submit patches. There is also Lustre Script Coding Style for writing test scripts.

Patches and landing status can be tracked via some Gerrit searches courtesy of John Hammond:

Kernel Policy

As Lustre runs in the Linux kernel it can be sensitive to changes between Linux kernel version. While Lustre is used in production on a wide variety of Linux distributions, testing efforts are concentrated on a limited number of primary Linux distributions. There are usually some secondary Linux distributions that either have been demonstrated to work to some reasonable degree or are work-in-progress towards becoming a primary distribution in the future. The primary Linux distributions are what is called out in the release page and the Lustre support matrix, but both the primary and secondary Linux distributions will be mentioned in the Lustre change log.

At the time of writing, the primary Linux distribution for Lustre servers is the latest Red Hat 9.3 kernel and the primary Linux distributions for Lustre clients are the latest RHEL 8.9, RHEL 9.3, Ubuntu 22.04 and SLES 15 SP5. Please consult lustre/Changelog in the Lustre tree for the latest details and known secondary Linux distributions.

Please note that wherever Red Hat or RHEL is mentioned that it includes RHEL-like Linux distributions like CentOS and Rocky Linux.

Lustre Internals Documentation

To gain a better understanding of the Lustre code and how it works, a good resource is Understanding Lustre Internals.