Lustre Upstreaming to Linux Kernel - Revision history

Adilger: link more major orgs using Lustre

2026-07-23T22:25:16Z

link more major orgs using Lustre

← Older revision		Revision as of 15:25, 23 July 2026
Line 70:		Line 70:
	\| Responsiveness		\| Responsiveness
	\| "The maintainer is expected to respond to bug reports, address review feedback, and adapt the filesystem to VFS infrastructure changes such as folio conversions, iomap migration, and mount API updates."		\| "The maintainer is expected to respond to bug reports, address review feedback, and adapt the filesystem to VFS infrastructure changes such as folio conversions, iomap migration, and mount API updates."
	\| There is a longstanding [https://jira.whamcloud.com/projects/LU/issues Lustre issue tracking system] that is open to the public (need free account to post issues) as well as ~~lustre-discuss~~ mailing ~~list~~ for users, lustre-devel list for developers, and [https://join.slack.com/t/lustreupstreamclient/shared_invite/zt-23pl0xxsi-1PwfkpjP_vphW8nqoUnUBA Slack channel].		\| There is a longstanding [https://jira.whamcloud.com/projects/LU/issues Lustre issue tracking system] that is open to the public (need free account to post issues) as well as [[mailing lists\|Mailing_Lists_and_IRC]] for users, lustre-devel list for developers, and [https://join.slack.com/t/lustreupstreamclient/shared_invite/zt-23pl0xxsi-1PwfkpjP_vphW8nqoUnUBA Slack channel].
	\|-		\|-
	\| User base		\| User base
	\| "Clearly describe who the users of this filesystem are and the scale of the user base."		\| "Clearly describe who the users of this filesystem are and the scale of the user base."
	\| Lustre users are widespread across universities and research organizations around the world (e.g. Stanford, UFlorida, ORNL, LLNL, LANL, Cambridge, HLRS, CINECA, SFU, NASA), weather forecasting (NOAA, ~~NREL~~, DKRZ), ~~financial modelling~~ (~~HRT~~), LLM (NVIDIA, Softbank), and many other organizations that need high performance and high capacity storage. About 70% of the Top-500 supercomputers ~~use Lustre~~ for the past 15 ~~or more~~ years, which implies at least many thousands of systems ~~using~~ it. Lustre is freely available ~~for all users~~, so there is no way to tell the total number of users. There are also multiple vendors selling Lustre-based storage hardware and support offerings: [https://github.com/Cray/lustre HPE], [https://aws.amazon.com/fsx/lustre/ AWS], [https://www.ddn.com/products/lustre-file-system-exascaler/ DDN], [https://cloud.google.com/products/managed-lustre Google], [https://learn.microsoft.com/en-us/azure/azure-managed-lustre/amlfs-overview Azure], [https://www.oracle.com/cloud/storage/file-storage-with-lustre/ Oracle], [https://www.aeoncomputing.com/ Aeon], [https://thelustrecollective.com/ TLC]		\| Lustre users are widespread across universities and research organizations around the world (e.g. Stanford, UFlorida, ORNL, LLNL, LANL, NCSA, Cambridge, HLRS, CINECA, SFU, NASA), weather forecasting (NOAA, NCAR, DKRZ, ECMWF, BoM, UKMET), energy (CEA, BP, Total), medicine/genomics (Sanger, BGI, Cornell), LLM (NVIDIA, xAI, Anthropic, Softbank), and many other organizations that need high performance and high capacity storage. About 70% of the Top-500 supercomputers for the past 15+ years use Lustre, which implies at least many thousands of large systems use it, and at least hundreds of thousands of direct users. Lustre is freely available through multiple channels, so there is no way to tell the total number of users. There are also multiple vendors selling Lustre-based storage hardware and support offerings directly or indirectly: [https://github.com/Cray/lustre HPE], [https://aws.amazon.com/fsx/lustre/ AWS], [https://www.ddn.com/products/lustre-file-system-exascaler/ DDN], [https://cloud.google.com/products/managed-lustre Google], [https://learn.microsoft.com/en-us/azure/azure-managed-lustre/amlfs-overview Azure], [https://www.oracle.com/cloud/storage/file-storage-with-lustre/ Oracle], [https://www.aeoncomputing.com/ Aeon], [https://thelustrecollective.com/ TLC]
	\|-		\|-
	\| Building your track record		\| Building your track record

Adilger: add link to Development page, fill in and expand several sections

2026-07-23T22:01:15Z

add link to Development page, fill in and expand several sections

Show changes

Tim-day-387 at 20:53, 23 July 2026

2026-07-23T20:53:58Z

← Older revision		Revision as of 13:53, 23 July 2026
Line 1:		Line 1:
	The Lustre upstreaming effort is currently being managed as part of the <code>master</code> Lustre development branch, and submitted patches should follow the normal Lustre patch submission process, which includes fixing issues reported by checkpatch.pl, so that the code follows the upstream kernel style guidelines. Once all issues in the Lustre code that are known to be blocking upstream submission have been addressed, then the filesystem will be resubmitted to the upstream kernel.		The Lustre upstreaming effort is currently being managed as part of the <code>master</code> Lustre development branch, and submitted patches should follow the normal Lustre patch submission process, which includes fixing issues reported by checkpatch.pl, so that the code follows the upstream kernel style guidelines. Once all issues in the Lustre code that are known to be blocking upstream submission have been addressed, then the filesystem will be resubmitted to the upstream kernel.


			== Upstream Filesystem Requirements ==

			The Linux kernel has [https://github.com/torvalds/linux/blob/master/Documentation/filesystems/adding-new-filesystems.rst documented requirements] for accepting new filesystems. The following section discussed those requirements.

			=== Do You Need a New In-Kernel Filesystem? ===

			<blockquote>"Before proposing a new in-kernel filesystem, consider whether one of the alternatives might be more appropriate."</blockquote>

			{\| class="wikitable"
			! Question !! Answer
			\|-
			\| Does an existing in-kernel filesystem cover the same use case?
			\| Lustre targets high performance scale-out network file storage use-cases. The closest analogs in the kernel are pNFS and Ceph. Lustre has a more mature implementation and is generally higher performance than either pNFS or Ceph.<ref>https://io500.org/</ref>
			\|-
			\| Can we use FUSE?
			\| One of Lustre's highest priorities is performance. FUSE presents an unacceptable performance tradeoff.
			\|}

			=== Technical Requirements ===

			{\| class="wikitable"
			! Requirement !! Upstream guidance !! Lustre status
			\|-
			\| rowspan="3" \| Use modern VFS interfaces
			\| "Do not use interfaces listed in Documentation/process/deprecated.rst."
			\|
			\|-
			\| "Use folios rather than raw page operations for page cache management."
			\| Lustre is in the process of converting to folios.<ref>[https://jira.whamcloud.com/browse/LU-17916 LU-17916]</ref>
			\|-
			\| "Network filesystems should consider using the netfs library Documentation/filesystems/netfs_library.rst, or be prepared to explain why it is not a good fit."
			\| Lustre stripes files over several storage servers. It needs to maintain cache coherence whenever any server revokes a lock for a file range. This requires special callbacks that netfs doesn't currently support.
			\|-
			\| rowspan="2" \| Provide userspace utilities
			\| "A mkfs tool is expected ..."
			\| Lustre maintains a mkfs.lustre tool.<ref>[https://github.com/lustre/lustre-release/blob/master/lustre/utils/mkfs_lustre.c mkfs_lustre.c]</ref>
			\|-
			\| "A fsck tool is strongly recommended ..."
			\| Lustre has an online repair tool, [https://doc.lustre.org/lustre_manual.xhtml#lfsckadmin lfsck].
			\|-
			\| Be testable
			\| "The filesystem must be testable in a meaningful way." "The fstests framework (also known as xfstests) is the standard testing infrastructure for Linux filesystems and its use is highly recommended." "At a minimum, there must be a credible and documented way to test the filesystem and detect regressions. When submitting, include a summary of test results indicating which tests pass, fail, or are not applicable."
			\| Lustre has a large body of existing tests.<ref>[https://github.com/lustre/lustre-release/blob/master/lustre/tests/ lustre/tests/]</ref> Lustre is testable via fstests. We intend to submit fstests support after Lustre is accepted upstream.
			\|-
			\| Provide documentation
			\| "A documentation file under Documentation/filesystems/ describing the filesystem, its on-disk format, mount options, and any notable design decisions is recommended."
			\| Lustre has a lot of existing documentation<ref>[https://github.com/lustre/lustre-release/blob/master/Documentation/ lustre-release Documentation]</ref> that we will submit alongside the filesystem as appropriate. Lustre also has extensive online documentation.<ref>[https://doc.lustre.org/lustre_manual.xhtml Lustre Operations Manual]</ref><ref>[https://wiki.lustre.org/Main_Page Lustre Wiki]</ref>
			\|}

			=== Community and Maintainership Expectations ===

			{\| class="wikitable"
			! Requirement !! Upstream guidance !! Lustre status
			\|-
			\| Identified maintainers
			\|
			\| In no particular order - James Simmons, Timothy Day, Andreas Dilger, Oleg Drokin
			\|-
			\| Demonstrated commitment
			\| "A track record of maintaining kernel code -- for example, in other subsystems -- significantly strengthens the case for a new filesystem."
			\| Andreas Dilger has a history of involvement in ext4.
			\|-
			\| Sustained backing
			\| "Major filesystems in Linux have organizational or corporate support behind their development."
			\| A non-exhaustive list of domains of contributing organizations in the past 3 years: whamcloud.com, ddn.com, hpe.com, aeoncomputing.com, amazon.com, thelustrecollective.com, infradead.org, cea.fr, nvidia.com, oracle.com, suse.de, lanl.gov, google.com, llnl.gov, linaro.org, openchip.com, microsoft.com, tintri.com, gluesys.com, stanford.edu, cray.com, ornl.gov, jlab.org, intel.com, iastate.edu, eviden.com, codewreck.org, canonical.com, atos.net
			\|-
			\| Responsiveness
			\| "The maintainer is expected to respond to bug reports, address review feedback, and adapt the filesystem to VFS infrastructure changes such as folio conversions, iomap migration, and mount API updates."
			\|
			\|-
			\| User base
			\| "Clearly describe who the users of this filesystem are and the scale of the user base."
			\| A few examples of vendors selling Lustre offerings: [https://github.com/Cray/lustre HPE], [https://aws.amazon.com/fsx/lustre/ AWS], [https://www.ddn.com/products/lustre-file-system-exascaler/ DDN], [https://cloud.google.com/products/managed-lustre Google], [https://learn.microsoft.com/en-us/azure/azure-managed-lustre/amlfs-overview Azure], [https://www.oracle.com/cloud/storage/file-storage-with-lustre/ Oracle], [https://www.aeoncomputing.com/ Aeon], [https://thelustrecollective.com/ TLC]
			\|-
			\| Building your track record
			\| "A practical way to demonstrate many of the qualities above is to maintain the filesystem out-of-tree for a period before requesting a merge."
			\| Lustre has been in active development out-of-tree for over 25 years.
			\|}

			=== References ===

			<references />


	== Pending Work ==		== Pending Work ==

Tim-day-387 at 20:51, 6 July 2026

2026-07-06T20:51:25Z

← Older revision		Revision as of 13:51, 6 July 2026
Line 188:		Line 188:

	==== Lustre User Group (LUG) ====		==== Lustre User Group (LUG) ====

			[https://www.opensfs.org/wp-content/uploads/03-LUG_2026_Upstream_Efforts_Progress.pdf LUG 2026: Upstream Efforts Progress - James Simmons]

	[https://www.youtube.com/watch?v=nOPpWygOreI LUG 2025: Lustre Upstreaming Efforts - James Simmons]		[https://www.youtube.com/watch?v=nOPpWygOreI LUG 2025: Lustre Upstreaming Efforts - James Simmons]

Tim-day-387 at 16:35, 24 February 2026

2026-02-24T16:35:41Z

← Older revision		Revision as of 09:35, 24 February 2026
Line 209:		Line 209:
	This is a collection of mailing list threads of interest - any useful discussion related to upstreaming Lustre.		This is a collection of mailing list threads of interest - any useful discussion related to upstreaming Lustre.

	[https://lore.kernel.org/linux-fsdevel/DA28F0FE-ACB6-486E-BF3D-85AF328FE2AD@amazon.com/T/#t LSF/MM 2025]		[https://lore.kernel.org/linux-fsdevel/B588ECCE-7CBF-4111-841F-0510A8F5886B@amazon.com/T/#u Lustre Upstreaming Project Update (2026)]

			[https://lore.kernel.org/linux-fsdevel/DA28F0FE-ACB6-486E-BF3D-85AF328FE2AD@amazon.com/T/#t LSF/MM (2025)]

	[http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2025-January/thread.html Discussing Lustre Upstreaming with Lustre Developers (2025)]		[http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2025-January/thread.html Discussing Lustre Upstreaming with Lustre Developers (2025)]

	[https://lwn.net/ml/linux-fsdevel/20180601091133.GA27521@kroah.com/ Lustre removed from staging]		[https://lwn.net/ml/linux-fsdevel/20180601091133.GA27521@kroah.com/ Lustre removed from staging]

Tim-day-387 at 06:05, 20 February 2026

2026-02-20T06:05:42Z

← Older revision		Revision as of 23:05, 19 February 2026
Line 167:		Line 167:
	<br>		<br>


			== Testing ==

			Currently, [https://tim-day-387.github.io/upstream-patch-review/# ktest] is used to validate that Lustre continues to build and run against the latest mainline kernels. In the future, it will validate various kernel configurations and architecture builds. It will also perform various static checks and validations.

			Lustre also has extensive [https://build.whamcloud.com/ build] and [https://testing.whamcloud.com/test_sessions testing] against LTS kernels.

			<br>

Tim-day-387: /* [II] Get lustre/ and lnet/ to natively compile on a mainline kernel */

2026-02-20T06:00:34Z

[II] Get lustre/ and lnet/ to natively compile on a mainline kernel

← Older revision		Revision as of 23:00, 19 February 2026
Line 125:		Line 125:
	==== [II] Get lustre/ and lnet/ to natively compile on a mainline kernel ====		==== [II] Get lustre/ and lnet/ to natively compile on a mainline kernel ====

	'''''ETA: ~~November 2025~~'''''		'''''ETA: April 2026'''''

	Once the compatibility code is isolated, we must get <code>lustre/</code> and <code>lnet/</code> to natively compile on a mainline kernel - without any Autotools magic, #ifdefs, or <code>lustre_compat/</code> layer.		Once the compatibility code is isolated, we must get <code>lustre/</code> and <code>lnet/</code> to natively compile on a mainline kernel - without any Autotools magic, #ifdefs, or <code>lustre_compat/</code> layer.

Tim-day-387: /* Pending Work */

2026-02-20T05:52:55Z

Pending Work

← Older revision		Revision as of 22:52, 19 February 2026
Line 24:		Line 24:

	\|-		\|-
	\| Convert to kernel-doc comments \|\| ~~Incomplete~~ \|\| Work is currently underway. \|\| [https://jira.whamcloud.com/browse/LU-9633 LU-9633] \|\| Various		\| Convert to kernel-doc comments \|\| Partially Complete \|\| Work is currently underway. \|\| [https://jira.whamcloud.com/browse/LU-9633 LU-9633] \|\| Various
	\|-		\|-

Line 64:		Line 64:

	\|-		\|-
	\| Don't use Autotools \|\| Mostly Complete \|\| Lustre kernel modules should not be built with Autotools \|\| [https://jira.whamcloud.com/browse/LU-18876 LU-18876] \|\| Various		\| Don't use Autotools for kernel modules \|\| Mostly Complete \|\| Lustre kernel modules should not be built with Autotools \|\| [https://jira.whamcloud.com/browse/LU-18876 LU-18876] \|\| Various
	\|-		\|-

	\|-		\|-
	\| Don't use inline #ifdef for kernel compatibility \|\| Incomplete \|\| The core Lustre kernel docs should be free from extraneous #ifdefs \|\| - \|\| Various		\| Don't use inline #ifdef for kernel compatibility \|\| Incomplete \|\| The core Lustre kernel docs should be free from extraneous #ifdefs \|\| - \|\| Various
			\|-

			\|-
			\| Ensure that Lustre builds on all support architectures \|\| Incomplete \|\| - \|\| - \|\| Various
			\|-

			\|-
			\| Ensure Lustre can build !MODULE \|\| Mostly Complete \|\| Fix bugs when building Lustre into the kernel image \|\| [https://jira.whamcloud.com/browse/LU-19874 LU-19874] \|\| Various
	\|-		\|-

Tim-day-387: /* Mailing List Archives */

2026-02-20T04:54:05Z

Mailing List Archives

← Older revision		Revision as of 21:54, 19 February 2026
Line 194:		Line 194:

	[https://lore.kernel.org/linux-fsdevel/DA28F0FE-ACB6-486E-BF3D-85AF328FE2AD@amazon.com/T/#t LSF/MM 2025]		[https://lore.kernel.org/linux-fsdevel/DA28F0FE-ACB6-486E-BF3D-85AF328FE2AD@amazon.com/T/#t LSF/MM 2025]

			[http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2025-January/thread.html Discussing Lustre Upstreaming with Lustre Developers (2025)]

	[https://lwn.net/ml/linux-fsdevel/20180601091133.GA27521@kroah.com/ Lustre removed from staging]		[https://lwn.net/ml/linux-fsdevel/20180601091133.GA27521@kroah.com/ Lustre removed from staging]

Tim-day-387 at 04:50, 20 February 2026

2026-02-20T04:50:20Z

← Older revision		Revision as of 21:50, 19 February 2026
Line 74:		Line 74:

	== Proposed Transition to an Upstream-first Development Model ==		== Proposed Transition to an Upstream-first Development Model ==

			[[Image:Split_lustre_tree.png]]

	==== Background ====		==== Background ====