Lustre Upstreaming to Linux Kernel

The Lustre upstreaming effort is currently being managed as part of the master Lustre development branch, and submitted patches should follow the normal Lustre patch submission process, which includes fixing issues reported by checkpatch.pl, so that the code follows the upstream kernel style guidelines. Once all issues in the Lustre code that are known to be blocking upstream submission have been addressed, then the filesystem will be resubmitted to the upstream kernel.

Pending Work

Significant work has gone into improving the Lustre codebase since the Lustre client was removed from staging. Below are listed some of the major areas of ongoing work. This list is not comprehensive. The first ticket LU-12511 links to a lot of work that has been completed historically. This table will be updated periodically as new requirements emerge and as work becomes completed.

Task	Status	Notes	Jira	Patch(es)
Prepare Lustre for Adoption in the Linux kernel	Incomplete	This is the main Jira ticket - which links to other subtasks	LU-12511	Various
IPv6 Support	Mostly Complete	Initial support in 2.16, finalized support in 2.17	LU-18417	Various
Enforce Linux kernel coding style	Mostly Complete	The majority of checkpatch.pl warnings have been addressed	LU-6142	Various
Convert to kernel-doc comments	Incomplete	work underway	LU-9633	Various
Unify Lustre kernel modules	Incomplete	Reduce the total number of Lustre kernel modules	LU-17862	55153
Split userspace, compat, and kernel code	Incomplete	Split the core filesystem from the compatibility code needed to support older kernels	LU-18687	Various
Replace libcfs hashing code with rhashtable	Incomplete	Most of the users on the client have been removed; the majority of the work is server-side	LU-8130	Various
Adapt Lustre Infiniband driver to latest APIs	Deferred	The initial submission of Lustre to Linux will be TCP/IP only	LU-8874	Various
Remove ProcFS users	Incomplete	Most of the users on the client have been removed; the majority of the work is server-side	LU-8066	Various
Remove custom interval-tree	Complete	-	LU-11085	Various
Transition completely to folios	Incomplete	A lot of callbacks have been implemented; still have dependency on mm/folio-compat.c; large folio support is in-flight	LU-17916	Various
->writepage could likely be deprecated	Incomplete	Mostly a factor of ensure Lustre still behaves - context	LU-18675	57894

Proposed Transition to an Upstream-first Development Model

Background

Currently, the Lustre kernel modules are maintained as an out-of-tree driver. This driver supports a variety of vendor kernels (primarily on RHEL and Ubuntu). Code is developed first for these vendors kernels. It is later updated to support newer kernel as they are release. The core filesystem code and the compatibility code are intermixed.

For upstreaming to work, the normal development flow has to generate patches to mainline Linux. However, we must still support the older distribution kernels that most people use to run Lustre. To achieve this - we must invert the development model. Code must be developed first for the upstream kernel. Afterwards, this code will be supported on older kernels via a compatibility layer maintained by the Lustre development community. These next sections provide a high-level overview and defines a clear roadmap with tangible milestones on our path to upstream. This transition will occur in stages and will proceed in parallel with normal development - without interrupting normal feature work.

The timelines provided are approximate.

Tenets

There are several important considerations we must account for when undertaking this work (in order of importance):

• Mainline Linux is the source of truth - Every Lustre release must be derived from a mainline Linux release. Code change to Lustre should target mainline Linux first - and later be backported as needed. We should not block upstream improvements to facilitate backporting.

• Do not complicate the process of contributing to Lustre - Lustre has an active developer community and a large amount of pre-existing test infrastructure. Our existing processes have worked well. We should maintain the beneficial aspects of our existing workflows while transitioning to an upstream-first model.

• Integrate with the wider filesystem community - We should strive to share Lustre's improvements and advancements with the wider filesystem community. We should regularly collaborate with others in the community to facilitate kernel-wide improvements.

[I] Separate the kernel code, compatibility code, and userspace code

ETA: June 2025

The Lustre tree should be reorganized to have a clear separation of concerns:

fs/lustre/
net/lnet/
net/lnet/libcfs/
lustre_compat/
tests/
utils/

The functional components of libcfs/ would stay in that directory and the compatibility components would live in lustre_compat/. Centralizing the compatibility code makes it easier to maintain and update and allows us to start removing the compatibility code from the modules themselves. lustre_compat/ will be compiled into libcfs.ko. The functional components of libcfs/ will be compiled into lnet.ko.

The compatibility code will mirror the upstream kernel-tree. After an upstream kernel is released, we will update the compatibility layer as-needed to support that release on older kernels.

[II] Get fs/ and net/ to compile on a mainline kernel

ETA: November 2025

Once the compatibility code is isolated, we must get fs/ and net/ to compile on a mainline kernel - without any configuration or lustre_compat/ layer.

This would enforced by adding build validation to each patch submitted to Gerrit. The kernel version would be pinned in the contrib/lbuild/ scripts and we'd periodically update it and fix any new build failures.

Once this is achieved, we'll have a native mainline Linux client/server that can be run on older distributions via a compatibility layer.

[III] Move fs/ and net/ to a separate kernel tree

ETA: November 2026 (This would immediately precede submission for inclusion)

Transition to maintaining fs/ and net/ as a series on patches on top of a mainline kernel release. At this point, we'll generating patches to mainline Linux while retaining the ability to support older distro kernels via lustre_compat/. Similar to the previous step, we periodically rebase our Lustre patch series - fixing lustre_compat/ as needed.

This is the only step that requires a significant change the Lustre development workflow - patches would have to be split and sent to two different repositories. This step will be delayed until we have some confidence that Lustre has a path to be accepted to mainline.

[IV] Submit the patch series for inclusion

ETA: November 2026

Once we are comfortable with the above process, we can submit the initial patches to add Lustre support to the kernel. Our normal development flow will generate a batch of patches to be submitted during each merge window. After the merge window, we can focus on testing and making sure that our backport to older distro kernels is still functional.

FAQ

Who will actually run the Lustre code in mainline Linux?

Everyone. Releases for older distros will be a combination of the upstream Lustre combined with lustre_compat/ and whatever stuff the kernel won't allow.

What will a Lustre release look like?

We can generate a tarball by combining an upstream Lustre release from mainline along with lustre_compat/ and the userspace stuff. Vendors and third-parties can base their versions of Lustre on those tarballs. Every time a new kernel releases - a new Lustre release tarball will be created. LTS releases can center around the LTS kernel releases.

How will we validate that fs/ and net/ build on mainline?

It would probably be easiest to create a minimalist mainline kernel build in Jenkins. This would allow us to reuse most of the existing lbuild scripting. The build would be non-enforced at first. Testing would remain on distribution kernels, since most people use those.

Does anyone else have a similar model? Does this even work?

AMD GPU seems to have a similar approach, at least. We should talk to other developers working in a model similar to this.

Past Talks and Coverage

This section covers previous talks and coverage about the Lustre upstreaming process.

Lustre Administrators and Developers (LAD)

LAD 2022: Native Linux client Status - James Simmons

LAD 2020: Linux Lustre Client State - James Simmons

Lustre User Group (LUG)

LUG 2024: Native Linux client Status - James Simmons

LUG 2019: Lustre - A view from the outside - Neil Brown

Linux Weekly News (LWN)

Will staging lose its Lustre?

Mailing List Archives

This is a collection of mailing list threads of interest - any useful discussion related to upstreaming Lustre.

LSF/MM 2025

Lustre removed from staging