Lustre Upstreaming to Linux Kernel

The Lustre upstreaming effort is currently being managed as part of the 'master' Lustre development branch, and submitted patches should follow the normal Lustre patch submission process. Once all issues in the Lustre code that are known to be blocking upstream submission have been addressed, then the filesystem will be resubmitted to the upstream kernel.

Proposed Transition to an Upstream-first Development Model

For upstreaming to work, the normal development flow has to generate patches to mainline Linux - while still supporting the older distribution kernels that most people use to run Lustre. This transition can occur in stages. This section provides a high-level overview. This won't be without challenges - but the majority of the transition could happen without interrupting feature work or normal development.

[I] Separate the kernel code, compatibility code, and userspace code

The Lustre tree should be reorganized to have a clear separation of concerns:

fs/lustre/
net/lnet/
net/libcfs/
lustre_compat/
tests/
utils/

The functional components of libcfs/ would stay in that directory and the compatibility components would live in lustre_compat/. Centralizing the compatibility code makes it easier to maintain and update and allows us to start removing the compatibility code from the modules themselves. lustre_compat/ could still be compiled into libcfs.ko, if we want to avoid creating even more modules.

[II] Get fs/ and net/ to compile on a mainline kernel

Once the compatibility code is isolated, we must get fs/ and net/ to compile on a mainline kernel - without any configuration or lustre_compat/ layer.

This would validated by adding build validation to each patch submitted to Gerrit. The kernel version would be pinned in the contrib/lbuild/ scripts and we'd periodically update it and fix any new build failures.

Once this is achieved, we'll have a native Linux client/server that can be run on older distributions via a compatibility layer.

[III] Move fs/ and net/ to a separate kernel tree

Transition to maintaining fs/ and net/ as a series on patches on top of a mainline kernel release. At this point, we'll generating patches to mainline Linux while retaining the ability to support older distro kernels via lustre_compat/. Similar to the previous step, we periodically rebase our Lustre patch series - fixing lustre_compat/ as needed.

This is the only step that requires a significant change the Lustre development workflow - patches would have to be split and sent to two different repos. We can delay this step until we have some confidence that Lustre has a path to be accepted to mainline.

[IV] Submit the patch series for inclusion

Once we are comfortable with the above process, we can submit the initial patches to add Lustre support to the kernel. Our normal development flow will generate a batch of patches to be submitted during each merge window. After the merge window, we can focus on testing and making sure that our backport to older distro kernels it still working.

FAQ

Who will actually run the Lustre code in mainline Linux?

Everyone. Releases for older distros will be a combination of the upstream Lustre combined with lustre_compat/ and whatever stuff the kernel won't allow.

What does a Lustre release look like?

We can generate a tarball by combining an upstream Lustre release from mainline along with lustre_compat/ and the userspace stuff. Vendors and third-parties can base their versions of Lustre on those tarballs. Every time a new kernel releases - a new Lustre release tarball will be created. LTS releases can center around the LTS kernel releases.

How will we validate that fs/ and net/ build on mainline?

It would probably be easiest to create a minimalist mainline kernel build in Jenkins. This would allow us to reuse most of the existing lbuild scripting. The build would be non-enforced at first. Testing would remain on distro kernels, since most people use those.

Does anyone else have a similar model? Does this even work?

AMD GPU seems to have a similar approach, at least. We should talk to other developers working in a model similar to this.

This is still a high level sketch, but I think this is a feasible path to upstreaming Lustre. We need to define a clear roadmap with tangible milestones to have a hope of upstreaming working.

But it's important that we don't disrupt developers established workflows. We don't want to complicate contributing to Lustre and we don't want to discourage people from contributing their changes upstream.

Pending Work

Significant work has gone into improving the Lustre codebase since the Lustre client was removed from staging. Below are listed some of the major areas of ongoing work. This list is not comprehensive. The first ticket `LU-12511` links to a lot of work that has been completed historically. This table will be updated periodically as new requirements emerge and as work becomes completed.

Task	Status	Notes	Jira	Patch(es)
Prepare Lustre for Adoption in the Linux kernel	Incomplete	This is the main Jira ticket - which links to other subtasks	LU-12511	Various
IPv6 Support	Mostly Complete	Initial support in 2.16, finalized support in 2.17	LU-18417	Various
Enforce Linux kernel coding/doc style	Mostly complete	Doxygen to kernel-doc conversion is the largest outstanding task	LU-6142	Various
Unify Lustre kernel modules	Incomplete	Reduce the total number of Lustre kernel modules	LU-17862	55153
Remove libcfs hashtable	Incomplete	Most of the users on the client have been removed; the majority of the work is server-side	LU-8130	Various
Adapt Lustre Infiniband driver to latest APIs	Deferred	The initial submission of Lustre to Linux will be TCP/IP only	LU-8874	Various
Remove ProcFS users	Incomplete	Most of the users on the client have been removed; the majority of the work is server-side	LU-8066	Various
Remove custom interval-tree	Complete	-	LU-11085	Various

Past Talks and Coverage

This section covers previous talks and coverage about the Lustre upstreaming process.

Lustre Administrators and Developers (LAD)

LAD 2022: Native Linux client Status - James Simmons

LAD 2020: Linux Lustre Client State - James Simmons

Lustre User Group (LUG)

LUG 2024: Native Linux client Status - James Simmons

LUG 2019: Lustre - A view from the outside - Neil Brown

Linux Weekly News (LWN)

Will staging lose its Lustre?