File:LUG2019-Lustre View from Outside-Brown.pdf

A little under two years ago I joined the Lustre development community with one very specific goal: to get the lustre code fully integrated with the Linux kernel. Combining my experience of nearly two decades of active kernel development with a strong desire to see the integration effort succeed has given me a unique perspective into the lustre code base, a perspective that I would like to share.

Lustre contains much that is good, but could be better. The strengths come from a focused development community, a demanding user community, and effective feedback between these group allowing (and requiring) problems to be identified and resolved. The weaknesses come in part from a history of trying to support multiple operating systems and in part from being isolated from the much large communities which can both provide valuable expertise, and can change the base OS kernel to better support the needs of Lustre. Lustre can clearly benefit from improved integration with both the Linux code and the Linux community, and Linux itself is likely to benefit too.

Through an exploration of various subsystems include wait queues, hash tables, linked lists, locking, build infrastructure, fault injection, trace logging and more, an external perspective on the Lustre code base will be presented, and a case will be made that if full integration can be achieved then there will be clear benefits for Lustre in maintainability, in performance, in memory usage, and in test coverage, and benefits for Linux through improved cross-community collaboration and through enhancements that Lustre demands and others could benefit from.