Upstream contributing

From Lustre Wiki
Revision as of 17:13, 24 March 2016 by Jsimmons (talk | contribs) (Restore some lost material)
Jump to navigation Jump to search

Getting started

The upstream Lustre client code is currently hosted in the staging area of the linux kernel tree. All current efforts for the upstream client are done with the git staging repo and that work then goes into Linus's tree during the merge window shortly after a Linux kernel release is cut. The git staging tree has several branches but the one of interest for contributing is the staging-testing branch. Any patches created against the other branches will likely not even apply correctly so please ensure you are working and testing with the staging-testing branch. Before we can start developing patches we need to obtain the source repo, build and install the Lustre enabled kernel with the proper utilities. The steps to getting started are listed below. In my example I named by git repo top directory lustre-upstream but you can name it whatever you want.

cd your-work-directory
git clone git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git lustre-upstream
cd lustre-upstream
git checkout -b staging-testing origin/staging-testing

Kernel configuration

With the lustre client source tree available the next step will be configuring your kernel to enable Lustre. The easiest way to configure a kernel is to run make menuconfig. If this fails it most likely is due to the missing libncurses library that is required. Running the make menuconfig will present you the top level menu. The section for Lustre is located in the Device Drivers section:

< > Volume Management Device Driver
[*] Networking support --->
Device Drivers --->
Firmware Drivers --->

Once in the Device Drivers menu you want to enable the Staging driver menu as shown here:

Virtio drivers --->
Microsoft Hyper-V guest support ----
[*] Staging drivers --->
[ ] X86 Platform Specific Device Drivers --->

Enable the staging driver allows you to display the options of that menu. Scroll down to the Staging drivers option and hit enter to go to the Staging menu. In the menu you will see:

< > GCT GDM724x LTE support>
< > TTY over Firewire
< > Lustre networking subsystem (LNet)
< > Digi Neo and Classic PCI Products

You want to select Lustre networking subsystem which will in turn enable several other options.

< > GCT GDM724x LTE support
< > TTY over Firewire
<M> Lustre networking subsystem (LNet)
 (1048576) Lustre lnet max transfer payload (default 1MB) <M> Lustre networking self testing
<M> LNET infiniband support
<M> Lustre file system client support
(8192) Lustre obd max ioctl buffer bytes (default 8KB)
[ ] Enable Lustre DEBUG checks
<M> Lustre virtual block device
< > Digi Neo and Classic PCI Products
< > Xilinx FPGA firmware download module

These default values will be enough to get you on your way to building a lustre client. Help sections are provided for all the entries to help you understand your options. Once you have made your selections then the normal build process of make bzImage and make modules on x86 machines will produce the lustre client modules. For other platforms or if you want to create debian or rpm packages please consult freely available documentation about kernel building on the web. Also feel free to ask for help on the lustre-devel mailing list. Once your kernel is ready just install it as you normally would a new kernel and reboot.

Building the Lustre utilities

To use the lustre upstream client properly you will need to install the lustre utilities rpm or debian packages. First you will need to download the latest Lustre OpenSFS/Intel repository. Like the upstream repo checkout you can name it something different than lustre-intel-branch in this example.

cd my-work-space
git clone git://git.hpdd.intel.com/fs/lustre-release lustre-intel-branch
cd lustre-intel-branch
sh ./autogen.sh
./configure --disable-server
make rpm - make debs

Once it completes you will have the debian based packages stored in the debs directory or the rpms in the top level source directory. In either case you only need the utilities package. In the rpm case it is the lustre-'version'*.rpm and the debian package of interest is lustre-utils_*.deb. Besides the utility package the other package that could be of interest is the lustre-tests-* package which contains an test suite used to validate Lustre to avoid potential regressions observed in the past. If you are interested in full fledge testing then I recommend reading the How to test Lustre Code section.

Once all your packages are installed you will need to setup a lustre file system. An excellent link to get the newcomer going is https://wiki.hpdd.intel.com/display/PUB/Create+and+Mount+a+Lustre+Filesystem. As always if you have questions feel free to ask on the lustre-devel mailing and people will gladly help you.

Developing for upstream client

This section goes over how to become actively involved in the upstream client work. The process of creating and sending patches is a very different from what is done for the OpenSFS/Intel branch. We will go over the setup of sending patches as well as meeting the coding standards for acceptance.  

git setup for patch submitting

Now that you are familiar with Lustre setup and would like to contribute the next step is too configure .gitconfig to send patches to Greg and the staging tree. I will show my .gitconfig I use for pushing patches upstream.

[sendemail]

to = Greg Kroah-Hartman <[email protected]>
to = [email protected]
to = Andreas Dilger <[email protected]>
to = Oleg Drokin <[email protected]>
cc = Linux Kernel Mailing List <[email protected]>
cc = Lustre Development List <[email protected]>

[diff]  :renames = true

Basically the .gitconfig is set to email patches to the correct people. I also add a [diff] field to handle renaming of files for when it does happen. Most users will most like never use that functionality. After you merge your patches the easiest way to prepare to send them by:

git format-patch -"# of patches" --cover-letter -o patch-directory/
Edit your cover letter git send-email patch-directory/


This is one approach but people that are much better at git know of ways to do the above more efficiently.

git setup for patch submitting

Now that you are familiar with Lustre setup and would like to contribute the next step is too configure .gitconfig to send patches to Greg and the staging tree. I will show my .gitconfig I use for pushing patches upstream. This is a one time process.

[sendemail]

to = Greg Kroah-Hartman <[email protected]>
to = [email protected]
to = Andreas Dilger <[email protected]>
to = Oleg Drokin <[email protected]>
cc = Linux Kernel Mailing List <[email protected]>
cc = Lustre Development List <[email protected]>

[diff]

renames = true

Basically the .gitconfig is set to email patches to the correct people. I also a [diff] field to handle renaming of files for when it does happen. Most users will most like never use that functionality. After you merge your patches the easiest way to prepare to send them by:

git format-patch -"# of patches" --cover-letter -o patch-directory/
Edit your cover letter
git send-email patch-directory/


This is one approach but people that are much better at git know of ways to do the above more efficiently.

Coding Style