Editing the LID

1. Required Software

To access the LID sources and format the document you need to install these utilities on your computer:

git: The git version control system is required to access the document sources. It is recommended that you install a recent git release.

For non-Windows systems, obtain the git source from here, build and install.

For Windows systems, it is recommended that you obtain the msysgit binary from here. The msysgit package includes a shell (bash) and other familiar utility programs (including perl) and it provides an adequate environment for executing the script that builds the LID from the source files.

If you are new to git, you can find good general documentation and tutorial material here.
python: Python is required to run the asciidoc program. You can find source and binary distributions here. Currently, python version 3+ doesn’t appear to be compatible with asciidoc so install the most recent python in the 2.6 series. You may already have python installed on your system. If so, check that it is at least version 2.4 (asciidoc requirement).
asciidoc: The document is formatted using asciidoc. You can obtain asciidoc from here. It is recommended you use the latest version (currently 8.5.0). Versions older than 8.4.5 can be used but expect some formatting problems. The main asciidoc website is here — it contains much useful information and describes how to install asciidoc on various systems.
graphviz: This graph visualization package is required to produce the graph diagrams. You can obtain it from here. Most non-Windows systems will have this available as a package.

1.1. Adding the utilities to your executable path

On non-Windows systems, once the packages have been built and installed. The required executables should already be accessible through your normal path.

For Windows systems, you will need to specify where the git, python and asciidoc executables reside (not required for the graphviz executables). If you followed the advice above and installed the msysgit package, then this is easy. When you install msysgit, it installs a bash shell which you can use to run the scripts required to build the LID. You must add the directories containing the python executable and the asciidoc utility to the bash shell’s path. Do that by creating a .profile file in the bash shell’s home directory that contains (at least) a line that looks like this:

PATH=$PATH:.:/c/Python26:/c/asciidoc

This example assumes that python was installed into C:\\Python26 and asciidoc was installed in C:\\asciidoc. Adjust as required to suit your installation.

The .profile file must reside in the bash shell home directory which is the directory that the bash shell starts up in when you click on the Git Bash desktop icon. This will be C:\\Documents and Settings\USER, where USER is your username. An easy way to create that file would be to type the following into the git shell:

echo PATH=\$PATH:.:/c/Python26:/c/asciidoc > .profile

Note the \ before the $PATH.

Once you have done this, close the bash shell and start a new shell. Invoking asciidoc.py should produce this output:

Man page:     asciidoc --help manpage
Syntax:       asciidoc --help syntax

And you can read the asciidoc manpage with:

asciidoc.py --help manpage | less

2. Obtaining the LID source

The source for the LID can be obtained by cloning the upstream repository, like this:

git clone git://git.lustre.org/lid.git

This will create a local directory, lid, that contains the source tree and the scripts. You should see diagnostic output a bit like this (obviously, the numbers will be different):

Initialized empty Git repository in /home/markb/lid/.git/
remote: Counting objects: 112, done.
remote: Compressing objects: 100% (110/110), done.
remote: Total 112 (delta 34), reused 0 (delta 0)
Receiving objects: 100% (112/112), 168.32 KiB | 36 KiB/s, done.
Resolving deltas: 100% (34/34), done.

3. Keeping the LID source up to date

Once the repository has been created, you can incorporate the latest updates by executing (within the lid directory):

git pull

This fetches the latest upstream changes and merges them into your master branch.

Using fetch/merge instead of pull

Git pull is equivalent to:

git fetch                  # fetches upstream changes into branch origin/master
git merge origin/master    # merge those changes into current branch

Some people prefer to separate the two operations because that gives them the opportunity to inspect the incoming changes (using git diff) before they are merged into the master branch. Like this:

git fetch                  # fetches upstream changes into branch origin/master
git diff ..origin/master   # show differences from current branch
git merge origin/master    # merge those changes into current branch

The ..origin/master syntax is described below.

4. Formatting the LID

In the lid directory you will find a subdirectory src that contains the document source files. You will also find a Perl script, build.pl, and an asciidoc configuration file (asciidoc.conf).

To process the source files to produce the HTML output, do (in lid):

build.pl

This script does the minimum work required to bring the output files up to date with respect to the sources.

The output goes into a directory called html which will be created as required. The top-level file is html/index.html. To view the document you can simply open html/index.html with your web browser. If you want to install the document on a web server, just copy everything under the html directory.

To remove the generated files completely, do:

build.pl clean

You should only need to do that before building if you have changed either asciidoc.conf or config.pl.

For convenience, a trivial Makefile is supplied. So on systems that have make installed, you can build the document or remove the generated files by saying ‘make’ and ‘make clean’, respectively.

5. Editing the LID and submitting patches upstream

As mentioned above, you can edit the LID source files with any text editor.

5.1. Telling git who you are

Before you commit any changes to your local repository you should tell git what your user name and email address are so that any commits you make will be identified. You do this with the git config utility like this:

git config --global user.name "Mickey Mouse"
git config --global user.email mickey@disney.com

The --global option tells git that these configuration values should be used for all repositories you work with rather than just the lid repository.

5.2. Creating new branches

It is a good idea to make your edits on a new branch rather than editing the master branch directly. Branching (and merging) using git is straightforward. To create a new branch called branchname based on the current branch, do:

git branch branchname

Having created your new branch, you can check it out with:

git checkout branchname

To list the branches in the repository, do:

git branch

The current branch will be indicated with an asterisk.

You can switch to another branch at anytime with git checkout branchname.

If when you switch to another branch you have uncommitted changes and those changes are in files that are the same in both branches, the checkout will go ahead as expected. If the changes are in files that differ between the branches, the checkout will be refused so as to preserve your changes in context. You can force the checkout by specifying the -m option to merge your changes into the file(s) in the checked out branch.

5.3. Commiting changes

Now you can edit the sources as you wish and commit the changes to the current branch. Commiting changes in git is a 2-part process that involves adding the changes into a staging area (often referred to as the ‘index’ in git documentation) and then committing the contents of the index.

This 2-part commit process may appear complex but it is one of git’s strongpoints and you will soon come to appreciate how useful it is.

So, to add all of the changes in one or more files into the index, do:

git add filenames

You can then commit those changes with:

git commit -m "commit message"

Don’t forget to put quotes around the message if it contains spaces.

You can also tell git to automatically add all changed files to the index and commit them with a message by including the -a option:

git commit -a -m "commit message"

So git commit -a works very much like commiting changes with legacy SCM systems.

Remember, committing only commits to your local repository and does not effect the upstream repository.

There is also a GUI (git gui) that you can use to select the changes and make the commit. The great thing about git gui is that although you can use it to add all the changes in a given file into the index (just like using git add), you can also use it to add selected changes into the index. This is very useful because quite often you make some number of changes to a file’s contents that, ideally, would be committed either separately or in groups. With legacy SCM systems you nearly always end up committing all the changes together because it’s too much like hard work to separate them. With git gui, you can easily select which changes you want to put into a given commit and so the commits tend to be more specific.

This following image shows a typical git gui screen. The box in the top-left lists the files that contain changes that have not yet been added to the index (in this example, src/gitk.png is a new file not yet under version control). The box in the bottom-left lists the files that contain changes that have been added to the index. Selecting a line in either of those two boxes will display the changes for that file in the box on the top-right.

To commit changes using git gui, select the changes you wish to commit from the list on the top-left by clicking on the page icon to the left of the file name. This will move the file name down to the bottom box and the changes will be put into in the index. If you change your mind and decide that something that is in the index should not be committed yet, just click on the page icon on the appropriate line in the lower box and it will be moved out of the index (and reappear in the top box).

When the index contains the changes you want to commit, enter the commit message in the bottom-right box and press the Commit button.

Good quality commit messages are very desirable and the first line in the message should always summarise the changes made by the commit. If further information is to go into the message, place a blank line between the first line and the following text.

5.4. Specifying revisions

Many git commands require a revision or range of revisions to be specified and git provides a rich syntax for specifying revisions. Full details can be found in the git manual pages and tutorials. The most commonly used ways of specifying a single revision are:

A branch’s current revision is specified as HEAD. This can often be ommitted as it is the default revision for many commands.
A branch or tag is specified simply by using its name.
Any commit can be specified using its SHA1 identifier (a unique substring is sufficient).

To specify a range of revisions from r1 to r2, use: r1..r2

For example, to specify the range of revisions from the head of the current branch to, say, origin/master, use HEAD..origin/master or, more simply, ..origin/master.

5.5. Viewing history

To see a list of commits on a given branch do:

git log branchname

If you leave out the branch name you see the commits on the current branch. By default, this just shows you the first line of the commit message and a few other details. It doesn’t show the changes (see below).

There is a gui program (gitk) that allows you to browse the commit history and see the full commit messages and the changes made by the commit. Here is a typical gitk window:

The top portion of the window lists the commits on the current branch. Selecting one of the commits will display in lower portion of the window the changes made by that commit.

5.6. Creating diffs

To see the changes between your current working tree and the current branch, do:

git diff

To compare your current working tree (or current branch) against another branch (e.g. master), do:

git diff branchname

By default, git diff produces patch output so a very simple way to generate a patch file that you could then email to someone else would be:

git diff master > mychanges.patch

That works but it will squash all of the changes into a single patch and not include useful information such as the commit messages. A much better way to send your changes to someone else is to use git format-patch.

5.7. Emailing patches

To prepare a set of patches (one for each commit) that can then be emailed, use git format-patch. Here’s an example that formats all the revisions between the master branch and HEAD (as individual files):

git format-patch -C -M -s -o outdir master

Like most git commands, git format-patch can take a wide range of options but we only need to use a few of them. In our example, the -M and -C options specify that file renames and copies are to be detected and the -s option ‘signs off’ the patches so people know who produced them. The -o outdir specifies the name of the directory that the patches are output to (default the current directory).

By default, each patch goes into a separate file whose name is based on the commit message for the patch prefixed with a sequence number. Eg:

0001-Changed-meta-elements.patch
0002-Tweaks.patch
0003-Put-links-into-table.patch
0004-Build-formatting-tweaks.patch
0005-New-file.patch

Specifying the --stdout option, will send all the patches to standard output so to put them in a single file do this:

git format-patch -C -M -s --stdout master > outfile

The patch files are in UNIX mbox format and can be sent using regular email programs. Alternatively, on non-Windows systems, you can use git send-email to send out the patches like this:

git send-email --to recipient dir-or-files

Where recipient is the email address of the recipient (e.g. lustre-lid-ext@sun.com) and dir-or-files is either the name of a directory that contains the files to be sent (and nothing else) or a list of files to send.

If someone sends you patches in this format, save the message(s) into a mbox file and apply them to the current branch using git am, like this:

git am mbox_file