»LID Home
»LID History
»Editing the LID

This page describes how to:

  • obtain the LID source from the upstream git repository

  • format the source to produce the browsable document

  • create patches which can then be emailed to the upstream repository gatekeepers (lustre-lid-ext@sun.com)

The majority of the LID is written as asciidoc text files. Asciidoc is a simple markup language that is easy to use. Special editing software is not required — you can edit the source files with your favorite text editor.

A Perl script is provided to automate the process of building the LID from the source files. The build process formats the asciidoc sources to produce the HTML output, copies any files that are used verbatim (images, pre-written HTML, etc.) and (re)builds the various indices.

1. Required Software

To access the LID sources and format the document you need to install these utilities on your computer:

git

The git version control system is required to access the document sources. It is recommended that you install a recent git release.

For non-Windows systems, obtain the git source from here, build and install.

For Windows systems, it is recommended that you obtain the msysgit binary from here. The msysgit package includes a shell (bash) and other familiar utility programs (including perl) and it provides an adequate environment for executing the script that builds the LID from the source files.

If you are new to git, you can find good general documentation and tutorial material here.

python

Python is required to run the asciidoc program. You can find source and binary distributions here. Currently, python version 3+ doesn’t appear to be compatible with asciidoc so install the most recent python in the 2.6 series. You may already have python installed on your system. If so, check that it is at least version 2.4 (asciidoc requirement).

asciidoc

The document is formatted using asciidoc. You can obtain asciidoc from here. It is recommended you use the latest version (currently 8.5.0). Versions older than 8.4.5 can be used but expect some formatting problems. The main asciidoc website is here — it contains much useful information and describes how to install asciidoc on various systems.

graphviz

This graph visualization package is required to produce the graph diagrams. You can obtain it from here. Most non-Windows systems will have this available as a package.

1.1. Adding the utilities to your executable path

On non-Windows systems, once the packages have been built and installed. The required executables should already be accessible through your normal path.

For Windows systems, you will need to specify where the git, python and asciidoc executables reside (not required for the graphviz executables). If you followed the advice above and installed the msysgit package, then this is easy. When you install msysgit, it installs a bash shell which you can use to run the scripts required to build the LID. You must add the directories containing the python executable and the asciidoc utility to the bash shell’s path. Do that by creating a .profile file in the bash shell’s home directory that contains (at least) a line that looks like this:

PATH=$PATH:.:/c/Python26:/c/asciidoc

This example assumes that python was installed into C:\\Python26 and asciidoc was installed in C:\\asciidoc. Adjust as required to suit your installation.

The .profile file must reside in the bash shell home directory which is the directory that the bash shell starts up in when you click on the Git Bash desktop icon. This will be C:\\Documents and Settings\USER, where USER is your username. An easy way to create that file would be to type the following into the git shell:

echo PATH=\$PATH:.:/c/Python26:/c/asciidoc > .profile

Note the \ before the $PATH.

Once you have done this, close the bash shell and start a new shell. Invoking asciidoc.py should produce this output:

Man page:     asciidoc --help manpage
Syntax:       asciidoc --help syntax

And you can read the asciidoc manpage with:

asciidoc.py --help manpage | less

2. Obtaining the LID source

The source for the LID can be obtained by cloning the upstream repository, like this:

git clone git://git.lustre.org/lid.git

This will create a local directory, lid, that contains the source tree and the scripts. You should see diagnostic output a bit like this (obviously, the numbers will be different):

Initialized empty Git repository in /home/markb/lid/.git/
remote: Counting objects: 112, done.
remote: Compressing objects: 100% (110/110), done.
remote: Total 112 (delta 34), reused 0 (delta 0)
Receiving objects: 100% (112/112), 168.32 KiB | 36 KiB/s, done.
Resolving deltas: 100% (34/34), done.

3. Keeping the LID source up to date

Once the repository has been created, you can incorporate the latest updates by executing (within the lid directory):

git pull

This fetches the latest upstream changes and merges them into your master branch.

Note
Using fetch/merge instead of pull

Git pull is equivalent to:

git fetch                  # fetches upstream changes into branch origin/master
git merge origin/master    # merge those changes into current branch

Some people prefer to separate the two operations because that gives them the opportunity to inspect the incoming changes (using git diff) before they are merged into the master branch. Like this:

git fetch                  # fetches upstream changes into branch origin/master
git diff ..origin/master   # show differences from current branch
git merge origin/master    # merge those changes into current branch

The ..origin/master syntax is described below.

4. Formatting the LID

In the lid directory you will find a subdirectory src that contains the document source files. You will also find a Perl script, build.pl, and an asciidoc configuration file (asciidoc.conf).

To process the source files to produce the HTML output, do (in lid):

build.pl

This script does the minimum work required to bring the output files up to date with respect to the sources.

The output goes into a directory called html which will be created as required. The top-level file is html/index.html. To view the document you can simply open html/index.html with your web browser. If you want to install the document on a web server, just copy everything under the html directory.

To remove the generated files completely, do:

build.pl clean

You should only need to do that before building if you have changed either asciidoc.conf or config.pl.

For convenience, a trivial Makefile is supplied. So on systems that have make installed, you can build the document or remove the generated files by saying ‘make’ and ‘make clean’, respectively.

5. Editing the LID and submitting patches upstream

As mentioned above, you can edit the LID source files with any text editor.

5.1. Telling git who you are

Before you commit any changes to your local repository you should tell git what your user name and email address are so that any commits you make will be identified. You do this with the git config utility like this:

git config --global user.name "Mickey Mouse"
git config --global user.email mickey@disney.com

The --global option tells git that these configuration values should be used for all repositories you work with rather than just the lid repository.

5.2. Creating new branches

It is a good idea to make your edits on a new branch rather than editing the master branch directly. Branching (and merging) using git is straightforward. To create a new branch called branchname based on the current branch, do:

git branch branchname

Having created your new branch, you can check it out with:

git checkout branchname

To list the branches in the repository, do:

git branch

The current branch will be indicated with an asterisk.

You can switch to another branch at anytime with git checkout branchname.

Note If when you switch to another branch you have uncommitted changes and those changes are in files that are the same in both branches, the checkout will go ahead as expected. If the changes are in files that differ between the branches, the checkout will be refused so as to preserve your changes in context. You can force the checkout by specifying the -m option to merge your changes into the file(s) in the checked out branch.

5.3. Commiting changes

Now you can edit the sources as you wish and commit the changes to the current branch. Commiting changes in git is a 2-part process that involves adding the changes into a staging area (often referred to as the ‘index’ in git documentation) and then committing the contents of the index.

This 2-part commit process may appear complex but it is one of git’s strongpoints and you will soon come to appreciate how useful it is.

So, to add all of the changes in one or more files into the index, do:

git add filenames

You can then commit those changes with:

git commit -m "commit message"

Don’t forget to put quotes around the message if it contains spaces.

You can also tell git to automatically add all changed files to the index and commit them with a message by including the -a option:

git commit -a -m "commit message"

So git commit -a works very much like commiting changes with legacy SCM systems.

Note Remember, committing only commits to your local repository and does not effect the upstream repository.

5.4. Specifying revisions

Many git commands require a revision or range of revisions to be specified and git provides a rich syntax for specifying revisions. Full details can be found in the git manual pages and tutorials. The most commonly used ways of specifying a single revision are:

  • A branch’s current revision is specified as HEAD. This can often be ommitted as it is the default revision for many commands.

  • A branch or tag is specified simply by using its name.

  • Any commit can be specified using its SHA1 identifier (a unique substring is sufficient).

To specify a range of revisions from r1 to r2, use: r1..r2

For example, to specify the range of revisions from the head of the current branch to, say, origin/master, use HEAD..origin/master or, more simply, ..origin/master.

5.5. Viewing history

To see a list of commits on a given branch do:

git log branchname

If you leave out the branch name you see the commits on the current branch. By default, this just shows you the first line of the commit message and a few other details. It doesn’t show the changes (see below).

5.6. Creating diffs

To see the changes between your current working tree and the current branch, do:

git diff

To compare your current working tree (or current branch) against another branch (e.g. master), do:

git diff branchname

By default, git diff produces patch output so a very simple way to generate a patch file that you could then email to someone else would be:

git diff master > mychanges.patch

That works but it will squash all of the changes into a single patch and not include useful information such as the commit messages. A much better way to send your changes to someone else is to use git format-patch.

5.7. Emailing patches

To prepare a set of patches (one for each commit) that can then be emailed, use git format-patch. Here’s an example that formats all the revisions between the master branch and HEAD (as individual files):

git format-patch -C -M -s -o outdir master

Like most git commands, git format-patch can take a wide range of options but we only need to use a few of them. In our example, the -M and -C options specify that file renames and copies are to be detected and the -s option ‘signs off’ the patches so people know who produced them. The -o outdir specifies the name of the directory that the patches are output to (default the current directory).

By default, each patch goes into a separate file whose name is based on the commit message for the patch prefixed with a sequence number. Eg:

0001-Changed-meta-elements.patch
0002-Tweaks.patch
0003-Put-links-into-table.patch
0004-Build-formatting-tweaks.patch
0005-New-file.patch

Specifying the --stdout option, will send all the patches to standard output so to put them in a single file do this:

git format-patch -C -M -s --stdout master > outfile

The patch files are in UNIX mbox format and can be sent using regular email programs. Alternatively, on non-Windows systems, you can use git send-email to send out the patches like this:

git send-email --to recipient dir-or-files

Where recipient is the email address of the recipient (e.g. lustre-lid-ext@sun.com) and dir-or-files is either the name of a directory that contains the files to be sent (and nothing else) or a list of files to send.

If someone sends you patches in this format, save the message(s) into a mbox file and apply them to the current branch using git am, like this:

git am mbox_file