Lustre Manual Changes

Lustre includes an extensive manual. This is developed much like the source code: anyone can submit patches, review changes, and suggest improvements. The manual workflow includes Gerrit and continuous integration practice. This enables changes to be automatically verified and conveniently reviewed using Gerrit.

Requesting a Change or Improvement
Manual corrections and enhancements are tracked in the same manor as code changes. A Jira project exists, LUDOC, where changes are created and tracked. LUDOC tickets can be created, updated or commented by anyone with an account on JIRA. Once you have an account, go to JIRA and create your LUDOC ticket:

To browse all existing issues, please visit: https://jira.whamcloud.com/browse/LUDOC/

Minor changes
Minor changes and fixes can be committed against the existing LUDOC-11 issue. The intention is to minimize the overhead for quick fixes. For such a change, your commit message may be:  LUDOC-11 quota: spelling corrections Brief description of the change if it is more extensive. Signed-off-by: Random User 

Submitting a Change to the Manual Source
Anyone can create changes that modify the current Lustre manual. The manual is written in Docbook XML version 5, with each chapter residing in a separate xml file. Reference material is readily available on the web for Docbook XML syntax.

Development Environment Setup
The Docbook 5 development environment has been tested with Ubuntu 18.04 (previous Ubuntu versions back to 10.0 are known to work), RHEL7 Linux, and Mac OS X. The auto build process occurs on RHEL7. The original development work for the manual was completed using Ubuntu.

Ubuntu 18.04
Install the following packages (last validated 2018-06-20):

RHEL 7
Install the following packages:

Install from EPEL, CentOS, source or Whamcloud toolkit:

fop and xmlgraphics
 * A suitably recent version of fop (>= 1.0) is available for EL7 from Whamcloud toolkit in the 'x86_64' folder.
 * A suitably recent version xmlgraphics-commons to provide dependencies on fop >= 1.0 is a available for EL7 from Whamcloud toolkit in the 'noarch' folder.

Fedora 18
Install the following packages:

Max OS X
It is possible to use either the MacPorts or Homebrew (Brew) package managers on Mac OS X to install the required 3rd-party DocBook tools, instead of compiling them yourself. Only one of these two package managers should be used at one time.

If using the MacPorts package, install the following packages:

You may experience problems if you don't use MacPorts. These problems typically can be resolved by editing the Makefile to give the correct path to 'docbookxi.rng'. If using Brew, install following packages:

Changing the Manual
The manual is controlled using, Jira, git, and Gerrit (details below). The general workflow is:  git clone git://git.whamcloud.com/doc/manual lustre-manual  git checkout -b b_new_change; vi XXXX.xml A few notes regarding edits:
 * Clone the manual from the "doc/manual" repository:
 * Make change(s) in a local branch, editing .xml files as needed:
 * Vim has syntax highlighting, as does emacs. Limited success has been reported with WYSIWYG XML editors.
 * If using a WYSIWYG editor, run "git diff" to ensure it didn't reformat the whole .xml file and that only the intended text was modified.
 * Text should be wrapped at 80 columns so that it is easier to comment on changes during review. Otherwise, it is difficult to make specific comments on an issue after a whole long paragraph of text that is a single line in the .xml document.
 * Ensure you are familiar with Change Review Guidelines specified in this document so that you understand the writing criteria reviewers will apply to qualify your change.

 make check See Verify the Docbook for more information on validation.
 * Check that the document still validates:

 make pdf open lustre_manual.pdf  git commit -a -v -s  make push Note: 'make push' simply calls One inspector should be someone with a technical background related to the changes and another person should review for style/English.
 * Build a target (i.e. html or pdf) and visually check that the document looks good:
 * Commit the change locally:
 * Push the change for review:
 * Add two inspectors to the manual

Verify the Docbook
Docbook 5 is verfied using a relaxNG schema with schematron rules. The manual is built using separate xml files for each chapter, which use XInclude elements. Two complementary tools support relaxNG schemas with XInclude, xmllint and jing. The tools can be used together to identify problems as shown here. The easiest way to verify the changes are valid XML is to use the make check target:  make check
 * This will internally verify with xmllint the ./index.xml of doc/manual.
 * OPTIONAL additional verification with jing the ./index.xml of doc/manual:

Fixing Problems
If make check fails, an error will be reported. Finding the source of the problem can be tricky if incorrectly positioned tags are the underlying issue, i.e. a misplaced. In this case, experience suggests that the problem may be below the indicated problem in the returned error.

Building The Manual
Docbook manuals can be transformed into a number of formats automatically. Two formats of interest for testing are HTML and PDF. HTML transforms are made directly from the Docbook xml (with xsltproc). PDF transforms require transforming into an intermediatory fop xml format first, and then into a PDF with using fop. The easiest way to generate the output files is via the make targets:  make html  make pdf Building an xhtml and epub version of the manual is also supported by the makefile.
 * To build a HTML version of the manual:
 * Similarly, to build a PDF version of the manual:

Debugging
There are two types of errors that are common:
 * 1) well formed xml
 * 'make check' will do a good job of discovering and precisely identifying the location of these types of errors.
 * 1) valid xml
 * 'make check' will validate the xml but does a poor job of identifying errors when they occur. The first complication is that xinclude is used to include separate xml files. As a result, xmllint is unable to really spot where the problem is.

Example
 $ make check xmllint --noout --xinclude --noent --relaxng /usr/share/xml/docbook5/schema/rng/5.0/docbookxi.rng ./index.xml ./index.xml:6: element itemizedlist: Relax-NG validity error : Expecting an element listitem, got nothing ./index.xml:6: element itemizedlist: Relax-NG validity error : Expecting element example, got itemizedlist ./index.xml:63: element section: Relax-NG validity error : Element chapter has extra content: section ./index.xml:1: element chapter: Relax-NG validity error : Did not expect element chapter there ./index.xml:1: element chapter: Relax-NG validity error : Expecting element include, got chapter ./index.xml fails to validate

This complication can be avoided by first flattening the manual into a single xml file:  $ xmllint --xinclude  index.xml  >  /tmp/resolved.xml

Now, you can get a slightly better idea of where the problem is using xmllint – but it's still not that great:  $ xmllint --noout --xinclude --noent --relaxng /usr/share/xml/docbook5/schema/rng/5.0/docbookxi.rng /tmp/resolved.xml /tmp/resolved.xml:8757: element itemizedlist: Relax-NG validity error : Expecting an element listitem, got nothing /tmp/resolved.xml:8757: element itemizedlist: Relax-NG validity error : Expecting element example, got itemizedlist /tmp/resolved.xml:8812: element section: Relax-NG validity error : Element chapter has extra content: section /tmp/resolved.xml:8752: element chapter: Relax-NG validity error : Did not expect element chapter there /tmp/resolved.xml:8752: element chapter: Relax-NG validity error : Expecting element include, got chapter /tmp/resolved.xml fails to validate

However, if you have Jing installed, you can get an even better idea of what the problem is - as well as where it is:  $ jing /usr/share/xml/docbook5/schema/rng/5.0/docbookxi.rng /tmp/resolved.xml /tmp/resolved.xml:9127:29: error: element "para" not allowed here; expected the element end-tag, text or element "abbrev", "accel", "acronym", "address", "alt", "anchor", "annotation", "application", "author", "bibliolist", "biblioref", "blockquote", "bridgehead", "calloutlist", "caution", "citation", "citebiblioid", "citerefentry", "citetitle", "classname", "classsynopsis", "cmdsynopsis", "code", "command", "computeroutput", "constant", "constraintdef", "constructorsynopsis", "coref", "database", "date", "destructorsynopsis", "editor", "email", "emphasis", "envar", "epigraph", "equation", "errorcode", "errorname", "errortext", "errortype", "example", "exceptionname", "fieldsynopsis", "figure", "filename", "firstterm", "footnote", "footnoteref", "foreignphrase", "funcsynopsis", "function", "glosslist", "glossterm", "guibutton", "guiicon", "guilabel", "guimenu", "guimenuitem", "guisubmenu", "hardware", "important", "indexterm", "informalequation", "informalexample", "informalfigure", "informaltable", "initializer", "inlineequation", "inlinemediaobject", "interfacename", "itemizedlist", "jobtitle", "keycap", "keycode", "keycombo", "keysym", "link", "literal", "literallayout", "markup", "mediaobject", "menuchoice", "methodname", "methodsynopsis", "modifier", "mousebutton", "msgset", "nonterminal", "note", "olink", "ooclass", "ooexception", "oointerface", "option", "optional", "orderedlist", "org", "orgname", "package", "parameter", "person", "personname", "phrase", "procedure", "productionset", "productname", "productnumber", "programlisting", "programlistingco", "prompt", "property", "qandaset", "quote", "remark", "replaceable", "returnvalue", "revhistory", "screen", "screenco", "screenshot", "segmentedlist", "shortcut", "sidebar", "simplelist", "subscript", "superscript", "symbol", "synopsis", "systemitem", "table", "tag", "task", "termdef", "tip", "token", "trademark", "type", "uri", "userinput", "variablelist", "varname", "warning", "wordasword", "xi:include" or "xref"

Tool Reference
Jira has a project specifically for Manual changes: LUDOC. See Changing the Manual for command line reference. Using Gerrit describes how to submit changes to the lustre-release repository. The same git configuration will work for the manual. Docbook is xml and as such can be edited with a xml, or text editor. xsd aware editors provide additional functionality that users may find advantageous. The following observations are recorded for the tools below: oXygen make a number of xml editing products that support docbook5. See: [http://www.oxygenxml.com] for more information. Serna supports XSD and has been found to work well for editing blocks of existing text. However, it can be tricky to add significant new content. It is kind of WYSIWYG but allows an unsuspecting user to shift elements into invalid positions. The version tested was the free version on Ubuntu 11.10. Serna doesn't have Docbook 5 support. Fortunately, you can tell Serna to use the Docbook 5 XSD (on Ubuntu): Problem: Serna doesn't seem to like xinclude. xlinclue is used to break the manual into multiple separate xml files. The result of this is that Serna complains that the document is not valid and it cannot resolve xrefs. The screenshot below illustrates this: Work-around: Use the   command line for validation. Vim works well, and is a good choice for developers that are already making changes to the Lustre code, since it is configured to provide XML syntax highlighting by default. Any other text editor would also be usable. You might find the following commands useful for turning on xml sensitive spell checking: 
 * Jira
 * Git
 * Gerrit
 * Editing Tools
 * oXygen XML editor
 * serna
 * 1) open 'index.xml' from the manual.
 * 2) 'choose template'
 * 3) choose 'Docbook V4.5'
 * 4) 'Details...' 'Schema:' /usr/share/xml/docbook/schema/xsd/5.0/docbook.xsd
 * vim
 * syn spell toplevel
 * redraw
 * setlocal spell



= Change Review Guidelines = Like code, Manual changes are submitted to a review process.
 * All the manual patches that are in Gerrit for review are publicly accessible for viewing. Logged-in users can review previous versions of the Manual.
 * Updates to the Manual are automatically built from the change and available as a review Manual build. The links to these builds are in the comments of a change.

Rules on Proper Use of Other Party Trademarks

 * Designate other party trademarks with a superscript asterisk (*)
 * In the first appearance in a headline, and/or the first prominent use.
 * In the first appearance in the body of text.
 * The asterisk (*) may go at the end of the other party's trademarked name or at the end of the product name.
 * EXAMPLES:
 * Lustre* software
 * HP Pavilion system*
 * Adobe Acrobat* software
 * Use an appropriate noun after the trademark.
 * EXAMPLE:
 * On a Lustre* file system

Writing Guidelines
<pre style="overflow-x:auto;"> **Allocate, modify, and free a system buffer. **Send and receive data messages with modules. **Manage timers and scheduling. <pre style="overflow-x:auto;"> **Allocating, modifying, and freeing a system buffer **Sending and receiving data messages with modules **Managing timers and scheduling
 * Use simple sentence structure. Break up long sentences.
 * Describe a series of steps as a numbered list.
 * Use active voice. You can often avoid passive voice by using second person ("you") or imperative voice ("Do this...").
 * Wrong example (passive voice):
 * The module files should be created before they are referenced by another program.
 * Correct example (active voice):
 * Make sure that your program creates the module files before another program references them.
 * Use present tense whenever possible.
 * Wrong example:
 * When you press <CR>, the program will use the default value of zero
 * Correct example:
 * When you press <CR>, the program uses the default value of zero
 * Use parallel constructs for heading titles, table entries, numbered lists, and bulleted lists.
 * Correct example (all bullets start with a verb):
 * The relay uses the system services interface to perform these functions:
 * Correct example (all bullets start with a gerund):
 * The relay uses the system services interface for the following functions:
 * Use second person ("you") when referring to the target reader.
 * Use third person entities for other than the targeted reader ("users", "programmers").
 * Do not use first person references ("we").
 * Gender neutrality should be maintained; do not use these terms: "he", "she", "his", "her", or "him". Rewrite to avoid these terms if possible. If necessary, use third person ("their" rather than "his/her").
 * Lustre version specific hints
 * [LUDOC-104] introduces a set of conditional element values that declares a section, para, etc... as only relevant for a particular Lustre release. The currently supported values for the 'condition' attribute are: l21, l22, l23, l24, l25, l26, l27, l28, l29, l2A, l2B.. These values correspond to 'Introduced in Lustre 2.3' up through 'Introduced in Lustre 2.11', respectively. An example of the decoration created by including a conditional value of 'l2B' is available in the Manual if you search for 'Lustre 2.11'.
 * Always visit the HTML or PDF version of the document.
 * In addition to using the 'diff all side-by-side' to inspect changes within Gerrit, it is vital to check the document rendering well.
 * The number of lines that a typical change touches can be numerous. Viewing these changes in Gerrit is useful to ensure readability, grammar and style. Inspecting the change after a HTML or PDF build ensures that formatting is correct. Checking formatting within Gerrit is time-consuming and tricky, it is much easier once formatting is rendered.
 * Ensure that xml readability and format is not degraded.
 * Changes may be submitted from GUI xml editors. This is fine - as long as the XML is well formatted for humans to read.