KenRawlings at 18:46, 4 March 2016

2016-03-04T18:46:19Z

← Older revision		Revision as of 11:46, 4 March 2016
Line 142:		Line 142:
	http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf		http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf

	[[Category: ~~Architecure~~]]		[[Category: Architecture]]

KenRawlings: assigning category

2016-03-04T18:46:01Z

assigning category

← Older revision		Revision as of 11:46, 4 March 2016
Line 141:		Line 141:

	http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf		http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf

			[[Category: Architecure]]

Adilger: Copy from original page on old wiki

2015-09-14T23:06:44Z

Copy from original page on old wiki

New page

'''''Note:''''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.''

== Definitions ==

; '''ZAP''' : ZFS Attribute Processor, a hashed name=value lookup table that can be used to do efficient and scalable attribute storage
; '''MicroZAP''' : a form of ZAP used by ZFS that only allows value to be a __u64
; '''TinyZAP''' : a compact ZAP format that allows arbitrary values to be stored
; '''FatZAP''' : a form of ZAP used by ZFS that allows arbitrary name/value pairs but (as name implies) consumes a lot of space

== Use Cases ==

{| border=1 cellspacing=0
|-
!id !! quality attribute !! summary
|-
|fast_ea || performance || TinyZAP must be flexible to allow storage in arbitrary-sized containers such as large dnode or ancillary dbuf, and not waste too much space per attribute
|-
|mdt_fid || performance || ZAP must allow FID storage for MDT directories in a manner that is compatible with ZFS directories
|-
|compatible-zfs || usability || integration should be done at the DMU level so that any ZAP users (including ZFS) can use TinyZAPs to store name/value data
|}

=== Efficient EA storage ===

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || Efficient EA storage in large dnode
|-align="left"
|colspan=2|'''Business Goals:''' || Fast access to Lustre EA values
|-align="left"
|colspan=2|'''Relevant QA's:'''|| Performance
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| EA needs to be stored in large dnode
|-align="left"
|'''Stimulus source:'''|| Lustre OSD (MDT/OST) storing EA data to object
|-align="left"
|'''Environment:'''|| EA being stored on a specific dnode within a transaction
|-align="left"
|'''Artifact:'''|| EA is stored in the dnode in ZAP
|-align="left"
|'''Response:'''|| size of EA in ZAP
|-align="left"
|'''Response measure:'''|| overhead should not be more than approximately 128 bytes + 48 bytes/EA
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''|| None.
|-
|}

=== MDT FIDs in directories ===

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || Storage of MDT FIDs in directories
|-align="left"
|colspan=2|'''Business Goals:''' || Able to store FID data along with name and (in some cases) local DMU objid in a single directory entry to avoid extra FID lookup overhead
|-align="left"
|colspan=2|'''Relevant QA's:'''|| Performance
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| Storing MDT FID in a directory entry
|-align="left"
|'''Stimulus source:'''|| Lustre MDT creating new directory entry
|-align="left"
|'''Environment:'''|| Directory entry being created within a transaction with objid and FID
|-align="left"
|'''Artifact:'''|| FID is stored in directory entry after __u64 objid value
|-align="left"
|'''Response:'''|| size of EA in ZAP
|-align="left"
|'''Response measure:'''|| overhead should not be more than approximately 128 bytes + 48 bytes/EA
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''|| None.
|-
|}

=== ZFS compatible directories ===

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || ZFS reading TinyZAP directory with FID
|-align="left"
|colspan=2|'''Business Goals:''' || Compatible with (possibly modified) ZFS code
|-align="left"
|colspan=2|'''Relevant QA's:'''|| Usability
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| ZFS reading TinyZAP entry
|-align="left"
|'''Stimulus source:'''|| ZFS doing name lookup in directory
|-align="left"
|'''Environment:'''|| Lustre MDT mounted with ZFS for diagnostic reasons
|-align="left"
|'''Artifact:'''|| ZFS is able to read directory and find objid (if local)
|-align="left"
|'''Response:'''|| ZFS does not fail in object lookup
|-align="left"
|'''Response measure:'''||
|-align="left"
|colspan=2|'''Questions:'''|| What should stored in the objid in a CMD environment where the object is on a remote MDT?
|-align="left"
|colspan=2|'''Issues:'''|| There will be some small amount of code change needed in ZFS to only access the first __u64 of the value, because it currently only provides enough space to return a single __u64 of data. We will also need to handle the case for a remote MDT by either storing objid=0 or some other well-defined value.
|-
|}

== Implementation constraints ==

TinyZAP needs to be flexible enough to store arbitrary name/value data, including both Lustre LOV EA, and also MDT directories with extended FID data. Using a MicroZAP is not possible because this only allows storage of a single __u64 value with each entry. Using a FatZAP is wasteful as it requires a full block just for the header and a separate block for the leaf data.

A preferred implementation would have a structure similar to the existing zap_leaf_{phys_t,chunk} for the TinyZAP, since the leaf structure is reasonably compact, and may avoid a large amount of almost-identical code in the ZAP.

ZFS should be adapted (if necessary) to be able to handle directories created with TinyZAP layout, so they can can get the objid from the first __u64 and ignore the FID component of the directory entry.

The current ZAP implementation uses an object set and object number as parameters and we will need to interface using a buffer that might be located in the dnode or in an external block. So this might require some refactoring of the ZAP code.

This needs to handle endian swabbing issues correctly, as does all ZFS code.

== Questions and Issues ==

Should we "wrap" the FID data after the DMU object id in an MDT directory so that it is possible in the future to add other extra data in a directory without compromising compatibility? Something like:

<nowiki>
#define ZAP_LUSTRE_FID 0x110f1d0f1d0f1d10

struct zap_dir_fid {
__u64 zdf_magic;
struct lu_fid zdf_fid; /* or other data as appropriate */
};

#define zdf_len (zdf_magic & 0xff).
</nowiki>
This means we can skip (possibly unknown) extra directory info by skipping (zdf_len) bytes at a time looking for zdf_magic == ZAP_LUSTRE_FID.

== References ==

[[Architecture - ZFS large dnodes|ZFS large dnodes]]

http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf

Architecture - ZFS TinyZAP - Revision history

KenRawlings at 18:46, 4 March 2016

KenRawlings: assigning category

Adilger: Copy from original page on old wiki