Lustre 2.14.0 Changelog

Server support for kernels:

• RHEL 8.3 (4.18.0-240.1.1.el8_3)

Client support for unpatched kernels:

• RHEL 8.3 (4.18.0-240.1.1.el8_3)
• SLES15 SP2 (5.3.18-24.24.1)
• Ubuntu 20.04 (5.4.0-40)

Recommended e2fsprogs version:

• v1.45.6.wc5

Interoperability Support:

• Clients & Servers: Latest 2.12.X, 2.13

Issues fixed between 2.13.0 and 2.14.0

Test

   * [LU-13586] - check quota pools with PFL and SEL
   * [LU-13810] - Check OST pool quota hard limit at file with wide striping
   * [LU-13817] - Make lfs quota -uv --pool more verbose

New Feature

   * [LU-3606] - Implement fallocate() support for ldiskfs Lustre
   * [LU-10997] - Ubuntu 18 support
   * [LU-11023] - OST Pool Quotas
   * [LU-11025] - DNE3: directory restripe
   * [LU-11200] - Centos 8 arm64 server support
   * [LU-12036] - Add option to create new OSTs inactive
   * [LU-12275] - Client-side file data encryption
   * [LU-12461] - Contribute epython scripts to aid crash dump analysis
   * [LU-13180] - Reserve bit for RDMA-only memory RPC
   * [LU-13385] - lnetctl: calculate Lustre traffic service-id 
   * [LU-13946] - OpenZFS 2.0 Compatibility

Task

   * [LU-12861] - Add and use scnprintf() where using snprintf() is not handling return correctly	
   * [LU-12904] - Support for linux kernel version 5.4
   * [LU-13006] - need test for /sys/fs/lustre/jobid_this_session
   * [LU-13238] - add OFD layer object access logs
   * [LU-13255] - Abstract code for wait-with-periodic-warning
   * [LU-13359] - change lfs quota --pool to print used based only on OSTs from the pool
   * [LU-13376] - add batching to ofd_access_log_reader
   * [LU-13479] - "could not allocate data integrity bioset" on centos7
   * [LU-13904] - Support non-modular builds
   * [LU-14034] - test-framework and sanity-hsm use libtool nonsensically

Improvement

   * [LU-1538] - cleanup test scripts
   * [LU-2225] - sanity/27m to poll for specific state
   * [LU-4198] - Improve IO performance when using DIRECT IO using libaio
   * [LU-8465] - parallel e2fsck performance at scale
   * [LU-9019] - Migrate lustre to standard 64 bit time kernel API
   * [LU-9091] - Replace lprocfs_str_with_units_to_s64 with string_get_size()
   * [LU-9114] - Make MDS (And other server threads?) hog CPU less
   * [LU-9441] - Use kernel threads in predictable fashion to confine OS noise
   * [LU-9555] - "df /path" and "lfs df /path" should return projid-specific values
   * [LU-10235] - mkdir should check for directory existence on client before taking write lock
   * [LU-10428] - LNet events should generated without resource lock held
   * [LU-10447] - Replace use of $SETSTRIPE and $GETSTRIPE with '$LFS setstripe' and '$LFS getstripe' in test suites
   * [LU-10467] - use standard linux wait_event macros
   * [LU-10810] - SEEK_HOLE and SEEK_DATA support for lseek
   * [LU-10934] - integrate statx() API with Lustre
   * [LU-11310] - support for SLES 15
   * [LU-11365] - test-framework::add() should load ldiskfs
   * [LU-11505] - improve functions() run mpi tests with ability to set the composite layout
   * [LU-11607] - Reduce repeated function calls in Lustre test suites
   * [LU-11623] - Allow caching of open-created dentries
   * [LU-11631] - symlink migration should not create remote inodes
   * [LU-11668] - mdd_parent_fid()) ASSERTION( (((mdd_object_type(obj)) & 00170000) == 0040000) ) failed
   * [LU-11963] - Add nonrotational flag to obd_statfs
   * [LU-11971] - Send file creation time to clients 
   * [LU-12043] - improve Lustre single thread read performances
   * [LU-12071] - bypass pagecache for large files
   * [LU-12133] - sanityn test_35() defect
   * [LU-12137] - update client to use iterate_shared
   * [LU-12222] - LNet should select loopback NI when possible
   * [LU-12280] - Provide option to disable soft quota expiration
   * [LU-12303] - Use lnet_health_sensitivity for restoring health for each lnet_recovery_internal
   * [LU-12332] - Add a liblustreapi call for IOC_MDC_GETFILEINFO
   * [LU-12353] - optimizations for ldiskfs quota updates
   * [LU-12355] - Support for linux kernel version 5.0
   * [LU-12400] - Support for linux kernel version 5.2
   * [LU-12410] - Refactor lnetctl peer nid handling to utilize nidstrings library
   * [LU-12477] - Remove obsolete config checks
   * [LU-12518] - improve Lustre unaligned IO read performances
   * [LU-12521] - print_instance() incorrect if fsname contains a dash
   * [LU-12542] - LDLM improvements form linux lustre client work
   * [LU-12546] - add option to abort recovery between MDTs but not between client/MDT
   * [LU-12631] - Report latency of client operations
   * [LU-12634] - Support for linux kernel version 5.3
   * [LU-12662] - Run pjdfstest POSIX test in review and full testing
   * [LU-12722] - exclude local client mounted on MDS/OSS from recovery
   * [LU-12785] - DOM2: dynamic DoM component size as MDT becomes full
   * [LU-12806] - llapi: have llapi_fd2fid rely on name_to_handle_at() rather than ioctl()
   * [LU-12820] - obd_dirty_transit_pages is always zero and can be removed
   * [LU-12871] - enable changelog garbage collection by default
   * [LU-12899] - Make Lustre build report the required package: kernel-rpm-macros
   * [LU-12905] - sanity/51e can take less with debugging disabled
   * [LU-12910] - osc.*.short_io_bytes parameter cannot be set to the maximum
   * [LU-12929] - Field net_state in struct lnet_net is never used, only set.  It should be discarded
   * [LU-12930] - use schedule_timeout_{un,}interruptible where possible.
   * [LU-12931] - Fix minor errors with cfs_time_seconds usage
   * [LU-12936] - Discard pt_number from lnet_peer_table
   * [LU-12941] - Track aliveness at the peer level
   * [LU-12942] - Optimize check for routing feature flag
   * [LU-12950] - Сhange force_over_512tb lustre mount option to force_over_1024tb for ldiskfs
   * [LU-12966] - print logs when lfsck detected inconsistent_owner
   * [LU-12973] - lustre/doc/.gitignore ignores relevant files
   * [LU-12977] - fix i_mutex for ldiskfs_truncate() in osd_execute_truncate()
   * [LU-12984] - Add --newerXY support for lfs find
   * [LU-12988] - improve mount time on huge ldiskfs filesystem
   * [LU-13004] - Use a single style of IO buf descriptor  - iovec
   * [LU-13005] - Remove LNet event queuing subsystem (lib-eq)
   * [LU-13013] - osd can save credits on mapped blocks
   * [LU-13036] - Reduce memory consumption by lnet.
   * [LU-13056] - lctl llog_catlist, llog_print, etc. should default to MGS device
   * [LU-13059] - kernel update [RHEL7.7 3.10.0-1062.9.1.el7]
   * [LU-13098] - supress connection restore message on idling connections
   * [LU-13110] - kernel update [SLES12 SP4 4.12.14-95.45.1]
   * [LU-13111] - support for SLES12 SP5
   * [LU-13120] - lustre-osd-zfs-mount depends on zfs, kmod-lustre-osd-zfs
   * [LU-13122] - osd-zfs to use 8K blocksize for llog objects
   * [LU-13127] - make Lustre error when hash algo is not available
   * [LU-13130] - sanity-scrub to use total device size with ZFS
   * [LU-13134] - try to use slab allocation for cl_page
   * [LU-13141] - Include block allocator improvements to additional distributions
   * [LU-13151] - Add parent FID to MTIME changelogs
   * [LU-13164] - remove unused LUSTRE_DIRECTIO_FL
   * [LU-13169] - Allow test-framework to repeat subtests
   * [LU-13178] - Update ZFS Version to 0.8.3
   * [LU-13224] - Expose lctl get/set/list param functionality in lustreapi
   * [LU-13226] - SUSE 15 SP2 server ldiskfs
   * [LU-13241] - mount.lustre for large filesystem runs slow debugfs commands
   * [LU-13258] - Bind linux workqueues to specific core
   * [LU-13268] - customize lnet-selftest for performance
   * [LU-13269] - make lnet-selftest.sh more flexible
   * [LU-13272] - customise dom-performance
   * [LU-13274] - Building againt lustreapi using -std=c99
   * [LU-13275] - enhance racer with various FOFB scenarios
   * [LU-13280] - customize run_mdtest() for several mount points
   * [LU-13281] - ha.sh improvements: different layouts  for clients directories
   * [LU-13283] - ha.sh improvement: add racer to nonmpi load
   * [LU-13297] - parallel-scale enhancement
   * [LU-13312] - Optimized RA for stride read under memory pressure
   * [LU-13347] - kernel update RHEL 7.8 
   * [LU-13366] - SEL addons
   * [LU-13369] - kernel update [RHEL7.7 3.10.0-1062.18.1.el7]
   * [LU-13383] - lazy atime update on OST
   * [LU-13388] - racer crash: general protection fault: 0000 RIP: lod_obj_for_each_stripe+0xf7/0x2d0 [lod]
   * [LU-13394] - avoid sending OST RPC for project setting
   * [LU-13405] - kernel update [SLES15 SP1 4.12.14-197.37.1]
   * [LU-13412] - fix read if readahead window smaller than rpc size
   * [LU-13421] - kernel update [RHEL8.1 4.18.0-147.8.1.el8_1]
   * [LU-13445] - enhance ha.sh to support different users
   * [LU-13488] - Support RHEL 8.2
   * [LU-13501] - Disable health on single-rail deployments
   * [LU-13502] - Provide ability to fine-tune response tracking
   * [LU-13503] - allow setting larger max_mod_rpcs_in_flight dynamically
   * [LU-13509] - Improve ptlrpc_register_bulk() behavior
   * [LU-13510] - Allow control over LND timeouts independent of lnet_transaction_timeout and retry_count
   * [LU-13533] - Disable lazy_itable_init
   * [LU-13556] - kernel update [RHEL7.8 3.10.0-1127.8.2.el7]
   * [LU-13571] - Refine which network errors result in LNet Health activity
   * [LU-13585] - add mustfail check to ha.sh
   * [LU-13590] - support for RHEL 7.9
   * [LU-13595] - add option to lustre_rmmod to dump debug logs
   * [LU-13597] - add processing time/latency, IO sizes, stats to job_stats
   * [LU-13600] - limit number of RPCs in flight during recovery
   * [LU-13651] - Conditionally skip finding compatible HSM requests
   * [LU-13657] - kernel update [RHEL8.2 4.18.0-193.6.3.el8_2]
   * [LU-13676] - script to show unique backtraces from crash utility
   * [LU-13699] - Improve: mdt_identity.c Error handling and Error message reporting
   * [LU-13723] - Use ERR_CAST() instead of ERR_PTR(PTR_ERR()) 
   * [LU-13734] - Allow duplicate nets in kernel ip2nets syntax
   * [LU-13735] - Loosen restrictions on LNet Health parameter settings
   * [LU-13740] - Ubuntu 20.04 LTS release
   * [LU-13761] - MOFED 5.1 support
   * [LU-13782] - LNet Routers should monitor the ni_fatal flag to inform peers of changes to route status
   * [LU-13819] - Update ZFS Version to 0.8.4
   * [LU-13827] - ofd_access_log_reader to print top hot files
   * [LU-13837] - Introduce constant for LNET_NIDNET(LNET_NID_ANY)
   * [LU-13839] - support for RHEL 8.3
   * [LU-13919] - kernel update [RHEL7.8 3.10.0-1127.19.1.el7]
   * [LU-13921] - change output format for ofd_access_log_reader
   * [LU-13944] - t-f: add ability to set clients parameters
   * [LU-13961] - kernel update [RHEL8.2 4.18.0-193.19.1.el8_2]
   * [LU-13969] - Updates to lustre-release yaml.sh
   * [LU-13977] - sanity test_255a uses integer comparison for non-integers
   * [LU-14020] - Update SUSE SP2 ldiskfs
   * [LU-14029] - kernel update [SLES15 SP2 5.3.18-24.24.1]
   * [LU-14043] - lfs mirror extend need not use O_DIRECT on source
   * [LU-14052] - ofd_access_log_reader support for multiple MDTs
   * [LU-14054] - add option to ofd_access_log_reader to exit on close of last log
   * [LU-14071] - add OST pool quota options for lfs-quota.1 and lfs-setquota.1
   * [LU-14075] - kernel update [RHEL8.2 4.18.0-193.28.1.el8_2]
   * [LU-14077] - kernel update [SLES15 SP1 4.12.14-197.64.1]
   * [LU-14129] - kernel update [RHEL7.9 3.10.0-1160.6.1.el7]
   * [LU-14134] - reduce credits for new writing potentially
   * [LU-14140] - osd_idc_find_and_init() should dump details
   * [LU-14154] - ldiskfs doesn't builld with gcc10
   * [LU-14170] - Update ZFS version to 2.0.0
   * [LU-14194] - add mdc/mdd related checksum parameter files
   * [LU-14218] - kernel update [SLES15 SP1 4.12.14-197.75.1]
   * [LU-14265] - Allow "lfs mkdir" without arguments
   * [LU-14334] - update lnet/ChangeLog with current information

Technical task

   * [LU-4423] - Tracking of patches from upstream kernel to Lustre client
   * [LU-10664] - DoM: make DoM lock enqueue non-blocking
   * [LU-11300] - LNet: Router Aliveness and Health
   * [LU-12923] - replace CLASSERT() with BUILD_BUG_ON()
   * [LU-13415] - DoM: keep optimizations even when OST stripe is initialized