HP-UX File Systems Architecture Enhancements

HPUX 11i v2 provided support for large file systems (currently up to 16TB). However, mainly because specific file system types did not provide support for individual files larger than 2TB, this feature was not available to customers until HP-UX 11i v3.

In HP-UX 11i v3, customers using VxFs file systems will have support for larger file systems up to 40TB, and individual files up to 16TB.

Enables a variety of new file system features, and multi-level stacking will allow those features to be combined by customers in arbitrary ways.

The architecture will also make it easy for third-parties (ISVs) to add new stacked modules and extend HP-UX FS functionality after releases.

This allows a new subsystem to be installed on a running system and be immediately recognized without reboot. The daemon calls message-related routines for each subsystem in designated /sbin directories, until the device is claimed. The identity and status is then returned from the daemon to caller. The install program simply needs to (over)write the executable file in /sbin.

This provides better time-to-market for new or upgraded file system types because the commands and libraries that used to contain dependent code (and all the code statically-linked to them) will no longer need to be patched.

Third-party file systems can be supported for these functions.

VXFS, HFS and CDFS file systems take advantage of the new file caching technology; in particular, issues existing in previous HP-UX releases with file data coherency in VXFS and HFS file systems are removed.

In conjunction with the UFC and new Physical Memory Control, the use of the new file cache tunables provide improved file caching technology as well as improved physical memory management associated to the file I/O data cache.

File cache tunables are reduced in number from 5 to 2, and provide improved usability and default behavior.

File cache tunables will automatically adjust with OL* (Online Addition or Deletion) of memory events.

Application-disclosed hints can dramatically reduce data access latency in I/O intensive applications as well as reduce overall system resource impact.

Ease migration of Tru64 applications to HP-UX 11i v3

New interface for Unix 2003 compliance: pselect(2)

Expanded capacity PIDs, UNAME and hostname

Changes in the HP-UX VFS layer to support the porting of AutoFS, CacheFS, and NFS file systems to HP-UX 11i v3. Removal of incompatibilities between the HP-UX and Solaris VFS layers, without disturbing user space compatibility with previous HP-UX releases.

Enabling CIFS server integration to HP-UX 11i v3. The CIFS Server will have improved availability and manageability in heterogeneous HP-UX and NT environment.

HP CIFS Server will avoid data corruption by providing Lock interoperability among CIFS Clients, NFS Clients and local HP-UX processes.

Fixes synchronization delays that existed in previous releases between the /etc/mnttab user file and the in kernel mount table.

In previous releases the HFS file system did not handle write errors and could loop indefinitely retrying I/O’s in the SCSI stack. With the new enhancements, HFS safely halts the affected file system. This change minimizes the possibility of file system corruption caused by I/O errors.

The multi-thread syncer is converted to a kernel daemon in HP-UX 11i v3. The new syncer solves previously existing synchronization and performance issues with CPU allocation and de-allocation, and provides general performance improvements for mount/umount and other commands and utilities using the sync() system call.

Significant improvement in performance of asynchronous I/O (AIO) for applications that use of aio_reap(2)

Customers will be able to archive, restore, copy and list files that are of multi-terabyte size using the pax backup utility.

Customers will be able to use pax on link files that have a filename length that is >200 characters (previously, a limit of 100 characters existed).

The pre-HP-UX 11i v3 DNLC cached pathname components of length up to 39 characters. This has been extended to 256 characters, which provides (kernel) name look-up performance improvement when the pathname includes long component names. NFS file systems in particular use long pathname component names, and benefit from this performance improvement.

The pre-HP-UX 11i v3 HFS fsclean command would retry on non-responding devices for a long period of time. This caused a large delay on system start-up time while attempting to perform file consistency checks on HFS file systems resident on iSCSI devices that were not online. The fsclean command was modified to eliminate the re-tries, resulting in performance improvements in the start-up circumstances described above. A backward compatibility option was added to the command to revert to old behavior.

The system open file table was restructured in HP-UX 11i v3 to remove previous architectural limitations, and, therefore, the need to tune the nfile tunable is removed.

The nfile tunable is now private and deprecated. The recommended way to regulate the maximum number of open files on the system is to set the values of maxfiles_lim and nproc appropriately; the theoretical system maximum can be assumed to be maxfiles_lim * nproc.

The nfile tunable is now private and deprecated. It should no longer be used, and may be removed in a future HP-UX release.

Please note that the default value returned by the tunable infrastructure for the private tunable nfile is 0 (zero). The value of zero means that the system limit usually enforced by nfile will be disabled (that is, the number of system-wide open files is limited only by available memory). So applications depending on a non-zero tunable value for nfile must be modified immediately to remove this dependency. Applications may want to use instead the pstat interface pstat_getstatic()which will return MAX_INT for pst_max_nfile if nfile is left to default.

These tunables are obsolete and removed. Attempting to tune any of the obsolete buffer cache tunables, bufcache_max_pct, bufpages, dbc_min_pct, dbc_max_pct, or nbuf results in an error.

If needed, customers should use the tunables filecache_max(5) and filecache_min(5) to set limits on the file cache. Note that, on any given system, the optimum values of these two new file cache tunables are not necessarily equivalent to the optimum values of the obsolete buffer cache tunable values in the older systems. Customers should first determine if the default/automatic values yield acceptable performance on their system, before attempting to change the values of the new file cache tunables.

This tunable is obsolete and removed. Do not attempt to use this tunable, as it will result in an error. In previous releases this tunable was used to limit how the number of HP-UX buffer cache pages the sendfile()system call could use. The sendfile() operations no longer utilize the HP-UX traditional buffer cache and this tunable becomes obsolete. Please refer to fcache_seqlimit_file(5) and related file cache tunables to control the use of physical memory by sendfile() and other file system operations.

The multi-thread syncer is converted to a kernel daemon in HP-UX 11i v3. The new syncer solves previously existing synchronization and performance issues with CPU allocation and de-allocation, and provides general performance improvements for mount/umount and other commands and utilities using the sync() system call.

Significant improvement in performance of asynchronous I/O (AIO) for applications that use of aio_reap(2)

The pre-HP-UX 11i v3 DNLC cached pathname components of length up to 39 characters. This has been extended to 256 characters, which provides (kernel) name look-up performance improvement when the pathname includes long component names. NFS file systems in particular use long pathname component names, and benefit from this performance improvement.

The pre-HP-UX 11i v3 HFS fsclean command would retry on non-responding devices for a long period of time. This caused a large delay on system start-up time while attempting to perform file consistency checks on HFS file systems resident on iSCSI devices that were not online. The fsclean command was modified to eliminate the re-tries, resulting in performance improvements in the start-up circumstances described above. A backward compatibility option was added to the command to revert to old behavior.

setmnt (/etc/mnttab converted to pseudo driver)

pstat_getfile()(interface expansion)

tunables nbuf5, bufpages5, bufcache_max_pct5, dbc_min_pct5, dbc_max_pct5, bcvmap_size_factor5, sendfile_max5, dskless_node(5)

nfile(5), file_pad(5), o_sync_is_o_dsync(5), hpux_aes_override(5)