Copying the Template Files

Select the NFS Server package during Red Hat Linux installation and verify that the NFS is properly installed.

After Red Hat installation is complete, check for the NFS utility to verify NFS installation:

Set up your Serviceguard cluster according to the instructions in the Managing HP Serviceguard for Linux user’s guide.

Configure the disk hardware for high availability. Data disks associated with Serviceguard NFS must be external disks. All the nodes that support the Serviceguard NFS package must have access to the external disks. For most disks this means they must be attached to a shared bus that is connected to all nodes which support the package. The disk on which NFS volume is configured can be either a single lun or a split site RAID 1 array, which provides a level of disaster tolerance to the NFS volume. This can be achieved by setting up a Serviceguard Extended Distance Cluster which uses the linux software RAID as the basic building block. For information on setting up the Extended Distance Cluster, see the HP Serviceguard Extended Distance Cluster for Linux Deployment Guide.

Create a directory for each NFS package. For example:

/usr/local/cmcluster/nfs1

The names of the volume groups must be unique within the cluster, and the major and minor numbers associated with the volume groups must be the same on all nodes. In addition, the mounting points and exported file system names must be the same on all nodes.

The preceding requirements exist because NFS uses the major number, minor number, inode number, and exported directory as part of a file handle to uniquely identify each NFS file. If differences exist between the primary and adoptive nodes, the client’s file handle would no longer point to the correct file location after movement of the package to a different node.

Use LVM commands to set up volume groups, logical volumes, and file systems as needed for the data that will be exported to clients. Refer to the Managing HP Serviceguard for Linux manual.

Make sure the user IDs and group IDs of those who access the Serviceguard NFS file system are the same on all nodes that can run the package.

Make sure the user IDs and group IDs in the /etc/passwd and /etc/group files are the same on the primary node and all adoptive nodes, or use NIS to manage the passwd and group databases.

Create an entry for the name of the package in the DNS or NIS name resolution files, or in /etc/hosts, so that users will mount the exported file systems from the correct node. This entry maps the package name to the package’s relocatable IP address.

Decide whether to place executables locally on each client or on the NFS server. There are a number of trade-offs to be aware of regarding the location of executables with Serviceguard NFS.

The advantages of keeping executables local to each client are as follows:

The advantage of putting the executables on the NFS server is as follows:

Except for the variables listed below, use the default values for the variables in the package configuration file, or change them as needed.

For instructions on modifying the default values, see the Managing HP Serviceguard for Linux manual, or read the comments in the package configuration file.

Set the PACKAGE_NAME variable. For example:

PACKAGE_NAME pkg01

Each package must have a unique name.

Create a NODE_NAME variable for each node that will run the package. The first NODE_NAME should specify the primary node. All the NODE_NAME variables following the primary node should specify the adoptive nodes, in the order in which they will be tried. For example:

NODE_NAME thyme 
NODE_NAME basil
NODE_NAME sage

Set the RUN_SCRIPT and HALT_SCRIPT variables to the full path name of the control script. You do not have to specify a timeout for either script. For example:

RUN_SCRIPT /usr/local/cmcluster/pkg01/pkg01.cntl
RUN_SCRIPT_TIMEOUT NO_TIMEOUT
HALT_SCRIPT /usr/local/cmcluster/pkg01/pkg01.cntl
HALT_SCRIPT_TIMEOUT NO_TIMEOUT

If you want to run the NFS monitor script, set the SERVICE_NAME variable. For example:

SERVICE_NAME nfs1.monitor

Each package must have a unique service name. The SERVICE_NAME variable in the package configuration file must match the SERVICE_NAME variable in the package control script.

If you do not wish to run the NFS monitor script, comment out the SERVICE_NAME variable. For example:

SERVICE_NAME nfs1.monitor

If your NFS package configuration file specifies AUTO_RUN as YES, the package switches to the next adoptive node in the event of a node or package failure. The NFS monitor script causes the package to fail over if any of the monitored NFS services fail.

Set the SUBNET variable to the subnet that is monitored for the package. For example:

SUBNET 192.100.112.0

Create a separate VG[n] variable for each volume group. For example:

VG[0]=vg01
VG[1]=vg02

Create a separate LV[n], FS[n], FS_TYPE[n] and FS_MOUNT_OPT[n]variable for each volume group and file system that will be mounted on the server. For example:

LV[0]=/dev/vg01/lvol1;FS[0]=/ha_root; FS_TYPE[0]=ext3;FS_MOUNT_OPT[0]=”-o rw”
LV[1]=/dev/vg01/lvol2;FS[1]=/users/scaf; FS_TYPE[1]=ext3;FS_MOUNT_OPT[1]=”-o rw”
LV[2]=/dev/vg02/lvol1;FS[2]=/ha_data; FS_TYPE[2]=ext3;FS_MOUNT_OPT[2]=”-o ro”

This example defines the variable for three NFS mounted file systems, ha_root, users/scaf and ha_data.

Specify the IP address for the package and the address of the subnet to which the IP address belongs. For example:

IP[0]=192.100.112.10
SUBNET[0]=192.100.112.0

The IP address you specify is the relocatable IP address for the package. NFS clients that mount the file systems in the package will use this IP address to identify the server. You should configure a name for this address in the DNS or NIS database, or in the /etc/hosts file.

Specify that this package uses the high availability NFS server by uncommenting the HA_APP_SERVER variable. Uncomment the following line:

HA_APP_SERVER=”pre-IP”

If two packages have the same adoptive node, and you want to prevent a shared adoptive node from adopting both packages at once, specify the cmmodpkg command with the package control option (-d) in the function customer_defined_run_cmds. For example:

function customer_defined_run_cmds
{
    cmmodpkg -d -n ‘hostname‘ pkg02 &
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail.

In this example, if a host is an adoptive node for both pkg01 and pkg02, the above cmmodpkg -d command, in the control script for pkg01, would prevent the host that is running pkg01 from adopting pkg02. If pkg02 fails, it will failover to another adoptive node (if configured) where pkg01 is not running.

Add a similar line in the control script for pkg02 to prevent the host that is running pkg02 from adopting pkg01.

The ampersand (&) causes the cmmodpkg command to run in the background. The cmmodpkg command in the background allows the control script to complete and finish bringing up the package.

	NOTE: There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it. In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified.

See “Configuring One Adoptive Node to Support Failover of Multiple Packages” for a sample configuration using the package control option.

Use the default values for the rest of the variables in the control script, or change them as needed. For instructions on modifying the default values, see the Managing HP Serviceguard for Linux manual, or read the comments in the pkg.cntl template file.

Create a separate XFS[n] variable for each NFS directory to be exported. Specify the directory name and any export options. For example:

XFS[0]=”*:/ha_root”	
XFS[1]="*:/users/scaf"	
XFS[2]="-o ro *:/ha_data"

Do not configure these exported directories in the /etc/exports file. When an NFS server boots up, it attempts to export all file systems in its/etc/exports file. If those file systems are not currently present on the NFS server node, the node cannot boot properly. This happens if the server is an adoptive node for a file system, and the file system is available on the server only after failover of the primary node.

If you want to start and monitor rpc.quotad daemon, set QUOTA_MON to YES. For example:

QUOTA_MON=YES

If you do not want to start and monitor rpc.quotad daemon, set QUOTA_MON to NO. For example:

QUOTA_MON=NO

In the package configuration file, set the SERVICE_FAIL_FAST_ENABLED to “YES” for the NFS monitoring service. Example:

SERVICE_NAME nfs1.monitor	
SERVICE_FAIL_FAST_ENABLED YES
SERVICE_HALT_TIMEOUT    300

	NOTE: In Red Hat, there are times when sending SIGKILL to the kernel ‘lockd’ thread might not release all the file locks and cause the failure of the unmounting of filesystem. To force unmount of the filesystem, the machine has to be rebooted. In such cases, it is recommended to set SERVICE_FAIL_FAST_ENABLED to “YES” which reboots the machine upon service failure. In SLES, the SM directory does not consistently update with the client entries. This is due to the client entry being made for the first time only after the system has booted. After a fail back of the package, the NFS fails to create SM directory entries. After a fail back, if the client attempts to reclaim his locks, fresh entries for the clients will not be made in the /var/lib/nfs/sm directory of the server. For SLES, it is mandatory to set SERVICE_FAIL_FAST_ENABLED to “YES”, so the server reboots in order to have lock migration feature work consistently. If you halt the package manually on any node configured for an NFS package, you must reboot the machine before the package is run again on the same node.

In the Package Control Script, edit the customer_defined_run_cmds function to execute the toolkit.sh script for file lock migration. In the example below, the following line was added, $HA_APP_SCRIPT lock_migration:<Package IP>.

	NOTE: HA_APP_SCRIPT is initialized to the toolkit.sh path in the package control script.

Example:

function customer_defined_run_cmds        
{		
$HA_APP_SCRIPT lock_migration:${IP[0]}		
test_return 51
}

	NOTE: The argument to be passed to the HA_APP_SCRIPT for lock migration should contain the same name or IP address used by the clients while mounting the exported file system.

Example: If client mounts the NFS file system using the package name, specify the same in customer_defined_run_cmds as shown below:

function customer_defined_run_cmds        
{		
$HA_APP_SCRIPT lock_migration:
<package name>		
test_return 51
}

	NOTE: A Serviceguard package can support multiple re-locatable IP addresses associated with it. However, if lock migration feature is enabled then only one IP is allowed to be specified in the NFS package control script since the NFS limits server reboot/failover notification to be sent using only a single IP. In the package control script, only one IP/Subnet address pair is allowed to be specified under IP_ADDRESS section.

Configure the following variables in hanfs.conf:

Use the cmapplyconf command to verify the content of your cluster and package configuration and copy the binary configuration file to all the nodes in the cluster.

In the following example, the cluster configuration file is /usr/local/cmcluster/cluster.conf. On your system, use the names of your own cluster configuration and package configuration files.

# cmapplyconf -v -C /usr/local/cmcluster/cluster.conf -P \ /usr/local/cmcluster/pkg01/pkg01.conf