Creating a Storage Infrastructure with CVM

Initializing the VERITAS Volume Manager

If you are about to create disk groups for the first time, you need to initialize the Volume Manager. This is done by creating a disk group known as rootdg that contains at least one disk. Use the following command after installing CVM on each node:

# vxinstall

This displays a menu-driven program that steps you through the CVM initialization sequence. From the main menu, choose the “Custom” option, and specify the disk you wish to include in rootdg.

	IMPORTANT: The rootdg in the VERITAS Volume Manager is not the same as the HP-UX root disk, and cannot be used for the HP-UX root file system (/). Note also that rootdg cannot be used for shared storage. However, rootdg can be used for other local filesystems (e.g., /export/home), so it need not be wasted. Creating a rootdg disk group is only necessary the first time you use the Volume Manager.

Preparing the Cluster for Use with CVM

In order to use the VERITAS Cluster Volume Manager (CVM), you need a cluster that is running with a special CVM package. This means that the cluster must already be configured and running before you create disk groups.

	NOTE: Cluster configuration is described in the previous section.

To prepare the cluster for CVM disk group configuration, you need to set MAX_CONFIGURED_PACKAGES to 1 or greater in the cluster ASCII configuration file, and ensure that only one heartbeat subnet is configured. Then use the following command, which creates the special package that communicates cluster information to CVM:

# cmapplyconf -P /etc/cmcluster/cvm/VxVM-CVM-pkg.conf

	WARNING! This file should never be edited.

After this command completes successfully, you can start the cluster and create disk groups for shared use as described in the following sections. When you start the cluster, it will now run with a special system multi-node package named VxVM-CVM-pkg, which is on all nodes. This package is shown in the following output of the cmviewcl -v command:

CLUSTER      STATUS       
bowls        up           
 
NODE         STATUS       STATE        
spare        up           running 
split        up           running 
strike       up           running    
 
SYSTEM_MULTI_NODE_PACKAGES:
 
PACKAGE      STATUS       STATE        
VxVM-CVM-pkg up           running

Starting the Cluster and Identifying the Master Node

Run the cluster, which will activate the special CVM package:

# cmruncl

When CVM starts up, it selects a master node, and this is the node from which you must issue the disk group configuration commands. To determine the master node, issue the following command from each node in the cluster:

# vxdctl -c mode

One node will identify itself as the master. Create disk groups from this node.

Converting Disks from LVM to CVM

You can use the vxvmconvert utility to convert LVM volume groups into CVM disk groups. Before you can do this, the volume group must be deactivated, which means that any package that uses the volume group must be halted. The procedure is described in Appendix G of the Managing Serviceguard user’s guide.

Initializing Disks for CVM

You need to initialize the physical disks that will be employed in CVM disk groups. If a physical disk has been previously used with LVM, you should use the pvremove command to delete the LVM header data from all the disks in the volume group (this is not necessary if you have not previously used the disk with LVM).

To initialize a disk for CVM, log on to the master node, then use the vxdiskadm program to initialize multiple disks, or use the vxdisksetup command to initialize one disk at a time, as in the following example:

# /usr/lib/vxvm/bin/vxdisksetup -i /dev/dsk/c0t3d2

Creating Disk Groups for RAC

Use the vxdg command to create disk groups. Use the -s option to specify shared mode, as in the following example:

# vxdg -s init ops_dg c0t3d2

Verify the configuration with the following command:

# vxdg list

NAME         STATE                  ID
 
rootdg        enabled             971995699.1025.node1
ops_dg        enabled,shared      972078742.1084.node2

Creating Volumes

Use the vxassist command to create logical volumes. The following is an example:

# vxassist -g log_files make ops_dg 1024m

This command creates a 1024 MB volume named log_files in a disk group named ops_dg. The volume can be referenced with the block device file /dev/vx/dsk/ops_dg/log_files or the raw (character) device file /dev/vx/rdsk/ops_dg/log_files.

Verify the configuration with the following command:

# vxdg list

	IMPORTANT: After creating these files, use the vxedit command to change the ownership of the raw volume files to oracle and the group membership to dba, and to change the permissions to 660. Example: # cd /dev/vx/rdsk/ops_dg # vxedit -g ops_dg set user=oracle * # vxedit -g ops_dg set group=dba * # vxedit -g ops_dg set mode=660 * The logical volumes are now available on the primary node, and the raw logical volume names can now be used by the Oracle DBA.

Mirror Detachment Policies with CVM

The required CVM disk mirror detachment policy is ‘global’, which means that as soon as one node cannot see a specific mirror copy (plex), all nodes cannot see it as well. The alternate policy is ‘local’, which means that if one node cannot see a specific mirror copy, then CVM will deactivate access to the volume for that node only.

This policy can be re-set on a disk group basis by using the vxedit command, as follows:

# vxedit set diskdetpolicy=global <DiskGroupName>

	NOTE: The specific commands for creating mirrored and multi-path storage using CVM are described in the HP-UX documentation for the VERITAS Volume Manager.

Oracle Demo Database Files

The following set of volumes is required for the Oracle demo database which you can create during the installation process.

Create these files if you wish to build the demo database. The three logical volumes at the bottom of the table are included as additional data files, which you can create as needed, supplying the appropriate sizes. If your naming conventions require, you can include the Oracle SID and/or the database name to distinguish files for different instances and different databases. If you are using the ORACLE_BASE directory structure, create symbolic links to the ORACLE_BASE files from the appropriate directory.

Example, Oracle8:

# ln -s /dev/vx/rdsk/ops_dg/opsctl1.ctl \
    /u01/ORACLE/db001/ctrl01_1.ctl

Example, Oracle9:

For more information about Oracle directories, refer to the Oracle Server for HP 9000 Installation and Configuration Guide. More information about the maximum sizes of data files is found in the Oracle Reference.

Adding Disk Groups to the Cluster Configuration

After creating units of CVM storage with VxVM commands, you need to specify the disk groups in each package configuration ASCII file. Use one DISK_GROUP parameter for each CVM disk group the package will use. You also need to identify the CVM disk groups, file systems, logical volumes, and mount options in the package control script. The package configuration process is described in detail in Chapter 6 of the Managing Serviceguard user’s guide.