Task: Planning Server Consolidation

This section starts with a generic procedure (“Understanding the Process”) for consolidating servers followed by an example of server consolidation (“Example: Consolidating to an Existing Server: Stacking Programs”).

Understanding the Process

Consolidating workloads to a new server or to free up an existing server is a common task. Capacity Advisor can help make the decisions about how to consolidate the workloads informed ones.

There are three fundamental approaches to consolidating servers:

Stacking workloads (representing applications) into standalone servers or nPartitions.
Stacking virtual machines into standalone systems or nPartitions.
Stacking nPartitions and virtual partitions into complexes.

The task description below is based on stacking Virtual Machines. For other approaches, the step describing editing the scenario would be different.

Prerequisites. To plan for consolidating servers:

You should be familiar with Capacity Advisor operations (see “Basic Procedures”).
You must be logged in to HP Systems Insight Manager. (See “Accessing Capacity Advisor”.)
You must have already collected data on the systems of interest (see “Gathering Data for Capacity Advisor”).

Procedure 4-1 To Consolidate Server Loads

Determine the Systems to Consolidate
This should include:
- The servers with loads you wish to consolidate.
- The server to which you wish to consolidate the loads (this can be a new server or an existing one).
Be sure to take into consideration the following:
- Connectivity: are the LANs and SANs needed by all the systems to be merged available to the server being targeted as the new host?
- Security: do any of the systems require isolated networks; are any of the systems subject to HEPA requirements?
- Ownership: are all of the applications and systems to be combined owned by a common organization or are all the owners agreeable to the consolidation?
- Licensing: are there any licensing restrictions that will prevent moving the applications; are there any advantages to moving the applications to a common server (such as combining applications using SAP or Oracle licenses to a single server)?
- Quality of Service: are the Quality of Service requirements for each application and server well understood?

Create a Scenario

Follow the procedure in “Creating a Scenario”. Remember to select the servers containing the applications that you wish to consolidate and, if it is not already in the scenario, the server you are targeting for the consolidated loads.




	NOTE: You cannot specify a Virtual Machine (VM) directly. Specifying a VM Host will cause all VMs hosted by the host to be included in the scenario. There is no way to include a single VM within a scenario without including the VM Host and all VMs managed by that VM Host. However, once a VM Host is added to a scenario, its VM guests are accessible individually.

Edit the Scenario
In this part of the process, you will build a model of the new configuration you wish to evaluate.
Follow the procedure in “Editing a Scenario” to edit the newly created scenario. While editing the new scenario, you will need to:
1. Set up the host system, including making it a VM host. This involves following the procedures in either “Creating a System” to create a system with the characteristics of the new system or “Editing a System” to change the characteristics of an existing system to reflect any changes being made in the existing host.
2. Create a VM for each of the systems you are planning to consolidate by following the procedure in “Creating a System”.
3. Move the workloads from the systems you are planning to consolidate to the appropriate VMs by following the procedure in “Moving a Workload”. At this point, you can adjust resource allocations (for example, CPU cores and speed, and memory) to model any projected change in the required resources or to accommodate differences in architecture and software configuration, using the procedure described in “Editing a Workload”. An example of applying the platform multipliers to accommodate differences in architecture is provided in theonline help for the Move Workload screen.
  NOTE: The bar graphs in the table that show resource usage provide a quick estimate of the effect of moving each workload.
Estimate the New Quality of Service
Generate a report on the scenario following the procedure in “Producing Graphs and Reports”
Repeat the Editing and Estimating Steps if Necessary
It is sometimes necessary to run through a number of variations on the scenario to determine the best consolidation strategy. You can repeat the preceding two steps to determine the optimum configuration, including whether adding Temporary Instant Capacity (TiCap) processors could help optimize your cost structure.

Example: Consolidating to an Existing Server: Stacking Programs

This example demonstrates how Capacity Advisor can be used to plan for the simplest method of server consolidation: stacking programs. It illustrates many screens that you will encounter in Capacity Advisor.




	NOTE: Try the HP Smart Solver. Once you understand the basics of how Capacity Advisor works, you will realize that many of the steps illustrated in this example can be done faster and on a broader scale using the Smart Solver to automate the analysis of multiple possibilities. See “Using the Smart Solver” to learn about the Smart Solver.

Assume that several virtual machines were set up over the last few months to support applications. There is good data on resource utilization, and it is a good time to see if consolidating the applications can free up resources for other uses.

The steps referenced in the following titles are from the procedure “Understanding the Process”.

Step 1: Determine Which Systems to Consolidate

This step requires a profound knowledge about the systems. Many questions need to be answered about each system. Different consolidation models make different questions appropriate and others inconsequential. The questions listed below are derived from the questions in the generic procedure provided above.

What set of patches (patch level) is required by the applications? (This does not matter if virtual machines are used for consolidation.)
What changes to settings of kernel tunables have been made? (This does not matter if virtual machines are used for consolidation.)
What are the licensing requirements for the applications?
Who owns each of the systems and, if they are owned by different organizations, are they agreeable to the consolidation?
What are the security requirements?
What are the networking requirements (LAN and WAN)?
Are there Storage Area Network (SAN) requirements?
How stable are the applications? All of them should be test and development systems or productions systems.

This list is simply illustrative, many other questions may need to be answered for your particular environment.

For the purposes of this exercise, the following three virtual machines running on puny03 are being considered for consolidation:

puny03v7
puny03v8
puny03v9

Each of the three systems has two processors and:

Is running applications based on the same database program, which is licensed per CPU
Is running the same operating system, with the same set of patches
Requires the same settings of kernel tunables for the database application
Uses the same LAN
Uses the same SAN
Has minimal security requirements that are adequately provided by the corporate firewall

Step 2: Create a Scenario

Create a new scenario:

Select OptimizeCapacity AdvisorCreate Scenario... from the upper menu bar.
On the Select Systems for the new Scenario screen, click in the check box beside the desired system to include it in the scenario. This system is the VM Host for the virtual machines to be consolidated.
Click the Next button.
Fill in the Scenario Name and the Description fields.
Click the Next button.
Check the values presented on the Verify Scenario Parameters screen of the Create New Scenario wizard and then click the Finish button.

Step 3: Edit the Scenario

When finished creating a new scenario, the Edit Scenario screen automatically opens for editing the new scenario.




	TIP: To Return to a Scenario at a Later Time... Locate the name of the scenario in the list presented on the VSE Management: Capacity Advisor tab screen. Click the scenario name to open the Capacity Advisor - Edit Scenario window for that scenario.

The current systems in the scenario are listed on the System tab. The bar graphs in the table show the peak resource utilization from data collected for the current week. The following image shows the three virtual machines mentioned at the beginning of this planning example.

The weekly data is representative of recent utilization and can be calculated quickly, but doesn't always give a comprehensive picture.

For a more comprehensive picture, look at a month's worth of data. Change the simulation interval to a month by clicking the first drop-down list of the Simulation Interval (where you see the word “Week”) and selecting Month from the list.

Once the screen has refreshed, new information about resource utilization is available. In this case, you can see that the utilization peaks for CPU, memory, and I/O disk bandwidth have increased.

Look at the CPU utilization profile for the system with the heaviest load (in this case, puny03v8), by clicking the first bar graph beside its name. A profile viewer for the selected resource and system opens.

There are several periodic peaks that use a little over half of the available CPU resource. The majority of the time usage is below 25 percent of the available cores. Examining the tabular summary can provide more insight into the resource usage on this server:

From the table, you can see the average utilization is less than 7 percent of the available resource, while the 90th Percentile value is less than 9 percent of two cores. Also note that there are times when applications are using a little over half of the CPU allocation for the allowed 15–minute duration.

Adding Processors

Returning to the Edit Scenario screen through the “Go back to....” link, you can review the number of cores on each of the systems in a scenario in the Scenario Systems table. (See “Step 3: Edit the Scenario” for example screen shots.)

In this example, all of the VMs have the same number of cores allocated (2 cores at 1.10 GHz for each VM). A look at the month long accumulation of peaks shows that the usage of the VM CPU core allocation ranges from 60% to 84% among the three servers. By mousing over the machine name in the left column of the systems table, you can see the current utilization limits set for these VMs (they are allowed to exceed 85 percent utilization 0 percent of the time).

Altogether, this indicates that for best fit, a consolidated VM will need more than 2 cores to support all the applications.

Based on the one month of data in this example, it seems unlikely that the consolidated VM will need the 6 cores currently allocated to the three separate VMs. Taking into consideration the historic CPU usage and the utilization limit for all three virtual machines, one additional core is added to the target consolidation VM, puny03v8




	NOTE: It is possible to refine this estimate further by changing the utilization limits or by forecasting future growth. Also, the longer the period of data analyzed, the better the analysis of resource utilization.

Add one processor to puny03v8:

Click the check box preceding puny03v8 in the list of systems.
Select What-If ActionEdit System... from the Edit Scenario-System tab menu bar.

In the resulting Edit Scenario - Edit System screen, increase the number of processors to three. This will represent moving an additional processor from the other VMs into this one.




	NOTE: You will soon realize that all of the operations done separately in this example can be done at once on the Edit System screen if you have taken care to estimate the potential desired change up front for each of the resources tracked by Capacity Advisor. Thus, you can tweak the resource allocations at the same time via this screen as you refine the fit through additional changes. Also, note that you can use What-If ActionsUndo/Edit/View Applied What-If Actions to disable or re-enable changes that you have made or to easily edit multiplier values. (See “To Edit Applied What-If Actions” and “Scaling Multipliers”.)

Click the OK button.
You are returned to the Edit Scenario-System tab screen.
The CPU Utilization bar graph for puny03v8 now shows about 55% utilization of the 3 cores.

Move Workloads

Next, move the workloads from selected servers (puny03v7 and puny03v9) to the consolidation server (puny03v8).

Click the Edit Scenario-Workload tab.
Click the check boxes preceding the virtual machines that contain the workloads that you are going to move.
Select What-If ActionMove Workload... from the Edit Scenario-Workload tab menu bar.
When the screen repaints, locate the workloads that are to be moved. You can see that it is possible to modify the default multiplier values for these systems at this location. For this example, it is not necessary to adjust the CPU or memory utilization multipliers because the systems are already virtualized, and the processors and operating systems are the same for all the workloads.
Now locate the target system for the consolidation.
Study the changes to the utilization meters and note the 5–star headroom ratings in this table. (See “Headroom” to understand the headroom rating and how to interpret it.)
For this example, the following information is seen:
None of the stars are green for the target system. This indicates that the new workloads do not fit the current configuration of the target system. Also notice:
- Adding the workloads only incrementally increased the utilization of the cores on the target system.
- The problem is that the current settings for memory allocation are insufficient for the additional workloads. The memory allocation should be increased, though the utilization limit on the memory could also be adjusted, if desired.
- Note that disk IO usage has increased to 94 percent of that allocated to puny03v8. The utilization limit for disk I/O on puny03v8 is set to allow disk I/O activity to exceed the 95 percent limit, 0 percent of the time.
Click the radio button preceding the target system.
Click the OK button.
Click the Edit Scenario-System tab and locate the puny03v8 system. The current result of the consolidation simulation looks like this:

Increase the Memory

Returning to the Edit Scenario screen through the “Go back to....” link, you can review the amount of memory on each of the virtual machines in a scenario. (See “Step 3: Edit the Scenario” for an example screen shot.)

In this example, all of the VMs have the same amount of memory allocation (.99 GB). However, all of the VMs are using over half of their allocation, which means that a consolidated VM will need more memory.

To provide adequate memory, edit the system again.

Click the System tab to open it.
Check that the consolidated system (puny03v8) is still selected, and click What-If ActionEdit System... from the System tab menu bar.
On the resulting Edit Scenario - Edit System screen, change the memory size to 3 GB, which represents moving memory from the other two servers to this one. While it might appear that a smaller addition of memory would be adequate, the scenario values for memory utilization only account for the memory used by the workloads, not the operating system overhead memory usage such as the data buffer cache.
Click the OK button.

The list of systems with new values is displayed on the System tab.

Step 4: Estimate the New Quality of Service

To quickly estimate the quality of service of the new configuration, it is useful to review both the utilization limits on the workloads and the combined resource utilization history of the workloads now on puny03v8.

To see the current utilization limits assigned to the workloads on the consolidated server, mouse over the workload name in the name column of the System table on the Edit Scenario-System tab.

Because all of the workloads have the same limits in this example, we can review the limits for the puny03v8 workload.

Now click on the CPU Utilization bar graph to open the Profile Viewer for this resource. Notice in the next example screen shot, in the Interval Metric Summary table, that peak utilization on this system is now at 62 percent of 3 cores. As the limit for CPU utilization is 85 percent for any workload, it appears that more CPU is assigned than is needed for this system. In this example, the number of CPU cores probably could be set to 2.5 instead of 3 and still meet the utilization limits for this resource.

Click the Memory radio button above the graph to switch to the memory utilization Profile Viewer. The graph indicates that memory utilization is steady, and the table data shows that measured utilization comfortably approaches the utilization limit set at 65 percent for this resource. Collecting data on the system after consolidation can provide a better picture of how much memory is actually used by the system once it is deployed.

In Step 8 of “Moving Workloads”, it appeared that I/O disk utilization would be very near the utilization limit set for the VM Host (95 percent). Click the I/O Disk radio button to check on I/O disk bandwidth utilization.

The graph and table data indicate that I/O disk bandwidth utilization is well within the utilization limit for the VM Host.

Further Analysis

The simulation scenario just completed shows that combining applications from the three virtual machines can lead to a significant reduction of the number of processors required (from 6 CPU cores to 2.5 or 3). Using peak-of-sums modeling provides a clearer picture of resource usage as compared to the traditional sum-of-peaks model. Depending on the licensing model for this setup, the number of licenses required for the underlying database application may have been reduced also. Would it be possible to reduce the number of processors further?

Return to the CPU utilization profile by clicking on the CPU radio button and locate the 90th Percentile value in the Interval Metric Summary table:

This value indicates that 90 percent of processing is occurring using only 19 percent (1/5) of the available CPU cores. A detailed report can give more insight into what this implies for the number of CPU cores needed.

Generate a Report on the Scenario

To investigate further reduction of CPU allocation, generate a report on the scenario:

Close the Capacity Advisor - Edit Scenario window.
From the Capacity Advisor menu bar on the Capacity Advisor tab in Virtualization Manager, select ReportScenario Utilization Report....
Place a check mark on both the Utilization Summary and Details to see all possible data reported.
Click the Next button.
The Select Report Targets screen opens.
Click the “Edit Scenario Selection” link.
The list of scenarios is displayed.
Click the check box to the left of the scenario name and OK.
The Select Report Targets screen opens with systems listed from the scenario that you selected.
Specify the date range for the report.
Click Next.
The Select Report Details screen opens.
Select the type(s) of resources for which you want to see data.
Select the type of scale for data presentation: percentage, absolute numbers, or both.
Select the graph size, and click Finish.
The Report Created screen opens.
Click the “Browse Report” link, and view the report.

Other variations on creating a report are available. See “Stepping Through the Report Wizard”.

Using the Report

This report combines many tabular summaries that provide detailed descriptions of the resource utilization. The one most helpful in this instance is the Time spent at or above each percent of allocation table for CPU utilization. Scroll down to see it.

From the 60% line in the table, with 0.02% of the time spent at that level, only an absolutely mission critical application would require more than three processors. Dropping down to the 40% row, with 1.64% of the time spent at or below that value, some applications might be able to tolerate dropping down to two processors. What final allocations are decided upon will be a business decision once the alternatives are understood.