HP HyperFabric Installationshandbuch

Installing and Administering HyperFabricHP-UX 11i v1 and HP-UX 11i v2 Edition 13Manufacturing Part Number: B6257-90060October 2006Printed in U.S.A.© C

Figures8

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4100Figure 4-2 An MC/ServiceGuard Configuration (with Two HyperFabric Switc

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4101adapter is now handling the network traffic.) This rearrangement is han

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4102Example 1: This example, illustrated by Figure 4-3 below, presents an H

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4103Example 2: This example, illustrated by Figure 4-4 below, shows the sam

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4104Example 3:This final example, illustrated by Figure 4-5 below, shows a

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4105NOTE You should configure HyperFabric with MC/ServiceGuard before runni

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 4106Configuring MC/ServiceGuard for HyperFabric Relocatable IP AddressesIf

Chapter 4834 Managing HyperFabricThis chapter contains the following sections that give information about managing HyperFabric:•“Starting HyperFabric”

Managing HyperFabricChapter 484•“Displaying Status and Statistics” on page 91•“Viewing man Pages” on page 98•“Stopping HyperFabric” on page 99

Managing HyperFabricStarting HyperFabricChapter 485Starting HyperFabricHyperFabric is started in one of these three ways:• As part of the normal local

11Printing History The manual printing date and part number indicate its current edition. The printing date will change when a new edition is printed

Managing HyperFabricStarting HyperFabricChapter 486The clic_start -? command can be issued to display the online help for clic_start, or look at the c

Managing HyperFabricVerifying Communications within the FabricChapter 487Verifying Communications within the FabricYou can verify the communications w

Managing HyperFabricVerifying Communications within the FabricChapter 488• -r specifies that VRID switch_hopcount is the routing information for the a

Managing HyperFabricVerifying Communications within the FabricChapter 489NOTE Also see the clic_diag command to:Probe a specific remote node.Dump an

Managing HyperFabricVerifying Communications within the FabricChapter 490• Example 3If the local node is bently6, and you want to send five packets to

Managing HyperFabricDisplaying Status and StatisticsChapter 491Displaying Status and StatisticsYou can get the status of and statistics associated wit

Managing HyperFabricDisplaying Status and StatisticsChapter 492Note that turning on performance statistics gathering will affect the performance of Hy

Managing HyperFabricDisplaying Status and StatisticsChapter 493============================================================================• Example 2

Managing HyperFabricDisplaying Status and StatisticsChapter 494============================================================================Date: Sat A

Managing HyperFabricDisplaying Status and StatisticsChapter 495• Example 4If the local node is bently6, and you want to display the VRIDs, IP addresse

12

Managing HyperFabricDisplaying Status and StatisticsChapter 496============================================================================Date: Sat A

Managing HyperFabricDisplaying Status and StatisticsChapter 497---------------------------------------------------------------------------- No

Managing HyperFabricViewing man PagesChapter 498Viewing man PagesIf you want to be able to view the HyperFabric man pages, you must first add /opt/cli

Managing HyperFabricStopping HyperFabricChapter 499Stopping HyperFabricYou can stop HyperFabric only on a local node. Stopping HyperFabric on a node m

Managing HyperFabricStopping HyperFabricChapter 4100Using SAM—HP-UX 11.0 and 11i 0nlyTo use SAM to stop HyperFabric on a local HP 9000 system running

Chapter 51015 Troubleshooting HyperFabricThis chapter contains these sections that describe troubleshooting HyperFabric:•“Running Diagnostics” on page

Troubleshooting HyperFabricChapter 5102•“LED Colors and Their Meanings” on page 111•“Determining Whether an Adapter or a Cable is Faulty” on page 122•

Troubleshooting HyperFabricRunning DiagnosticsChapter 5103Running DiagnosticsBefore running HyperFabric diagnostics:1. Confirm HyperFabric adapters ar

Troubleshooting HyperFabricRunning DiagnosticsChapter 5104clic_shutdownclic_init (The answer to the interoperability question must be consistent on al

Troubleshooting HyperFabricRunning DiagnosticsChapter 5105If you are running an Oracle application using HMP and traffic does not appear to be flowing

Chapter 1131 OverviewThis chapter contains the following sections that give general information about HyperFabric:• “Overview” on page 15• “HyperFabri

Troubleshooting HyperFabricRunning DiagnosticsChapter 5106• -r specifies that you want to probe a specific remote node identified by remote_node_name.

Troubleshooting HyperFabricRunning DiagnosticsChapter 5107Example of clic_diagAn example of the clic_diag command is shown below.If the local node is

Troubleshooting HyperFabricUsing Support Tools ManagerChapter 5108Using Support Tools ManagerUse Support Tools Manager (STM) with HyperFabric to gathe

Troubleshooting HyperFabricUseful FilesChapter 5109Useful FilesWhen you are troubleshooting HyperFabric-related problems, you might find it useful to

Troubleshooting HyperFabricUseful FilesChapter 5110/var/adm/clic_fw.dumpxThis is the default file for a memory dump of an HyperFabric adapter, created

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5111LED Colors and Their MeaningsListed below are the possible colors (and the corresp

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5112— The adapter is connected to a non-operational adapter in the remote node (in a n

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5113Table 5-2 HyperFabric Adapter LED Colors and MeaningsLED Color Meaning Notes“Conne

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5114HF1 Switch LEDsThe HF1 switch LED colors and meanings are explained below.❏ For ea

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5115❏ For the “Ethernet” port and LED on the switch:✓ If the Ethernet port is not conn

OverviewChapter 114•“HyperFabric Concepts” on page 19

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5116Table 5-3 HF1 Switch LED Colors and MeaningsLED Color Meaning NotesPort None Port

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5117“Ethernet” None Ethernet port is disconnected.This can happen if the Ethernet port

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5118HF2 Switch LEDsThe HF2 switch LED colors and meanings are explained below.❏ For ea

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5119❏ For each port on the cards/switch module in the switch:✓ If the port is not oper

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5120Table 5-4 HF2 Switch LED Colors and MeaningsLED Color Meaning Notes“Status” None T

Troubleshooting HyperFabricLED Colors and Their MeaningsChapter 5121“Port x” None Port is not operational.• A cable is not attached correctly to the p

Troubleshooting HyperFabricDetermining Whether an Adapter or a Cable is FaultyChapter 5122Determining Whether an Adapter or a Cable is FaultyIf you su

Troubleshooting HyperFabricDetermining Whether a Switch is FaultyChapter 5123Determining Whether a Switch is FaultyIf you suspect that a HyperFabric s

Troubleshooting HyperFabricDetermining Whether a Switch is FaultyChapter 5124a. If the port is connected to an operational Ethernet network, the port’

Troubleshooting HyperFabricReplacing a HyperFabric AdapterChapter 5125Replacing a HyperFabric AdapterIf OLAR is supported for the HP 9000 system and t

OverviewOverviewChapter 115OverviewHyperFabric is a Hewlett-Packard high-speed, packet-based interconnect for node-to-node communications. HyperFabric

Troubleshooting HyperFabricReplacing a HyperFabric SwitchChapter 5126Replacing a HyperFabric SwitchIf you need to replace a faulty HyperFabric switch,

151IndexAadaptersdeconfiguring, 96faulty, testing for, 146installing, 54LED colors and meanings, 135, 137OLA of, 57OLR of, 58

152replacing, 149Ccables, testing for faulty, 146CLIC, definition of, 19, 60clic_diag command, 113, 129clic_init command, 92after OLA, 58clic_ping com

153current map of, 115, 116definition of, 19planning, 23filesglobal configurationcontents of, 85creating, 91softwaredescription of, 60structure of, 6

154Pplanning the fabric, 23prerequisites for installation, 53Rrelocatable IP addresses, 19configuring, 106SSAMusing for configuration, 94using for dec

155viewing man pages, 122VRID, 86

156

OverviewHyperFabric ProductsChapter 116HyperFabric ProductsHyperFabric hardware consists of host-based interface adapter cards, interconnect cables an

OverviewHyperFabric ProductsChapter 117The A6092A HyperFabric adapter is supported beginning with the following HyperFabric software versions:• HP-UX

OverviewHyperFabric ProductsChapter 118To determine the version of HyperFabric you have, issue this command:swlist | grep -i hyperfabricOther Product

OverviewHyperFabric ConceptsChapter 119HyperFabric ConceptsSome basic HyperFabric concepts and terms are briefly described below.The fabric is the phy

2Legal NoticesThe information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this

OverviewHyperFabric ConceptsChapter 120

Chapter 2212 Planning the FabricThis chapter contains the following sections offering general guidelines and protocol specific considerations for plan

Planning the FabricChapter 222•“HyperFabric Functionality for TCP/IP and HMP Applications” on page 24•“TCP / IP” on page 25•“Hyper Messaging Protocol

Planning the FabricPreliminary ConsiderationsChapter 223Preliminary ConsiderationsBefore beginning to physically assemble a fabric, follow the steps b

Planning the FabricHyperFabric Functionality for TCP/IP and HMP ApplicationsChapter 224HyperFabric Functionality for TCP/IP and HMP ApplicationsThe fo

Planning the FabricTCP / IPChapter 225TCP / IPTCP/IP is supported on all HF1 (copper) and HF2 (fibre) hardware. Although some of the HyperFabric adapt

Planning the FabricTCP / IPChapter 226the package to another node in the cluster, allowing services to remain available with minimal interruption. MC/

Planning the FabricTCP / IPChapter 227• Load Balancing: SupportedWhen a HP 9000 HyperFabric cluster is running TCP/IP applications, the HyperFabric dr

Planning the FabricTCP / IPChapter 228• Maximum Cable Lengths:HF1 (copper): The maximum distance between two nodes or between a node and a switch is 6

Planning the FabricTCP / IPChapter 229Speed and Latency:Table 2-1 HF1 Speed and Latency w/ TCP/IP ApplicationsServer Class Maximum Speed Latencyrp7400

Contents11. OverviewOverview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Planning the FabricTCP / IPChapter 230Table 2-3 Supported Configurations for A6386A HF2 Adapter On PCI (4X) Supported HP 9000 SystemsHP-UX VersionOLAR

Planning the FabricTCP / IPChapter 231TCP/IP Supported ConfigurationsMultiple TCP/IP HyperFabric configurations are supported to match the cost, scali

Planning the FabricTCP / IPChapter 232Figure 2-1 TCP/IP Point-To-Point Configurations

Planning the FabricTCP / IPChapter 233SwitchedThis configuration offers the same benefits as the point to point configurations illustrated in figure 1

Planning the FabricTCP / IPChapter 234High Availability SwitchedThis configuration has no single point of failure. The HyperFabric driver provides end

Planning the FabricTCP / IPChapter 235HybridServers and workstations can be interconnected in a single heterogeneous HyperFabric cluster. In this conf

Planning the FabricTCP / IPChapter 236Mixed HF1 / HF2 (Copper & fibre)All currently available HyperFabric products can be interconnected in a sing

Planning the FabricHyper Messaging Protocol (HMP)Chapter 237Hyper Messaging Protocol (HMP)Hyper Messaging protocol (HMP) is Hewlett-Packard’s patented

Planning the FabricHyper Messaging Protocol (HMP)Chapter 238Starting with the December 2000 releases B.11.00.11 and B.11.11.01, the HyperFabric EMS mo

Planning the FabricHyper Messaging Protocol (HMP)Chapter 239• Switch Management: Not SupportedSwitch Management is not supported. Switch management wi

Contents2With the Rail Kit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Planning the FabricHyper Messaging Protocol (HMP)Chapter 240Instead, all of the traffic will be sent over a single connection with no failover capabil

Planning the FabricHyper Messaging Protocol (HMP)Chapter 241Table 2-6 Supported Configurations for A6386A HF2 Adapter On PCI (4X) Supported HP 9000 Sy

Planning the FabricHyper Messaging Protocol (HMP)Chapter 242HMP Supported ConfigurationsMultiple HMP HyperFabric configurations are supported to match

Planning the FabricHyper Messaging Protocol (HMP)Chapter 243Figure 2-6 HMP Point-To-Point Configurations

Planning the FabricHyper Messaging Protocol (HMP)Chapter 244Enterprise (Database)The HMP enterprise configuration illustrated in Figure 2-7 is very po

Planning the FabricHyper Messaging Protocol (HMP)Chapter 245Figure 2-7 HMP Enterprise (Database) Configuration, Single Connection Between Nodes

Planning the FabricHyper Messaging Protocol (HMP)Chapter 246Figure 2-8 HMP Enterprise (Database) Configuration, Multiple Connections Between Nodes

Planning the FabricHyper Messaging Protocol (HMP)Chapter 247Technical Computing (Work Stations)This configuration is typically used to run technical c

Planning the FabricHyper Messaging Protocol (HMP)Chapter 248Figure 2-9 Technical Computing Configuration

Planning the FabricHyper Messaging Protocol (HMP)Chapter 249Figure 2-10 Large Technical Computing Configuration

Contents3Determining Whether an Adapter or a Cable is Faulty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146Determining

Planning the FabricHyper Messaging Protocol (HMP)Chapter 250

Chapter 3513 Installing HyperFabricThis chapter contains the following sections that describe installing HyperFabric:•“Checking HyperFabric Installati

Installing HyperFabricChapter 352•“Installing HyperFabric Switches” on page 66

Installing HyperFabricChecking HyperFabric Installation PrerequisitesChapter 353Checking HyperFabric Installation PrerequisitesBefore installing Hyper

Installing HyperFabricInstalling HyperFabric AdaptersChapter 354Installing HyperFabric AdaptersThis section contains information about installing Hype

Installing HyperFabricInstalling HyperFabric AdaptersChapter 355Online Addition and Replacement—HP-UX 11i OnlyOnline Addition and Replacement (OLAR) a

Installing HyperFabricInstalling HyperFabric AdaptersChapter 356There are two methods to add or replace OLAR-compatible cards:• Using the SAM utility.

Installing HyperFabricInstalling HyperFabric AdaptersChapter 357IMPORTANT In many cases, other interface cards and slots within the system are depende

Installing HyperFabricInstalling HyperFabric AdaptersChapter 358The card must also operate at the slot’s bus frequency. A PCI card must run at any fre

Installing HyperFabricInstalling HyperFabric AdaptersChapter 359• If the driver is static and not configured in the kernel, then the card cannot be ad

Contents4

Installing HyperFabricInstalling the SoftwareChapter 360Installing the SoftwareThis section describes the HyperFabric file structure and the steps nec

Installing HyperFabricInstalling the SoftwareChapter 361The global configuration file, which contains the IP addresses for each adapter and each Hyper

Installing HyperFabricInstalling the SoftwareChapter 362• /var/adm/OLDclic_logThe log file from the previous time the clic_start command was executed.

Installing HyperFabricInstalling the SoftwareChapter 363• /opt/clic/firmware/clic_fw_4x8cThe 4X PCI HyperFabric 8-bit CRC firmware. Note that this fil

Installing HyperFabricInstalling the SoftwareChapter 364Loading the SoftwareListed below are the steps you must follow to load the HyperFabric softwar

Installing HyperFabricInstalling the SoftwareChapter 365• hp-clic 3384/udp #clic switch managementNote that these lines are used by the HyperFabric so

Installing HyperFabricInstalling HyperFabric SwitchesChapter 366Installing HyperFabric SwitchesThis section contains the information you need to insta

Installing HyperFabricInstalling HyperFabric SwitchesChapter 367❏ After the HyperFabric switch is mounted in the rack, you attach the various cables t

Installing HyperFabricInstalling HyperFabric SwitchesChapter 368Steps for Installing the HF1 SwitchThis section contains information for installing a

Installing HyperFabricInstalling HyperFabric SwitchesChapter 369Step 2. Install and secure the rails in the rack, using two screws per rail. The figur

Tables5HF1 Speed and Latency w/ TCP/IP Applications 30HF2 Speed and Latency w/ TCP/IP Applications 30Supported Configurations for A6386A HF2 Adapter O

Installing HyperFabricInstalling HyperFabric SwitchesChapter 370Step 4. From the back of the rack, slide the switch—with the front of the switch facin

Installing HyperFabricInstalling HyperFabric SwitchesChapter 371that you cannot easily slide the switch. If so, try lifting the switch over the rail s

Installing HyperFabricInstalling HyperFabric SwitchesChapter 372Step 6. Tighten all four screws in brackets 3 and 4.bracket 3bracket 4front of switchf

Installing HyperFabricInstalling HyperFabric SwitchesChapter 373Step 7. From the front of the rack, push the switch so that it barely touches brackets

Installing HyperFabricInstalling HyperFabric SwitchesChapter 374Step 8. Once the switch is snug against brackets 3 and 4, push brackets 1 and 2 in tow

Installing HyperFabricInstalling HyperFabric SwitchesChapter 375Step 12. Plug the switch’s power cord into the rack’s power distribution unit (PDU), i

Installing HyperFabricInstalling HyperFabric SwitchesChapter 376Installing the HF2 SwitchThis section contains information for installing an HF2 switc

Installing HyperFabricInstalling HyperFabric SwitchesChapter 377Figure 3-4 Front of HF2 Switch (A6389A Switch Module Installed)You can install the HF2

Installing HyperFabricInstalling HyperFabric SwitchesChapter 378With the Rail KitHP recommends that you install the HF2 switch using the rail kit that

Installing HyperFabricInstalling HyperFabric SwitchesChapter 379One end of each rail has six screw-holes (End-A), the other end has two screw holes (E

Tables6

Installing HyperFabricInstalling HyperFabric SwitchesChapter 380NOTE HP recommends employing two people to support the weight of the switch because En

Installing HyperFabricInstalling HyperFabric SwitchesChapter 381Without the Rail KitAs mentioned earlier, HP strongly recommends installing the HF2 sw

Installing HyperFabricInstalling HyperFabric SwitchesChapter 382If you want to install another HF2 switch using the rail kit, go to “With the Rail Kit

Chapter 4834 Configuring HyperFabricThis chapter contains the following sections that describe configuring HyperFabric:•“Configuration Overview” on pa

Configuring HyperFabricChapter 484•“Deconfiguring a HyperFabric Adapter with SAM—HP-UX 11.0 and 11i 0nly” on page 96•“Configuring the HyperFabric EMS

Configuring HyperFabricConfiguration OverviewChapter 485Configuration OverviewYou do not need to configure the HyperFabric switch because the HyperFab

Configuring HyperFabricInformation You NeedChapter 486Information You NeedWhen you run the clic_init command or use SAM for configuration, you have to

Configuring HyperFabricInformation You NeedChapter 487❏ For each node in the fabric, you need the IP address of the node’s Ethernet LAN interface that

Configuring HyperFabricInformation You NeedChapter 488Configuration Information ExampleFor this example, we have added some “dummy” (that is, not vali

Configuring HyperFabricInformation You NeedChapter 4894. What is the subnet mask of the first adapter? (255.255.255.0)If you do not specify a value fo

Figures7TCP/IP Point-To-Point Configurations 33TCP/IP Basic Switched Configuration 34TCP/IP High Availability Switched Configuration 35TCP/IP Hybrid C

Configuring HyperFabricInformation You NeedChapter 4908. If switch management has been enabled, how many switches will be configured? As stated earlie

Configuring HyperFabricDoing the ConfigurationChapter 491Doing the ConfigurationAs explained in “Configuration Overview” on page 85, you must create t

Configuring HyperFabricDoing the ConfigurationChapter 492Using the clic_init CommandRun the clic_init command to create the global configuration file.

Configuring HyperFabricDoing the ConfigurationChapter 493Examples of clic_initSome examples of using the clic_init command are shown below.• Example 1

Configuring HyperFabricDoing the ConfigurationChapter 494Using SAM—HP-UX 11.0 and HP-UX 11iThis section describes how to use SAM to configure HyperFab

Configuring HyperFabricDoing the ConfigurationChapter 495• If you have just created the global configuration file on the local node for the first time

Configuring HyperFabricDeconfiguring a HyperFabric Adapter with SAM—HP-UX 11.0 and 11i 0nlyChapter 496Deconfiguring a HyperFabric Adapter with SAM—HP-

Configuring HyperFabricConfiguring the HyperFabric EMS MonitorChapter 497Configuring the HyperFabric EMS Monitor Starting with the December 2000 relea

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 498Configuring HyperFabric with MC/ServiceGuardHyperFabric supports the MC/

Configuring HyperFabricConfiguring HyperFabric with MC/ServiceGuardChapter 499NOTE Because the HyperFabric network does not currently support MC/Servi