IBM p5-p575 - Bluevista

The Computational and Information Systems Laboratory (CISL) took delivery of a new IBM p5-575 supercomputer on August 27, 2005, in Phase 3 of NCAR's Advanced Research Computing System (ARCS) subcontract with IBM.

Bluevista was based on IBM's POWER5 processor and high-performance switch technology, had approximately the same sustained computing capacity as Bluesky, an IBM POWER4 system, and nearly doubled the number of computational cycles available to NCAR's users.

At 1.9 gigahertz (GHz), the POWER5 processor ran at a faster clock speed than the POWER4 processor (1.3 GHz). The POWER5 processor could also sustain a higher memory bandwidth. As a result, NCAR applications on Bluevista showed enhanced performance, sustaining a higher percentage of the system's peak computational capacity.

Bluevista consisted of 78 IBM POWER5 p5-575 symmetric multiprocessor (SMP) nodes. Each node had eight POWER5 simultaneous multithreading (SMT) processors and 16 gigabytes of memory.

Bluevista's nodes were allocated as follows:

• 72 nodes for computational workload
• 2 nodes for interactive login sessions
• 4 nodes for file system input/output, management of the 55-terabyte disk subsystem, and connectivity to NCAR's Mass Storage System

All nodes were interconnected with IBM's pSeries High-Performance Switch (HPS), which was known previously as the IBM “Federation” switch. The HPS provided unidirectional, point-to-point communication bandwidth of 2 gigabytes per second per link with a latency of seven microseconds. Each node had two bidirectional link interfaces to the HPS.

While Bluevista provided roughly the same sustained computational capacity as Bluesky on NCAR models, it was a denser, hotter system. Bluevista required 276 kilowatts of power to operate (the average U.S. home consumes around 10.5 kilowatts). It occupied only a third of the floor space as Bluesky, and required just two-thirds of Bluesky's power and cooling.

CISL initially allocated Bluevista to experimental research for the Nested Regional Climate Model (NRCM) project. NRCM was a joint collaboration between NCAR's Mesoscale and Microscale Meteorology Division and the Climate and Global Dynamics Division. Researchers used Bluevista to study the effects of tropical convection on hurricanes and other tropical weather systems, using a regional climate model embedded within a global model. This work was relevant to the question of whether global climate change could induce more Category 4 and 5 hurricanes (the most powerful). The problem was so computationally demanding that it occupied all of Bluevista's available time from October through December 2005. CISL made Bluevista available for general use in January 2006.