Building and operating a supercomputer at more than three miles above sea level poses some unique problems, the designers of the recently installed Atacama Large Millimeter/submillimeter Array (ALMA) Correlator discovered.
Why build a supercomputer at 16,000 feet? Because the ALMA computer serves as the brains behind the ALMA astronomical telescope, a partnership between Europe, North American, and South American agencies. It’s the largest such project in existence. Based high in the Andes mountains in northern Chile, the telescope includes an array of 66 dish-shaped antennas in two groups. The telescope correlator’s 134 million processors continually combine and compare faint celestial signals received by the antennas in the ALMA array, which are separated by up to 16 kilometers, enabling the antennas to work together as a single, enormous telescope, according to Space Daily.
Funded by the US National Science Foundation (NSF), and designed, constructed, and installed primarily by the National Radio Astronomy Observatory (NRAO), the back-end correlator is tuned for signal processing. The four “quadrants” of the correlator can each process data coming from 504 antenna pairs. ALMA’s Website reports that signals are processed via tunable filter bank cards, four of which are needed per antenna. Each tuner card can “slice and dice” the perceived spectrum that the antennas “see,” allowing them to make specialized observations.
(Fifty radio antennas make up the main ALMA array; an additional array of 16 antennas, called the Atacama Compact Array (ACA), is provided by the National Astronomical Observatory of Japan (NAOJ) and has its own Fujitsu-designed correlator; there’s a 2010 paper describing that design.)
All told, the supercomputer includes 134 million processors and performs up to 17 quadrillion operations per second, according to Space Daily. That would put it close to the top of the semiannual TOP500 supercomputer list, which ranks the world’s most powerful supercomputers; since ALMA Correlator is a special-purpose machine, however, it won’t qualify.
The extreme high altitude makes it nearly impossible to maintain on-site support staff for significant lengths of time, with ALMA reporting that human intervention will be kept to an absolute minimum. Data acquired via the array is archived at a lower-altitude support site.
But the altitude also poses other challenges. For one thing, the actuators that glide above the surface of a hard disk no longer operate properly, requiring the use of solid-state disks. The thin air also poses a cooling problem, requiring twice the normal airflow to sufficiently cool the machines (which draw some 140 kilowatts of power). Seismic activity is also common in the area, so the correlator had to be designed to withstand the vibrations associated with earthquakes.
The altitude also limited the construction crew’s ability to actually build the thing, requiring 20 weeks of human effort just to unpack and install it. “There are thousands upon thousands of cable connections that we had to make, and every one of our cables is the same color blue, so I’m just glad we devised a good labeling system while at sea level,” NRAO’s Rich Lacasse, leader of the ALMA Correlator Team, wrote in a statement released by the agency.
What lessons can data-center designers learn from ALMA? Probably not too many, given the specialized nature of the machine. But if nothing else, building a high-altitude facility to take advantage of lower ambient temperatures may be more trouble than it’s worth. The facility will be formally inaugurated in March.