NSF-funded leadership-class computing center boosts U.S. science with largest academic supercomputer in the world

Today, the Texas Advanced Computing Center (TACC) at The University of Texas at Austin (UT Austin) launched Frontera, the fastest supercomputer at any university and the fifth most powerful system in the world. The system is the result of a $60 million investment by the National Science Foundation (NSF) to advance the next generation of leadership class computing.

Joined by representatives from NSF, UT Austin and technology partners Dell EMC, Intel, Mellanox, DataDirect Networks, NVIDIA, IBM, CoolIT and Green Revolution Cooling, TACC inaugurated a new era of academic supercomputing with a resource that will help the nation's top scientists explore science at the largest scale and make the next generation of discoveries.

"Scientific challenges demand computing and data at the largest and most complex scales possible. That's what Frontera is all about," said Jim Kurose, assistant director for Computer and Information Science and Engineering at NSF. "Frontera's leadership-class computing capability will support the most computationally challenging science applications that U.S. scientists are working on today."

First announced in August 2018, Frontera was built in early 2019 and earned the number five spot on the twice-annual Top500 list in June, achieving 23.5 PetaFLOPS (one thousand million million floating-point operations-per-second) on the high-performance LINPACK benchmark, a measure of the system's computing power.

Frontera has been supporting science applications since June and has already enabled more than three dozen teams to conduct research on a range of topics from black hole physics to climate modeling to drug design, employing simulation, data analysis and artificial intelligence (AI) at a scale not previously possible.

Olexandr Isayev, a chemist from the University of North Carolina, used Frontera to run more than 3 million atomic force field calculations in less than 24 hours -- a major achievement in high-speed quantum computation. The calculations are part of an effort to train an AI system that can predict the likely characteristics of new drug compounds and identify compounds with the ability to target specific cells.

"It's a great machine, especially for quantum mechanics applications," Isayev said. "We're really looking forward to running large-scale calculations that were not possible before."

Ganesh Balasubramanian, an assistant professor of mechanical engineering and mechanics at Lehigh University, has been using Frontera to study the dynamics of organic photovoltaic materials and model manufacturing conditions.

"The lightning speed at which Frontera performs computations is very beneficial," said Balasubramanian, who during the early user period experienced a five time speed-up in his simulations of solar material manufacturing. "Overall, the entire pace of computational research will be increased by the arrival of Frontera."

Manuela Campanelli, an astrophysicist at the Rochester Institute of Technology, has been using Frontera to perform the longest simulations ever of the merger of neutron stars, including for the 2017 event detected by the Laser Interferometer Gravitational-Wave Observatory (LIGO), the Europe-based Virgo detector, and 70 ground- and space-based observatories.

"Frontera is an amazing system because it gives us a very large number of computer nodes that we can use to solve very complex problems," Campanelli said. "These types of resources are unavailable on most university campuses, so we really need to have Frontera in order to be able to do the simulation we do."

Frontera combines Dell EMC PowerEdge servers with 8,008 compute nodes, each of which contains two, second generation Intel Xeon scalable ("Cascade Lake") processors, totaling more than 16,000 processors and nearly half a million cores, connected by a 200 gigbit-per-second HDR Mellanox InfiniBand high-speed network.

The system incorporates innovative flash storage from DataDirect Networks and novel cooling systems from CoolIT, Cooltera and Green Revolution Cooling (GRC) and employs several emerging technologies at unprecedented scales, including high-powered, high-clock rate versions of the latest Intel Xeon processors, Intel Deep Learning Boost, Intel Optane memory and several kinds of liquid cooling.

In August, Frontera added two new subsystems to provide additional performance and to explore alternate computational architectures for the future. A 360 NVIDIA Quadro RTX 5000 GPU (graphics processing unit) system submerged in liquid coolant racks developed by GRC will explore more efficient ways to cool future systems, as well as explore single-precision optimized computing. An IBM POWER9-hosted system with 448 NVIDIA V100 GPUs will provide additional performance and provide the top-performing GPU architecture with tight coupling to the processor and memory subsystems. These additional systems will accelerate AI, machine learning and molecular dynamics research for Frontera researchers in areas ranging from cancer treatment to biophysics.

In the coming months, Frontera will integrate with cloud providers Microsoft, Google and Amazon to provide researchers access to emerging computing technologies and long-term storage.

Frontera (Spanish for "frontier") will operate for at least five years and will support hundreds of research projects and thousands of researchers in nearly every field of science over its lifetime. It is expected to have a major impact on:

• Natural hazards modeling -- predicting the trajectory and intensity of storms, and helping to design infrastructure that can withstand the strongest disasters;
• Genomics -- including precision agriculture to feed the world's growing population;
• Energy research -- from fusion to solar power to cleaner coal.
• Astrophysics -- including multimessenger astronomy and gravitational wave modeling.
• Materials sciences -- using a combination of modeling and deep learning to accelerate the development of new molecules for medicine and engineering.

Projects will be selected through a competitive application process and researchers will need to show that they require a computer at the scale of Frontera to solve their problems.

Faculty at the Oden Institute for Computational Engineering and Sciences at UT Austin are leading the world-class Frontera science applications and technology team, with partners from the California Institute of Technology, Cornell University, Georgia Tech, Ohio State University, Princeton University, Stanford University, Texas A&M University, the University of Chicago, the University of Utah and the University of California, Davis.

Frontera will serve as a workhorse for the largest and most experienced computational users in the nation and a training ground for the next generation of scientists.

"Academic researchers have never had a tool this powerful to solve the problems that matter to them," said Dan Stanzione, TACC executive director. "We are proud to launch Frontera -- a new, national resource for science that will power the discoveries of the future."