Software/ Open Source

Clustering on SUSE Linux

HRI employs SUSE Linux Enterprise Server to power its supercomputing clusters, enabling globally recognized research projects in cosmology, high-energy physics and condensed matter physics, writes Nivedan Prakash

Founded in 1975 and located in Allahabad, India, the Harish-Chandra Research Institute (HRI) is dedicated to advanced research in mathematics and theoretical physics. The academic community at HRI consists of more than 30 faculty members, 20 post-doctoral fellows, and 40 graduate students. The institute is funded by the Department of Atomic Energy.

Pain points

Since 2000, when it created a 12-node cluster running Linux, HRI has made significant use of cluster computing to perform large and complex calculations. The institution is engaged in a variety of research projects, and requires different types of computing resources tuned for different computational tasks, including linear algebra involving very large matrices, Monte Carlo methods in up to ten-dimensional parameter spaces, and fast Fourier transforms. Jobs may vary from a single computation for 10 hours on a single core, to a full month on 64 cores.

When the time came to add a new 240-core cluster to its facilities, HRI needed to ensure that the new system would offer excellent price-performance, easy management and be a good fit with its existing clusters. The institution also wanted to ensure that the new cluster would be able to make full use of the latest CPU performance optimization features from Intel.

Choosing Novell

After reviewing options for its new cluster, HRI chose SUSE Linux Enterprise Server running on Intel Xeon processors and it also deployed SUSE Linux Enterprise Desktop on eight workstations for data visualization.

“HRI has used Linux-based clusters for nine years, and we have always run a SUSE distribution. This led us to move to SUSE Linux Enterprise Server,” said Jasjeet Bagla, Professor at the Harish-Chandra Research Institute.

Bagla added, “As we look to squeeze the best possible performance out of our new hardware, it’s vital to have an up-to-date C compiler and libraries that can take full advantage of the latest CPU optimizations. SUSE Linux Enterprise Server is the only Linux distribution that combines the most up-to-date compiler with full enterprise-class support. ”

HRI regularly tests other distributions of Linux to ensure that its chosen platform remains on top after nine years. For the latest cluster, the institute saw as much as a 20% performance differential between SUSE Linux Enterprise and other distributions on dual-core, dual-processor systems. After recompiling the kernel for its quad-core systems, HRI saw even larger gains.

“We have experimented with a variety of other Linux distributions in the past, and we have always found the tuning of the kernel in SUSE Linux Enterprise to be much better, in particular for clusters. Equally, SUSE Linux Enterprise offers the stability we need to keep maintenance to a minimum, and impressive speed in terms of upgrades and bug fixes,” pointed out Bagla.

With four distinct clusters, ranging in size from 32 cores up to 640 cores, across approximately 200 physical servers, the IT infrastructure at HRI is becoming more complex. The institute has also selected Novell ZENworks Linux Management to help manage software updates across clusters.

Additionally, HRI plans to use the Novell software to create a local repository for patches and updates, avoiding excessive Web traffic caused by numerous machines, all downloading the same code at the same time.

Reaping the benefits

HRI is a relatively small organization and is totally focused on research. As its clusters are designed and managed by the scientists who work on them, it is crucial to ensure not only high performance, but also stability and ease of management. With SUSE Linux Enterprise Server, the institute has the right combination of performance and availability.

“SUSE Linux Enterprise Server gives us all the performance we need without distracting us from our research. The availability is excellent and we run for months on end with no interruptions. HRI contributes to a number of important global research projects, including models for the Large Hadron Collider and data analysis using the cosmic microwave background data from NASA’s WMAP mission, and so it’s enormously valuable to have a reliable cluster operating system,” asserted Bagla.

“At HRI, we are using cluster computing to help us understand events that took place shortly after the Big Bang, tapping into global sources of data to improve our model,” said Bagla. “The more detailed our model, the more points we need to simulate and the performance of SUSE Linux Enterprise Server means we can keep improving our capabilities.”

HRI is helping to push back the boundaries of knowledge in a number of fields, including modeling the birth of galaxies during the early history of the universe. Jasjeet Bagla’s team uses the SUSE Linux Enterprise-based cluster to simulate the formation and subsequent evolution of galaxies.

nivedan.prakash@expressindia.com