Technologies

Supercomputing for the masses

HPC has been a catalyst for advanced research in India and its adoption is rising across verticals says Varun Aggarwal

HPC has been at the forefront of many prestigious national projects in the country. Our enhanced capabilities in atmospheric and ocean research (which helps predicting weather with increasing accuracy), or harnessing information from space research and experiments are standing testaments to this fact. In the private sector, various HPC installations play a major role in helping both multinationals and Indian companies succeed in setting-up research facilities in the areas of automotive and aerospace engineering and pharmaceuticals. The evident success in many areas resulted from smart leverage of skills and superior infrastructure including those provided by HPC.

The market today is showing a strong influx of HPC users in the industry, as rapidly increasing demand for more processing power and storage capabilities comes up. Also related is the growing usage of commodity components in processors, interconnects etc.

IDC estimates that APAC is one of the world’s largest markets for HPC installations and is the fastest growing HPC market worldwide. HPC usage in the region is being propelled by strong demand from the Indian market. With the initial market barriers of cost and complexity resolved, Indian enterprises, just like the global scenario, are heading for wider and more mainstream adoption of HPC solutions. The advent and rapid rise of outsourcing engineering, simulation and animation projects is giving further impetus to the Indian HPC story. Additionally, the partner ecosystem is today indeed more capable than ever to deliver and integrate these solutions.

Karthik Ramarao, Director Technology, Systems, Sun Microsystems, opined, “HPC started out as a cluster-of-workstations solution. However, today it has evolved into a business where customers are looking at dedicated resources, which are able to churn out high compute performance. HPC has been prevalent largely in the education and research and development space but what we have seen in the last couple of years is that HPC is there in pharmaceuticals, medical, healthcare, BFSI and engineering.” Therefore, HPC is pretty much there in the commercial side of the business. In fact, there is so much potential for this technology in the commercial side that there is a large opportunity for growth, added Ramarao.

In the media and entertainment (M&E) sector, increasing digital data and content brings an ever-increasing need for systems to process, store, analyze and manage data. “With the growing demand for CGI, animation and gaming content, companies in the M&E industry such as Maya Entertainment have been early adopters of HPC solutions. In fact, Dawning Information Industry Company, that manufactures some of the world’s fastest high-performance servers and supercomputers (based out of China) decided to switch to Windows HPC Server 2008 to run its next-generation computer, the Dawning 5000A,” Pallavi Kathuria, Director, Server Business Group, Microsoft India said. Dawning owns more than 30% of China’s HPC market.

There is a need for a high performance computing infrastructure in the healthcare industry to conduct research, develop new drugs, collaborate among institutions, comply with regulations, and handle the explosion of digitally generated medical and clinical information. Today, many of the healthcare institutions are looking to deploy Grid Medical Archive Systems, which goes a long way in enabling medical image management systems.

x86 drives the HPC market

x86-based systems will accelerate adoption and deployment of HPC clusters. First, from a cost of ownership perspective–the barrier to large scale adoption of HPC has been the prohibitive cost of setting up these clusters and x86-based systems and other commodity components in interconnects help address that. Second, the ease of managing and deploying clusters is an important requirement that most customers talk about and this is something that commodity components will help address. Lastly it is easier to manage.

Kathuria opined, “Poor management can lead to increased downtime, reduced agility, and increased cost of ownership. Microsoft System Center Server Management Suite Enterprise (SMSE) contains a set of solutions which work together to transform a company’s HPC environment into a strategic asset allowing efficient, automated management of the entire HPC data center from a single set of tools. Windows HPC Server 2008 provides a centralized management and deployment interface with template-based deployment capabilities, and a Network Configuration Wizard that helps to simplify network and topology setup and configuration. It includes key features, such as high-speed networking, scalable cluster management tools, advanced failover capabilities, a service-oriented architecture (SOA) job scheduler, and support for partners’ clustered file systems.”

Subram Natarajan, Senior Consultant, STG–IBM India/SA, said, “The advancement of clustering technology on the x86 platforms, the affordability factor and stack of tools that are available in the open domain provides an unique environment for innovations in this area. x86-based HPC clusters lower the entry barrier for people who want to set up HPC clusters; and that has lead to the proliferation of this technology.”

Storage gets bundled with HPC

Along with the commoditization of x86 in the HPC space, what has also happened is that companies have had an equal interest in storage to get better performance. Ramarao said, “Customers are looking at not just at distributed clusters for compute but also distributed storage so that they can build storage out of off-the-shelf products and get extremely high performance from [relatively inexpensive devices]. The parallel file system called Lustre is becoming popular. We are using Lustre in a way that storage has never been used in the past.”

Many high performance applications require heavy storage communication. CPU horsepower alone is not enough. Just as you need a lot of memory, you also need a lot of fast storage, to deliver the kind of performance required. Time wasted in getting the data from a dedicated storage system to where the computing is done means that there is a loss or wastage of compute cycles. Therefore, splitting storage across multiple computers and running file systems in parallel so that data becomes available faster is important.

According to Ramarao there are high performance applications such as Linpack that do not require storage. Unfortunately, some of the biggest measurement criterions for HPC are based on those benchmarks that are CPU cache sensitive or at the most CPU memory sensitive. However, in the real world, not many applications fall under this category—they use humongous data sets.

There is an interesting development happening on the networking side as well. We used to look at Ethernet in the past, then we had Infiniband, then we have the 10 GbE today. So the networking is becoming important. It is also cheap. Infiniband used to be quite expensive in the past—it is much more affordable today. The 10 GbE has all the benefits of the Infiniband plus the bandwidth in 10 GbE is higher. Therefore, high performance as an environment is not just about compute, several of these other components need to mature. The CPU side has matured but other parts need to evolve, as storage and networking are weak links today. Once we have good technologies to help evolve these components of HPC, then we will be able to extract better performance out of the clusters.

R Ravichandran, Director–Sales, Intel South Asia, added, “At the platform level, today there is a higher level of maturity for a standards based platform, interconnect, memory and I/O. Intel has been following Moore’s law and we continue to drive significant performance on a standard building block.” He continued “Four years back, the bulk of computing power that was available was on single core; today quad-core is pretty much the mainstream, and the price points are extremely competitive on not only the performance side, but also performance per watt and power consumption. The greatest benefit of a standard platform is that you can scale as much as you want without any trouble.”

He continued that you see a lot of easy to use programming tools and parallel programming is not a challenge anymore as many applications are ready to take advantage of multi-core and multi-threading capability of a platform. Intel’s software solutions group helps ISVs with tools, with programming techniques and technologies, so that developers can take advantage of the platform. A combination of these tools has brought HPC into the mainstream today and there is a higher level of comfort with HPC than was the case about three years ago.

IBM continues to invest in the HPC industry with offerings such as IBM BladeCenter and Blue Gene, which address critical issues of power and space consumption, scaling, integrated networking, and centralized systems management. Natarajan explained the company’s HPC strategy which is simple—IBM is dedicated to solving the most challenging and complex problems at lower cost, lower energy consumption, enable its clients to innovate and gain competitive advantage with innovative technology. IBM’s portfolio of Deep Computing solutions is extensive and includes systems based on POWER, the Cell Broadband Engine, as well as Intel and AMD processors. IBM strongly supports open standards, and the entire server product line is enabled to run Linux. The Deep Computing portfolio includes high performance systems, storage hardware and software, HPC software and tools, on demand and grid computing, and visualization solutions.

Going ahead

Talking about IBM’s HPC roadmap, Natarajan said that the fastest supercomputer in the world ‘Roadrunner’ can deliver sustainable 1 PF performance. In Dec 2008, IBM announced plans to develop two new supercomputers—Sequoia and Dawn—to be used at the Lawrence Livermore National Lab. Sequoia, with speeds exceeding 20 petaflops, will be about 15 times more powerful than any supercomputer in use today. It is also more powerful than the combined power of today’s entire Top500 list of supercomputers.

Products based on Intel’s Larrabee will target the personal computer graphics market and are expected in 2009 or 2010. Larrabee will be the industry’s first many-core x86 Intel architecture, meaning it will be based on an array of many processors. The individual processors are similar to the Intel processors that power the Internet and the laptops, PCs and servers that access it.

Larrabee is expected to kick start an industry-wide effort to create and optimize software for the dozens, hundreds and thousands of cores expected to power future computers. Intel has a number of internal teams, projects and software-related efforts underway to speed the transition, but the tera-scale research program has been the single largest investment in Intel’s technology research and has partnered with more than 400 universities, DARPA and companies such as Microsoft and HP to move the industry in this direction.

In the next few years as the HPC systems becomes affordable a large number of projects will be deployed making it a commodity.

varun.aggarwal@expressindia.com