Data reduction shows promise

The adoption of deduplication is slowly gaining ground in India on account of the savings in terms of space and energy utilization that accrue as a direct result of applying this technology. By Nivedan Prakash

In the past, tape has been the be-all and end-all when it comes to archiving the mountains of data that any large business generates. Over the years, these data mountains have grown to the point where the need for something more has been felt. Data deduplication technology, also called data reduction or commonality factoring, allows users to store more information on fewer physical disks. Used in conjunction with tape, it helps manage the data explosion by allowing organizations to retain frequently accessed data on disk and use tape for medium- and long-term archival.

Another factor spurring enterprises to go in for deduplication is the fact that with worries about data loss due to natural disasters, theft etc. on the rise, companies feel the need to store multiple copies of their data whereupon it is to their advantage to reduce the size of the data set that has to be replicated to begin with which is where deduplication can help.

Emerging trends

Indian companies are starting to capitalize on data reduction technologies like deduplication to reduce the disk capacity needed in the IT environment. While better manageability of data is an expected outcome, decision makers are also exploring this technology for cost benefits such as reduced electricity consumption and lower physical footprint.

Sanchit Vir Gogia, Associate Research Manager - Software, Springboard Research, asserted, “While deduplication has chiefly been integrated with backup systems, enterprises are now exploring the option of replicating this technology (and others) for primary storage data systems as well. Springboard believes that the drive by enterprises to reduce data duplication at the source will drive investments in deduplication over the next two to three years.”

Sandeep K. Dutta, VP - Storage, Systems and Technology Group, IBM India/SA, said, “By 2010, there will be a six-fold increase in the world’s information. The amount that vendors are charging for adding deduplication to existing backup software is another differentiator for some offerings. The trend is toward sub-record level deduplication. A wide range of systems that provide deduplication are already available. Along with dramatically reducing the consumption of backup storage space, these technologies cut restore time and eliminate the need to wade through incremental backup tapes. Most systems allow users to restore back to a specific date and time, and some make decentralized backups possible.”

“Data deduplication is the hottest technology in the storage area today. While it has primarily been significant in the backup and archival market, an emerging trend is to do data deduplication at the source and there are some vendors offering primary storage deduplication,” added B. Narayanan, Project Manager - Storage, American Megatrends India.

P.K. Gupta, Director and Chief Architect - BRS Practice, EMC, pointed out that the vendor was seeing considerable interest in all implementations of data deduplication. The maximum interest was coming from a need to provide single step recovery. With traditional backup software solutions, customers had the need to restore the last full backup and all the incremental ones that were taken after that. This is both time consuming and inefficient.

The Indian scene

Although businesses in the BFSI, telecom, and manufacturing segments have found high value in adopting this technology, new-age industries like retail and media have been slow in adopting the same. Software development firms and IT/ITES in general are also adopting this technology.

According to Aman Munglani, Principal Research Analyst at Gartner, although deduplication technology has been in the market for quite some time, it is still in a nascent stage of adoption. While some large enterprises have already adopted it, Gartner had yet to witness mass-scale adoption in India. The reason was that the customers had not been educated about this technology and they looked at it with skepticism.

While large enterprises remain the key buyer of deduplication, mid-market firms in India are increasingly grappling with compliance and data loss issues and are hence are starting to look at such technologies.

The tipping point for spending on deduplication solutions stems from larger projects around improving storage performance, virtualizing servers, and disaster recovery.

S. Sriram, CEO of iValue InfoSolutions, said, “Deduplication solutions are not only designed for larger projects or companies but also for any business which sees information as its asset and business differentiator and hence is keen to identify, manage and protect information.”

“The primary value proposition that deduplication offers is savings in storage space and network bandwidth. Both of these play a major role in determining operational cost when implementing disaster recovery in terms of the amount of storage space required on the DR side and the network bandwidth required when replicating between the primary site and the DR site,” opined Lakshman Narayanaswamy, Co-founder and VP - Products, Sanovi Technologies.

Pushing the cause

As data growth continues, so does the increase in the use of data deduplication solutions since the cost of storing and managing data multiplies silently. The cost of retention, compliance and management are the three primary driving factors here.

Adoption of deduplication also arises from the growing need for CIOs to invest in their overall backup and business continuity projects. The need to adhere to regulations and the resultant business risks are also prompting Indian CIOs to look at such technologies. Reducing the amount of data flowing on the corporate network reduces stress on internal networks and improves the availability of bandwidth; it also reduces the cost involved in managing large pools of data.

“The key reason as to why this technology has found acceptance is because organizations are experiencing tremendous growth in data captured at various points like customer care centers, retail outlets, etc. Having a unified view of data is increasingly becoming important to ensure that customers can be served in the best possible manner. Projects like UID and other e-governance projects are expected to push the demand for data deduplication,” added Gogia.

This technology helps reduce floor space required by enterprise storage arrays and lower energy consumption by decreasing storage capacity requirements (in the case of source-based deduplication where the data reduction occurs on the storage array itself). It also helps minimize administrative overhead by retaining data on disk for longer periods of time. Deduplication can provide substantial cost benefits through reduced storage capacity requirements.

Meanwhile, the increasing demand by IT buyers for greater storage efficiencies will drive the adoption of deduplication solutions over the next 12 months. With virtual tape libraries using deduplication, companies can save on labor as it does away with backup job failures on account of bad tapes.

Moreover, companies are spending less money on storage capacity because they are storing less data, so there’s a cost-efficiency aspect as well.

Another benefit is that more data can be kept available and online in a virtual tape library with deduplication. Instead of only keeping a week’s worth of backups on the virtual tape library and relying on tapes for older data, companies can keep several months of backup data on their virtual tape library, making it easier to restore older data.

Other advantages

Some of the potential advantages of this technology include reduced storage capacity required for a given amount of data, ability to store significantly more data on given amount of disk, restore from disk rather than tape may improve ability to meet recovery time objective (RTO), network bandwidth savings (some implementations), and lower storage-management cost resulting from reduced storage resource requirements.

“Data deduplication at source eliminates the unnecessary transmission of redundant backup data that is sent over the network and stored. This defuses the enormous amount of data growth both in core data centers and at remote offices. Deduplication at the source can shrink the amount of time required for backup, improve performance of production storage (it gets freed faster) as well as shrink network utilization substantially. Deduplication at the source stops the avalanche of full and incremental backups before it forms, reducing backup times, client resource consumption and network utilization,” asserted Gupta.

IBM is delivering deduplication across its storage portfolio. “The IBM ProtecTIER family of deduplication solutions are easy to deploy and cost effective for Open Systems, System i and System z environments,” said Dutta.

Growth prospects

In the last few years, India Inc. has seen epic growth in data. Information is the key asset and differentiator for every business and hence success or failure depends on how it is managed. Management costs have tripled over the last few years while hardware costs have remained almost static. Hence, most companies have started realizing the relevance and importance of deduplication.

As per industry estimates, since companies are in the early stages of adopting deduplication, the growth potential for this technology is expected to be on the higher side with a CAGR of 50%+ for the next few years. After making its mark worldwide, deduplication has been adopted across industries and by companies of various sizes in India as well.

Data deduplication will be driven by storage growth even in India. It can automate the disaster recovery process by providing the ability to perform low-bandwidth site-to-site replication at a lower cost. The emergence of data deduplication technology is helping to change the fundamental nature and economics of data protection for enterprise IT organizations. In particular, it is at the forefront of a major shift in the way that organizations deploy and maintain their backup infrastructure.

nivedan.prakash@expressindia.com