Vendor Accent

ILM and intelligent data management

ILM helps companies strategise on how to manage data right from the time the data is generated to the time it is deleted from the systems,says Sai Gundavelli.

We are living in the age of ‘information overload’. Companies are relying heavily on enterprise planning systems like ERP, SCM and CRM to automate and manage their resources. These systems generate and host a vast amount of data which is well structured in its own way to fulfill a specific need. Apart from this structured data, enterprise also store massive volumes of unstructured data in the form of e-mail, IM, documents and images. The structured and unstructured information is required to be stored and retained within these systems for various strategic, business and regulatory requirements.

As information accumulates in the production servers, the performance of the systems deteriorate; storage need grows; disaster recovery, backups and upgrades take longer—leading to extended business outages. Today’s dynamic enterprises cannot afford these outrages. No wonder Charlie Gary from Meta group says, “Data is growing at 125 percent per year. In a typical enterprise, up to 80 percent of this data is inactive and yet remains in production systems to cripple performance.” This is what I call the data tsunami and it can be avoided by intelligent data management.

Enter ILM

Information by itself is considered a resource and enterprises need to plan effectively and put the right combination of strategy, software and hardware tools in place to avoid a data tsunami. Apart from the new data that is generated everyday, the strict data retention policies and legal regulations to retain transactional data over long periods is fuelling data growth. These ever-increasing volumes of inactive data which is retained for compliance affect the application performance, limit data access and strain storage infrastructure. This has resulted in the increased complexities in mission-critical IT environment and is a growing concern among businesses. This is where the increasingly popular concept of information lifecycle management (ILM) comes into picture.

ILM helps companies strategise on how to manage data right from cradle to grave—from the time the data is generated/captured to the time it is deleted from the systems.

The value of information keeps changing with time, processes, business and regulatory needs. This in turn affects the probability of usage of data. Data reuse has been one of the key metrics of ILM which helps strategise the storing of data on different tiers to cost-effectively optimise the storage infrastructure and enhance performance. A well-planned ILM strategy will allow the enterprise to retain all the reporting and access capabilities as if the data were lying on the same server.

Analysts have been scouting through experiences to come out with the best practices that would guide companies through the changing times of ILM. The experiences of various organisations clearly state that there are no definitive best practices. ILM means different things for different organisations. Nevertheless, there seem to be enough common issues that every organisation is coming across during its implementation of ILM strategies. Some of the key issues are as under:

Classifying data

The importance of data retention policies during ILM implementation is of key significance. The data value otherwise called data classification forms the foundation for a successful and efficient information management. The data retention policies need to have a buy-in from all the entities which own or use the data. Classification of data, which helps organise the data onto different tiers is probably the most important step for ILM implementation for any organisation.

Choosing the right storage tier

In a recent conference in California, Data Base Administrators complained that their senior management was misinterpreting the hierarchical storage management (HSM) and was looking forward to totally removing Tier 1 (production tier) from their IT environment. But, the Tier 2 storage could not handle data request of any real-time production environment. It was only for the data which was rarely accessed. Tiering the data should be for eliminating the unnecessary load on the production servers, improving performance and achieving optimised storage utilisation.

Restoring data

Businesses need to expect the unexpected and be prepared for any eventuality. The archived data is always in ‘Read only’ mode for compliance reasons. The software which enables the company to archive the data needs to allow for de-archiving the data into the production database without losing data integrity. This is necessary in case of editing requirements of the archived data (e.g. product recall). This is a key component in any ILM strategy.

Data security and compliance

The need for setting apt user and management level access privileges for data increases as we classify the data into various tiers based on its value. Only required users need to be given access to production, archive, or both depending on their responsibilities. Also, sensitive data (e.g. financial, health data) needs to be protected in production, archive and non-production environments (testing, development, and outsourcing).

One of the reasons for ILM to come into existence is compliance. Various regulatory bodies across the world have been coming out with their own version of governing data retention. For today’s global companies, ILM software should allow for incorporating any number of regulations without overriding the other and help achieve compliance

Data integrity

ILM requires that data of any value be available for immediate access for reporting and compliance purposes. A few regulatory bodies also require all the tiered data—say production and archived data to be accessed through the same application which created the data. This online seamless availability of data can be achieved only if data integrity and referential integrity are maintained during the hierarchical staging of data.

Many vendors are attacking the archive market from a packaged application perspective (e.g. Oracle Applications, PeopleSoft, SAP). But most companies will have a need to archive more than a single application; for this reason, users should evaluate the scope of packaged solutions. What companies need is a comprehensive enterprise archiving solution which covers both structured data as in packaged application like Oracle Apps and unstructured data like e-mail, IM, and documents.

Database archiving as a gateway to effective information management is gaining ground and according to a recent study, the data lifecycle market has the potential to reach $4 billion by the year 2007.

The author is CEO, Solix Technologies. He can be reached at sai.gundavelli@solix.com