Lead

Detecting software defects

Software testing provides functional assurance and business value. Vinita Gupta writes how organizations are now looking at testing for providing not just quality control, but also quality assurance.

There have been several cases when businesses have suffered significant financial and credibility losses due to deployment of under-tested systems. In addition, the cost of fixing these systems during production is phenomenally higher. As per the National Institute of Standards and Technology (NIST) of the US Department of Commerce, industries in the country have incurred losses of about $60 billion in 2002, due to software defects.

It has become important for IT applications to run without any disruptions and be available at required times and places. Moreover, they must be easy to handle, be scalable to the loads and enforce the applicable regulatory compliances without compromising on the security of the system.

Customers have therefore started perceiving the need for doing independent testing, which would ensure that a third party who is not pressurized by the development load and has established strong testing processes be involved at an early stage of development/maintenance cycle.

Dilip Dhanuka, Senior VP and Head-Products and Technology Initiatives, Patni, points out that five years ago, there was no independent testing market. The testing was seen as an integral part of the development or maintenance process. “Testing was considered a fag-end activity and thought about last. However, due to increased complexity of the applications and the technology stacks; reduced tolerance for the defects due to deployment of IT systems in more critical and infrastructure applications; the situation has changed.”

As per Gartner estimates the global software testing market will be $13 billion by 2010. The current global market size of outsourced testing services could be pegged at around $6.1 billion. India has the potential to corner 70 percent of the worldwide market.

Testing in SDLC

The testing plan needs to begin in the early phase of Software Development Life Cycle (SDLC.) There is no need for organizations to wait till the development is over, to begin testing. The sooner the fault is found in the SDLC, the lesser it costs to fix it.

Gangadharaiah C P, Senior VP and World Wide Head of Testing Services, Wipro Technologies adds, “Involving the testing team early in the SDLC helps in fixing bugs and defects upstream and reducing the cost of fixing these defects. Also clear sign offs on test plans are extremely important to establish the deliverables, timelines, measurement metrics.”

At AppLabs, a software testing company, the team utilizes five-steps:

• AppLabs test methodology, wherein they perform test requirements analysis to understand the testing needs and environment
• Test planning to define test strategy and schedule
• Test designing to author test case/script
• Test execution of test cases/scripts and comprehensive
• Test reporting to provide an assessment on product quality
• The team typically comprises of test architects, test leads and test engineers.

Sai Chintala, VP, Testing Services, AppLabs, reveals that the test bed is created based on the application requirements. Some of the testing for large enterprise applications is performed over the Web.

Hexaware Technologies has implemented an enterprise-wide project management tool called Plan Arena, which covers end-to-end project management process, e.g. project planning, tracking, risk management, defect and SLA management. Testing projects are also being managed through this tool.

Slowly but surely, the software testing industry is starting to evolve using models such as Test Maturity Model (TMM), Metrics-based TMM and the Test Process Improvement (TPI), which address the gaps that the testing industry had felt but were not met by the CMM initiatives that they had embarked upon.

Dhanuka believes that the adoption of these processes would improve the testing system and help organizations take their testing processes to the next level. “Other areas that have significantly improved the testing processes are the advances in the test automation industry, where with the automation tools are being able to handle more and thereby produce significant productivity gains,” he states.

Companies have offshore, onshore, near shore options available for independent testing services. The ratio depends on the organization’s requirements, complexity of the project, ease of replicating environments and security issues. Typically, companies have 80 percent offshore and 20 percent onshore ratio mix for testing.

Scott R Sargent, Senior Executive, System Integration and Technology Group, Accenture, and Lead, Global Delivery Excellence Practice, Accenture, asserts, “Companies are outsourcing testing services depending on the skills, experience and capacity of third-party vendors. The stand-alone model for testing services has a huge potential in the market.”

Challenges faced in software testing

Lack of project-specific test processes No common test governance model Managing multiple projects across different divisions Definition of common SLA parameters across projects in the same engagement Lack of a common testing team across the lifecycle of the project Monitoring test tool usage/licenses management Consistent metrics tracking across all projects Perception of being low-end manual work

Software testing management

Software testing management is done by: Having a well-defined entry-exit criteria for each phase in SDLC Having a single point of contact from testing team and development team for formal communication purposes Sign-off process when development hands over the code to test and when testing team hands the tested code to the client for User Acceptance Testing (UAT). Presence of a dedicated test manager who manages the entire testing activities for the account Clear metrics defined for the entire project

The testers

As testing evolves, one can see interesting ramifications in the skill-sets that are required in the industry, such as increased analytical skills and higher client engagement abilities. Quality people with intensive skills are required in the field of application testing. India as a destination has quality talent with the right capabilities and has people with an inherent passion for testing.

Wipro has 6,300 plus testing professionals. Gangadharaiah adds, “We hire graduates from engineering, computer science or mathematics backgrounds. We have a rigorous training programme for all our new hires. This includes training on testing basics, tools and domain knowledge.”

Using testing tools is also a skill, but a different one from testing. Hence testers need to be trained and certified on these tools.

Patni has Test Engineers who are trained and proficient in execution of functional and automation tests. They are trained to use the common tools that are required in the testing industry. Later they grow to become either Test Analysts/Specialists or Test Team Leads, depending on whether they chose a delivery or a solutions track and are responsible for test planning, test case development and creating test frameworks and strategies.

Open Source testing

One of the chief concerns while choosing a testing tool is the cost of the tool itself. Since software testing usually attracts the least budget in an organization, many people look to the Open Source Software (OSS) community for good solutions, but OSS testing tools may not have all the features and technical support that are required.

Ramanan R V, Chief Software Architect, Hexaware Technologies, believes that OSS testing tools are better than the proprietary software testing tools. OSS tools can be matched to a customer’s requirement as most are specific to a technology. For example, Jmeter is a load testing tool specifically for testing Java applications. “On the other hand, proprietary software testing tools like QTP (Quick Test Pro), Load Runner which are Mercury products and tools from IBM Rational are better suited for testing multiple technologies. For example, QTP supports technologies like Java, .Net, VB and more,” adds Ramanan.

Patni uses various OSS testing tools such as XUnit Family, Wiki-Wiki, Fit/Fitnesse, DBfit, XML Starlet, etc. For the company, this experience has provided significant advantages as the source code is openly published and hence can be tweaked to meet specific requirements for its clients. However, as most of these tools are not supported they require a fair degree of investment in terms of training and maintenance.

“OSS testing tools find limited and specific usage, especially where proprietary tools have not yet delved in,” points out Dhanuka.

Wipro evaluates tools on a case to case basis. If an OSS tool suits test project requirements, they would use it.

Proprietary software is predominantly being used in the test management, functional and performance areas. For white box testing areas, OSS tools are used.

Using OSS technology, AppLabs has recently developed a test management tool named TEMA that provides dashboards on testing progress. As a part of its contribution to the OSS community, in the company would make TEMA available for global users in the near future.