Opinion

Measuring outcomes in e-Governance

Neel Ratan writes that it is necessary to measure the outcome of a project as it leads to systematic improvements in the same

- Neel Ratan Executive Director and e-governance practice leader, PricewaterhouseCoopers

Governments around the world are leveraging advances in Information and Communication Technologies (ICT) to enhance their productivity and efficiency while simultaneously, upgrading their service delivery mechanisms to meet the aspirations and expectations of citizens in the 21st century as well as gain competitive advantage over other nations in attracting investments. However, this indulgence in creation of IT infrastructure, training public servants and development of software applications entails huge expenditure. Making room for the required investments and then justifying the same becomes arduous because the return on investment for such projects cannot be measured in the same way as conventional commercial projects. Therefore, the identification of targeted outcomes and measuring the same becomes important.

Before talking about outcomes, it is essential to appreciate the difference between outputs and outcomes. While the output of an e-Governance program could be electronic delivery of ‘n’ number of services using Internet, mobile or call center the outcome would be a rise in satisfaction levels of the customers due to enhanced convenience, faster turnaround times, smaller queues or higher compliance on account of bringing down the cost of compliance or reduction in per transaction costs of the departments due to higher productivity and efficiency of the department. This understanding of outcomes ensures that the e-Governance practitioners remember the fact that e-Governance is not an end in itself but a means towards the end goal of ensuring higher customer satisfaction, strengthening the services delivery capabilities of the government and engaging citizens in governance.

In light of the above, desired outcomes for e-Governance projects may be measured in terms of two categories:

Government transformation: The introduction of e-Governance in government departments helps the departments to improve their performance in various ways. Some of these are:

• Better planning and informed decision making due to timely availability of information and trends
• Reduction in footfalls at the offices due to availability of other avenues such as the Internet, contact center and Common Service Centers
• Better utilization of time of the departmental staff due to lesser queries from customers as information related to procedures, status etc. may be made available through other channels
• Better compliance due to the ability to electronically verify cases and cull out defaulters
• Decrease in the cost of delivery of services or the ability to deliver services to more people without proportional increase in costs

Citizen satisfaction: It needs to be understood that e-Governance is not an end in itself, it is just a means and a tool to delight the customers and ensure that government processes and procedures leverage technology to improve the customer satisfaction with government service delivery. Some of the parameters that may help in evaluating outcomes related to this category include:

• Increase in percentage of people using services through the new channels
• Percentage of users with positive feedback in surveys on governance and public services
• Awareness of the citizens regarding e-Governance initiatives

In order to have the desired impact, it is essential that the above parameters be built in the project right from the design phase rather than as an afterthought post implementation. This ensures that the e-Governance projects achieve maximum impact and generate momentum for bigger, more complex engagements. To drive home this point, consider the case of development of e-Governance strategy for a nation. A strategy that does not account for monitoring and evaluation of outcomes might measure the success of the strategy only in terms of achievement of progress in terms of number of services made available online. Therefore, even if a small proportion of people use the online services and even those users are unhappy with the quality of service delivery, the decision makers will be under an impression that the strategy is a success. However, if outcome measures were a part of the strategy, mid-term corrective actions would be possible to ensure that adequate efforts are expended towards ensuring the online services meet the expectations of the citizens and improve the quality of service delivery thereby ensuring more impact of the initiative while also building a case for further investments and projects.

Measuring outcomes has another significant impact, especially on e-Governance projects. So, if only outputs are measured, then the launch of the project means that it is a success irrespective of the fact that desired objectives were met or not. However, measuring outcomes ensures that all stakeholders keep up their commitments even after the project is operational, as outcome measurement will only happen once the project is on stream and is a continuous process. This builds a feedback mechanism for further improvements against new demands and changed circumstances.

Additionally, outcome measurement aids in the following:

• Evaluation of organization and development learning
• Informed decision making
• Supporting substantive accountability
• Making a positive contribution to the overall effectiveness of development
• Designing standardized processes to identify and characterize the data used to formulate and compute various indicators
• Development of methods that may be used in measuring outcomes for performing surveys for taking a fast baseline measurement and a more comprehensive formal data collection
• Creation of institutional mechanisms for organization, formulation, activation, monitoring, reporting, controlling and dissemination of M&E results and associated best practices
• Designing different types of output measurements and analysis that could be performed including cost-benefit and cost–effectiveness

As evident from the above, outcome measurement leads to systematic improvement in projects over time while keeping the focus on the accomplishment of the objectives. In the absence of this, many promising e-Governance projects fail. A classic example being the Gyandoot project implemented in the Dhar district of Madhya Pradesh. The project established intranet-based kiosks in villages delivering government services such as copies of land records, online applications for caste, income, domicile certificates, prices of crops and providing an online system for redressal of public grievances. The project was considered path breaking at its time and won several awards at national and international levels including the Stockholm Challenge awards in 2000. However, within three years of its launch, it was found that the usage levels were extremely low and only a few services such as grievance redressal, applications for income, domicile, and caste certificates, or information regarding the list of people falling below the poverty line were being availed through the project. The most-requested service in the beginning, information on agricultural prices for a payment of Rs 10 lost its charm due to fall in telecom prices that allowed the farmers to make a call for Rs 2-3 to get the rates in real time while rates at gyandoot kiosks were not even updated for 2-3 days. Lack of outcome measurement meant that no further improvements in the project were undertaken such as computerization of backend, rationalization of the user charges etc. that would have led to improvement in service delivery capabilities and popularity of the project. If outcome measurement had been built in the project design and feedback was used for corrective action, the project could have fared better as additional services and content relevant to the target customers could be created to not only maintain but also enhance the user base.

As may be inferred from the above discussion, outcome measurement is extremely useful to ensure that the programs accomplish their intended objectives. However, there are certain things outcome measurement does not do. For example, it does not eliminate the need to monitor resources, activities, and outputs. Similarly, measuring and improving program-level outcomes does not improve community-level outcomes. Except in rare instances, an individual e-Governance program does not serve enough individuals to affect community-wide statistics, regardless of how successful the program is. Additionally, community-level conditions are the result of a constellation of influences (for example, economic conditions, environmental factors, demographic trends, public- and private-sector policies, cultural norms and expectations) that are far beyond the scope of influence of individual human service programs.

The author is Executive Director and e-governance practice leader, PricewaterhouseCoopers