Between 1988 and 2013, the number of cases of polio worldwide plummeted from 3.5 lakh to 406. The number of countries in which the disease was endemic also went down, from 125 to three. The world seemed on the verge of eradicating the disease once and for all.
In 2014, polio surged again to 359 cases, with 85% of the new cases appearing in Pakistan. According to the World Health Organization, people with the disease were crossing the border into Afghanistan, and strains of the virus that originated in Pakistan were found in Egyptian sewers.
Neighbouring India eliminated the disease in 2011, after switching to a more effective vaccine and ramping up vaccination efforts. Compared to Pakistan, India clearly had greater capacity to respond, and it didn’t face as much political unrest or a border with Afghanistan. Still, many other countries with weak central governments had eliminated the disease. So why not Pakistan?
The reason Pakistan was having so much trouble didn’t come down to not having enough doses of the vaccines or health workers to administer them – the country did. A key problem was that information about who was getting vaccinated wasn’t getting collected, and that the incentives health workers got didn’t actually motivate them to perform more vaccinations.
Our group of researchers wanted to find a way of making sure that more detailed information about vaccinations was collected and to develop better incentives for health workers. The solution we found (and which we detail in a draft paper) is cheap and portable: smartphones.
What happened in Pakistan?
Because Pakistan lacks data on which citizens needed to be vaccinated, the government aims to achieve complete vaccination coverage by going door-by-door. To do this, the government assigns four or five roving teams of health workers to vaccinate each neighborhood. These drives happen about once a month and usually last for at least two days.
Health workers on these vaccination drives don’t face an easy task, especially in a city of 60 lakh like Lahore.
Information about who has been vaccinated or where teams have gone isn’t centralised. Rather, teams use paper maps and forms to figure out where to go and to record their efforts. To show what houses have been visited, health workers use chalk to write on the walls of houses. The only information recorded centrally is a team’s daily total number of vaccinations.
The government paid workers for the door-to-door drive with one-size-fits-all incentives – 100 Pakistani rupees (about $1) per day regardless of who or how many they vaccinated. The incentives, in other words, weren’t based on performance.
Combine all of these factors, and the result was that some children were immunised several times and others, in small pockets isolated by geography or violence, were not vaccinated at all.
Getting the right data
In rich and poor countries, pay-for-performance systems that use incentives have proven particularly effective in health care.
In a pay-for-performance system, a worker’s pay is higher when they perform better. Implementing performance pay necessarily requires high-quality, detailed data on performance, which is hard or impossible to use when it is recorded on paper or chalked on the sides of houses.
In a pilot project funded by the UK Department for International Development, our group has been working with Lahore’s municipal government to develop a tool to help track information better and to find out what incentives provide the best motivation for health workers.
We distributed cheap smartphones to the health workers so they could send geo-coded information about their activities to a centralised dashboard. After they visited each home, they would send back data about how many children they had vaccinated, the time that the vaccination visit occurred, and the address it occurred at as coded by GPS location.
Better perks
We also changed the incentive for workers. Instead of a bonus that workers got regardless of how many vaccinations they did or households they visited, we introduced an incentive system that offered a bigger bonus for better performance.
Each worker needed to visit 300 households in the first two days of the campaign, but they could set a goal in advance for how many houses they wanted to visit on each of these days. This allowed workers some flexibility over when they performed their vaccinations. The app also recorded each time they visited a household to perform a vaccination in real time.
Meeting the target meant a health worker would get her bonus. Meanwhile, data about actual vaccinations were aggregated, visualised and sent to an online portal for policymakers to use.
By recording personal goals for first two days of the campaign, workers indicated how much they preferred to delay visits from the first day to the second day. Some workers set high targets for the first day, and other workers pushed more vaccinations to the second day. We were able to use this information to design incentives that were used in the second round of the campaign.
For instance, workers that tended to delay vaccinations to the second day got an incentive scheme, where vaccinations performed on the second day didn’t count as much toward their goal of 300 as vaccinations performed on the first day. This helped provide more incentive for these workers to do more on the first day.
More vaccinations
We piloted the system among 505 workers, divided into different treatments and control groups in a randomised trial. Our preliminary findings, based on data from the mobile phones, shows that the bonus did improve performance: incentivised workers completed 15% to 17% more vaccinations than workers just being monitored and not positively rewarded.
The individually tailored incentive structure achieved its intended goal of causing workers to even out their effort over the two days.
Preliminary results further suggest that individually tailoring can also be used to increase the total number of vaccinations, though this would require a different set of incentives than those we evaluated. And in the few cases where parents didn’t allow their children to be vaccinated, the online dashboard allowed policymakers to identify those households for a follow-up.
In upcoming vaccination drives, we will use these data to understand why certain pockets remain unvaccinated, and formulate a pay gradient that will push workers to fill in those gaps.
Mapping individual performance
Obviously people are individuals, and workers respond to incentives in very different ways. We can map out individual decision-making tendencies to target incentives – even exploit those differences for the public’s gain.
Our results suggest that this system can bridge the pay-for-performance information gap very quickly. Indeed, Pakistan’s government is scaling up the technology under the banner of Punjab Information Technology Board and aims to implement the program well beyond Lahore.
These tools are simple – just smartphones and apps – and could be used in any developing country to collect better data and target workers efforts more effectively.
Michael Callen, Assistant Professor of Public Policy, Harvard University.
This article was originally published on The Conversation.
In 2014, polio surged again to 359 cases, with 85% of the new cases appearing in Pakistan. According to the World Health Organization, people with the disease were crossing the border into Afghanistan, and strains of the virus that originated in Pakistan were found in Egyptian sewers.
Neighbouring India eliminated the disease in 2011, after switching to a more effective vaccine and ramping up vaccination efforts. Compared to Pakistan, India clearly had greater capacity to respond, and it didn’t face as much political unrest or a border with Afghanistan. Still, many other countries with weak central governments had eliminated the disease. So why not Pakistan?
The reason Pakistan was having so much trouble didn’t come down to not having enough doses of the vaccines or health workers to administer them – the country did. A key problem was that information about who was getting vaccinated wasn’t getting collected, and that the incentives health workers got didn’t actually motivate them to perform more vaccinations.
Our group of researchers wanted to find a way of making sure that more detailed information about vaccinations was collected and to develop better incentives for health workers. The solution we found (and which we detail in a draft paper) is cheap and portable: smartphones.
What happened in Pakistan?
Because Pakistan lacks data on which citizens needed to be vaccinated, the government aims to achieve complete vaccination coverage by going door-by-door. To do this, the government assigns four or five roving teams of health workers to vaccinate each neighborhood. These drives happen about once a month and usually last for at least two days.
An example of a map used by health workers for vaccination campaigns.
Source: Michael Callen, CC BY-NC
Health workers on these vaccination drives don’t face an easy task, especially in a city of 60 lakh like Lahore.
Information about who has been vaccinated or where teams have gone isn’t centralised. Rather, teams use paper maps and forms to figure out where to go and to record their efforts. To show what houses have been visited, health workers use chalk to write on the walls of houses. The only information recorded centrally is a team’s daily total number of vaccinations.
The government paid workers for the door-to-door drive with one-size-fits-all incentives – 100 Pakistani rupees (about $1) per day regardless of who or how many they vaccinated. The incentives, in other words, weren’t based on performance.
Combine all of these factors, and the result was that some children were immunised several times and others, in small pockets isolated by geography or violence, were not vaccinated at all.
Getting the right data
In rich and poor countries, pay-for-performance systems that use incentives have proven particularly effective in health care.
In a pay-for-performance system, a worker’s pay is higher when they perform better. Implementing performance pay necessarily requires high-quality, detailed data on performance, which is hard or impossible to use when it is recorded on paper or chalked on the sides of houses.
In a pilot project funded by the UK Department for International Development, our group has been working with Lahore’s municipal government to develop a tool to help track information better and to find out what incentives provide the best motivation for health workers.
An example of forms used by health workers to track vaccinations.
Source: Michael Callen, CC BY-NC
We distributed cheap smartphones to the health workers so they could send geo-coded information about their activities to a centralised dashboard. After they visited each home, they would send back data about how many children they had vaccinated, the time that the vaccination visit occurred, and the address it occurred at as coded by GPS location.
Better perks
We also changed the incentive for workers. Instead of a bonus that workers got regardless of how many vaccinations they did or households they visited, we introduced an incentive system that offered a bigger bonus for better performance.
Each worker needed to visit 300 households in the first two days of the campaign, but they could set a goal in advance for how many houses they wanted to visit on each of these days. This allowed workers some flexibility over when they performed their vaccinations. The app also recorded each time they visited a household to perform a vaccination in real time.
Meeting the target meant a health worker would get her bonus. Meanwhile, data about actual vaccinations were aggregated, visualised and sent to an online portal for policymakers to use.
By recording personal goals for first two days of the campaign, workers indicated how much they preferred to delay visits from the first day to the second day. Some workers set high targets for the first day, and other workers pushed more vaccinations to the second day. We were able to use this information to design incentives that were used in the second round of the campaign.
For instance, workers that tended to delay vaccinations to the second day got an incentive scheme, where vaccinations performed on the second day didn’t count as much toward their goal of 300 as vaccinations performed on the first day. This helped provide more incentive for these workers to do more on the first day.
More vaccinations
We piloted the system among 505 workers, divided into different treatments and control groups in a randomised trial. Our preliminary findings, based on data from the mobile phones, shows that the bonus did improve performance: incentivised workers completed 15% to 17% more vaccinations than workers just being monitored and not positively rewarded.
The individually tailored incentive structure achieved its intended goal of causing workers to even out their effort over the two days.
Preliminary results further suggest that individually tailoring can also be used to increase the total number of vaccinations, though this would require a different set of incentives than those we evaluated. And in the few cases where parents didn’t allow their children to be vaccinated, the online dashboard allowed policymakers to identify those households for a follow-up.
In upcoming vaccination drives, we will use these data to understand why certain pockets remain unvaccinated, and formulate a pay gradient that will push workers to fill in those gaps.
Mapping individual performance
Obviously people are individuals, and workers respond to incentives in very different ways. We can map out individual decision-making tendencies to target incentives – even exploit those differences for the public’s gain.
Our results suggest that this system can bridge the pay-for-performance information gap very quickly. Indeed, Pakistan’s government is scaling up the technology under the banner of Punjab Information Technology Board and aims to implement the program well beyond Lahore.
These tools are simple – just smartphones and apps – and could be used in any developing country to collect better data and target workers efforts more effectively.
Michael Callen, Assistant Professor of Public Policy, Harvard University.
This article was originally published on The Conversation.
Limited-time offer: Big stories, small price. Keep independent media alive. Become a Scroll member today!
Our journalism is for everyone. But you can get special privileges by buying an annual Scroll Membership. Sign up today!