On key health indicators, West Bengal outperforms many states that offer PM’s insurance scheme

In the Economic Survey 2020-’21, released ahead of the Union Budget, the central government singled out poll-bound West Bengal for comparison on various health indicators against three adjacent states, Bihar, Assam and Sikkim, which, unlike West Bengal, have implemented its health insurance scheme, the Pradhan Mantri Jan Arogya Yojana.

The PMJAY scheme aims to provide Rs 5 lakh health insurance coverage per year to nearly 11 crore economically vulnerable families or 50 crore people. PMJAY is jointly implemented by the central and state governments in 32 states and Union Territories. West Bengal, Telangana, Odisha and Delhi have not implemented PMJAY. The Telangana government has announced in December 2020 that it intends to participate in PMJAY.

As PMJAY implementation began in 2018, data from the fourth (2015-’16) and fifth (2019-’20) rounds of the National Family Health Survey, India’s largest official household health survey used for framing health policy, would adequately capture pre- and post-implementation health outcomes, the Economic Survey said. This would enable an assessment of the impact of PMJAY.

The Economic Survey concluded that of 22 states and UTs covered in Phase 1 of NFHS-5 (2019-’20), PMJAY states were showing greater improvements than non-PMJAY states on several indicators. West Bengal had been outperformed by PMJAY-implementing neighbours on many of these indicators, the Centre’s survey said.

“Most of the indicators used in the report have nothing to do with PMJAY,” Sulakshana Nandi, Chhattisgarh convenor of the Public Health Resource Network, and a researcher on state-funded insurance schemes, told IndiaSpend. “For example, PMJAY does not offer vaccination or ante-natal care visits. PMJAY is restricted to hospitalisations. So the Economic Survey’s assessment of PMJAY improving various such health parameters is flawed, to begin with.”

The Survey, all the same, has made a larger conclusion – that multiple health indicators have improved in states which have implemented the scheme. IndiaSpend analysed the same data used in the Economic Survey to verify the claims and found that the picture is more complex. We also analysed other indicators to see if the premise of the Economic Survey stands, and found that West Bengal was often performing higher than the states it has been compared with.

While the claims in the Economic Survey are technically correct, West Bengal’s health data have been compared to the average of its three neighbours’ after adding their health data together. It is unclear why data of one state should be compared to average aggregated data of three other states.

Being West Bengal’s neighbours, Bihar, Assam and Sikkim are, therefore, “similar on socio-economic dimensions”, the Economic Survey has claimed. None of these states, however, share a border with each other, and their socio-economic dissimilarities are underscored by the same NFHS data the Economic Survey has used.

Further, should health data of Bihar, India’s poorest and third-most populous state, situated in the Gangetic plain, be grouped with that of India’s second-wealthiest and least populous state Sikkim, a former Himalayan kingdom, to make a comparison with a third state? That would be akin to comparing upper-middle-income country Mexico’s gross domestic product to the combined GDP of its northern and southern neighbours, high income United States and upper-middle-income Guatemala, respectively. Mexico’s GDP would inevitably be lesser.

In fact, West Bengal outperformed its neighbours and other large states that implemented PMJAY, on several health indicators mentioned by the Economic Survey, including infant and child mortality, child vaccinations and use of family planning methods, per the same NFHS data disaggregated by individual state, our analysis shows. Bihar’s performance was poor on several indicators, but this was not evident from the aggregated outcomes. Many states have shown worsening infant and child mortality, despite implementing PMJAY.

Further, the Survey used “difference-in-difference” calculations – that is, the relative difference in improvement between states. In such analyses, states that have more ground to cover, to begin with, would seem to be performing better than those that were better off in the first place – a fact that the Survey alludes to when it mentions a “higher base” or a “lower base”.

Moreover, health insurance coverage is only one way to assess access to health. Affordability is another, health researchers tell us. For instance, increased coverage under PMJAY should reflect both higher institutional newborn deliveries and a decrease in out-of-pocket expenditure for the same, they say, but PMJAY has not had a positive bearing on either.

On these indicators, West Bengal has outperformed both its neighbours and all large states that have implemented PMJAY, NFHS data show. In 12 of 20 PMJAY states, out-of-pocket expenditure on delivery increased by between 3% and 130%.

Health insurance coverage

The central government claimed that PMJAY implementation enhanced health insurance coverage in states adjacent to West Bengal by 89%, while coverage fell 12% in the latter.

PMJAY enhanced health insurance coverage by 54% in states and UTs included in Phase 1 of NFHS-5 (2019-’20), in the four years since NHFS-4 (2015-’16), said the Economic Survey, while falling by nearly 10% in non-PMJAY West Bengal and Telangana.

In West Bengal’s neighbours Bihar, Assam and Sikkim, the proportion of households with health insurance coverage increased by 89%, while in West Bengal, the same metric declined by 12%, said the Economic Survey.

Disaggregating the four states on individual performance, health insurance coverage increased by 2.3 percentage points and 49.6 percentage points in Bihar and Assam, respectively. In Sikkim, coverage decreased by 4.6 percentage points and in West Bengal by 4.1 percentage points, our analysis shows.

It would not be appropriate to compare tiny Sikkim’s performance on health, however, with West Bengal, which has 150 times more people. Sikkim’s health indicators are better compared with all other North-Eastern states’ included in the NFHS-5.

Looking at the individual performance of large PMJAY states that have over 1 crore people, like West Bengal, Andhra Pradesh has also seen a decrease in households with health insurance cover and Karnataka has seen no change, despite both having implemented PMJAY.

Overall, the proportion of households with health insurance cover has increased 10 percentage points, on average, in large states that implemented PMJAY.

Newborn delivery expenditure

Measuring access to health insurance is only one way to assess the affordability of healthcare for people. Another is to look at the metric of out-of-pocket expenditure, ie how much people are paying from their own pockets for healthcare. It makes sense to look at out-of-pocket expenditure as the main indicator to show the performance of a cashless scheme such as PMJAY, said Nandi.

“Any insurance scheme that claims to be cashless must lead to zero out-of-pocket expenditure,” she said.

Average out-of-pocket expenditure per newborn delivery in a public health institution is the only out-of-pocket expenditure metric publicly available in NFHS-5 factsheets. On this particular metric, the same data set shows West Bengal to be the best performer among all states, by a substantial degree.

People in West Bengal paid on average Rs 7,919 per delivery in a public health facility in 2015-’16, per NFHS-4. By NFHS-5 (2019-’20), they paid Rs 2,683, a 66% reduction in cost. The state went from being the second-worst on this metric in 2015-’16 to fourth-best in 2019-’20, among the 22 states/UTs, large and small, covered in Phase 1 of NFHS-5.

Out-of-pocket expenditure per delivery rose in 12 out of 20 PMJAY states and UTs, by between 3% and 130%. Among large states, only Gujarat has a lower out-of-pocket expenditure per delivery in a public health facility than West Bengal.

PMJAY covers institutional deliveries of babies, including in public hospitals, so enrollment in PMJAY should reflect higher utilisation of health facilities for institutional delivery, and a decrease in out-of-pocket expenditure for the same.

“However, the data does not show that this has happened,” said Nandi, based on her own analysis of NFHS data. PMJAY has not had a positive bearing either on the levels of institutional deliveries or on the amount of money women spend on it, she concluded.

“The biggest failure of PMJAY and other insurance schemes in India has been their inability to decrease out-of-pocket expenditure,” said Nandi. “This has been proven time and again from numerous studies. A study from Chhattisgarh that analysed [National Sample Survey data] on utilisation of hospital-care pre- and post-PMJAY implementation found that enrollment under PMJAY or other schemes did not increase this in Chhattisgarh. The study also found that out-of-pocket expenditure and incidence of catastrophic health expenditure did not decrease with enrollment under PMJAY.”

Institutional births

The central government claimed that West Bengal outperformed its neighbours on the proportion of institutional births and that increased C-section deliveries show people accessed public healthcare more in PMJAY states.

PMJAY covers institutional deliveries of babies, including in public hospitals. Being enrolled in PMJAY should reflect in a high utilisation of health facilities, for the institutional delivery of babies, and a decrease in out-of-pocket expenditure for the same. “However the data does not show that this has happened,” said Nandi.

The Economic Survey noted that while the percentage of institutional births increased in all four states, West Bengal’s performance saw a larger increase at 22%, compared to its PMJAY neighbours Bihar, Assam and Sikkim, at 11%.

In terms of births in public health institutions, West Bengal’s performance (a 28% increase) was even better compared to its neighbours (10%), per the Economic Survey. This remains correct even when comparing West Bengal only with other large states. Non-PMJAY Telangana too has outperformed large PMJAY states Andhra Pradesh, Maharashtra and Karnataka on this metric. Thus, a link between PMJAY and increased institutional deliveries is not made clear by the data.

An increase in births via caesarean section (C-section) in public healthcare institutions has been higher in Bihar, Assam and Sikkim (46%), versus in West Bengal, (21%), said the Economic Survey, while noting that West Bengal began from a higher base. The survey concluded that PMJAY thus “seems to have enabled” people in Bihar, Assam and Sikkim to make greater use of public health facilities.

Increased use of C-sections may not, however, be the best metric to assess improved public health access, given this health ministry guidance from July 2019, quoting a World Health Organization finding that “at population level, caesarean section rates higher than 10% are not associated with reductions in maternal and new-born mortality rates”.

The ministry ordered a curb of unnecessary C-sections in public health institutions. Such births have increased in public health institutions in all large states, PMJAY and otherwise, in the four years to NFHS-5 (2019-20). Bihar (3.6%) is the only large state where C-section births are less than 10% of all deliveries.

Infant, child mortality

The central government claimed that West Bengal’s neighbours performed better in reducing infant and under-five mortality.

The Economic Survey said that in the four years between the two NFHS surveys, infant mortality had declined by 20% for West Bengal, but by 28% for its three neighbouring states. On under-five mortality as well, West Bengal had seen an improvement of 20% whereas its three neighbouring states improved by 27%. Lastly, the neonatal mortality rate declined by 22% in PMJAY states compared to 16% in non-PMJAY states, said the Economic Survey.

Once again, West Bengal’s standalone performance was judged, while infant and child mortality rates for its three neighbouring states were aggregated to provide their combined performance. When we look at individual performance, the Economic Survey’s claim is only partly correct.

In the four years to 2019-2020, West Bengal improved infant mortality by 20%. In Sikkim, infant mortality improved by 62% and in Assam, by 33%. In Bihar, infant mortality improved by just 2.7%. On under-five mortality, West Bengal improved by 20.1%, Sikkim by 65.2% and Assam by 30.8%.

In Bihar, under-five mortality dropped by just 2.9%. By 2019-2020, West Bengal improved neonatal mortality by 29.5%, while Sikkim and Assam did so by 76% and 31.4%. But it didn’t drop as much in Bihar (6%, from 36.7 deaths per 1,000 live births to 34.5).

Bihar’s actual performance was obscured behind the aggregated figures of all three states. It continues to be the worst-performer on neonatal, infant and under-five mortality among all the 22 states and UTs covered in NFHS-5 Phase 1.

What about other large states that have implemented PMJAY but were not included in the Economic Survey’s analysis? West Bengal has seen the third biggest improvement in neonatal, infant and under-five mortality compared to all other large PMJAY states. Only Assam and Jammu & Kashmir have performed better than West Bengal on these metrics. Non-PMJAY Telangana too has seen higher improvement than Maharashtra and Kerala.

Family planning

The central government claimed that adjacent PMJAY states saw a higher increase in the use of family planning methods than West Bengal.

West Bengal’s three neighbouring states had performed better on use of family planning methods as well, the Economic Survey said. Bihar, Sikkim and Assam saw a 51% increase in the use of family planning methods, while West Bengal improved by only 5%, said the survey. “[S]imilar to what we observed in the case of child mortality, the increase has been higher in states that have adopted PM-JAY,” concluded the survey.

Disaggregating NFHS-4 (2015-’16) data by individual state, 24.1%, 46.7%, 52.4% and 70.9% of married women aged 15 years-49 years in Bihar, Sikkim, Assam and West Bengal, respectively, had used at least one family planning method. This has improved in all four states by NFHS-5 (2019-’20), by 31.7 percentage points, 22.4 percentage points, 8.4 percentage points and 3.5 percentage points, respectively.

It appears that West Bengal has seen the least improvement on this important metric. But the base for West Bengal’s performance is much higher to begin with, as the data above show and the Economic Survey also notes.

It was and remains the best performer on uptake of family planning methods among all 22 states and UTs, large and small, covered in NFHS-5 Phase 1, with 74.4% of married women in West Bengal accessing family planning methods, without PMJAY.

BCG vaccine

The central government claimed that the proportion of children aged 12 months-23 months who have received the BCG vaccine increased 5% in PMJAY states, but decreased by 1% in non-PMJAY states.

The Economic Survey said the number of children aged 12 months to 23 months immunised with BCG vaccine (for tuberculosis and meningitis) increased in PMJAY states compared to non-PMJAY states. However, BCG is only one of five types of immunisation given to children assessed in the Economic Survey.

On fully vaccinating children, West Bengal has outperformed all the large PMJAY states. While Assam, Maharashtra and Karnataka have seen a higher level of increase than West Bengal, they are catching up to West Bengal’s higher base level of child vaccinations. Telangana too is ahead of all the large PMJAY states, except Jammu and Kashmir and Karnataka.

“The Economic Survey’s attribution of health indicators to PMJAY is faulty and unscientific as it does not use adequate statistical methods for such an analysis,” said Nandi. “States which have done well in terms of health indicators and universal healthcare must be recognised and we should learn lessons from them.”

“We should also emphasise strengthening the government health system, rather than schemes such as PMJAY that commercialise healthcare further,” Nandi said. “This will ensure more equitable and free healthcare access.”

This article first appeared on IndiaSpend, a data-driven and public-interest journalism non-profit.