How Modi government’s warning about private economic data ends up highlighting its own failures

The Modi government on January 4 cautioned against unemployment surveys conducted by private organisations.

The government’s press note titled “Rebuttal to the news on unemployment rate” came three days after the Centre for Monitoring Indian Economy, a private think tank, showed that India’s unemployment rate was at a 16-month high of 8.3% in December, up from 8% in November.

The government’s cautionary note added to a long-standing debate about alleged gaps in the CMIE’s economic data as well as the larger problem of what experts suggest is a breakdown in India’s crucial statistical system because of inadequate data.

‘Methodology biased’

The labour ministry said many private organisations conduct such surveys based on their methodologies and that they are “generally neither scientific nor based on internationally accepted norms”.

“The methodology used by these companies/organisations usually have a bias towards over-reporting unemployment or under-reporting employment due to their own sampling procedure and definitions used for collection of data on employment/unemployment,” the labour ministry stated. “The results of such surveys should be used with caution.”

Without directly referring to the Centre for Monitoring Indian Economy’s estimate of the unemployment rate, the ministry’s statement pointed out that the survey conducted by a private company has been reported by the media. The Centre for Monitoring Indian Economy is a private think tank that collects and compiles economic and business data for research and for customers. In case of India’s unemployment rate, its calculation is based on a sample size of about 1.7 lakh households.

The government instead highlighted the official employment data released by the Ministry of Statistical and Programme Implementation based on the quarterly Periodic Labour Force Survey.

Haryana’s Chief Minister Manohar Lal Khattar, on Monday, also criticised the methodology of the Centre for Monitoring Indian Economy’s survey after the data released by the private organisation on January 2 showed his state had the highest unemployment rate at 37.4% in December.

The Bharatiya Janata Party chief minister said the Centre for Monitoring Indian Economy’s unemployment figures “are based on a very small survey sample”.

This is not the first time Khattar had criticised the Centre for Monitoring Indian Economy. In July 2021, Khattar had said the survey was faulty. “CMIE is a privately-owned company created to make profits and whose decisions cannot be called fair and transparent,” he had said. “CMIE’s survey report is not based on employability, workforce, and labour force. There are many mistakes in the design, size, data and questionnaire in the survey of CMIE.”

Reliance on CMIE data

This cautionary note by the government assumes significance as economists and policy researchers have been increasingly relying on the Centre for Monitoring Indian Economy’s data in the absence of official monthly unemployment numbers, among other key economic data.

The Union government had discontinued the Labour Bureau’s quarterly enterprises surveys in March 2018 a year after scrapping the annual Employment-Unemployment Survey. Similarly, a 2017-’18 jobs report by the National Sample Survey Office, which had pegged the unemployment rate at a four-decade-high of 6.1%, was delayed by the Union government for several months during the 2019 general election. The National Sample Survey Office works under the Ministry of Statistics and Programme Implementation.

Yet, the government’s caution is not the first time economic data, in general, from the Centre for Monitoring Indian Economy has been questioned or alleged flaws in the organisation’s sampling methodology have been flagged.

For example, in May, economists had expressed caution about the Centre for Monitoring Indian Economy’s monthly unemployment data, particularly about the numbers from the country’s rural areas.

Economist Ajitava Roychowdhury had told The Economic Times that the organisation’s method of counting employed persons was not in accordance with the International Labour Organization’s definitions.

“If someone says on the day of the survey that he is doing something, for example, mobile hawking or rag picking, this person is considered as employed [by the Centre for Monitoring Indian Economy],” Roychowdhury, a professor of economics at Jadavpur University, said. “The CMIE does not differentiate between those who are in decent jobs and those who are not. If the ILO criteria of decent jobs are applied, the unemployment rate will be much higher.”

More prominently, explaining what they viewed as methodological flaws in the Centre for Monitoring Indian Economy’s broader triannual Consumer Pyramids Household Survey, economists Jean Drèze and Anmol Somanchi wrote in June 2021 that the sample has become more biased towards better-off households over time.

“The bias is, perhaps, not surprising, since the sampling method apparently consists of surveying the ‘main street’ first in each sample village or enumeration block, and proceeding to inner streets only if the sample size requires it,” they wrote. “If only for this reason, poor households are bound to be under-represented.”

This flaw leads to painting an incorrect picture of the Indian economy, they suggested.

Responding to Drèze and Somanchi’s criticism about the sample being biased towards the better-off, the Centre for Monitoring Indian Economy’s Managing Director Mahesh Vyas had said that “the reality is that it merely reflects the progress made by Indian households”.

Vyas said that “no bias creeps in because of our methodology”.

As not enough households can be found on the main street of average Indian villages to meet the sample size, accessing the inner streets is inevitable, Vyas explained. “The CPHS [Consumer Pyramids Household Survey] sample cannot escape including households from the outskirts.”

Vyas, while seeking to clarify concerns raised by Drèze and Somanchi on some sampling points, however, admitted that there are some limitations with respect to national representation such as exclusion of some border states and Union territories.

“There are practical limitations but no bias in the CPHS,” he added.

Drèze and Somanchi, however, said Vyas’ response to the criticism of their methodology only confirmed the survey is likely to be biased. “Even if the sample size often requires the survey to go beyond the main street, as Vyas points out, the fact remains that inner-street households will be under-represented,” Drèze and Somanchi said. “This may or may not be the main source of bias, but it is certainly a smoking gun.”

A crumbling data collection apparatus

The increased focus on Centre for Monitoring Indian Economy data, however, has a much larger problem behind it: a decline in India’s official data collection apparatus. For some time now, experts have been suggesting that there is an unavailability, unreliability and lack of critical socio-economic data. In May, The Economist noted, “India’s once-vaunted statistical infrastructure is crumbling”.

As a result, economists are finding it difficult to measure the true depth of India’s socio-economic problems because they do not have credible numbers to rely on, some highlight. Even former Reserve Bank of India governor Duvvuri Subbarao had pointed in his book that flawed data provided by the government were often leading to bad monetary policy decisions.

Nikhil Menon, a historian who has authored a book on how newly independent India set up world-class statistical infrastructure, warned that the country’s statistical institutions are falling apart. “There has been an institutional decay due to insufficient funding, and more perniciously, on account of political interference,” Menon said earlier. “As I say in my book, ‘good data isn’t always good politics’, and so we’ve seen National Sample Survey results suppressed, and other studies discontinued when it appears that they convey politically uncomfortable facts.”

Even in cases where robust economic data is available, questions have been raised about quality. Krishna Raj, economics professor at Institute for Social and Economic Change, Bengaluru, and researcher Kaibalyapati Mishra suggest that even the figures that are available cannot be compared internationally because they are not standardised.

“In addition to the unavailability of data, existing figures of employment are unworthy of international comparisons,” they wrote. “Most emerging and developed countries possess a strong and quality unemployment database as they endow the unemployed with several social security benefits.”

They added: “However, in the case of India, we suffer significantly from international standardisation and, thus, comparability.”