What ails India’s bedrock health survey: Exploited field workers, badly designed questionnaires

The National Family Health Survey, one of the pillars of health planning and policy making in the country, has been riddled with problems. The biggest shortcoming is an insufficient number of poorly trained and poorly paid field agents to collect data who receive little logistical support and work under harsh conditions. This affects the quality of the data collected as does the design of the survey itself.

The NFHS has been conducted since 1992-’93 and has provided state and national information on fertility, infant and child mortality, family planning practices, maternal and child health, reproductive health, nutrition, utilisation and quality of health and family planning services. The government is currently in the process of planning the fifth round of the survey and needs to address the gaps seen in previous rounds.

The Ministry of Health and Family Welfare has designated the International Institute for Population Sciences in Mumbai as the nodal agency for NFHS 5. The institute is expected to collaborate with field organisations to conduct the survey.

Labour law violations

One of the most serious problems with the NFHS is the that field researchers are often poorly trained, underpaid and exploited. Field agencies often violate labour laws with regard to payments, accommodation, travel and insurance.

For example, during NFHS 4, the field agency chosen in Karnataka was a consulting company from Gurgaon with no base in the state and no staff who spoke the regional language Kannada. The agency violated several labour norms. Researchers were not paid during their four-week training and were made to wait another month and a half without any emoluments before the survey was initiated. The agency retained their original marksheets to prevent them from seeking other employment. After the start of the survey, researchers were either not paid or paid less than mandated wages and had to work in unsafe conditions. They were not allowed holidays and were issued letters saying that if they left work mid-way, they were not entitled to any payment.

After days of this gruelling schedule, three researchers and a driver died and another researcher was injured in an accident in May 2015. Despite the IIPS bid document claiming that the field agency would pay for group insurance, the field agency issued letters saying that it would not be liable to cover any costs such as medical treatment.

On one hand, agreements signed by IIPS and field agencies have been violated in NFHS 4 and on the other hand grievance redressal mechanisms have been dysfunctional, with both IIPS and the health ministry absolving themselves of responsibilities. Researchers have sent repeated letters and emails to IIPS and the health ministry highlighting the violations but have not received any response.

The government must greatly improve NFHS systems to hold field agencies accountable and address grievances. Agencies that violate rights of field staff should be blacklisted even if they meet technical and financial bidding criteria.

Lengthy questionnaires

It is widely accepted that the longer a questionnaire, the poorer the quality of data. In NFHS 4, the household questionnaire had 74 questions. The women’s questionnaire is 93 pages long with 1,139 questions. Some of these questions are difficult for researchers to ask and for respondents to answer. Among the the 40 questions related to domestic violence including intimate sexual violence are ones like “how old were you the first time you were forced to have sexual intercourse or perform any other sexual acts by anyone, including (your/any) husband?”

The man’s questionnaire is 38 pages long with 843 questions. Of these 43 questions with several sub-sections are dedicated to HIV and STD. NFHS 4 field researchers have repeatedly asked how it is possible to ask intimate sexual questions, especially when it is almost impossible to ensure privacy during the survey. The only answer they received was that questionnaire had been finalised and it has been done before.

In a section that resembles market research more than public health, the questionnaire lists brands of mosquito nets – Dawa Plus, Duranet, Interceptor, Lifenet, Magnet, Netprotect, Olyset, Permanet, Royal Sentry, Yorkool. Researchers are supposed to read them out and tick mark which ones are used.

Data quality

When field researchers work in exploitative conditions, work suffers and this can affect the quality of data. This is relevant not only to the survey underway but also subsequent surveys and analyses that depend on this data.

A field agency that deploys data collectors should have a local presence, administrative and training capacity. If trainers do not know the local language, this can affect the training process and, in turn, data quality.

The government would also do well to involve Population Research Centres, a network of 23 centres established by the health ministry to conduct research on population stabilisation, socio-demographic surveys and study communication of family welfare policies. Currently, very few Population Research Centres are part of the NFHS despite being more eligible than the selected field agencies in terms of staff capacity, infrastructure, training facilities, experience and regional presence. The National Sample Survey Organisation can be involved in coordination with respective state governments. These organisations have experience of large scale surveys and being government agencies, can be held more accountable for violations and poor quality of data.

The NFHS has the potential to generate much-needed trends in health and reproductive indicators and yet it does not. Data must be made more systematically available from private and public hospitals to look at geographic, seasonal and demographic variations of diseases.

Public health researchers have raised concerns about inflated costs of NHFS pointing out that it has not translated to better quality data. (The budget for NFHS 4 was Rs 220 crore and funded by the Government of India with financial support from United States Agency for International Development, the United Kingdom’s Department for International Development, Bill and Melinda Gates Foundation, MacArthur Foundation, United Nations Children’s Fund and United Nations Population Fund). Anecdotal evidence indicates that about 50% of the costs have gone to consulting agencies.

Transparency in the tendering process

The criteria for the technical bid, which should precede the financial bid, should be clearly specified, transparent and developed in consultation with a larger group of public health and other experts who have an understanding of ground level challenges in conducting national and state level surveys.

Criteria that should be included in the technical bid are previous history, experience, researcher database, familiarity with the local language and experience of working in the state. Agencies that are technically competent should not be left out because they are unable to match the financial bid.

Technical and financial assistance for NFHS 4 was provided by several multinational agencies like ORC Macro, USAID, DFID, Bill and Melinda Gates Foundation, UNICEF and UNFPA. While consultation with a wide number of agencies can help to improve data quality, it is unclear on what the roles of these agencies are in deciding the questionnaire, selection of field agencies and ownership of data. The health ministry and IIPS must issue a clear statement on the roles and responsibilities of different agencies and have them declare any conflicts of interest.

To sum up, the overall design and planning of NFHS 5 can be improved with the following steps:

• Ensure that researchers employed for data collection are paid a reasonable wages and have financial support for meeting all expenses.
• Develop local capacity for well-trained researchers in India with institutional support.
• Have an expert committee of public health professionals from different states in India to draw up the criteria for the technical bidding process.
• Have strong grievance redressal mechanisms for the researchers.
• Have an evaluation mechanism for all large surveys to identify gaps and lessons for the future.
• Defining the role of multinational agencies and addressing  conflicts of interest.
• Compiling a national level disease database drawing from government and private hospitals, surveillance centres and other facilities.

The author is a public health doctor and researcher who was involved in the NFHS4 in Karnataka and documented the exploitative working condition of field researchers as well as gaps in data collection.