Statisticians Struggle To Keep Data Accurate Amid Lockdown Disruptions

Unemployment will spike. But by how much? Production will fall. Again, by how much and which categories? What will happen to prices? Over the next few months each incoming economic data point will be parsed to get the answer to these questions.

Except, compiling the data itself is proving to be a challenge amid restrictions of movement imposed across the country.

Researchers, data collection staff and statisticians are all trying to find ways to collect the right amount of data amid these restrictions and ensuring that it provides as accurate a picture as possible.

For now, field operations to collect data are at a near standstill, said a government official from the field operations division at the Ministry of Statistics and Programme Implementation. With field operations remaining suspended, the ministry has released data for March along with caveats and riders, this official said, speaking on the condition of anonymity.

Another official from the statistics ministry, who also spoke on the condition of anonymity, said the ministry is exploring strategies to work around the challenges posed by the lockdown and is in consultation with international organisations on the best way forward.

Host Of Indicators Impacted

The data most impacted includes purchasing managers’ indices, inflation, industrial output and unemployment.

The field work for price collection used in the Consumer Price Index was suspended from March 19, 2020. About 66 percent of all price quotations were received till then, according to a government press release dated March 13, 2020. The National Statistical Office used that data and followed “well established and internationally accepted methodology and practices” to put out the CPI inflation for the month, it said in the release.

Price data for CPI is collected across 1,114 urban markets and 1,181 villages using a weekly roster.

The wholesale price index data faced similar problems.

While much of this data is collected online, price data submission took a hit as factories manufacturing non-essential items remained shut. The data was released with the caveat that it was computed with a low response rate and is likely to be revised “in a significant manner” in the final index.

Private companies have also struggled with data collection but found ways to put out statistics important to judge the impact of the lockdown.

The Centre for Monitoring Indian Economy is the only agency which puts out high-frequency data on the unemployment rate in India. They also saw field surveys disrupted.

“We quickly reorganised ourselves and started the survey telephonically and have been able to bring out numbers which are reasonably reliable,” Mahesh Vyas, managing director and chief executive officer at CMIE, told BloombergQuint. The average sample size for the data fell to 2,289 observations in last week of March, and increased to 9,429 observations in the first week of April. Usually, the company gets 25,000 observations per week approximately, said Vyas.

While CMIE did not immediately release the data for the last week of March, which showed a spike in the unemployment rate to 23 percent, it decided to put it out after a wider sample for the first week of April showed similar results. We have a high degree of confidence in the data, Vyas said.

For the week ended April 12, CMIE has managed to collect 10,355 responses, which show an unemployment rate of over 24 percent, according to an article by Vyas on the CMIE website.

However, GDP data, when released, is unlikely to be impacted significantly.

The first estimate of GDP is based on agricultural production during the rabi season, industrial production, performance of the corporate sector, logistics sector, bank credit, union and state government expenditure and government revenue collections. As such, it is likely to be remain relatively unaffected, said Pronab Sen, former chief statistician of India.

From Face-To-Face To Phone

Nearly 80-90 percent of the data collection in South Asia is usually undertaken in person by about 1,200 enumerators, said Tithee Mukhopadhyay, associate director of research at the Abdul Latif Jameel Poverty Action Lab, a global research centre founded by Abhijit Banerjee, Esther Duflo and Sendhil Mullainathan.

The South Asian arm of J-PAL started winding down field operations around mid-March, Mukhopadhyay said. “Now, we have made a switch to phone surveys, engaging about 400 enumerators.”

Of the 40 field active projects that were being carried out, J-PAL has been able to move to phone surveys for about 17 of them. For other projects, such as those which require data collections from schools, research will restart once schools reopen.

It isn’t always easy to switch from face-to-face surveys to online or phone-based surveys.

V Sudarshan, chief marketing officer and senior vice-president at Hansa research, which conducts the RBI field surveys, said that first problem is incentivising interviewers to undertake the project in such difficult circumstances. Many of these interviewers work on individual projects and are not regular employees.

Hansa has stopped or postponed surveys being done face-to-face as they believes that a change in methodology will lead to questions about comparability. For instance, the central bank’s inflation expectations and consumer confidence surveys can’t be switched, Sudarshan said. He expects to disruption to continue over May and June, which means the next round of surveys scheduled to be released by the RBI in June may not come through.

Mukhopadhyay adds that the trickiest part is building trust over the phone.

Often, field agents will use measured scripts that have been tested to build trust. Doing that over the phone is not easy. As such, minor tweaks to questionnaires may be needed, she said.

Will It Hurt Data Accuracy?

How much will these factors impact accuracy of data?

According to Sen, the extent of impact on data accuracy will depend based on the data being collected.

Revisions are unlikely for March CPI, which would have been collected by the middle of the month.

However, for April, if at all the data is released, it is likely to be fairly unreliable, Sen said. He added that CPI data is collected from designated shops. Depending on whether these shops remain open and if field staff can visit these stores, data may not be comparable in any case, he said.

For data collected from companies, such as WPI and IIP, the accuracy will depend on responses that can be collected. Often, initial estimates are made on the basis of a response rate of 20-30 percent, he said.

Pravin Srivastava, India’s chief statistician, told BloombergQuint that the agency will decide on what to do about data for April when the lockdown is lifted.

In the meantime, what can the government do to get an accurate picture of conditions on the ground?

One way out for government organisations is to use an input-output model, Sen said.

Using an input-output model is a bottom-up approach, where data on raw materials and intermediaries can be used to to extrapolate overall production. For instance, data on coal can be used to gauge electricity generation. The method was used by the Planning Commission in the past, Sen said.

However, the statistical system is no longer geared for that, he added.

Corrects an earlier version where V Sudarshan’s designation was wrong.