Inside Bloomberg’s Covid Resilience Ranking
(Bloomberg) -- Everyone is fighting the same coronavirus, but a year into the pandemic, quality of life and control of the pathogen’s spread look vastly different across the world.
Bloomberg’s Covid Resilience Ranking scores the largest 53 economies on their success at containing the virus with the least amount of social and economic disruption.
We considered many datasets, indicators and indexes produced by organizations around the globe and applied three fundamental criteria in whittling the list down to the 10 components that make up our Ranking:
- How complete is the data? Many relevant indexes and databases -- for instance, measures on trust in government -- cover only a small number of places. We focused on indicators that cover the vast majority of the 53 economies in the Ranking, filling in the gaps with substitutions where reasonable.
- How current is the data? All datasets have a lag, some of up to a few years. Because of the pandemic’s pace and transformational impact, we chose to use the most up-to-date datasets where possible, with the maximum lag one year. Eight of our indicators are refreshed daily, while two are annual figures.
- Who collects the data? We decided to only use data compiled by Bloomberg or indicators from reputable third-party organizations with a track record of collation and analysis.
Why only rank 53 economies?
We decided for brevity and relevance to limit the Ranking to economies valued at more than $200 billion prior to the pandemic.
How often do you update?
The Ranking and the coverage is updated monthly, in the last week of every month.
How is the Ranking aggregated?
Each of the 10 data indicators are aggregated through the “max-min” method, which is used to convert metrics expressed in different scales into a common one, while maintaining the relative distance between values.
All the indicators are scored on a 0-100 scale, with 100 (blue) indicating the best performance and zero (orange) the worst. The rest fall in between, scaled by their distance from one another. The final Bloomberg Resilience Score is the average of a place’s performance across the 10 indicators, equally weighted.
The final score given to each place is a relative measurement on a given date. That score shouldn’t be compared in isolation to the economy’s previous scores as the max-min ceiling and floor values change in every update.
Can Covid-19 data be trusted?
Under-detection, under-reporting and manipulation of virus data on cases and deaths have been recurring issues across many economies during the pandemic.
In most places, gaps in the data are largely due to the chaotic and fast-moving nature of the crisis: the supply of test kits has been inadequate, leading to under-detection of cases. Official death tolls are also likely to be under-reported due to people dying at home before being diagnosed, lags in reporting from overwhelmed hospitals and Covid-19 deaths being recorded in some places as due to other causes.
Countries like Spain, the U.S., China and Turkey have adjusted their numbers throughout the pandemic, and credible reports have emerged that some nations, including Iran, Brazil and Russia, have intentionally concealed or downplayed data. Beyond that, excess mortality in some economies -- the additional number of deaths overall compared to previous years -- has outstripped official Covid-19 death tolls.
Still, given that it’s hard to distinguish between slow reporting, inadequate resourcing and the intentional concealment of data, we’re using cases and deaths data compiled by Johns Hopkins University, which draws largely from government sources, with the awareness that this is likely not the full picture.
A note on China’s numbers, which have been called into question by the U.S. and others. Throughout the pandemic, the country has repeatedly adjusted its virus data, adding nearly 15,000 new cases in one day in mid-February 2020 and raising its overall death toll by 40% to 4,632 in April 2020. These revisions arguably make its latest numbers more reliable. Researchers have used fraud detection techniques to conclude that, while China did manipulate its data in the early stages of the pandemic, its numbers have been accurate since.
Why do you have two datapoints on virus deaths?
The trailing one-month case-fatality rate is a good indicator of whether a place is effectively treating infected people and preventing Covid-19 deaths, a fundamental aspect of containing the virus. Over time, this ratio has improved among mostly developed economies as doctors and hospitals learned how to better fight the coronavirus.
The rate also captures the point economies are at on their individual pandemic curves. Australia had a high case-fatality rate during a brutal southern hemisphere winter wave in 2020 that triggered a three-month lockdown in the country’s Victoria state. As of December, with that outbreak largely quelled, the rate has fallen to a relatively low level. Correspondingly, the case-fatality ratios for nations in the northern hemisphere have deteriorated after they entered the cold season.
The one-month case-fatality rate doesn’t, however, capture the scar that the pandemic leaves on an economy as a whole, especially in places like Belgium, Sweden, Italy, the U.K. and the U.S., where Covid-19 cut a swathe through elderly populations in its initial phase in the spring. A higher percentage of Sweden’s population died from the virus than Vietnam’s, even if Sweden’s ability to save the lives of Covid-19 patients has since improved. This is why we’ve also included an indicator that reflects cumulative Covid-19 deaths as a share of the total population: Total Deaths Per 1 Million.
Economies with older demographics generally rank lower on this measure, given the way the virus ripped through aged-care homes in its first wave. In Belgium, officials decided to classify every nursing home death as being from Covid-19, even if the patient wasn’t officially diagnosed before they died. This approach has made Belgium one of the bottom-ranked economies on overall mortality though it’s not bottom-ranked in the one-month case-fatality ratio indicator.
What does the Positive Test Rate show?
The positive test rate is considered by experts as the most reliable way to determine if a place is testing enough.
A high rate of virus tests coming back positive indicates authorities are probably only testing the sickest patients who seek out medical attention and are not casting a wide enough net. It’s a sign there is likely to be undetected infections in the community. World Health Organization guidance is that governments should wait for the positive test rate to fall below 5% for at least 14 days before relaxing social distancing measures.
For economies that don’t report daily positive test rates, we derived their rates by dividing the number of cases over total tests administered on the last date that data was disclosed. We did this for China, Hong Kong, Vietnam and Austria. The correlation between an economy’s daily positive test rates and this measure is high, giving us confidence to substitute these values.
China’s latest-reported overall testing number is from August 2020 and officials have not disclosed a more recent positive test rate figure. However, its latest flareups confirm that the figure continues to hover close to zero in China. In January, in the northern city of Shijiazhuang, the population of about 11 million was tested twice with over 600 cases found. In April, China tested a border city neighboring Myanmar with nearly 400,000 people three times over after a cluster emerged, finding over 110 cases.
Our World in Data doesn’t collate figures for Egypt and Brazil. These two nations, therefore, don’t have scores for this indicator in Bloomberg’s Ranking.
One caveat on this dataset is that some places report the total number of people tested, while some report the total number of tests. The possibility of multiple tests being administered to the same person in a single day could mean they’re not exactly apples-to-apples, but these differences are not significant in scenarios where millions of tests are being conducted daily.
What are you measuring with the People Covered by Vaccines indicator?
The metric representing vaccines access has been repeatedly refined as progress on the ground becomes clearer. In the March Ranking, the Doses Given Per 100 indicator was replaced by People Covered By Vaccines, in accordance with Bloomberg’s Vaccine Tracker. This is a calculated figure looking at equivalently what percentage of the population can be covered in full regimens with the current number of doses administered and the mix of vaccine types used. This figure accounts for the doses required for each type of shot -- some require two doses, others just one.
Here are a few examples to show how it’s calculated. Keep in mind that Pfizer and Moderna require two doses per vaccination, while Johnson & Johnson’s is a single-dose vaccination:
- Pfizer: 10M doses administered = 5M people covered
- Moderna: 10M doses administered = 5M people covered
- J&J: 10M doses administered = 10M people covered
Previously we relied on per capita doses administered when comparing the pace of vaccinations between economies. The arrival of single-dose vaccines such as the Johnson & Johnson’s made that metric imprecise. Yet given single-dose vaccines were new to the market, the shift to this display had no material impact on the Ranking’s consistency: It basically halved the number for all the places to that date and didn’t change their scale of difference.
When the mix of vaccine type is not reported in an economy, we assume two doses are required per person for the people coverage calculation, as we take the more conservative estimate.
In earlier versions, the Ranking also included an indicator tracking vaccine supply agreements, named Access to Covid Vaccines. In March, we retired this metric, given that distribution is proving far more important than deals signed. Below is a table capturing how the vaccine access indicators have changed since the Ranking’s debut in November.
|November||Vaccine supply was represented by an indicator called Access to Covid Vaccines. This tracked the number of supply agreements each place had signed based on information compiled by Duke Global Health Innovation Center and Bloomberg|
|December||As governments disclosed more information on their negotiations and order sizes, Access to Covid Vaccines was fine-tuned with a Bloomberg tracker that collated contracts and what percentage of a place’s population is covered by its deals|
|January||As vaccine rollouts started in some places, a new indicator -- Doses Given Per 100 column -- was introduced as an additional metric|
|March||With the progress of distribution proving more important than supply deals, the Access to Covid Vaccines metric was removed and the Ranking reverted to 10 indicators, with vaccine access represented by People Covered By Vaccines|
What does Lockdown Severity measure?
This indicator is based on an index produced by the University of Oxford, which assesses the number and strictness of government policies that limit people’s movements as a way of containing spiraling outbreaks.
We’re interpreting restrictive government policies as a negative in the Ranking as the stricter the lockdown, the more disruption people are experiencing. Taiwan is among the top-ranked for this indicator as the population is subject to almost no restrictions, while European countries, many of which have had to impose repetitive lockdowns, rank lower. A higher score on Lockdown Severity denotes a less advantageous performance.
Some may argue that strict lockdowns should be viewed positively, a sign a government is acting aggressively to control the virus. But more than a year, and multiple waves, into this pandemic, the need to impose lockdowns reflects a failure to contain and manage Covid-19, and so we’re scoring it accordingly.
Stringent restrictions also correlate with the mental and economic toll of the virus on a population. Social disruption and isolation have been linked to higher suicide rates in some places, while school closures are raising concerns over child development and increases in hunger and drop-out rates among disadvantaged families.
Does Lockdown Severity reflect conditions across an entire economy?
The Oxford University index reflects the most stringent conditions in place in a given economy, regardless if those curbs are just being imposed in a specific region. That means a lockdown in one city or area will be the basis for an economy’s overall score.
This naturally penalizes large, expansive countries like China, India and even the U.S., where conditions can vary greatly from region to region and city to city. China’s conditions were scored as stringent in April on the Oxford index because some districts were in lockdown as local officials fought a resurgence near the Myanmar border. But the majority of China’s 1.4 billion population face barely any limits on their movement.
Still, while imperfect, we view Oxford’s economy-wide approach as a proxy for how aggressively governments are likely to react if an outbreak emerges. China’s playbook has been to impose some of the most oppressive measures in the world, from bans on people in specific areas from leaving their homes to mass, obligatory testing. So while life is relatively relaxed for many in China right now, that could shift abruptly if even one case is identified where they live or work.
What does Community Mobility show?
Given the broadness of the Lockdown Severity indicator, and the fact it reflects government policy and not its impact, we sought out another datapoint to better capture this picture. Google’s Covid-19 Community Mobility Reports, which underpins our Community Mobility measure, tracks people’s real-time movements and helps round out our understanding of how they are responding to virus restrictions in their everyday lives.
We track movement to and from retail, recreational and work places, taking a 30-day average to smooth out the effects of holiday periods. The closer movement levels are to the economy’s pre-pandemic baseline, the higher the score on this datapoint as part of Bloomberg’s Ranking.
Places where Stringency and Mobility scores are at odds reflects a lack of official ability to enforce restrictions, and a lack of compliance among the population.
As Google data is not available for mainland China, we used an estimate from an activity tracker created by Bloomberg Economics -- used in a weekly analysis of activity in 26 major economies -- to derive China’s corresponding score. The correlation between Google’s data and Bloomberg’s daily activity tracker for the other 25 economies is high, giving us confidence in using it as a substitute for China.
Google doesn’t track mobility data for Iran, either, and we couldn’t find a viable alternative source, so Iran doesn’t have a score for this indicator.
What does the GDP measure track?
When we first launched the Ranking in November, this indicator was based on the International Monetary Fund’s annual gross domestic product forecasts for 2020. The greater the expected contraction, the more challenging the economic reality is for people in these places, and therefore the weaker their performance on this measure.
China is an example of how largely eliminating the virus locally can lead to a rebound in economic activity. But its vast domestic market of more than 1 billion consumers means it’s an outlier. For almost every other place, containment and economic growth are in a complex trade-off relationship.
Places like Singapore and New Zealand have nearly eliminated virus transmission among their populations, but at a massive cost to their economies, which are reliant on tourism and travelers from abroad.
In 2021, we refreshed this indicator to projections for the year ahead. We also swapped out the IMF’s forecasts for a consensus figure compiled by Bloomberg from economist surveys, as this is more up-to-date than the IMF’s quarterly-updated number. For places with less than five economist forecasts, we revert to the IMF’s latest-available estimates instead. This was done for Bangladesh, Egypt, Iran and Iraq.
Our GDP measure reflects the percentage contraction or expansion projected this year in the 53 economies. We chose to use this indicator, despite its bias towards emerging, high-growth economies, instead of the “swing” between the current forecast compared to the pre-pandemic figures. This is because the Ranking aims to show the best places to be right now, not just how well they’ve deflected the economic blow from Covid-19.
For India, the forecast used refers to the year ending March 31, 2022, in alignment with the government’s fiscal cycle.
Why is the state of pre-pandemic health care relevant to Covid-19?
The Universal Healthcare Coverage indicator draws from a 2019 dataset published by the Institute for Health Metrics and Evaluation published in the Lancet in August 2020. It maps the effectiveness of 23 preventive and treatment measures, ranging from access to basic non-Covid vaccines to cancer care.
While the dataset was produced prior to the onset of the pandemic, it reflects an economy’s ability to effectively prevent, detect and treat illness across a population. Bloomberg’s deduction is that places with higher scores on this measure are in a better position to prevent deterioration and death in Covid-19 patients, and more able to maintain the provision of non-Covid related healthcare during the pandemic. Hong Kong isn’t tracked, so it doesn’t have a score for this indicator.
There are various measures and datapoints out there tracking the strength of health-care systems, including from the WHO. We chose this indicator due to its completeness: 204 countries and territories are tracked. It’s also the most up-to-date of those we surveyed.
What’s the point of including the Human Development Index?
This indicator, produced annually by the United Nations Development Programme, has three parts: life expectancy at birth, years of schooling, and wealth per capita. The three components were chosen to represent the overall well-being of a society.
Scores since the December Ranking reflect the recently-released 2020 report.
Like the Universal Healthcare Coverage datapoint, this measure captures an economy’s pre-pandemic performance. Still, the Human Development Index reflects a society’s ability to withstand the Covid-19 blow, and can be a proxy for how populations have reacted to the crisis:
- Years of schooling reflects access to education and acts as a proxy for a population’s trust in science, which experts say is a key determinant in whether people follow public health guidance on social distancing and mask-wearing. We considered other measures of trust in science, such as the proportion of STEM degrees among degree-holders, but didn’t find a suitably comprehensive and up-to-date alternative dataset.
- Wealth per capita reflects people’s income, adjusted for purchasing power.
- Life expectancy is a proxy for whether access to health-care is equitable across a population.
The UNDP doesn’t include Taiwan in this indicator, but the government there calculates its own Human Development score every year using the same methodology. We use Taiwan’s self-published score for this measure.
Was the Ranking vetted?
The Covid Resilience Ranking is the result of months of sifting through various information sources by Bloomberg reporters and data specialists. It was developed in consultation with experts in the data collation, economic and scientific fields. While the Ranking is naturally a subjective measure, we think it captures fairly and comprehensively the best and worst places to live and work right now as we continue to ride out this pandemic.
It will morph and be updated as circumstances change -- stay tuned.
©2021 Bloomberg L.P.