Here’s What a Real Meritocracy Looks Like

(Bloomberg Opinion) -- Meritocracy is a popular idea for structuring society, but whether committees are picking students to go to Yale or doling out coveted time on the Hubble Space Telescope to astronomers, it’s often nearly impossible in practice. Merit can be hard to measure — doable enough in tennis or swimming, but harder to define in art, science or college admissions.

When merit can’t be measured, nominally meritocratic institutions tend to assess it by past success, even if that success was based on subjective judgments and a dose of good luck. That may explain why parents have become desperate enough to resort to bribery to buy their kids one marker of merit — admission to an elite university. But now scientists running the Hubble Space Telescope have found a way to improve the assessment of merit. And they’ve tested it scientifically. The results could apply to many other areas of life.

The Hubble Telescope is a limited resource — it can only do so much science in a day — so a committee has to pick a fraction of winners from among hundreds of worthy proposals. Five years ago researchers discovered that projects with male leaders had a higher chance of getting approved than projects with female leaders. Was this because the male-led projects were better, or because there was some hidden bias?

To probe this question, they decided to try a round of selection where they removed the names and affiliations of the applicants. It’s akin to the blinding of studies so that researchers and subjects aren’t told who is getting a medication, say, and who is getting a placebo.

The result was that the sex ratio problem reversed, with women getting a higher rate of approvals for the first time in the 17 years that people have kept track. But there were other consequences as well — written up in detail in a piece for this month’s Physics Today. The authors were two astronomers, one of whom helped create the anonymous evaluation process, and the other, a close observer.

They realized that the nature of these “double anonymous” evaluations would be more challenging, because more than the genders would be obscured: “Reviewers would need to identify the important underlying intellectual questions and evaluate them, without the help of knowing the prior record of the proposer.”

What the authors of the paper found was a noticeable shift in the depth of discussion. The reviewers had to read the proposals carefully and think about them, rather than taking the shortcut and assuming the more famous astronomers should win. With evaluations based entirely on proposal merit, newcomers or people from lesser-known institutions get a chance, and those who are well established will continue to face critical review.

According to the Physics Today piece, this experiment was so successful that all future allotments of Hubble time will done with an anonymous evaluation. Something similar could improve the fairness and quality of science in many other competitive processes.

I’ve liked the idea of blinding so much that a few years ago, when I sometimes evaluated news coverage for a media criticism site called the Knight Science Journalism Tracker, I would compile the articles with their bylines and publication names removed. That way as I read the pieces, I wouldn’t be influenced by prestige — only by the quality of the information.

In his book “The Formula: The Universal Laws of Success,” Northeastern University physicist Albert-Laszlo Barabasi wrote of a similar experiment, in which people trying out for jobs in the Boston Symphony Orchestra would play behind a screen, so the judges could hear without seeing them. This did eventually even out a previously skewed sex ratio, but only after they added carpeting so the judges couldn’t recognize the women by the sound of their dress shoes.

As I talked to Barabasi for a previous column, he explained that being really good at something can take you only so far, because measuring quality gets tougher at higher levels. Often the same people continue to get all the high positions, the accolades and the awards, because it’s assumed they must be good, and it’s risky to go with an unknown quantity.

More blind selection would force judges of merit to think harder and deeper. That should improve the quality of the science or music or whatever is being judged and give all sorts of people a chance.

It might not stop people from thinking that getting into a top college is the only route to success, but more importantly, it would create more routes to success.

To contact the editor responsible for this story: Philip Gray at philipgray@bloomberg.net

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Faye Flam is a Bloomberg Opinion columnist. She has written for the Economist, the New York Times, the Washington Post, Psychology Today, Science and other publications. She has a degree in geophysics from the California Institute of Technology.

©2019 Bloomberg L.P.