When You Cannot Trust the Experts

How does one explore a complicated issue when you believe that many of the experts are biased and you do not know which are not? Past posts dealt with versions of that problem in two contexts, climate and the origin of Covid. I know of four solutions.

Keep it Simple, Stupid

Deduce as much as you can from facts that everyone agrees on. If you want to persuade other people, do it with arguments simple enough so that an intelligent layman can check them for himself. I did that in my calculation of how much more land would become warm enough for human use with a given amount of warming. I did it again in my Bayesian analysis of the origin of Covid.

Here is another example. When Brett Kavanaugh was proposed for the Supreme Court, Christine Blasey Ford, a professor at  Palo Alto University, charged that he had tried to rape her at a party when both of them were in high school. I knew nothing about Kavanaugh that would tell me how likely the charge was to be true, nothing about Ford that would tell me how likely she was to be telling the truth, and any reports I could get on either would be from people likely to be badly biased in one direction or the other. That raised the question of how, with information publicly available, I could evaluate the charge.

Most people would not lie about that sort of thing but some, with a sufficiently strong incentive, would — there have been several recent cases where stories along similar lines were shown to be false. When the story first came out, the only evidence for it, other than Ford’s claim, was that she and Kavanaugh were both high school students in the same area at the same time.

Checking online, about six million people live in the Washington-Arlington-Alexandria suburban area. Of six million people, about 150,000 would be female high school students. Given the political polarization over Kavanaugh’s candidacy, of the 150,000 who had been high school students when he was, at least 50,000 would be against it, many of them passionately. Out of 50,000 people, it would not be surprising if at least one was willing to invent a story designed to block the appointment. I concluded that I had no way of knowing whether Kavanaugh was guilty but that the fact that Ford said he was did not much change the odds.

I am ignoring complications that came up later, such as claims that Ford had told someone else of the incident prior to the controversy over Cavanaugh’s candidacy for the Court and later claims against Cavanaugh by other women. My purpose is to demonstrate how publicly available information can be used to make a rough judgement of a controversy, so consider the calculation to have been made shortly after Ford first accused Kavanaugh.

Mining Bias

People sometimes make it obvious which way their bias runs. When William Nordhaus reported his estimate of the cost of waiting fifty years to do anything about climate change as 4.1 trillion dollars, he added “Wars have been started over smaller sums.” He did not mention that 4.1 trillion dollars, spread over the entire globe and the rest of the century, was equivalent to a reduction of world GNP of about .05%. I conclude that if his numbers are biased they are biased high, not low, hence that I can take them as an upper bound on what his analysis finds the cost to be. A similar argument applies to the IPCC reports. In both cases the source might be wrong, but if it was wrong due to bias I know which direction it would be wrong in.

For a very different example, consider the question of the dangers of nanotechnology. I have been an observer of the conversation from very early, having spoken on the economic implications of nanotech at the first MIT Nanotechnology symposium back in 1987. I know that the political biases of the people at the Foresight Institute, the group that first raised awareness of the potentials of nanotech, are, like mine, in a generally libertarian direction. Hence when I observed some of them arguing that the potential dangers of nanotech were sufficiently serious that some sort of government regulation might be necessary I took the argument more seriously than I would have if it came from people with more conventional political views, left or right.

Prediction as the Test

One way of evaluating both theories and experts is by comparing their predictions to what happened. We know that the Ehrlichs’¹ predictions cannot be trusted because they predicted, with great confidence, mass famines in the 1970’s, hundreds of millions of people starving to death and it didn't happen. We know that the mechanisms that produce an apparent consensus on a controversial issue cannot be trusted, because such a consensus, of which the Ehrlichs were an extreme example, existed on the dangers of population growth in the nineteen sixties and what happened since then was the opposite of what that consensus predicted. That is a reason not to give much weight to the current apparent consensus on the consequences of climate change.

Sometimes it goes the other way. I did a calculation not long ago comparing the temperature projections of the first four IPCC reports with what happened thereafter. The first report badly overestimated future warming but since then there have been two overestimates, one underestimate, all three within the predicted range. I conclude that the IPCC is not badly biasing its projections. From that and reading a good deal of several of the IPCC reports I concluded that their science was probably honest although the presentation of the results, especially in the Summary for Policy Makers, which is all most commenters will ever read, was slanted to make climate change look more threatening than the science implied.

Searching for an Honest Man

Not everyone is biased. Even someone who is biased may also be honest. The problem of recognizing an honest expert is a special case of the general issue of judging sources of information on internal evidence, a very useful skill.² If someone writes an article arguing for a conclusion, does it offer and respond to all the arguments against that conclusion that you can think of? Does it qualify its conclusions where arguments and data provide only limited support for them? Does it use emotive writing to cover weak points in its argument? Is the author willing to reject parts of what his allies support because the arguments for it are weak? An example would be James Hansen, an early and prominent voice on the dangers of climate change. He has come out in support of nuclear power, an obvious position from a logical standpoint since it provides a substitute for coal that produces no CO₂, but one rejected by most of his allies. That he was willing to go with the logic rather than the ideology increases my confidence that he is willing to follow arguments where they lead. He may be, I suspect is, mistaken in his views, but I am pretty sure he is at least telling the truth about them. An earlier example would be George Orwell, a committed socialist willing to recognize and point out problems with socialism.³

The Poverty of Our Circumstances

Often the best you can do with these approaches is to recognize what you do not know. The fact of a demonstrably false past consensus on population growth implie that the consensus on climate change cannot be relied on, not that it is false; my best guess is that it is right on the cause of climate change but may be wrong on the costs. In the Cavanaugh case, my analysis implied that he might be innocent, not that he is innocent.

Sometimes I can do better, conclude that the origin of Covid probably was a lab leak and the increased habitable area due to global warming probably within a factor of two of the current area of the US. My reason in the former case is that the disproportion in the two conditional probabilities is so large that only an extreme prior in the opposite direction would reverse the conclusion, in the latter that the argument is a simple one.

But even in those cases I could be wrong.

1. The Population Bomb was originally credited to Paul Ehrlich but later to Paul and Anne.

2. A skill untaught, often anti-taught, in conventional schooling, a point I discussed in an earlier post under the subhead “Teaching the Wrong Lesson.”

3. “It cannot be said too often – at any rate, it is not being said nearly often enough – that collectivism is not inherently democratic, but, on the contrary, gives to a tyrannical minority such powers as the Spanish Inquisitors never dreamed of.” (Review by Orwell: The Road to Serfdom by F.A. Hayek / The Mirror of the Past by K. Zilliacus Observer, 9 April 1944)