Thursday, May 31, 2012

Experts 2: Questioning the Experts?


Monty Python’s The Meaning of Life opens with a woman giving birth in a hospital. The camera gives us her point of view as her trolley-bed is wheeled into the theatre and as the two doctors shout commands to one another and the hospital staff. At one point she bravely asks ‘what can I do, doctor?’ – to which the doctors instantly reply ‘Nothing, dear – you’re not qualified!’ The point is clear: we aren’t supposed to resist, or even to try to help, the expert. We just have to lie back and let them do what they are trained and paid to do. Even something as normal and natural as giving birth is now carried out under the supervision of white-coated experts.

The problem with the cult of expertise is that we have no way really to check the expert's advice. We can get another opinion, but then it is just a case of expert against expert, and we are not necessarily better off. We just have to believe them: which means they have an incredible amount of power over us. I might possibly gain consolidation by finding my own area of expertise, so that at least I have something to bring to the social table. But this does not mitigate the feeling that I have that if the expert is wrong, I am liable to suffer for their mistakes – and there’s nothing I can do about it.

The case of climate change is a good example. Is it the case that the climate is changing because of human created environmental damage? Well, many of the experts are agreed that this is the case. I might say ‘science says so’; but when I do, I am just saying that the expert scientists are saying this at the moment. I personally have no idea whether it is or not in a direct sense. I am completely unable to evaluate the scientific data for myself – and, if I could, I wouldn't have time. So I am in the hands of the experts, who will decide government policies and how I should bag my rubbish. 

Now, you could have an attempt by experts to democratize their knowledge so that it becomes the possession of the masses. Al Gore’s lecture-film An Inconvenient Truth (2006) was an example of precisely this strategy. Clearly, Gore realized that blather of scientific jargon wasn’t going to win over non-experts at a deep level to the degree that their behaviours were transformed. So he packaged the expert information in a mass medium and used clever diagrams and animations to communicate his message.

But it isn’t easy to do successfully. Gore’s film was inevitably open to the accusation that he had bent the truth to fit the medium. While he was, according to scientists who reviewed the film, accurate in his main thesis and in many of the details, in some areas he was clearly guilty of rhetorical excess. The connection between hurricanes and climate change, for example, is a contentious matter amongst scientists. Yet the images of the devastating impact of Hurricane Katrina provided an alarming backdrop to Gore’s message – one that was unjustified on the basis of the scientific evidence. Or at least: that’s what the experts I consulted said… 

So, the attempt to communicate expert knowledge in a non-expert way does not give a non-expert grounds to challenge the experts. I couldn’t justifiably contradict a climate-change scientist on the basis of a viewing of An Incontrovertible Truth. The film gives me knowledge that is agreed to be largely trustworthy, but watching it does not convey to me the kind of status and authority that we accord an expert.

But clearly, experts are not always right. For a start, it is almost in the nature of expert knowledge that it is disputed. Expert disagreement is to be expected, because the level of detail and the sophistication required of expertise, in almost any area, means there is room for differing interpretations. This is more likely to be the case in more speculative fields of knowledge, of course. Expert economists will make widely varying, even contradictory, pronouncements about the future. And clearly they can’t all be right, even though they are acknowledged experts in their field. Though they have all the qualifications in the world, they may be exactly wrong. 

What’s more, cognitive neuro-scientific studies have argued that the possession of expert knowledge may in fact make experts more prone to certain sorts of error. Psychologist Itiel E. Dror argues that through their long years of training, experts develop certain ‘cognitive architecture’ which enables them to quickly access the kind of knowledge they need. That’s what makes them expert. However, Dror writes:

These information processing mechanisms, the very making of expertise, entail computational trade-offs that sometimes result in paradoxical functional degradation. 

That is: all the short cuts that experts develop in their brain structure may result in a vulnerability to certain kinds of error. Experts rely on quickly surmising the context of information; but these very habits of mind 

restrict flexibility and control, may cause the experts to miss and ignore important information, introduce tunnel vision and bias and can cause other effects that degrade performance. Such phenomena are apparent in a wide range of expert domains, from medical professionals and forensic examiners, to military fighter pilots and financial traders.

Thirdly, experts may be as prone to groupthink as any other group of human beings. They may act in a collective way, such that a powerful set of underlying assumptions is held to be unchallengeable. This may have financial consequences in the area of research grants and publications and so forth – which makes for a powerful pressure on individual experts to conform to the assumptions of the group. Or it could be that a pre-existing ideological commitment leads to a skewing of the evidence away from what is in fact the case. The impact of Marxism on the academic world of Eastern Europe in the middle of the twentieth century is a case in point. 

Given the trust we place in expert knowledge, the consequences of expert error are often woeful. Consider the case of Sir Roy Meadow, the British professor of pediatrics who made his name as an expert in the field of child abuse. His name became associated with ‘Meadow’s Law’, which stated that ‘one sudden infant death is a tragedy, two is suspicious and three is murder, until proved otherwise’. Meadow appeared as an expert witness in a number of trials, and his testimony was instrumental in gaining convictions of a number of people who had lost babies to cot death. Among these was Sally Clark, a solicitor who was tried in 1999 for the murder of her two babies Christopher and Harry. Meadow testified that the odds of a family experiencing two cot deaths were 73,000,000 to 1. A far more likely explanation, he claimed, was that Clark had smothered her children while suffering from post-natal depression. This was later shown to be a statistical nonsense. Clark’s conviction was eventually overturned on a number of grounds and she was released from prison in 2003; but the damage to her was done, and she died of alcohol poisoning four years later. 

The flaws in Meadow’s expertise were discovered, but only through the rigorous application of expertise against him. One problem was endemic to the cult of experts: Meadow was an expert in his own field, but ranged in to an area – statistics - in which he was clearly only an amateur. Yet because of the aura of expertise, he was believed by intelligent and well-meaning people, from the police to the lawyers and the juries involved, whose job it was to scrutinize the evidence available. A person who knows a lot about one thing may easily convey the impression that they know a lot about everything, when in fact they do not at all.

But there’s no way around this impasse other than the rigorous application of…expert knowledge. The Meadow case shows us that expert knowledge has a narrowness which may give it blind spots. Exposure to other fields of expertise can remedy some of the damage an expert error may cause. However, it is a slow process of checking and re-checking data and of allowing public discussion of ideas. The best we can do is equip ourselves with the kind of healthy skepticism that prepares us for the discovery that what we considered to the considered consensus of expert opinion may turn out to be entirely mistaken. We can do no more that make ourselves aware of the humanness of all knowing.

12 comments:

Anonymous said...

You might like Future Babble. It's basically Tetlock's research on expert prediction accuracy rates, in a book, with an ugly cover. That and Fooled by Randomness are brilliant. It is important to differentiate between fields of prediction where chaos theory drowns out any predictive capacity (predicting oil prices, medium term weather forecasts, currency levels etc), especially in reflexive environments, from areas of relative simplicity. Since it has been shown empirically, ad nauseum, that public "experts" in multiple fields (particularly economics) have zero predictive capacity whatsoever (sometimes negative predictive capacity), I am amazed that they retain their audience.
Steve Driscoll

http://www.amazon.com/Future-Babble-Expert-Predictions-Worthless/dp/0525952055/ref=sr_1_1?s=books&ie=UTF8&qid=1338427412&sr=1-1

Ben said...

Yep, there's too much to know which means we have to outsource which involves trust. Bummer I guess. Good piece.

Shane Clifton said...

I suspect that the problem of the 21st-century is the inability to discern between experts and rhetoricians in a world of unrestricted publication of ideas. Some of the crises we face now are precisely because people do not listen to the experts

byron smith said...

Shane - yes, precisely. There is a positive epistemic obligation to trust the settled and shared understanding of accredited experts in reputable fields of knowledge beyond our ken.

"So I am in the hands of the experts, who will decide government policies and how I should bag my rubbish."
Michael, just checking, you do know that recycling has almost nothing to do with climate, don't you?

"it is almost in the nature of expert knowledge that it is disputed."
Actually, it is in the nature of expert knowledge that much of it is not particularly disputed. We only focus on the relatively small areas of disagreement. I don't know of a single expert who disputes (to pick a few random examples) that William of Normandy won the Battle of Hastings in 1066 or that the square root of 625 is 25 or that without greenhouse gases, the earth would be on average more than 30ºC cooler.

It is possible to overstate the difficulties with accepting expert knowledge. I don't dispute that there are genuine grey areas of contested claims and that sometimes these involve very significant moral issues (indeed, often the greyness relates to the status of certain moral claims). But we're not perpetually in a state of epistemic agony over every knowledge claim.

michael jensen said...

No, we aren't in a state of epistemic agony etc...my series says this. We most ordinarily simply take our expert knowledge without fuss.

The date of the battle of hastings is not really a matter of expert knowledge, is it?

As for bagging my rubbish: recycling (which as it turns out, I mostly don't do for whatever reason (I asked for a bin but it never came) ) is frequently associated in the public mind with climate and so on. It's all the environment, ain't it?

michael jensen said...

Just to make it clear: I am happy to believe expert knowledge, and not cranks!

michael jensen said...

Just to make it clear: I am happy to believe expert knowledge, and not cranks!

byron smith said...

The point is, if you are equating recycling with climate change, then perhaps you're not paying as much attention to the experts as you ought. If you were merely referring to the common mistake of assuming that "it's all the environment" in the public mind then why reinforce an error on an important issue when there is a chance to redirect our attention in more constructive ways?

If we take the experts seriously, then the scale and pace of changes (cultural, behavioural, economic, political, industrial, infrastructural) required to significantly reduce the chances of truly terrifying outcomes in the lifetime of those alive today means recycling is unlikely to make it onto the top 20 things that ought to command our attention.

michael jensen said...

Yes, true.

I'm not 'equating recycling with cc'... it was simply an expression by which I was trying to show how government policies, driven by experts (rightly expert), intersects with the life of the non-expert.

It could be of course that it's so bad that there's nothing we can do...

byron smith said...

Fair enough. I realise that at times shorthand references are required to gesture towards something you don't really want to discuss in detail. Sorry for being picky about this, but I guess I'd see it as an opportunity to sharpen your throw-away line into something a little more interesting. Like, "So I am in the hands of the experts, who will decide government policies and whether I'm allowed to fly/whether we're allowed to keep digging up coal/how we should rebuild our cities". Though I'd seriously question whether the experts are really having that much of an influence on government policies. There isn't a national government anywhere with targets compatible with a habitable world. The Maldives were probably closest, until a coup just removed their first democratically-elected president a few months ago. Australia (even post carbon legislation) remains towards the bottom of the pack amongst rich nations, and when we include our plans something like double our coal exporting over the next couple of decades, then we become one of the biggest roadblocks. We're already the world's largest coal exporter and the 5th largest excavator of fossil fuels globally - NB not per capita. You'd be hard pressed to say that either the ALP or Coalition have policies that are within cooee of what the experts say is necessary.

"It could be of course that it's so bad that there's nothing we can do..."
That could indeed be true, though it's worth asking the next question: nothing we can do to do what? Save industrial civilisation? Prevent the extinction of humanity? Avoid the end of all life on earth? I don't need to point out that these are very different outcomes and while the first may in some sense be possibly true, I certainly don't think the latter two are. Thus, even if it is not possible for us to prevent the collapse/utter transformation of industrial civilisation as we know it, surely we still have moral responsibilities towards those who will come after and whose conditions of existence will be very significantly influenced by the choices our society makes in the next few decades.

The choices that the church makes in such a situation will also have significant ramifications for the future witness of the faithful whose reputation will partially depend on our success or failure in facing such a situation well. Will we the generation to hang another millstone round the neck of future believers, who will somehow have to apologise for the church dragging its heels and very frequently being more part of the problem than part of a healthy response?

And if it is the case that industrial civilisation as we know it has received a "terminal diagnosis", doesn't this only make it more important that the church has a pastoral, evangelistic and prophetic message (and practice) that is appropriate to such a situation? There may be nothing we can do to keep our way of life ongoing, but there is still plenty we can do to respond well to a situation in which "there is nothing we can do".

byron smith said...

Typos:

"when we include our plans *for* something like a doubling of our coal export"

"Will we *be* the generation..."

byron smith said...

(As you can see, you've hit pretty close to my hobby horse/PhD topic here...)
:-)