Tuesday, June 09, 2015

Judging Outside Your Expertise

I have just been involved in a lengthy exchange on Facebook over my criticism of the claim that warming on the scale projected by the IPCC for 2100 can be expected to have large net negative consequences. The response I got was that the person I was arguing with was not interested in my arguments. He does not know enough to judge for himself whether the conclusion is true, so prefers to believe what the experts say.

Accepting the views of experts on a question you are not competent to answer for yourself, assuming that you can figure out who they are and what they believe, is often a sensible policy, but one can sometimes do better. Sometimes one can look at arguments and evaluate them not on the basis of the science but of internal evidence, what they themselves say. Here are three examples:

The widely cited 97% figure is based mostly on Cook et. al. 2013, which is webbed. It is often reported as the percentage of climate scientists who believe that humans are the main cause of warming and that warming will have very bad effects. Simply reading the article tells you that the second half is false. The article is about causes of warming and offers no evidence on consequences. Anyone who says it does is either ignorant or dishonest, and other things he says can be evaluated on that basis.

If you read the article carefully you discover that the 97% figure, which is a count of article abstracts not scientists, is the percentage of abstracts which say or imply that humans are *a* cause of warming (“contribute to” in the language of one example). The corresponding figure for humans as the principal cause, which is not given in the article but can be calculated from its webbed data, is 1.6%. That tells you that anyone who reports the 97% figure as the number of articles holding that humans are the main cause of warming is either ignorant or dishonest. One person who has done so, in print, is John Cook, the lead author of the article. John Cook runs skepticalscience.com, which is a major source for arguments for one side of the global warming dispute, so knowing that he is willing to lie in print about his own work is a reason not to believe things on that site without checking them. [My old blog post giving details]

One of the economists who has been active in estimating consequences of warming is William Nordhaus. He is, among other things, the original source for the 2° limit. A few years ago, he published an article in the New York Review of Books attacking a Wall Street Journal piece that argued that climate was not a catastrophic threat that required an immediate response. In it, he gave his figure for the cost of waiting fifty years instead of taking the optimal steps now—$4.1 trillion dollars—and commented that “Wars have been started over smaller sums.” What he did not mention was that that sum, spread out over the rest of the century and the entire world, came to about one twentieth of one percent of world GNP. He was attacking the WSJ authors for an argument which his own research, as he reported it, supported.

In a recent Facebook exchange on the consequences of AGW for agriculture, someone linked to an EPA piece on the subject. Reading it carefully, I noticed that the positive effects of warming and CO2 fertilization were facts, with numbers: “The yields for some crops, like wheat and soybeans, could increase by 30% or more under a doubling of CO2 concentrations. The yields for other crops, such as corn, exhibit a much smaller response (less than 10% increase).” The negative effects were vague and speculative: “some factors may counteract these potential increases in yield. For example, if temperature exceeds a crop's optimal level or if sufficient water and nutrients are not available, yield increases may be reduced or reversed.” The same pattern held through the article.

A careful reader might also notice that the piece referred to the negative effects of extreme weather without any attempt to distinguish between extreme weather that AGW made more likely (hot summers), less likely (cold winters), or would have an uncertain effect on (droughts, floods, hurricanes). It was reasonably clear that the article was designed to make it sound as though the effects of AGW would be negative without offering any good reason to believe it was true. One telling sentence: “Overall, climate change could make it more difficult to grow crops, raise animals, and catch fish in the same ways and same places as we have done in the past.” With most of a century to adjust, it is quite unlikely that farmers will continue to do everything in the same ways and the same places as in the past.

These are three examples of arguments for one side of the climate controversy by a source taken seriously by supporters of that side. Each can be evaluated on internal evidence, what it itself says, without requiring any expert knowledge of the subject. In each case, doing so gives you good reasons not to trust either the source or the conclusion.

Readers may reasonably suspect that I too am biased. But nothing I have said here depends on your trusting me. In each case, you can look at the evidence and evaluate it for yourself. And all of it is evidence provided by the people whose work I am criticizing.

15 comments:

Michael Strong said...

Concise, appropriately focused, and therefore brilliant.

Anonymous said...

Are you familiar with the work of Jose Duarte, who does a lot of work investigating scientific malpractice in studies and papers? He has a lot to say about how the Cook study is outright fraud.

http://www.joseduarte.com/

David Friedman said...

I'm not familiar with the work of Duarte. I gather that Tol has also criticized the study. I wouldn't be surprised if much of it was fraudulent, but demonstrating that to an interested third party would be considerably harder than demonstrating that, even if we assume it is entirely legitimate, Cook's later statement misrepresents its results by a lot—97% vs 1.6%.

Joey said...

David, I encourage you to check out his blog. He's really interesting.

Duarte uncovered a lot of foul play on that Cook paper including the messageboard where survey respondents engaged in blatant misconduct seemingly with Cook's approval.

He also frequently writes about discrimination against Conservatives in the social sciences (he's a libertarian), so you may have come across some of his work in the past.

Worth checking out his site.

The Cook article:

http://www.joseduarte.com/blog/cooking-stove-use-housing-associations-white-males-and-the-97

Anonymous said...

What is the expertise of economists?

In 1952 a PhD economist, Raymond Goldsmith, wrote that the depreciation of durable consumer goods is ignored. If you look up the equation for NET Domestic Product you find that depreciation is subtracted from GDP. But if you research a little further you find that depreciation is CAPITAL GOODS ONLY.

Look in some Economics 101 books for NDP and you find it get about half a page in a 400 page book.

So where is Planned Obsolescence in all this. How much unnecessary CO2 production came from that?

So are economists internally consistent?

Fred Mangels said...

One problem I've found is Believers (in AGW) won't accept any input from those who aren't in line with AGW theory.

There was some back and forth in the comments to a story related to climate change in the Santa Rosa Press- Democrat a while back. A partisan left wing Believer cited the 97% figure. I pointed him to your previous post debunking the 97% figure. He wouldn't read it, stating you aren't a "climate scientist" so your conclusion wasn't valid.

I pointed out you may not be a climate scientist but, unlike him and most of the rest of us, you've actually read the papers the supposed scientists have produced. He simply reiterated that you weren't a climate scientist so what you thought about AGW was irrelevant.

I've found that to be the rule rather than the exception among Believers. Anyone not in line with AGW dogma is either a paid mouthpiece of the fossil fuel industry, or has been debunked in some other way. Then they'll continue with the 97% of scientists agree line.

It happened with Judith Curry after she fell out of favor with Believers. Once she said something along the line of "Maybe we don't have this quite right", in regards AGW models not working as predicted, she was cast aside from the AGW movement. Now if you mention her name in that crowd, you'll usually get the "She's been debunked so many times..." line.

Thomas Woods said...

David, "not a catastrophic threat" should be "now a catastrophic threat." Thanks for writing this great post.

Thomas Woods said...

David, "not a catastrophic threat" should be "now a catastrophic threat." Thanks for writing this great post.

Joey said...

Tom, I think the original is right.

The authors of the WSJ article argued that AGW isn't a big deal; Nordhaus, who thinks action should be taken, critiqued the article.

EliRabett said...

Cook et al, reported a second survey which was the result of their asking the authors of the papers what their opinion was and the result matched the abstract survey.

Moreover, there have been several other surveys, the majority of which approximate the 97% figure. The Wikipedia summarizes them

David Friedman said...

Eli: The second survey also pooled categories 1-3, as you can see by looking at footnote a to table 4.

The crucial question I'm raising is not whether Cook's conclusion was correct but whether he lied about the results of a paper of which he was the lead author.

Of the other papers, the one I have looked carefully is Anderegg. It started with a sample of 1372 climate researchers, classified them by public statements pro or con the IPCC position. About two thirds were classified as pro, about one third con. To discover that you have to read the supporting information--unless I've missed it, the article itself omits those numbers and only gives you the statistics for the most published authors (50, 100 or 200), which is where the 97%-98% figure comes from. That might suggest that supporting the current orthodoxy gets you more publications than opposing it.

Further, Anderegg classifies all IPCC AR4 Working Group I contributors as "pro." So far as I am aware, contributing to some part of a large research project does not imply that you agree with the project's conclusion. That comes to 619 names--a sizable majority of the total pro group.

A commenter on my old blog post on Cook et. al. 2013 reports:

"The other source of the (questionable) 97% support statistic was a survey done in 2009 by the University of Illinois.( Here's a gated version- http://onlinelibrary.wiley.com/doi/10.1029/2009EO030002/abstract).

The final number was based on 75 out of 77 respondents (of significant eminence) who answered two questions, the first utterly uncontroversial even to a climate skeptic and the second very vague. "

You have to actually look at papers to see what they say. I suggest that the examples of Cook and Anderegg suggest papers designed to make the support for the position look stronger than their evidence shows it to be.

EliRabett said...


David, you still have not dealt with the second confirmatory survey in Cook, et al. As to whether he was correct or not, Eli has some experience with such surveys, having participated in one ten years ago which never was published. The design was less ambitious but there were better checks, such as ratings by two people with reconciliation and a test set. There were also several different categories. The results are contained in two posts

http://rabett.blogspot.com/2013/05/the-past-is-prologue.html
http://rabett.blogspot.com/2013/05/cross-tabs.html

The long and short is that out of 650 papers judged by their abstracts 14 papers explicitly or implicitly rejected human driven climate change and 480 accepted it. You can do the math.

Anderegg found that those rejecting AGW were only 2% of the top 50 climate researchers (e.g. 1 of 50 ), 3% of researchers of the top 100 (3 of 100), and 2.5% of the top 200 (5 of 200). http://www.pnas.org/content/107/27/12107.full (open)

The fact that the percentage did not vary much across the three categories is indicative that your guess is wrong, but if you want to try, take a look at the list of climate scientists by citation put together by Jim Prall. In looking through the list, you should be aware that some of the entries, such as Freeman Dyson, while they may be high on the list by citation, have never published on climate (much the same could be said about Eli who is also there under an alias). Moreover, Jim has been good enough to mark IPCC Report authors, so you can check that too (Hint: 97% agreeing with AGW looks to be a pretty good estimate) Eli will even spot you John Christy and even Chris Landsea.
http://www.eecg.utoronto.ca/~prall/climate/climate_authors_table.html.

Further, your characterization of the two questions asked in the U Illinois survey was not quite spot on:

1. When compared with pre- 1800s lev-els, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?

2. Do you think human activity is a significant contributing factor in changing mean global temperatures?

BTW, it appears to be open http://onlinelibrary.wiley.com/doi/10.1029/2009EO030002/epdf (open)

David Friedman said...

"David, you still have not dealt with the second confirmatory survey in Cook, et al."

Sure I have. My repeated point is that the 97% figure is the figure for categories 1-3, where only category 1 is "principal cause." Hence when Cook claims "main cause" in the second paper, he should be limiting himself to category 1.

The second survey lumps together categories 1-3, as you can see by footnote a in the table. So, again assuming the paper is honest, about 97% of researchers say humans are a cause of warming. But Cook in the second paper is making a claim about humans as the main cause of warming. I don't know how the second survey breaks down among the first three categories--the paper doesn't say. But the data for the count of abstracts are webbed, and Category 1 is only 1.6%.

As you should be able to see from reading both this blog post and the earlier one I linked to, the question I'm raising is not how many people support what but whether Cook lied about his own work. Cook et. al. 2013 is my source for 1.6% as the number that said humans were the principal cause.

Do you agree that Cook lied about the results in Cook et. al. 2013? If not, how do you explain the shift from 1.6% holding that humans are the principal cause to 97% holding that humans are the main cause? In the second paper, he claims that the first paper gave that result--and it didn't.

If you read the supplementary information in Anderegg, you discover that of the whole sample about 1/3 rejected. The high percentage accepting was the result of eliminating all save the ones who published a lot. That might mean experts all agree. It also might mean that it is harder to get published, or less useful to your career to get published, if you question the current orthodoxy. You might also note that Anderegg includes in its "pro" category everyone who contributed to the IPCC report--and they make up a sizable majority of the whole category. Contributing to one piece of a report doesn't imply that you do or don't endorse the conclusion.

"A signifant contributing factor" doesn't correspond to "the main cause."

Doctor Mist said...

@Joseph Miller- That post by Duarte was just brilliant. Anybody who has any position on the 97% paper should read it.

the inquisitive neurologist said...

I can add another article to this list of infamy. It was published in Science:

Science. 2002 Dec 6;298(5600):1987-90.
Grassland responses to global environmental changes suppressed by elevated CO2.
Shaw MR1, Zavaleta ES, Chiariello NR, Cleland EE, Mooney HA, Field CB.

At the time it was published I tended accept the claims about global warming, based on the consistent information I was receiving from multiple trusted sources. Then science blogs breathlessly reported that "Carbon dioxide reduces plant growth! We will all die! We must do something right now!", citing the article above. This triggered suspicion - Carbon dioxide reduces plant growth? Really? So, although not a climate scientist, I read the original article and it turns out that carbon dioxide actually increases plant growth (no surprise there), but the exact amount of increase depends on additional factors (moisture, temperature, nitrogen). Yet, both the authors in blog interviews and journalists doggedly tried to spin the data the other way, painting a picture of impending carbon dioxide doom.

Needless to say, this blatant and concerted propaganda effort had the opposite effect on me - I really dislike being lied to, and here I had the discrepancy between duly reported data and their interpretation cast in stark relief, completely unmistakable. I actively investigated additional climate information sources, ended up reading Steve McIntyre, Anthony Watts, Roy Spencer and other forbidden lore. I became a well-informed climate realist, all because a few people's brazen misrepresentations were easy to check online.