I read an article a long time ago, can't remember where: Supposedly some of "the hottest" examples were created by looking back at previous "hottest" years. Recalculating them and deciding they weren't really as "hot" as they were reported to be. The recalculations always showed them to be cooler and not warmer. Since they were now cooler than they were thought to be, at least originally, the new "hottest" years, which conveniently fell within the AGW period of now supported the arguments for AGW. If, and I stress if, this is true it seems to throw all of the statistical (mystical?) analysis in a cocked hat.
Further it seems that most of the US weather stations are not in compliance with the Weather Bureaus own standards (96%?) and the data is "adjusted" using algorithms. Have no idea about stations outside of the US though it was reported a long time ago that when the Soviet Union collapsed they shut down a number of the stations in Siberia so data from that region was suspect.
As I said I have no idea if any of this is true\not true and unfortunately there doesn't seem to be much interest in finding out. Maybe the first part of lying with statistics is lying about the initial data or the recalculated data used in the statistical analysis.
IIUC, what Lovejoy has actually shown is that, at the 5% significance level, SOMETHING happened to the climate in the last 150 years that didn't happen in the preceding 375; it remains to be seen what.
So for simplicity, let's take that as a fact:_something_ happened to the climate in the last 150 years that didn't happen in the preceding 375. What can we say about the possibilities? We can rule out natural cycles with a period of less than about 300 years, or more generally "black swan" events with an annual probability more than about 1/300, because we would have seen them in the 1500-1875 period; we can't rule out natural "black swan" events with a lower probability than that. In fact, let's take "annual probability < 1/300" as our definition of "black swan".
Now let's pretend we hadn't done any climatic observations yet, and were interested in such hypothetical natural black-swan climate-warming events (henceforth NBSCE). _A priori_, the likelihood of such an event happening in the past 150 years is at most 1 - (1-1/300)^150) ~= 0.4 (possibly much less, depending on how black-swan events are distributed, which we don't know). Meanwhile, the likelihood of human activity that could plausibly affect the climate happening in the past 150 years is 1: we have extensive records of it. But we don't know for certain that it _did_ warm the climate significantly. I don't know how one would estimate that likelihood _a priori_; perhaps 2/3, based on what we know about the greenhouse effect?
So these are our priors: a probability less than 0.4 of a natural black-swan event significantly warming the climate, and a (presumably independent) probability of perhaps 2/3 that the human activity we know about has significantly warmed the climate.
P[both] = .267
P[AGW only] = .4
P[NBSCE only] =.133
P[neither] = .2.
If (with Lovejoy) we add the observation that the Earth's climate _has_ warmed significantly more in the past 150 years than in the preceding 375, we can with high confidence rule out the scenarios in which nothing new happened to warm the climate in the past 150 years. Which is the .2 in the above list of scenarios. So we divide each of the remaining probabilities by 0.8, producing
P[both | warming] = 1/3
P[AGW only | warming] = 1/2
P[NBSCE only | warming] = 1/6
We can't reject (at, say, 5% significance) the hypothesis that the warming has been entirely due to natural causes, but if you had to put money on it, you would bet on AGW-only over NBSCE-only at three to one, and you would bet on AGW happening over AGW _not_ happening at five to one.
There are a lot of guesses in the above. Maybe the _a priori_ probability of human activity warming the climate is only 1/2 or 1/3, not 2/3, which would make the case for AGW weaker. Maybe the plausible NBSCE's have a probability of 1/500 per year, not their ceiling of 1/300, which would make the case for AGW stronger. All in all, I think the statistical case for significant AGW is pretty strong, though not a "slam-dunk".
Your blog seems to be offline, but surely the answer to your point is: Yes, you can alwqays legitimately say that a study over n years shows nothing about once in n*10 year or longer events. So it is his AGW black swan vs your unknown every n*10 year event black swan. The thing about very long shot odds is how quickly they shorten on very little evidence. Take two random pedestrians on the street and assess the odds that one of them has just won $10m on the lottery - say 1 billion to one. If one of them says "I have just won 10m on the lottery" he is almost certainly drunk, but he has already drastically improved his odds as against the other guy, and if he then says "and here's an announcement I made about it on twitter 10 minutes ago" that is still pretty rubbish evidence, but he has shortened his odds from billions to 100s to one while the other guy hasn't moved. AGW has evidence which is at least as good as that twitter announcement, while the unknown has nothing.
Richard Tol has some interesting pre prints that might we worth reading, (https://arxiv.org/abs/2304.08049) and (https://arxiv.org/abs/2207.12199v3), in the latter "I do four things in this paper. First, I update my earlier meta-analysis using the same methods but much more data. Three new results emerge from this. (1) The initially positive impacts have vanished: The central estimate of the economic impact of global warming is always negative. (2) The confidence interval about the estimates has widened considerably, and now includes no impact for very considerable warming. The more we learn about the economic impacts of climate change, the better we understand our ignorance. (3) Elicitation methods give the most pessimistic results, econometric studies the most optimistic ones, with the enumerative methods and computable general equilibrium models in the middle. Two old results remain. (4) The uncertainty about the impact is skewed towards negative surprises. (5) Poorer countries are much more vulnerable than richer ones." He seems to have revised his estimate that the the initial effects of climate change are on net positive.
I read an article a long time ago, can't remember where: Supposedly some of "the hottest" examples were created by looking back at previous "hottest" years. Recalculating them and deciding they weren't really as "hot" as they were reported to be. The recalculations always showed them to be cooler and not warmer. Since they were now cooler than they were thought to be, at least originally, the new "hottest" years, which conveniently fell within the AGW period of now supported the arguments for AGW. If, and I stress if, this is true it seems to throw all of the statistical (mystical?) analysis in a cocked hat.
Further it seems that most of the US weather stations are not in compliance with the Weather Bureaus own standards (96%?) and the data is "adjusted" using algorithms. Have no idea about stations outside of the US though it was reported a long time ago that when the Soviet Union collapsed they shut down a number of the stations in Siberia so data from that region was suspect.
As I said I have no idea if any of this is true\not true and unfortunately there doesn't seem to be much interest in finding out. Maybe the first part of lying with statistics is lying about the initial data or the recalculated data used in the statistical analysis.
IIUC, what Lovejoy has actually shown is that, at the 5% significance level, SOMETHING happened to the climate in the last 150 years that didn't happen in the preceding 375; it remains to be seen what.
So for simplicity, let's take that as a fact:_something_ happened to the climate in the last 150 years that didn't happen in the preceding 375. What can we say about the possibilities? We can rule out natural cycles with a period of less than about 300 years, or more generally "black swan" events with an annual probability more than about 1/300, because we would have seen them in the 1500-1875 period; we can't rule out natural "black swan" events with a lower probability than that. In fact, let's take "annual probability < 1/300" as our definition of "black swan".
Now let's pretend we hadn't done any climatic observations yet, and were interested in such hypothetical natural black-swan climate-warming events (henceforth NBSCE). _A priori_, the likelihood of such an event happening in the past 150 years is at most 1 - (1-1/300)^150) ~= 0.4 (possibly much less, depending on how black-swan events are distributed, which we don't know). Meanwhile, the likelihood of human activity that could plausibly affect the climate happening in the past 150 years is 1: we have extensive records of it. But we don't know for certain that it _did_ warm the climate significantly. I don't know how one would estimate that likelihood _a priori_; perhaps 2/3, based on what we know about the greenhouse effect?
So these are our priors: a probability less than 0.4 of a natural black-swan event significantly warming the climate, and a (presumably independent) probability of perhaps 2/3 that the human activity we know about has significantly warmed the climate.
P[both] = .267
P[AGW only] = .4
P[NBSCE only] =.133
P[neither] = .2.
If (with Lovejoy) we add the observation that the Earth's climate _has_ warmed significantly more in the past 150 years than in the preceding 375, we can with high confidence rule out the scenarios in which nothing new happened to warm the climate in the past 150 years. Which is the .2 in the above list of scenarios. So we divide each of the remaining probabilities by 0.8, producing
P[both | warming] = 1/3
P[AGW only | warming] = 1/2
P[NBSCE only | warming] = 1/6
We can't reject (at, say, 5% significance) the hypothesis that the warming has been entirely due to natural causes, but if you had to put money on it, you would bet on AGW-only over NBSCE-only at three to one, and you would bet on AGW happening over AGW _not_ happening at five to one.
There are a lot of guesses in the above. Maybe the _a priori_ probability of human activity warming the climate is only 1/2 or 1/3, not 2/3, which would make the case for AGW weaker. Maybe the plausible NBSCE's have a probability of 1/500 per year, not their ceiling of 1/300, which would make the case for AGW stronger. All in all, I think the statistical case for significant AGW is pretty strong, though not a "slam-dunk".
Your blog seems to be offline, but surely the answer to your point is: Yes, you can alwqays legitimately say that a study over n years shows nothing about once in n*10 year or longer events. So it is his AGW black swan vs your unknown every n*10 year event black swan. The thing about very long shot odds is how quickly they shorten on very little evidence. Take two random pedestrians on the street and assess the odds that one of them has just won $10m on the lottery - say 1 billion to one. If one of them says "I have just won 10m on the lottery" he is almost certainly drunk, but he has already drastically improved his odds as against the other guy, and if he then says "and here's an announcement I made about it on twitter 10 minutes ago" that is still pretty rubbish evidence, but he has shortened his odds from billions to 100s to one while the other guy hasn't moved. AGW has evidence which is at least as good as that twitter announcement, while the unknown has nothing.
I used the book "how to lie with statistics" to teach someone who wanted to go to medical school English. It was a very enjoyable experience.
That book is one of my favorite things. Along with the Mark Twain quote.
Thanks for this post. There are other things I fear more than global warming, but it's always nice to get a rational reminder.
Richard Tol has some interesting pre prints that might we worth reading, (https://arxiv.org/abs/2304.08049) and (https://arxiv.org/abs/2207.12199v3), in the latter "I do four things in this paper. First, I update my earlier meta-analysis using the same methods but much more data. Three new results emerge from this. (1) The initially positive impacts have vanished: The central estimate of the economic impact of global warming is always negative. (2) The confidence interval about the estimates has widened considerably, and now includes no impact for very considerable warming. The more we learn about the economic impacts of climate change, the better we understand our ignorance. (3) Elicitation methods give the most pessimistic results, econometric studies the most optimistic ones, with the enumerative methods and computable general equilibrium models in the middle. Two old results remain. (4) The uncertainty about the impact is skewed towards negative surprises. (5) Poorer countries are much more vulnerable than richer ones." He seems to have revised his estimate that the the initial effects of climate change are on net positive.