More on the Hockey Stick Fraud of Dr Mann

Hi, thanks for joining. Please do join our conversations. We want participants, not lurkers. 0-post lurkers will get regularly purged / blocked.

[rayra]

http://www.technologyreview.com/Energy/13830/

In the scientific and political debate over global warming, the latest wrong piece may be the hockey stick, the famous plot (shown below), published by University of Massachusetts geoscientist Michael Mann and colleagues. This plot purports to show that we are now experiencing the warmest climate in a millennium, and that the earth, after remaining cool for centuries during the medieval era, suddenly began to heat up about 100 years ago––just at the time that the burning of coal and oil led to an increase in atmospheric levels of carbon dioxide.

But now a shock: Canadian scientists Stephen McIntyre and Ross McKitrick have uncovered a fundamental mathematical flaw in the computer program that was used to produce the hockey stick. In his original publications of the stick, Mann purported to use a standard method known as principal component analysis, or PCA, to find the dominant features in a set of more than 70 different climate records.

But it wasnt so. McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but it handles data normalization in a way that can only be described as mistaken.

Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called Monte Carlo analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!
...

--

...In PCA and similar techniques, each of the (in this case, typically 70) different data sets have their averages subtracted (so they have a mean of zero), and then are multiplied by a number to make their average variation around that mean to be equal to one; in technical jargon, we say that each data set is normalized to zero mean and unit variance. In standard PCA, each data set is normalized over its complete data period; for key climate data sets that Mann used to create his hockey stick graph, this was the interval 1400-1980. But the computer program Mann used did not do that. Instead, it forced each data set to have zero mean for the time period 1902-1980, and to match the historical records for this interval. This is the time when the historical temperature is well known, so this procedure does guarantee the most accurate temperature scale. But it completely screws up PCA. PCA is mostly concerned with the data sets that have high variance, and the Mann normalization procedure tends to give very high variance to any data set with a hockey stick shape. (Such data sets have zero mean only over the 1902-1980 period, not over the longer 1400-1980 period.)

The net result: the principal component will have a hockey stick shape even if most of the data do not.

[Ed Mahmoud]

Probably not an accident.  Probably got exactly what they wanted.


Monte Carlo simulation is done in oilfield project economics, where variables like geologic success, currency risk, commodity price etc. can affect the economics, and I've seen papers where it is used to fill in data in reservoirs between known data points.

[mitchell301]

Note the date of the article, Oct 15, 2004.  That is not to disparage the article, it is important.  The details are in the Wegman Report (well worth reading).

One should be prepared for the warmist's counter-argument though.

Their argument is that after the Wegman Report pointed out the COMPLETELY INNOCENT errors that Mann et al. had made, the team scurried back to their offices and corrected these "minor discrepancies", after which, LO AND BEHOLD, the hockey stick was still there.  McIntyre and McKitrick, being the obstinate sort, continued to point out that the new hockey stick repeated many of the same errors as the old one (although not quite so much the PCA errors as their hand had been quite thoroughly slapped by Wegman, et al.).

The problem is that now the congress has (or had) lost the desire to convene another committee like Wegman et al.

What one should be prepared to say to an objective listener is, "If M&M were right once, isn't there a good chance they are right again" (they are).

Then one should point out that all the skeptics ask for is a fair hearing and that science should prevail.

Then point out that the "TEAM" have shown themselves in the CRUtape Letters to be compromised.

Then urge your listener to write his congresscritter demanding a fair hearing and promising hell and retribution to critters who vote otherwise.

[Mike C.]

Yup - rather old news.  But this is what made McIntyre and McKitrick the most hated men in the world amongst the AGW crowd.  And they did it without access to the original algorithms.  The managed to back-calculate it all, a rather impressive feat.

[nbpundit]

The could call this scam the Enron Redux.