A simple recipe for the manufacturing of doubt

By Klaus Oberauer
Posted on 19 September 2012
Filed under Cognition
and Stephan Lewandowsky
Professor, School of Experimental Psychology and Cabot Institute, University of Bristol

Mr. McIntyre, a self-declared expert in statistics, recently posted an ostensibly unsuccessful attempt to replicate several exploratory factor analyses in our study on the motivated rejection of (climate) science. His wordy post creates the appearance of potential problems with our analysis.

There are no such problems, and it is illustrative to examine how Mr. McIntyre manages to manufacture this erroneous impression.

Our explanation focuses on the factor analysis of the five “climate science” items as just one example, because this is the case where his re-“analysis” deviated most from our actual results.

The trick is simple when you know a bit about exploratory factor analysis (EFA). EFA serves to reduce the dimensionality in a data set. To this end, EFA represents the variance and covariance of a set of observed variables by a smaller number of latent variables (factors) that represent the variance shared among some or all observed variables.

EFA is a non-trivial analysis technique that requires considerable training to be used competently, and a full explanation is far beyond the scope of a single blog post. Suffice it to say that what EFA does is to take a bunch of variables, such as items on a questionnaire, and then replaces the multitude of items with a small number of “factors” that represent the common information that is picked up by those items. In a nutshell, EFA permits you to go from 100 items on an IQ test to a single factor that one might call “intelligence.” (It’s more nuanced than that, but that captures the essential idea for now).

One core aspect of EFA is that the researcher must decide on the number of factors to be extracted from a covariance matrix. There are several well-established criteria that guide this selection. In the case of our data, all acknowledged criteria yield the same conslusions.

For illustrative purposes we focus on the simplest and most straightforward criterion, which states one should extract factors with an eigenvalue > 1.  (If you don’t know what an eigenvalue is, that’s not a problem—all you need to know is that this quantity should be >1 for a factor to be extracted). The reason is that factors with eigenvalues < 1 represent less variance than a single variable, which negates the entire purpose of EFA, namely to represent the most important dimensions of variation in the data in an economical way.

Applied to the five “climate science” items, the first factor had an eigenvalue of 4.3, representing 86% of the variance. The second factor had an eigenvalue of only .30, representing a mere 6% of the variance. Factors are ordered by their eigenvalues, so all further factors represent even less variance. 

Our EFA of the climate items thus provides clear evidence that a single factor is sufficient to represent the largest part of the variance in the five “climate science” items.  Moreover, adding further factors with eigenvalues < 1 is counterproductive because they represent less information than the original individual items. (Remember that all acknowledged standard criteria yield the same conclusions.)

Practically, this means that people’s responses to the five questions regarding climate science were so highly correlated that they reflect, to the largest part, variability on a single dimension, namely the acceptance or rejection of climate science. The remaining variance in individual items is most likely mere measurement error.

How could Mr. McIntyre fail to reproduce our EFA?

Simple: In contravention of normal practice, he forced the analysis to extract two factors. This is obvious in his R command line:


In this and all other EFAs posted on Mr. McIntyre’s blog, the number of factors to be extracted was chosen by fiat and without justification.

Remember, the second factor in our EFA for the climate item had an eigenvalue much below 1, and hence its extraction is nonsensical. (As it is by all other criteria as well.)

But that’s not everything.

When more than one factor is extracted, researchers can rotate factors so that each factor represents a substantial, and approximately equal, part of the variance. In R, the default rotation method, which Mr. McIntyre did not overrule, is to use Varimax rotation, which forces the factors to be uncorrelated. As a result of rotation, the variance is split about evenly among the factors extracted.

Of course, this analysis is nonsensical because there is no justification for extracting more than one factor from the set of “climate change” items.

There are two explanations for this obvious flaw in Mr. McIntyre’s re-“analysis”. Either he made a beginner’s mistake, in which case he should stop posing as an expert in statistics and take a refresher of Multivariate Analysis 101. Or else, he intentionally rigged his re-“analysis” so that it deviated from our EFA’s in the hope that no one would see through his manufacture of doubt.

Bookmark and Share


Prev  1  2  3  4  5  6  7  8  9  10  11  12  Next

Comments 401 to 450 out of 589:

  1. Reading the last few threads that consider the results of LOG12 has been absolutely fascinating.

    Time and again I'm reminded of the monothematic delusion somatoparaphrenia, and more usually of Anton–Babinski syndrome or other anosognosias.

    Climate change denialism seems almost to be a peculiar version of anosognosia, and indeed something much like it is recognised as symptomatic of the Dunning-Kruger effect where the afflicted can't seem to perceive their lack of ability in particular intellectual domains.

    And just with these neurological pathologies, one can spend a month of Sundays talking at a climate change denialist and they won't admit their cognitive scotoma. Witness the insistence that the survey posted on WUWT is not contaminated. Witness the repeated revisitings of the memes that climatology is a conspiracy, a fraud, that the global isn't warming or that humans are not increasing the concentration of atmospheric CO2 or that CO2 is not a 'greenhouse' gas.

    Anton and Babinski would have been mightily interested to observe how climate change denialists can see the frauds and the conspiracies and the refutations of climatological science that are not there...
  2. Addendum.

    My use of the term "neurological pathologies" was in reference to somatoparaphrenia and anosognosia and not to the D-K effect.

    Just in case any Dunningly-Krugered individuals protest otherwise...

  3. Bernard J (#456).. I have never yet met a climate change denialist who believes climate change is a conspiracy etc and I trust I never will. Most sceptics with whom I discuss climate change agree the earth is warming, that human produced C02 has a warming effect and that CO2 is a greenhouse gas. That said however, it is hard to accept that the claim that increased CO2 (from whatever cause) is the sole cause of global warming. Similarly it is hard to accept that the science is settled. The role of water vapour is not yet fully understood for although the GCM use a positive radiative forcing for water vapour the literature does give examples of water vapour, in the form of clouds having a negative radiative effect. The balance between the positiver and negative forcing effect of clouds and the relationship of that to CO2 levels is, at least as far as I am aware, poorly understood. That said I think there may be a consensus forming that any positive radiative forcing from clouds may be weak. Similarly production of soot and sulphur dioxide from the burning of fossil fuels has a negative forcing effect as do the clouds formed as a result of these compounds in the atmosphere. I am acutely of the moderator's strictures and I hope I don't get severely snipped as being off topic. I don't think I have been offensive (if inadvertently I have, sincere apologies). I write to show that not all who are not entirely convinced of the mantra CO2 is the cause of global warming have any truck with the comments in your post that you ascribe to climate change denialists
  4. "I have never yet met a climate change denialist who believes climate change is a conspiracy etc..."

    They are not hard to find online - including in some of the places where commentary about LOG12 has been posted.

    "That said however, it is hard to accept that the claim that increased CO2 (from whatever cause) is the sole cause of global warming."

    Climate scientists don't accept that claim either - climate science never argues that CO2 is the only factor affecting climate or global average temperatures. But "skeptics" frequently attack this strawman, and then often use that attack to ignore the science-based concerns.

    On the other hand, IIRC recent studies attribute between 74% and 122% of warming over the last few decades to anthropogenic CO2 - as in, there's a plausible case for arguing that that forcing is more than large enough to account for all the warming of the last few decades, and then some.

    "The role of water vapour is not yet fully understood for although the GCM use a positive radiative forcing for water vapour the literature does give examples of water vapour, in the form of clouds having a negative radiative effect."

    Firstly, water vapour != clouds. Vapour is gas; clouds are liquid water. Obviously there are mechanisms to transition between the two, and some of these lead to clouds.

    Secondly the literature also shows examples of clouds having a positive effect.

    Thirdly, IIRC some GCMs have a net negative forcing due to clouds. They still generate results that are cause for concern.

    Fourthly, scientifically competent skeptics such as Lindzen have spent more than a decade trying to demonstrate a strong negative feedback from clouds - and they have failed. The chances of a strong negative feedback saving us from the concern are small - if only because there are a number of lines of evidence that lead to significant concern without requiring highly certain characterisations of the effect of water vapour and clouds.

    So despite all of the typical "skeptic" objections, the range of plausible magnitudes of the effect of water vapour doesn't rule out strong concern. Yet most "skeptics" tend to note the cloud uncertainty, and then presume that it will influence outcomes so as to bias them all one way - towards the low end of the concern range. And they almost never note that (as we have witnessed with Arctic sea ice decline and sea level rise) the observations are on the more concerning side than anticipated.
  5. O.K. Ian, if our understanding of the dominant role of anthropogenic CO2 (and nitrous oxides and methane) was based on "mantra" then you might have a point. But it isn't. There's very little uncertainty that essentially all of the massively increased atmospheric CO2 is from human sources (how could this not be the case since a huge amount of anthropogenic CO2 is being forced into the oceans - i.e. the latter are by far a net absorber of CO2 as opposed to being a source).

    Attribution studies rather strongly indicate that almost all 20th century and contemporary warming is forced by anthropogenic sources. In fact a large (and this is the major uncertainty) of the warming has been offset by anthropogenic aerosols. Try, for example, Lean and Rind (2008) and Hansen et al (2005). I have no problem accepting that almost all (or perhaps all) of 20th century and contemporary warming is anthropogenic because that's simply what the evidence shows.

    The role for water vapour is quite well understood. Despite some disgraceful efforts to misrepresent this, the atmospheric response to greenhouse warming is occurring much as expected - i.e. an increase in absolute humidity broadly in line with expectations from physics.

    Cloud response is something else, and is indeed uncertain, but we know the effect can't be significantly to mitigate warming else we wouldn't see large warming during the contemporary period and in the deeper past. Indeed if we wished to assess what direct contemporary empirical observations of cloud response to surface warming was, we'd find that the direct evidence supports a small positive feedback (see recent research papers by Amy Clement and Andrew Dessler, for example; i.e. try Googling these names with "cloud").

    As for climate change "denialists" that believe climate change is a conspiracy, there are surely lots of these. Some of them say as much in their very public pronouncements (Mr. C. Monckton is a prime, if rather embarrassing, example 'though much admired in some quarters!). Many people that reject the scientific evidence and consider that massive enhancement of greenhouse forcing is of no concern presumably implicitly consider that "climate change" is a conspiracy, else how otherwise do they address the dichotomy beween their belief, and the interpretations of entire scientific fields and the statements from academic societies of the supposedly well-informed?
  6. ""Climate change denialism" is itself a delusion."

    It's shorthand for a range of denials within the space of the conclusions about "anthropogenic climate change" generated by climate science.

    These most certainly exist.

    ...your use of the word "symptomatic" is oxymoronic."

    "Symptomatic of" is a common idiom outside of psychiatry that means pretty much what Bernard appears to mean by it.

    But by all means have at those straw men.
  7. I'd just like to add that I find chris' contributions here to be excellent.
  8. Brad, one more thing. Even if we accept your wildly varying regard for dictionary definitions, the (say) Merriam-Webster dictionary disagrees with you:

    3: characteristic, indicative ("his behavior was symptomatic of his character")
  9. But Bernard #465 writes fun if somewhat sardonic posts. I loved his gift of new words to add to my vocabulary and laughed out loud at his imagery.

    For others, the gap widens :(
  10. @- Ian
    " I have never yet met a climate change denialist who believes climate change is a conspiracy etc and I trust I never will. Most sceptics with whom I discuss climate change agree the earth is warming, that human produced C02 has a warming effect and that CO2 is a greenhouse gas."

    You have obviously lead a surprisingly sheltered life.
    I will refrain from exposing you to egregious examples of the conspiracists by linking to examples that can be found in every days postings on certain websites like WUWT, ClimateAudit Judith Currys etc....

    But a more common symptom of climate science rejection can seen in A.Scott's post around #429 about the glacial cycles. There are rather vague statements like -
    "How does the current inter-glacial data look compared to prior inter-glacial warm peaks? Here's a hint - they come to a pretty sharp spike, and rapidly descend into another long glacial period. ...
    What about the entire "climate cycle" time frame - from glacial to and thru inter-glacial and back? They've been pretty regular for at least the past 500,000 years or more."

    Interglacial periods actually vary quite a bit, and the glacial cycle length is also variable, depending on the interaction of several orbital parameters and the size and duration of the ice sheets.
    But the intent of the hand-waving about the glacial cycle is to imply that e present warming might be some expression of a natural cycle, or that the big changes in climate in the past somehow negate the anthropogenic source of present climate change. It is not stated explicitly, but the only reason for posting such rhetorical statements with a conspicuous lack of detail is to encourage doubt in climate science, although probably with some level of deniability.
    But nothing was mentioned about the known trigger from orbital variations, or the fact that glacial cycles require the role of CO2 to explain the magnitude of warming, and that is pne strong indication for the magnitude of climate sensitivity.

    Now it could be that A.Scott had no intention of rejecting climate science with this post and I have read it completely wrong. That he was posting to support the idea that present CO2 levels will prevent at least the next glacial cycle. If so no doubt he will come and correct my error.
    But in the meantime perhaps you can see that while full blown conspiracy theorists like Monckton may be a minority, the rejection of climate science, albeit in a covert, coded manner is extremely common on the 'skeptic' blogs.
  11. Not only has Monckton suggested scientists are conspiring (to change data etc), he has reportedly promoted the 'one world government' conspiracy re climate change - and worse. He has also promoted the 'birther' conspiracy - nought to do with climate.

    Monckton would have most likely have been one of the conspiracy theory outliers on the survey had he completed it in line with what he's said in public.

    Whether he believes all the guff he comes up with or not is another matter. I wouldn't know. He's primarily an entertainer.

    (Google is your friend. I don't want to link to crazy conspiracy sites.)
  12. "Has Monckton ever said climate change is a conspiracy?"

    Depends if you're insisting on the literal interpretation of "climate change" in this context - a literalness that you don't impose on yourself in other contexts when you insist on private definitions of terms - or the common understanding of the term as shorthand, as I explained in another comment above.

    If the former, well, that's just being a prat.

    If the latter, well, you haven't tried very hard to find Monckton saying something about conspiracy theories in the context of climate change.
  13. Thanks Lotharsson and Chris for your replies. I have never "met" a sceptic with the views Bernard J ascribes to them. I'm sure there are many on the web (including Lord Monckton) but that is a different entity. With regard to sceptics looking at clouds and biasing all one way, I did note that there may be a consensus forming that clouds have a weak positive effect so I guess that is saying the effect is all one way. Chris as for the statement from academic societies, I think that some members of those societies have protested that these statements do not reflect their views. Also as mentioned above I did comment that clouds may well have a positive bias. I'm surprised you didn't mention Dr Roy Spencer who also has some comments on the effects of clouds that aren't entirely mainstream
  14. lot's of examples from the repellent and disgraceful Mr. Monckton:

    In answer to the question "Why would the climate change lobby and governments either exaggerate or totally invent the threat of global warming"...

    Mr. Monckton enters into an unsubstantiated rant starting with these words:

    "Well we know that the class politique and the class scientific, the great and the good are conspiring together to make up the evidence, because... "

    Have a listen if you have the stomach for it. The above question and answer starts around 3:20 minutes into this interview.

  15. #470 - yes. Don't you know how to use google?
  16. "Don't you know how to use google?"

    Go easy on him - he doesn't know how to read the IPCC reports and follow citations either. If it isn't handed to him on a plate, then he reckons it probably doesn't exist ;-)
  17. "Such as?"

    There have been a number of discussions on threads at this site about this over the last week or two. You don't remember?
  18. @- Brad Keyes
    "Has Monckton ever said climate change is a conspiracy? Quote?"

    Have you been living under a rock or is this a 'POE' ?!


    .. Interviewed on Monday morning by Alan Jones on Sydney radio station 2GB, Monckton warned that the aim of the Copenhagen draft treaty was to set up a transnational government on a scale the world has never before seen.
  19. There's a number of comments about sceptical blogs and those that run them but the number of people that are aware of their presence surely isn't a significant percentage of the global population compared with those that read the MSM. As is apparent from polls, in America most may believe global warming is real but base this on unusual weather rather than on long term climate changes. Many respondents also don't think it is caused by humans and only around 8% mentioned scientific research.
  20. And to think that some people have been claiming the outliers on the survey must be faked responses. They've not been following the posts here (or elsewhere)!
  21. If I'm not mistaken, Monckton believes climate change is a physical reality.

    Yep, you're insisting on being a prat!
  22. "Which is kinda the opposite of what you allege"

    Nope, I was pointing out a common usage.

    And you appear to have conveniently erased memories of people pointing out your private definitions that are not in common usage.

    Kind of the opposite of the opposite that you allege, or in other words, what I said.
  23. Ian, sadly Dr. Roy Spencer has plenty of "comments" but zero evidence on the effects of clouds as either having a positive or negative feedback.

    This is rather typical unfortunately. Dr. Spencer (with his collaborator Dr. John Christy), insisted over and over through a period lasting around 15 years from the early 1990's that the troposphere was not warming (in fact in the early days they asserted that it was cooling) on the basis of astonishingly misguided misanalyses of their satellite MSU data. Competent scientists made a serious of corrections culminating in an embarrassing climbdown by Spencer and Christy in the pages of Science (Science 310, 972-973, 2005).

    As two of the competent scientists stated in the discussion on those pages:

    "Once we realized that the diurnal correction being used by Christy and Spencer for the lower troposphere had the opposite sign from their correction for the middle troposphere sign, we knew that something was amiss. Clearly, the lower troposphere does not warm at night and cool in the middle of the day. We question why Christy and Spencer adopted an obviously wrong diurnal correction in the first place. They first implemented it in 1998 in response to Wentz and Schabel (1), which found a previous error in their methodology: neglecting the effects of orbit decay."
    C. A. Means and F. J. Wentz on page 973 of the issue of Science

    That scientists spend the better part of half a career getting their analyses hopelessly wrong is sad, if forgivable. That they might find it almost impossible to cast off long held incorrect interpretations about Earth response to greenhouse forcing is understandable (since scientists make strong emotional investment in their work and its interpretations). That they continue to misrepresent the science is less acceptable. One wonders what the appeal is to the opinions of Dr. Spencer...

    And note that with respect to the relevance to the subject of these threads there is lots of insight into various aspects of motivated rejection of science in this particlar example...
  24. A Scott @438, the question is whether comments exist which the majority of your respondents are likely to have read, and which are likely to have influenced their responses. The answer is yes - and clearly so. What is more, those comments are likely to have influenced your respondents to score lower levels of agreement on the FM questions than they might have done without that influence. That is critical to the issue of whether the results of your survey have any value, and specifically whether they have any value for comparison with the results of Lewandowsky's survey as has been done by McIntyre.

    I quite agree that discussion of the questions surrounding Lewandowsky's survey also call into question the results of that survey - although the discussion was nowhere near as extensive at the time of the survey, and hence far less of a factor.

    A Scott @439, you are apt at missing the point.

    I have previously voiced my opinion of the 4 point scale used by Lewandowsky, and agree that the five point scale is better, though still inadequate to the task. That is not the point. The point is that McIntyre compares the results of your survey with the results of Lewandowsky's and explicitly bases his analysis on the assumption that a score of 5 (Strongly Agree) on your survey is equivalent to a score of 4 (Strongly Agree) on Lewandowsky's. That assumption is not valid. People assign value to responses not just based on the terms used, but also based on the number of available responses. Because of that, the value assigned to 5 on your survey will over lap with, but not be coterminous with a value of 4 in Lewandowsky.

    These two issues, ie, direct discussion of the FM questions suggesting that accepting them was an extreme position, thereby discouraging strong agreement, and a different range of values which gives "strongly agree" a stronger relative value in the five choice survey mean we would not expect equivalent distributions between the two surveys for "strongly agreed" responses to FM questions. That we would expect an equivalent distribution is, however, McIntyre's key premise, without which his whole structure of dodgy inferences does not even start. Ergo his analysis is rather pointless, and certainly does not prove that responses with a large number of "strongly agreed" in the Lewandowsky survey are scammed responses.

    As it happens, there is independent evidence that they are not - but that is beside the point.
  25. "Otherwise how will I learn?"

    Well, just like you could Google or read the IPCC reports and citations if you chose, you could go back and read these threads.

    As it stands you're getting a reputation as the guy who wants other people to do your homework, and blames them if you fail the course.
  26. I'm sure it does, the operative word being "outside." It's a dead giveaway that he's never studied the field in which he's holding forth.

    It might be a "giveaway" if I hadn't already stated on Shaping Tomorrow's World that my area of expertise is ecology, and prior to that immunology.

    Oh, I've studied psychology for several years as a postgrad at university, but I never pretend to practise it.

    As Lotharsson notes, your latest squirrle is but another straw man.

    And as for the actual use of "symptomatic", it seems to have escaped your notice that I have a propensity for lumbering metaphors of greater or lesser degree. I know that comprehension of metaphor requires higher cognitive ability than some other linguistic instruments, but if you really don't understand the context in which the word was used, it's no surprise that you struggle with understanding the very basics of climate science.
  27. And as you're so fixated on fields of study, Brad Keyes, what is the extent of your educational/professional exposure to the sciences of climatology and psychology?

    Just as a matter of interest...
  28. @- Brad Keyes
    "If I'm not mistaken, Monckton believes climate change is a physical reality. Not a conspiracy."

    You are mistaken.
    There is abundant evidence that Monckton does not believe physical reality, never mind climate change.
    In fact he gives every indication of living in a parallel universe in which the laws of physics, chemistry and biology operate in some entirely distinct manner rendering the science research on this planet wrong so that he has to restate the explicit findings of the scientists to prevent the dissonance with his own beliefs.

    As the debacle with his 'Eternity' game indicates even his mathematics is not of this world....
  29. Lotharsson at 14:15 PM on 26 September, 2012
    Rather amusing that A. Scott goes off on a riff about the use of the word "bowdlerized" and totally ignores the clarification of the concerns about pump-priming that were made in the same comment.

    Nope - another outright falsehood. Addressed in deatil in a reply to Tom Curtis
  30. Sou at 14:50 PM on 26 September, 2012
    I find many of the assumptions peculiar, for example:

    (-snip-) they aren't worth a response. But I'll oblige anyway.

    You attack using "someone else's survey instrument (even if not specifically labelled proprietary)." Which perfectly illustrates how completely ridiculous your claims are. Answered many times, yet you and others just keep repeating the same tired attacks. It is not going to become any more true by repeating it.

    You clearly have not apparently read the paper - the authors own words:
    The free-market items were taken from Heath and Gi ord (2006). Most of the conspiracy items were adapted from [no they were essentially identical-and they would be of little worth if changed previous research (e.g., Swami et al., 2009)

    This replicates previous work (e.g., Heath &
    Gi ord, 2006)

    Finally, we replicated the finding that perceived scientific consensus is associated with acceptance of science
    They used the work of others and replicated that work - plain and simple.

    Regardless - your attack also fails on the pure common sense grounds. It would be (-snip-) to attempt to gather data that accurately represents skeptic views on this survey - if one used a different survey.

    I've already repeatedly answered the question about the choice of Watts Up With That (WUWT) ... It is one of, or the, largest site discussing climate change on the net. No other climate change related site I have found has even a fraction of the reach and coverage as WUWT.

    I compared each and every site noted by the original authors in Alexa and posted the results here in prior posts. WUWT is ranked apprx. 19,000 (smaller number being better) in the world - the ranking on the 8 pro and 5 skeptic sites ranged from appx 140,000 to 1,500,000 largest sites.

    The combined reach of all 13 sites noted by the orig authors is a fraction of the global reach of WUWT.


    WUWT is also a skeptic leaning site - exactly the group the authors were studying.


    Which makes zero sense - either from a common sense perspective - nor most certainly from a "scientific study"

    WUWT is, as I have proven - both by research, and action, the biggest and best source to obtain skeptic leaning views.

    Please explain exactly why going to one of the largest online climate science communities, well known to represent skeptical views, by any measure could be consider an improper place to obtain data on skeptic views of climate science?

    That you would challenge the fast response shows you have nothing left to attack with. (-snip-). Each and every visit was tracked and every survey timestamped. (-snip-).

    Your attack challenging whether a "five point scale is an improvement on a four point scale" is not worth yet another response. Read the literature yourself if you don't believe the answers I've given.



    This is a study whose stated purpose is to define what skeptics think regarding climate science - to prove the authors theory about skeptics "motivated ejection of climate science" ...

    Yet you ridicule using one of the largest skeptic leaning sites, where you yourself claim "99% of visitors reject climate science", as a source for data on what skeptic think and believe.

    Moderator Response:

    "Crying wolf" via claims of "ad hominem" is a hallmark complaint of those losing an online discussion thread. Patently transparent, false and snipped. Cease.

    Future comments constructed thusly will be deleted in their entirety. FYI.

  31. Tony Curtis #487, it would be useful to see the independent evidence that the extreme free market results are not scammed.

    It would also be interesting to compare the distribution obtained on Free Market from L. et al with those from Heath et al cited in L. et al.. They both used the same questions (Heath's cut down list, L. et al's pre-PCA), and the latter's sample frame and method of response avoids many of the pitfalls in L. et al.

    Putting aside scamming it would give some indication of just how "normal" the L. et al. sample was when it came to Free Market attitudes.
  32. izen at 15:49 PM on 26 September, 2012
    Do you have any serious expectation that your results will show anything different from the common finding that the ideological beliefs {and conspiracist ideation} of a person shape their tendency to accept of reject climate science ?

    I've stated my goals and intent many times. To do what the original authors failed in my and many others opinions, to do; collect data from skeptics that reflects the views of skeptics, in a sufficient quantity to accurately reflect skeptics true beliefs and views.

    Had the original authors done this, instead of sourcing all their data on skeptics thru strongly anti-skeptic sites, this exercise would have been likely unnecessary.
  33. HAS @494, you may find the following table interesting. It shows the correlation of the six FM items (FM) with either the five climate change items (CC) or the 12 Conspiracy Theory Items (CY12) using either the full data set, removing the two most obviously scammed responses, removing 35 FM outliers (defined as having a summed FM score greater than 21, but not either of the 2 worst), and removing both sets from the data. You will notice that removing bot sets reduces the negative correlation with CC items by only an additional 0.004, hardly consequential. More importantly, removing both sets restores the CY12/FM correlation to that of the full set showing that the FM outliers excluding the to worst actually reduce the negative correlation with the CY12 items. In other words, if they were scammed, the were scammed to make "skeptics" look good rather than to make them look bad. There is more, but that should be enough to show how implausible McIntyre's theory is.

    Correl Full -2 Worst -FM Outliers -Both
    CC/FM -0.768 -0.765 -0.728 -0.724
    CC/CY12 -0.139 -0.114 -0.168 -0.139
    CY12/FM 0.01 -0.019 0.043 0.009
    Affirm 0.089 0.087 0.089 0.087
    Absol 0.529 0.528 0.525 0.524

    With regard to Heath 2006, I compared their reported correlations for Human Causes with CauseCO2 responses from Lewandowsky, and Negative Consequences with the mean CCWillNeg and CO2HasNeg. The approx 0.3 difference in the correlations is easily explained by the facts that:

    1) Heath 2006 surveyed a Canadian city, while the majority of respondents to Lewandowsky would have been from the US. US citizens in general are far more in favour of free markets than those of other nations including Canada.

    2) Heath 2006 took a random sample of the entire population of the city, whereas Lewandowsky by design sampled people who exhibit considerable ideological/political commitment. That means we expect Lewandowsky's data to exhibit more extreme positions.

    Both of these factors are large. Whether they are large enough to account for all of the difference is a matter of conjecture, but it is certainly not the case that the difference between Heath 2006 and Lewandosky (In press) is sufficient to require the assumption of scammed responses to explain the difference (in this case).

    Correl Heath 2006 -2 Worst -Both
    HuCauses* -0.43 -0.722 -0.679
    NegCons* -0.4 -0.761 -0.727

    Finally, I am not the famous actor.
  34. "Addressed in deatil in a reply to Tom Curtis"


    Your reply to Tom Curtis at #433 was fairly weak, but I was referring to your non-reply to even more serious issues outlined by John Sully at #434 - focusing instead on John's use of "bowdlerized". You still appear to lack understanding of what the issue is or how detrimental it is to your claim to be merely replicating the previous survey on a larger scale.

    "Finally, we replicated the finding..."

    As a further example of your lack of scientific competence, I think you'll find this quote, which you took the time to bold for emphasis [differently from my bolding above] doesn't mean what you claim it means.

    "This is a study whose stated purpose is to define what skeptics think regarding climate science..."

    Really?! I expect that will come as a shock to the LOG12 authors.

    Because it's pretty well known what skeptics think regarding climate science and I don't recall that being a stated aim of the paper. Wasn't there something about relationships between certain sets of beliefs covering both "skeptics" and non-"skeptics"...wait a minute, let me look at the abstract. OK, here it is [my emphasis]:

    We report a survey (N > 1100) of climate blog users to identify the variables underlying acceptance and rejection of climate science. ... This provides empirical con firmation of previous suggestions that conspiracist ideation contributes to the rejection of science. Acceptance of science, by contrast, was strongly associated with the perception of a consensus among scientists.

    Hmmm, definitely doing more than figuring out "what 'skeptics' believe". Reporting relationships between various sets of beliefs, some of which aren't at all "skeptical", tends to give that away.

    You give every impression of not knowing what you need to know to achieve your stated aims, and of being unaware of the fact despite a fair bit of information being provided to you on these threads.
  35. Brad Keyes, you write: "And yet you can't produce a single quote to this effect."

    It is odd to me that you cited Monckton and yet don't seem to know the most pertinent facts about him.

    But since you're interested in learning more about Monckton's denial, and seem to have great trouble using Teh Google, here's another classic Monckton denial of physical reality. That "graph" is a classic of the genre.

    There's a whole series of posts on Monckton at Deltoid, if you're further interested. Knock yourself out!
  36. AScott #493 - You've made assumptions about the assumptions I wrote.

    In most cases, it looks as if you've validated my list as your assumptions but missed the mark completely in regard to why I thought they were peculiar.
    Moderator Response: As a general note, comment numbering is off on this thread due to Mr. Keyes opting to recuse himself from all participation in this venue.
  37. re Mr. Keyes,

    I did wonder whether he might be someone's experiment, e.g. to explore the extent to which one individual might sustain an "argument" based entirely on the delightful possibilities for contrived semantic confusion inherent in variable meaning of words and phrases. Whether or not it was an experiment, it was certainly an extreme example of a strategy much used (if far less blatantly usually) in motivated rejection of science.

    As I'm sure someone has said in a similar context "The word is not the thing"!
  38. Tom Curtis (anywhere else I would have said it was a Freudian slip), I had thought you had reached the view that potentially scammed responses had made that paper problematic. The R2 isn't the only issue as as I recall McIntyre wasn't criticising on this account when drawing attention to the outliers. BTW I went for a look for the raw data on line and couldn't find.

    Similarly I could locate Heath et al's data, but your comments on the reason why the correlation might be smaller in this study compared with L. et al. hide some complexity.

    If Canadians are less free market than US and there are (as some are attempting to claim) various causual relationships with other attitudes and beliefs, then those relationships should remain, but Canadians should exhibit those attitudes less strongly. The correlation should remain.

    But what this result is telling us (if the presenting rationale is accepted) is that the relationship between free market attitudes and beliefs is a uniquely US phenomenon. Elsewhere in the world the free-marketeers are less susceptible - interesting thought but a long bow as an explanation I'd have thought.

    In the case of the second rationale if the correlation in the population is higher at the extremes, then samples that are biased in those directions will naturally show higher correlations. But the antecedent is contingent and we have the means to test it - we could use Heath et al's data to match L. et al's distribution on FM and see what happens.

    But looking at the R2 in both cases is running before we can walk, and unnecessarily bringing other spurious constructs (eg the other vbles) into the analysis.

    Simply looking at the FM data the from the two surveys is the place to start, as I noted earlier.
  39. .. that's "couldn't locate" second para
  40. Lotharsson at 10:33 AM on 27 September, 2012

    The paper is subtitled "An Anatomy of the Motivated Rejection of Science" .. a "skeptic" is one who rejects climate science - who denies the consensus on climate science

    Direct quotes from the authors:
    In this article we investigate predictors of the rejection of climate science ...

    Rejection of science must be distinguished from true skepticism ...

    Another common attribute of the contemporary rejection of science is its reliance on the internet. By definition, denial is difficult to practice in the peer-reviewed literature

    Accordingly, climate "skeptic" blogs have become a major staging post for denial ...
    The authors clearly associate their intent/definition that "rejection of climate science" means those in "denial" on the "consensus" on climate science - and that "skeptic" blog "denizens" are a key "staging post" for such "denial"

    It really is quite simple - "skeptics" reject climate science - non-skeptics do not.

    But why parse other comments when the authors own words are direct and to the point:
    ...we designed the study to investigate what motivates the rejection of science in individuals who choose to get involved in the ongoing debate about one scientific topic, climate change.

    My comment: "This is a study whose stated purpose is to define what skeptics think regarding climate science..."

    ... is exactly accurate.
  41. HAS @438, the two almost definitely scammed responses do make the paper problematic, but only in conjunction with other factors. Just eliminating those two responses only reduces the negative correlation between conspiracy ideation and acceptance of global warming by between one quarter and a third (by my estimate). Should that reduction carry through into the correlation between latent constructs, as seems likely, then the reported correlation should be reduced to between -0.12 to -0.15. That result would still be statistically significant, and still be large enough to be worth reporting. However, it lowers it to the range where other flaws in the survey may account for the remainder of the result.

    Of these, I suspect but cannot show that the most significant is the wording of the choices for conspiracy (CY) items where the choice is between items being "Absolutely true", "Probably true", "Probably false", and "Absolutely false". There is a distinct possibility that people who were most undecided about climate change were also most uncomfortable with calling something "Absolutely false", which is very strong language. If that is the case, more neutral language (eg,"Almost certainly false"; "False", "Strongly disagreed") as the equivalent to "Strongly disagreed" elsewhere in the survey may have resulted in substantially more 1s, and fewer 2s in responses in that region. This conjectured effect could account for half of Lewandowsky's result by itself. Of course, while it is probable that this effect exists, the strength of that effect is pure conjecture (including Lewandowsky's supposition that it is negligible).

    It should be noted that the result with regard to Free Market ideation is robust. Eliminating FM outliers plus the two most probably scammed responses still leaves a CC/FM correlation of -0.724.

    That is of course germaine to the discussion of the difference between Heath 2006 and Lewandowsky 2012. Even an absurdly robust pruning of outliers still leaves a substantial difference between the two results, which must therefore be explained by differences in the demographic and or differences in the CC questions. I believe I have identified two of the most plausible reasons for the difference, but of course cannot prove it. Even accessing Heath 2006's data would not resolve the issue. With N=185 for the entire survey, the number of "extreme" respondents are will be too small to test any interesting hypothesis in that regard. Whether you accept my conjectures or not, however, there is clearly to much difference between the demographics and surveys to infer from the difference between the CC/FM correlations between the to that there is something wrong with that correlation in Lewandowsky.
  42. "The paper is subtitled "An Anatomy of the Motivated Rejection of Science"

    Yes, and?

    When one bothers to actually read the paper one does NOT find a definition of the term "skeptic". The only uses of the word are double-quoted - which from other text is used to distinguish from true scientific skepticism, a convention that is widely used.

    "It really is quite simple - "skeptics" reject climate science - non-skeptics do not."

    This inference from the usage in the paper seems reasonable, especially given that (a) this usage is common enough elsewhere, and (b) the authors stick to the term "rejection of science" when discussing their subjects and research. I'm not seeing any controversy thus far.

    So what's your actual point? You reiterate your main claim and assert that it is entirely accurate:

    "My comment: "This is a study whose stated purpose is to define what skeptics think regarding climate science..."

    And you do so after undermining it by quoting the title of the study, and immediately after undermining it by quoting the study itself:

    "......we designed the study to investigate what motivates the rejection of science..."

    That says the study's purpose is to investigate what motivates rejection of science. It clearly does not say the purpose is to find out what beliefs constitute that rejection of (climate) science.

    It shouldn't be hard to see these are two different concepts. One concept is the content of the beliefs, and the other is the motivation for (acquiring and) holding those beliefs. You keep claiming the study is designed to investigate the former; it clearly says it is investigating the latter - and clearly does NOT report as results the contents of the mish-mash of mutually inconsistent beliefs found in any reasonable sample of "skeptics".

    (It's also interesting once more which parts of a comment you choose to focus on - and to ignore.)
  43. AScott, despite gloating and frequently patting himself on the back over how quickly he got responses to his survey on WUWT (after all the kerfuffle), is taking a long time to post any data or results.

    Is his team's competence restricted to 'copy and paste' (and making unsupported and false allegations)? Or is it, as indicated by his posts, that he hasn't figured out the purpose of the survey? Or another reason, perhaps?

    Don't see too many 'skeptics' calling for the data or methodology or code or anything else for that matter, either. Maybe they'd rather not know!
  44. Sou - feel free to keep making the same attack - (-snip-).

    Lewandowsky took 2 years from collection of the data to complete his work and release the findings publicly. We are a little over 2 weeks since first publication of the survey.

    Moderator Response: It is oft considered the hallmark of a losing proposition to view questions as personal attacks. It does you ill credit and detracts from the discussion. Please cease.
  45. Tom Curtis ...

    I would respectfully encourage you to have a long look at Heath & Gifford 2006 closely again - regarding the provenance of the FM questions. I suspect you might agree with me the likelihood is quite high that they used a 5 point scale. I could well be wrong or simply missing it, but after they noted in detail the scale for all other questions, they seem for some reason mysteriously silent, throughout the paper, on the scale of the FM questions.

    There are perhaps some other interesting issues, with Lewandowskys representations, and in my layman's perspective, possibly with some key underlying methods.
  46. AScott, that's not a valid excuse nor a valid comparison.

    You know that prelim results were discussed by Prof Lewandowsky within only a couple of weeks of the request to blog owners to post the survey link. He probably had them within days of putting out the invitation, given that most responses would have come in from any blog within hours of the link being posted.

    faustusnotes did his full analysis of LOG12 data over only a day or two and you want two years?

    You've still not posted results or data.

    I can understand why you are having trouble with the stats, given it's not your field. And I appreciate that it could take you and your 'team' months to years to learn EFA and SEM if they are the techniques you plan to use.

    What I can't figure out is what you're afraid of in that you refuse to release the data and let other people look at it. Or can I?

    (Or are you seriously thinking of publishing your 'analysis' in the literature - and which field, not psych surely. Maybe you're looking for a psych researcher to add to your 'team'?)
  47. A Scott @445, in appendix A of Heath and Gifford, it is states that "The response format for all of the items below is the same: strongly agree to strongly disagree." The items below include the climate change items and the free market items, which therefore use the same response format. Earlier it is stated that, "The response format ranged from 1 (strongly disagree or very unlikely, depending on the wording of the question) to 5 (strongly agree or very likely) for all questions." That statement is made explicitly with reference to the global warming questions, but given the statement in the appendix it applies also to the free market questions. It is certainly possible that the switch from a 5 to a 4 point Likert scale by Lewandowsky is responsible for some of the increase in the CC/FM correlation.
  48. Correction of "faustusnotes did his full analysis of LOG12 data over only a day or two..."

    I believe faustusnotes did his version of analysis of LOG12 in his 'spare' time in a couple of nights after work.
  49. Tom Curtis (somewhere above), the problem with L. et al. is of course the sample selection, so it would be a long bow to draw any inference from the study about any wider population. Heath et al has less of a problem (but small sample and no follow up on non-respondent characteristics), so the dat is more interesting.

    This issue of sample selection makes L. et al basically worthless (as I think you noted earlier elsewhere). Under these circumstances looking for convoluted explanations of some of the results in the methods is a mugs game (or just shooting ducks in a barrel in SM's case).
  50. HAS @449, while the results of Lewandowsky's paper re conspiracy theories are dubious at best, the same cannot be said for the results re the CC/FM correlation, which are very robust to pruning for suspected "scammed responses". As noted above, even removing the two most suspect responses plus the thirty five additional responses with the highest FM score across all six items retains a CC /FM correlation of -0.724, compared to a correlation of -0.768 using all 1145 responses. Because the mean of the CY12 and FM items are not correlated (correlation of 0.01 with all 1145 responses, -0.04 with the two most probably scammed removed) additional pruning based on CY response is unlikely to noticeably change the CC/FM correlation.

    It must be understood, however, that they sample, and are only intended to sample, a specific population, ie, people who are active in internet discussion on climate change. As such, they show that the split on economic theory between acceptors and rejectors of climate science is very marked in that group. More so than in the general population; but how much more so is open to question based on the comparison to Heath and Gifford based on other confounding factors.

    As to Steve McIntyre, while some of his posts have been interesting, he currently appears to be on a campaign of ad hoc pruning of the data set so that no "skeptic" response that does not fit his preferred image of what AGW skeptics should look like survives. Another name for that is doctoring the data. So while criticizing Lewandowsky's article should be like shooting ducks in a barrel for SM, it appears that he is currently far to busy shooting himself in the foot.

Prev  1  2  3  4  5  6  7  8  9  10  11  12  Next

Comments Policy

Post a Comment

You need to be logged in to post a comment. Login via the left margin or register a new account.