Transparency and Scrutiny vs. Harassment and Intimidation: The Triage

By Stephan Lewandowsky
Professor, School of Experimental Psychology and Cabot Institute, University of Bristol
Posted on 25 January 2016

A commentary appeared today online in Nature, co-authored by me and Dorothy Bishop, which took up the issue of transparency in science, with a particular emphasis on the “dark” side of transparency and openness. The article is freely available, and in this post I condense our argument into a few words and then offer some additional issues that are likely to arise during discussion.

Both authors of the commentary are strongly committed to open science: Dorothy made some strong arguments in favor of sharing data here, and I am a co-author of a recent paper on openness published in Royal Society Open Science, which introduced the Peer Reviewers Openness initiative or PRO for short.

However, like many good things such as red wine or healthy dieting, openness and transparency, when taken to an extreme, may also have adverse consequences for the conduct of science. Much has been written—including by me—about the harassment of scientists in contested areas by interminable freedom-of-information (FOI) requests, requests for data when those data are already in the public domain, and so on. A collection of testimonials about such harassment, as well as the need to preserve transparency and openness, arose out of a meeting sponsored by the Royal Society that I organized in June last year: a common thread that emerged from that meeting and the testimonials is the need for a system of “triage” that differentiates legitimate scrutiny and healthy debate from problematic research practices or harassment campaigns that masquerade as scientific inquiry.  

Our commentary proposed a set of “red flags” that can be used to approach this triage. They are not definite criteria but we felt they constitute a worthwhile initial contribution to what will surely be an on-going discussion.

The basic principle underlying our commentary is one of symmetry: while scientists ought to disclose all relevant funding and their data, and while their science must be subject to scrutiny, the same rules ought to apply to critics. For example, if researchers are encouraged to preregister their research and their analysis plan, the same should apply to critics who seek to re-analyze data.  If scientists must disclose their conflicts of interest, why should requestors for data not do likewise?

I find the idea of symmetry self-evidently fair and reasonable. However, that does not mean that everybody will agree: after all, fairness and reason may motivate some critics of scientific work but clearly not all of them. Conversely, while many scientists may openly declare their conflicts of interest, not all of them do, and we may therefore encounter continued resistance to openness.

There are a few specific questions that we touched on in the commentary but did not have time or space to do full justice. I therefore provide some additional thoughts here.

Researchers’ control over their (behavioral or medical) data during reanalysis.

In our commentary we note that "Researchers also need control over how data is to be used if it goes beyond what participants agreed to (for example, analysis of ethnic, race or gender differences in data collected for different purposes)."

What does this mean and is it even enforceable? If I make available a multi-dimensional data set with many variables and someone else wants to do some post-hoc analysis of variables that I didn't consider, should I, as a researcher, have any control over that?

This question may be best examined in a very concrete, if rather stark, hypothetical context: Suppose I have collected data on a specific cognitive task in an experiment that examined the efficacy of a new training regime. To control for potential covariates, the data set includes numerous demographic variables, including race, gender, political affiliation, and religious denomination. The data are convincingly anonymized and participants have given consent that their data will be made publicly available.

So far, so open and transparent.

Now suppose the Ku-Klux-Klan (which, alas, exists) and the Anti-Muslim-Bigotry League (which likely exists in some form if not by that name) demand access to the data for reanalysis along racial and religious lines.

Did my participants really give consent to have their data used in that manner? Would anyone from a minority group ever again give consent to participate in an ostensibly “harmless” experiment to discover better training techniques if those data can be exploited by a clever post-hoc fishing expedition to score a political point?

To my knowledge, this problem has been largely ignored in the open data debate and it urgently requires attention.

By the way, the problem would be manageable, and the original researcher’s control enforceable, if requestors have to preregister analyses in the same way that the original researchers hopefully did in the first place. (And yes, we should move towards a culture in which pre-registration becomes a strong normative expectation, if not a requirement, of research.)

Do the requestor’s motives matter?

This is another tricky and nuanced issue: if I have made available my data from a potentially controversial research project, does it matter if they are being re-analyzed by someone who is opposed to my results for political or ideological reasons?

At first glance, the answer should be a clear “no, motives should not matter.” If a re-analysis is really driven by ideological motives, then its flaws will be readily identifiable and can be corrected by the usual scholarly means (such as peer-reviewed publications).

There is, however, a problem: Many areas of science that are contentious involve a political component in which the public’s opinion matters a great deal. For example, it matters whether the public supports labeling of genetically-modified (GM) foods, it matters whether the public supports non-smoking policies in public places, and so on. Now, as a rule of thumb, it is fair to assume that the public will not demand political action on any such problem while they perceive there to be scientific disagreement. After all, the tobacco industry famously stated that “doubt is our product” because they knew that the appearance of a scientific disagreement—even where there was none—would forestall tobacco control.

This creates a dilemma for open data that, to my knowledge, has not been satisfactorily resolved: In contested arenas, the motives underlying a request for data do matter because an illegitimate re-analysis can have far-reaching flow-on consequences. Is it really ethical to let the tobacco industry cherry-pick public-health data to death, thereby delaying tobacco control legislation at a huge cost of human lives and health? Not surprisingly, public-health researchers are therefore very concerned about who has access to raw data.

There is no easy resolution to this issue but it is worthy of further discussion and examination.

Do the requestor’s abilities matter?

Setting aside motivation, does it matter who requests data? Should there be a competence criterion? Or are all requestors equal under the transparency umbrella?

At first glance, the answer should again be a clear “ability or competence should not matter,” for the reasons already noted.

There is, however, a problem: what if the data contain information that challenges meaningful anonymization? Suppose the research involves some medical condition that has a social stigma attached to it, and as part of the research many medically-relevant items of information are collected (e.g., the name or post code of the participant’s physician, the participant’s income and profession, and so on).

It is a challenge to anonymize data at that level of granularity—especially if the sample is small or limited to a small geographic area—although various solutions exist, for example through “delinking” of identifying information from research-relevant information. (Even de-linking is not an entirely trivial matter because unless the linking key has been destroyed or is held by another institution, data are not considered anonymized under the U.K. data protection act.)

Supposing the challenges to anonymization have been met, for example by irreversible delinking, then sensitivity of data alone need not—indeed should not—preclude sharing of the data with other researchers working in an institutional framework in which ethical strictures apply and non-disclosure agreements are meaningful and enforceable.

However, should such data be released to Mr. Tom D. Harry who hails from Widgiemooltha and runs a Center for Transparency in his dunny?

On this issue, I come down on a clear “no”. Sensitive medical or psychological data whose anonymization is challenging ought not to be released to people whose facility to keep them confidential cannot be reasonably established. The U.K. Medical Research Council’s guidelines explicitly state: “The custodian [of the data] must ensure that the group [receiving the data] accepts a duty of confidence and protects confidentiality through training procedures, etc, to the same standards as the custodian [my emphasis].”

Mr. Tom D. Harry is unlikely to meet those stipulations, and if he does not, then he ought not to receive the data.

Of course, procedures must be put in place that balance transparency and concerns about violations of privacy in those instances. Arguably, this should not be left to the original researchers—who may have their own ulterior motives—but must be resolved by some independent arbitration process.

The institutional response to harassment

As we note in the commentary, universities have complaints processes for good reasons. However, complaints are also a known tool of harassment that are amply documented in the context of tobacco research.

How can institutions respond? Universities—by law—must not tolerate harassment of academics or students based on race or gender. So why should they tolerate harassment of academics based on contentious science? Once the triage has been conducted and harassment has been identified, the university’s duty of care should naturally extend to offering protection.

This can be achieved in a number of ways that deserve further discussion. One technique, briefly identified in our commentary, is a public declaration of support by the university for an academic and, importantly, for the status of the scientific issue that is being attacked.

A relevant precedent involves the Rochester Institute of Technology, which affirmed the overwhelming scientific consensus on climate change when one of its academics, philosopher Dr. Lawrence Torcello, became the subject of a hate campaign after he published an opinion piece in an online newspaper.

Dr. Torcello summed up the situation thus in an email to me, which I am citing with permission:

“In fact, RIT didn't just endorse my academic freedom and the scientific consensus on climate change, the statement published by the institute also acknowledged that my work had been misrepresented by certain media outlets, it encouraged people to read my actual piece, and it provided a link to my piece. Additionally, a motion was raised at academic senate to endorse the university's statement supporting me, which passed unanimously. The dean of my particular college also sent around an official communication to liberal arts faculty condemning the harassment and making faculty aware that his office is prepared to support any faculty harassed for their research. Finally, a generalized version of the statement issued in my defense was placed permanently on the Provost's website in order to direct any future harassers to the statement. I was consulted and kept in the loop at every stage of the university’s response.  The dean's office has also offered to help sponsor a conference on such academic harassment. … I think it makes a pretty good case study of how universities ought to respond in such situations.”

Let the conversation continue, without harassment and with an emphasis on transparency, open data, and full disclosure of potential conflicts of interest.

Bookmark and Share


Comments 1 to 14:

  1. geoffchambers at 09:12 AM on 27 January, 2016
    In your comment under your Nature article you state: “Readers are welcome to comment on that post” i.e. here. Here is the comment I left at Nature:

    Richard Tol: “It would be good if Professor Lewandowsky would come clean about his research on climate change...”

    Professor Lewandowsky is a signatory to the Peer-Review Openness Initiative, calling for openness and transparency in science, which states “All data needed for evaluation and reproduction of the published research should be made publicly available, online”.

    However, as Shub Niggurath notes, he has never published the complete data for Lewandowsky Oberauer Gignac 2013. A more recent paper of his - “Recurrent Fury” is a rewrite of a retracted paper “Recursive Fury” which contained defamatory remarks about my mental state and that of several other named individuals. I wrote to Professor Lewandowsky requesting the data for “Recurrent Fury” but he didn't reply.
    I was one of a small number of individuals who wrote to the journal which published “Recursive Fury” requesting its retraction, an action which Professor Lewandowsky described as “harassment”, a term which he repeats in the title of the article to which he links in the comment below this article.

    The tenor of this article is directed against those who have pointed out flaws in his many papers aimed at denigrating climate sceptics or “deniers”. It is a call for censorship, nothing less.
  2. geoffchambers at 23:41 PM on 27 January, 2016
    In a recent Royal Society symposium you said: “I am deeply concerned about failures to replicate, failures to release data, and other harmful practices in science. Full disclosure: I always release my data (although that hasn’t stopped accusations of me “covering up” results)...”

    A few months ago I asked for the data to your recent “Recurrent Fury” paper, but received no reply. Would you please release this data, or alternatively confirm that it is the same as the data for the Recursive Fury” paper?
  3. Geoff Chambers, you must have said some crazy stuff to end up in the recursive fury paper about conspiratorial discourse in the blogosphere. What was it he called you on?
  4. Comparision of Transparency and Scrutiny vs. Harassment and Intimidation contain that page. Yes its a crazy stuff regarding on that topics which you chose to explain here. These are have its own value while studying the envormental study. A blog help me to know about each like Transparency, Scrutiny and other Two as well.
  5. This webpage for iTunes codes free that can give you gift cards and codes can give you apps and games with music and TV show on the go.
  6. These 3D recreations are likewise viewed as indoor amusements.
    play here
  7. In spite of the fact that you don't generally need to apply vitality to play these recreations,
    game homepage
  8. they could help in redirecting one's consideration from strains to fun.
  9. Sebastiansa at 22:45 PM on 14 June, 2016
    Everything matter in those manners, clash royale online hack.
  10. This is jonesdruman This blog was simply great post byb you. I really appreciate your blog. Really awesome blog I would like to know more about this. Please give some ideas about your blog. Thanks…………………………………….
    Happy Friendship Day 2016 Quotes
    Friendship Day 2016 Date in India
    Happy Friendship Day Images
  11. catherinesimon at 16:08 PM on 14 July, 2016
    RRB NTPC Result
    RRB NTPC Admit Card
    RRB NTPC 2016 Sylabus
    RRB NTPC 2016
    RRB NTPC Result
    RRB NTPC Result 2016
    SSC CGL Result 2016
    Happy New Year 2017 Wishes
    IBPS PO 2016 Result
    GCSE Results 2016
    SBI PO Result 2016
    NEET 2016 Result
    RRB Result
    New Year 2017 Wallpaper
    Good Morning Quotes
    Merry Christmas wishes Christmas wishes
    Happy Birthday Wishes
    Happy Birthday Messages
    Good Morning Quotes
  12. friendshipday at 18:27 PM on 18 July, 2016
    Hear Friendship day 2016 quotes

    Check Happy Friendship day quotes

    20 Famous Friendship day quotes

    Happy friendship day 2016 Wishes

    Wishes SMS Friendship day

    Wishes 20 Happy friendship day 2016
  13. levillnickn at 03:27 AM on 22 July, 2016

    Links of London Hope white topaz pendant - $74.40 : links of london,




    italiano \}RD4RE1RD2z}z}z}z}z}z}z}RD4\RD3REE4REE\RD3REE0REERD2z}z}z}z}z}RD4\RD3REE(REE\RD3REERD6RE1RE1YYRE1ZZZZZ
    \\?\ \#
  14. syaifulanam77 at 00:43 AM on 28 July, 2016
    Positive site, where did u come up with the information on this posting ?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. grosir payung murah payung murah grosir jual payung murah grosir bikin kaos murah choirose parfume
Comments Policy

Post a Comment

You need to be logged in to post a comment. Login via the left margin or register a new account.