ReferenceError: "department" is not defined.
We aim to set up a hub-and-spoke, international network provisionally titled ‘The Institute for Ascertaining Scientific Consensus’ (IASC). IASC will be capable of emailing >100,000 scientists, asking for an agree/disagree response regarding a specific statement of interest, a candidate scientific fact such as ‘Covid is caused by a virus’. Emails will be sent internally within each participating institution, by a spoke representative, and the time-demand to read the email and respond will be less than two minutes. In this way the response rate will be very high, compared with the usual scientific opinion surveys. The responses will be instantly and anonymously recorded in a database, and the strength of consensus calculated. The network will be humanity’s premier means for measuring strength of scientific consensus regarding a specific statement of interest. It will thus stand as a useful tool for policymakers, especially given the proven[1] ability of consensus announcements to influence opinions and actions. It will also serve to inform laypersons, fighting against ‘fake news’ and misinformation. In other cases, it will serve to illuminate where experts in different countries, or different parts of the world, see things differently.
During 2022-23 a miniature version of IASC is operating out of Durham University, UK, led by Professor Peter Vickers. The hub-and-spoke network currently in place (December 2022) looks as follows:
During the pilot project, the method will be fully tested twice: the first statement to be tested will be a rather uncontentious statement, so that we can set a baseline for a strong consensus, and iron out any teething problems with the methodology and ICT architecture. The current front-runner is:
The second statement to be tested is yet to be decided. Amongst the huge variety of options are the following:
Advisory Board (project strategy)
The second advisory board consists of scientists from a range of fields, who will participate in a test run prior to any large-scale international rollout:
Advisory Board (scientists)
Method, including advantages over currently available state-of-the-art methodologies
The vast majority of methodologies currently being used to survey scientists fail to meet any of the following three criteria, all of which will be met by the new method:
In a pre-pilot-project (June 2022) scientists at Durham, UK, were provided with a statement, and merely asked to choose between two options ‘A’ and ‘B’; many responded with nothing more than a letter, either ‘A’ or ‘B’. In the proposed pilot project, scientists will be able to simply click a button (‘agree’ or ‘disagree’) embedded within the email. Their (totally anonymous) answer will be automatically recorded in a database.
Brief comparisons with state-of-the-art methodologies for surveying scientists’ opinions are here considered. First, in the well-known Myers et al (2021) paper reporting a solid consensus on the question of anthropogenic global warming, an 8-page questionnaire was sent to 10,929 scientists, and 2,780 responses were eventually received – a return rate of 25%. To achieve this return rate, three reminder emails were sent. The timeline between sending out the questionnaire and analysing the results was 36 days: 10th September to 16th October 2019. Whilst this methodology is effective up to a point, it is slow, and the slow return rate is a barrier to a significant scale-up. Sending personal, ‘internal’ emails to individual scientists, asking a simple ‘agree or disagree’ question, ensures a much higher return rate, and could in principle be scaled up to quickly access the opinions of many tens of thousands of scientists.
Second, in the Work and Well-Being Survey (Vaidyanathan 2021), out of 22,840 scientists contacted only 3,442 replied, which is a return rate of 15%. To get even that return rate required three months, during which time 13 reminder emails were sent (across two waves of solicitation) as well as reminder snail-mail postcards, and incentives were offered such as gift cards and a raffle to win an iPad.
The new methodology envisioned in this pilot project is quicker, simpler, more repeatable, and has the potential to pool a far greater number of opinions. In the pre-pilot-study Vickers achieved a 62% return rate based on nothing more than one email, indicating that with even just one or two reminder emails a very high return rate (>75%) might well be achieved.
Objections and replies
A few possible objections to the proposed programme, and initial responses, are as follows:
[1] Bartoš et al. (2022): Communicating doctors’ consensus persistently increases COVID-19 vaccinations. Nature, Vol. 606.