Scientific publishing is a confusing place to be these days, and if that’s not the most gripping opening sentence I’ve ever written then bear with me for a second because it is important. Yesterday, we submitted the second of the Galaxy Zoo papers to the Monthly Notices of the Royal Astronomical Society, the most prominent of the UK journals (it’s not monthly, and it doesn’t publish society notices, but still…). It follows the first of our papers, which was submitted a few weeks ago and we’re thus waiting for a referee’s report for. However, you can already read that paper here, thanks to an archive system that allows authors to make their papers public.

When the paper appears on the archive is a decision that’s left to the author, and it tends to depend on the field. In cosmology, it’s become common to put a paper up in advance of submitting to the journal, to take advantage of any helpful comments that might be made by people who see it before it has to face the acid test of peer review. However, the people I work with on astrochemical matters, for example, would be horrified by this and it’s really not the done thing to post to the archive until your work has been accepted by the journal. As the whole process of arguing with referees can take a long time, it’s certainly tempting to post early (while, of course, always being honest about what stage the paper is at).

Things have got more complicated, because journalists have learnt that the archive is a reasonable place to lurk when looking for the newest research. There’s been a gradual shift in the news pages of, say, New Scientist, to reporting submitted work rather than waiting for the journals to publish. Why am I writing about this now? One of the UK’s leading political bloggers, Iain Dale has today strongly criticised the BBC for posting this story about a predicted sea level rise based only on a submitted paper.

Iain – who despite his skepticism about the scientific evidence for global warming is normally quite sensible – says :

…the research in question, which goes against the consensus view, has not been peer reviewed or accepted for publication but merely submitted to a journal. This means that it is very probably wrong and may never be published (on the basis that almost all articles submitted to journals subsequently turn out to be wrong and many are turned down after failing peer review).

…which strikes me as a very strong statement. It’s certainly true that most articles submitted to journals turn out to be wrong; science is hard and our ideas change over time, but I don’t think that’s what Iain means. I think he means that peer review will reject most of these wrong articles, but that’s just not true. A peer reviewer (and I write as someone who referees papers and has also had articles rejected) is supposed to assess whether the science is well done, not whether it is 100% correct and ready to be written in the Big Book of Scientific Fact (where most people assume that ‘true’ theories end up). It’s perfectly possible to approve a paper which analyses the evidence correctly, but then reaches a conclusion that one disagrees with. For example, recent results indicated that the southern hemisphere of Mars was once covered in salty lakes. If I were competent in the field and assessing that paper, my job would be to check that the data analysis had been done correctly and that the conclusion was reasonable in that context; ie did the evidence point to salt lakes? I may well disagree with the authors about whether this is good or bad for the prospects of ancient life on Mars, but disagreeing with them doesn’t mean the paper gets rejected. Similarly, peer review in the case of the global warming results will check whether the modeling completed by the team is – given the assumptions that they must have put into it – reasonable, and whether their conclusions do in fact come from their results, not whether their model reflects what will really happen. The only way to make that assessment is to consider the whole set of peer reviewed models and evidence, and try and make an educated guess.

All of which is a long way from the question of whether the BBC should publish such an article or not. Personally, I’m happy with the fact that they say that the results come from a submitted paper, and this can then influence our judgement of the results. I don’t see the difference between this and reporting a conference talk, such as the story today about plants thriving in lunar soils. Of course, when it’s politicians and not scientists making a statement, the media reports what’s said without any hope of eventual peer review; we should be careful not to hold scientists to a higher standard all the time. It’s perfectly reasonable to report a scientific opinion, as long as you label it as such rather than as a tried and tested paper.

So I will make our second Galaxy Zoo paper public; but noone should mistake it for the Truth either before or after it’s been peer reviewed.