blogging the latest developments in microbiology

Guest Post: Measuring Impact in the field of Microbiology

with 2 comments


How do we judge whether a research paper is any good? One straightforward way is to measure how often the article is downloaded online – but that doesn’t tell us if the readers actually thought the article was significant once they’d read it. Instead we can monitor how often an article is cited in the writings of other scientists. We can even do this for all the articles published by a particular journal (for example, in the last two or five years) and divide the total cites by the number of articles published to get an impact factor: Bingo, journal impact instantly measured!

Impact factors are a flawed and derided metric but – whisper it – by and large they also reflect many microbiologists’ perceptions and prejudices about the status of the journals in which we publish. When drafting a manuscript, it’s likely that each of us approaches choosing which journal we want to submit our work to in much the same way: we assess the scope and significance of the piece of work to be written up and then have a gut instinct as to which journal will accept it.

Typically we’ll aim as high as we can and reassure ourselves that if it’s ‘bounced’ (rejected) by X we can always submit to Y. Broadly this hierarchy of journals maps pretty well onto the impact factor rankings and, within that, it’s possible to see some clear divisions that be compared to football leagues.

Aside from the ‘Champions League’ of general science journals (Nature, Science, PNAS), there’s a well-established Premier League, which even has its own ‘Big Four’ (PLoS Pathogens, Molecular Microbiology, Cellular Microbiology, Environmental Microbiology) and its steady performers (notably the suite of ASM Journals such as Journal of Bacteriology). The tier below contains some well-regarded players (the SGM’s Microbiology, FEMS Microbiology Ecology), and the lower leagues perform valuable functions by providing a home for sound but less stellar science (and in many cases provides an outlet for the work of developing scientists, much as the lower leagues can develop football talent).

Occasionally, specific articles from the lower league journals can have a big individual impact, the equivalent (in this increasingly strained metaphor) of a Cup ‘giant killing’. Antonie van Leeuwenhoek, the journal I edit, bobbles around as a steady performer in the lower leagues, and I like to think we score some good away goals (a recent paper on bacterial evolution by Radhey Gupta may be one such article: only time will tell).

One difference between the league table for journals and the football leagues is with regard to promotion and relegation, as typically there’s not too much movement between divisions. However, recently new Open Access (OA) journals (notably the PLoS portfolio) have moved the goalposts and broken into the rankings, and others (such as ASMs recently launched mBIO) will undoubtedly follow. will undoubtedly follow. The Wellcome Trust is part of a consortium that has recently announced its intention to launch an OA Champions League contender.

Even though the annual impact factor updates don’t tend to throw up many surprises, it’s interesting to see a few in the recently released 2010 figures. Most surprising is that the impact factor for Molecular Microbiology seems to be in decline, despite the fact recent years have seen an apparent increase in the scale and scope required of studies by the journal’s editors. In contrast, this year has seen a notably improved impact factor for Environmental Microbiology and a respectable first impact factor for its new sister journal Environmental Microbiology Reports. Within the lower leagues, there are some steady improvers (Systematic and Applied Microbiology; Anaerobe) whereas the International Journal of Systematics and Evolutionary Microbiology continues its decline, most likely due to the increasingly large numbers of descriptions of new microbial species it publishes each year. Disappointingly, the Antonie van Leeuwenhoek impact factor dropped this year but only within our normal ‘wobble’ – with luck next year we’ll have bounced back, perhaps driven by the work of the microbelog contributors!

Iain Sutcliffe is Chair of Microbiology at Northumbria University (Newcastle upon Tyne) and Editor-in-Chief of Antonie van Leeuwenhoek Journal of Microbiology. His research focuses on the cell envelopes of bacteria, in particular membrane-anchored molecules.

Image credit: usfdonsathletics on Flickr


Written by microbelog

25/07/2011 at 1:45 pm

Posted in Journals, Posts

Tagged with , ,

2 Responses

Subscribe to comments with RSS.

  1. Nice piece on the ups and downs of journals. However, I think there are some other points worth exploring in this debate when discussing impact. What is Impact?

    If we use scientists citation of other scientists as our measure of impact, then we’re only measuring the impact of scientists research on each other. This correctly measures impact between peers, and indeed that is how we measure ourselves against other scientists – our own un-tabulated league table (exposed by RAE). But it falls way short of measuring ‘impact’ defined as useful scientific knowledge, i.e. how the term impact is now being used to assess the value of science.

    Personally this is why I think impact as will be measured in the REF is redefining how scientists measure their value – it’s breaking away from the peer-to-peer measure.

    This leads to my second point. One explanation for a decline in impact (factor) is that the size of the population of scientists in a given niche (often associated with niche journals) is in decline too, hence fewer citations per article. Numbers of articles within a journal don’t track submissions. This doesn’t mean the journal has less impact if we broaden the definition of what is valuable to society, it’s just harder to measure as a statistic.

    This isn’t to say that microbiology is in decline, but the niche within which scientists publish changes, hence emerging niches with microbiology. So the population submitting, reading and citing science changes too, hence the ups and downs of journals.

    So my closing statement is – journal impact factors measure volume – not impact. Scientists with highly cited papers have reached the top of a big pyramid and join the self-congratulatory FRS club as a result, but let’s not pretend they’re always the ones delivering for society, and with real impact.


    26/07/2011 at 7:44 pm

  2. Thanks Nick, some good points.

    Firstly, I think it would be hard to find any piece of published scientific research that has had no impact, in fact Doug Kell the Chief Executive of BBSRC pointed this out in his talk at the “Excellence with Impact” BBSRC roadshow last year. As I understand it the REF 2014 will measure Impact in two different ways. Research impact rates four papers submitted by active research group leaders between 1 to 4 (with 4 being the best, i.e. Science or Nature level) and Socioeconomic impact, which has to demonstrate that the research has led to benefits for society. Creating wealth, jobs or influencing government policy are good examples.

    This doesn’t mean all research should be focussed on having some commercial impact but we do have a responsibility to exploit our research for human benefit wherever possible. This is easier in some fields than in others.

    On your second point, it seems to me that more and more journals are published every year, which makes it even more difficult to find relevant papers unless they are in high profile journals that we as scientists routinely read (or at least, we read the electronic tables of contents). This could be one reason why the high profile journals, Science. Nature and PNAS, maintain high impact factors while lower ranked journals have decreasing impact factors. I don’t think it necessarily reflects their standing in their respective fields? I fear that microbiology as a distinct field is in decline, we certainly see decreasing numbers of undergraduate students signing up to study for microbiology degrees. Most want to do biomedical science – which is a large and vague grouping that can include many areas of microbiology as well as anything to do with cell biology. Part of the reason we started microbelog is to highlight to people how important microbes and microbial processes are to humans and to the environment.


    27/07/2011 at 9:43 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: