​

​

Hello fellow journalologists,

This is the first newsletter of 2023 — I hope you’re settling back into work after a relaxing Christmas break. This week I’ve been in Berlin attending the APE meeting, which was time very well spent. I thoroughly enjoyed catching up with old friends and making new acquaintances. I returned home last night and haven’t had a chance to properly digest what I learned at the meeting. Perhaps that will be a topic for next week’s newsletter.

Yesterday we released the January issue of The Brief, which covered:

• ​Chat — our view on what ChatGPT, and similar technologies, might mean for scholarly communication.
• ​Springer Nature Reboots Acquisitions — an assessment of the company’s recent acquisition of Research Square and Cureus.
• ​NASA Goes First — a summary of the first post-Nelson Memo policy on public access to federally funded research.

If you’re one of the few people in scholarly publishing who hasn’t subscribed to The Brief yet, you can do so here. Please also encourage your colleagues to subscribe to the Journalology newsletter here.

KPIs for 2023

January can be a highly productive time of the year. The budget has (hopefully!) been signed off and the monthly management reporting hasn’t started yet. So there should be plenty of time to plan for the year ahead and to set objectives for journals and staff alike.

At the APE conference this week Judy Verses (President, Academic and Government Markets at Elsevier) repeatedly made the point that ‘what gets measured gets done’. I couldn’t agree more, so I thought it might be helpful to give a summary of the key performance indicators (KPIs) that I’d be monitoring on a regular basis if I was still responsible for a journal portfolio.

I’m largely focusing here on non-financial KPIs. Keeping on top of revenues and costs is bread and butter for publishers, and has been a constant over the years. However, the move to OA means that new non-financial indicators are needed to create high performing journals.

What follows is by no means comprehensive, but hopefully covers the main bases.

Core KPIs

• Number of submissions. These need to be broken down by article type and the geographical location of the corresponding author. If you can get information on the subject area too that can be helpful.
• Number of acceptances. Broken down by geography and subject area.
• Number of publications. This KPI should largely match ‘acceptances’ but may differ if the journal is growing rapidly and the time between acceptance and publication (see below) is long.

Editor performance

• Time to first decision. Authors often don’t mind being rejected without review as long as it’s done quickly. Editors who sit on submissions for a month before deciding whether to send a paper out for review are unlikely to get submissions from that author again.
• Proportion of submissions sent out for review. Publishers and editors should agree on the proportion of submissions that are sent out for peer review. Editors get to decide which papers are reviewed (with no influence from the publishers), but the targets needs to be set collaboratively, I’d argue.
• Proportion of papers accepted after review (acceptance rate). This metric is incredibly important, but is also difficult to track accurately. The best way is to follow the fate of a cohort of papers over time, but dividing the submissions received in a month by the number of acceptances in that same month might be a reasonable proxy if the monthly number of submissions is constant.
• Proportion of peer reviewed papers that are accepted. Editors who peer review five papers and only accept one of them for publication (20% acceptance) are providing a bad author experience and are being inefficient themselves. This KPI can vary by field: in some subjects editors may accept 3 papers for every 4 peer reviewed (75% acceptance); for others accepting 30-40% of papers sent out for review might be reasonable.
• Editor workload. As I explained in a previous newsletter, monitoring editor workload (number of papers being handled at a given point of time) is important. Overworked editors often have slower turnaround times (see below).
• Editorial decisions made per month. This KPI can help managers of editorial teams get a sense of workload. Editors working on journals that send out 80% of papers for peer review will likely have a higher workload than editors who only send out 20% of papers for peer review. Remember that rejecting a paper without review takes far less time than shepharding a paper through multiple rounds of peer review. You will need to take this into account when calculating the KPI (perhaps by using a weighting system).
• Referee requests made per paper. Finding peer reviewers is increasingly difficult. Monitoring how many referees are being invited to get to the required number of peer reviewers is important. How is this changing over time?
• Diversity of editorial board. Every journal should have a KPI related to this. Editorial boards should repreent the communities they serve in terms of gender, geography, age and subject area.

Turnaround times (TATs)

Rapid publication is a competitive advantage and some publishers have gained market share by publishing papers quickly. If you can knock a month off the time from submission to publication this year for your fully OA journals, you will increase revenues by ~8%. This is good for the publisher and is also a good author service.

• Time to first decision (see above)
• Time from submission to acceptance
  • % of time paper is with editors
  • % of time paper is with peer reviewers
  • % of time paper is with authors (being revised after review)
• Time from acceptance to publication. Authors tend to care most about submission to acceptance times, but a fast production process can provide a competitive advantage.

Pipelines

Monitoring the publication pipeline is vital, especially for larger journals. Editorial and publishing teams need to ensure that there is not a ‘blockage’ in the pipeline, and if there is to redeploy resources accordingly. The pipeline might be broken down in the following way:

• Number of papers in QC
• Number of papers waiting for first decision (reject or out to review)
• Number of papers waiting for reviewers
• Number of papers under peer review (round 1, round 2, round 3 etc)
• Number of papers being revised by authors
• Number of papers with editors (waiting for a reject, revise, accept decision)

Transfers

Increasingly publishers are developing portfolio strategies where the journals in the portfolio support each other by providing transfers from more to less selective journals. This can help the publisher commercially and also provides a better author experience (yes, this is a recurring theme). Publishers and editorial managers should be assessing:

• Number (and proportion) of rejected papers offered a transfer
• Number (and proportion) of authors who are offered a transfer who agree to the transfer
• Number (and proportion) of transferred papers that are sent out for review
• Number (and proporition of transferred papers that are accepted

Author and reader experience

For fully OA journals a good author experience means more repeat business and, equally importantly, more recommendations to colleagues. Measuring author experience is vital. Net promotor scores are problematic in many ways, but are better than nothing.

Revenue and cost derivatives

• Revenue per submission. This is an important metric for a journal portfolio. If the transfer cascade is working well then the revenue per submission should increase over time, as more of the submissions are finding a home within the portfolio. APC price increases will also affect this KPI, of course, and you may want to correct for that to determine whether portfolio efficiency is improving.
• Revenue per publication. This KPI is important to measure for journals transitioning from a subscription to an open access model. Revenue per publication is likely to be high under a subscription model and lower under an OA model; being clear how this KPI is changing over time is vital for the financial success of a journal and the wider portfolio. A publisher may need to publish more papers under a fully OA model to generate similar revenues.
• Indirect OA revenue. Transfers are an important part of a portfolio strategy and are dependent on receiving good submissions to journals at the top of the transfer pyramid. Those journals will transfer papers downstream to (most likely) fully OA journals. The accepted transfers will generate OA revenue, which will be accounted for in the P&L of the downstream journal. However, the donor journal (upstream) deserves some credit for providing the transfer even if that isn’t formally accounted for in its management reports. Editors who transfer papers downstream need to get credit for their actions and monitoring indirect OA revenue is one way to do that.
• Cost per download (CPD). This has long been an important metric for subscription journals. Librarians want to know that they’re getting good value for money. Publishers should be looking to increase institutional usage at a rate that’s higher than the institutional subscription price increase. That way the CPD decreases each year. Publishers of OA journals should consider publishing the CPD. For example, a journal that with an APC of $2000 that generates 200 page views per article over the first year of publication would have a CPD of $10 whereas a journal that has a $5000 APC and generates 5000 page views would have a CPD of $1. Authors and funders should have visibility on the CPD for OA papers too based on COUNTER compliant methodology.

I hope this list will give you some inspiration for 2023. Measuring and monitoring KPIs is important, but taking action on KPIs that fall short of expectations is key. The publishing landscape is increasingly competitive and keeping on top of the performance of the journals you’re responsible for is vital; many of your competitors will certainly be measuring KPIs carefully.

If you need help, support or guidance setting KPIs for 2023 please get in contact (james@journalology.com). The Clarke & Esposito team regularly works with publishers to help them to improve the performance of their portfolio and to develop a comprehensive portfolio strategy. We would be delighted to assist in any way that we can.

Briefly quoted

… will return soon. I chose to spend time at the APE bar socialising this week rather than reading the news wires. It wasn't a difficult decision to make...

Until next week,

James

​