JournalTOCs Blog

News and Opinions about current awareness on new research

Crowdsourcing the journal selection process

without comments

Selecting the Best Journals with Crowdsourcing

Since this year JournalTOCs has started to move on to a crowdsourcing model to maintain its growing database of journals.

Reaching the 24,000 journals milestone was the turning point. This number practically represents the bulk of relevant journals that have been selected and added by the selection team of JournalTOCs. In May we recognized that the selection process would greatly benefit from the contributions from professionals interested in having all the relevant journals in JournalTOCs.

The decision of using crowdsourcing was mainly based on two facts:

  1. Our small selection team cannot cope with the hundreds of requests we receive every day, most of them from relatively new Open Access (OA) publishers, asking us to add their journals to JournalTOCs. Very few of those journals pass the selection process.
  2. We had a growing number of talented and enthusiastic users, principally professional academic librarians, who have been helping us with the discovery and evaluation of new journals. Almost all the journals suggested by those users have passed the selection criteria.

Crowdselection works for JournalTOCs because the selection process relies upon the knowledge and requirements of those who actually need to use or provide access for the missing journals. In some way our approach is inspired in a crowdsourcing strategy used in the investment market, where the average price produced by ‘grey markets’ have demonstrated to be more accurate than the predictions made by the experts.

For example: Grey markets ran last year on both the Royal Mail and Twitter IPOs were more accurate in predicting prices than bankers and their advisers. On the Twitter IPO, the grey market predicted shares at the end of first day of trading would be worth $44. They actually ended up at $45.06 – incredibly close, particularly when you consider the price set the “expert” bankers was $26.

It was natural then to provide our valued users with the means to add and edit journals. Without realizing we started to use crowdsourcing to expand and update JournalTOCs. Thus gradually, crowdselection is effectively accomplishing the selection process that was once the province of the specialized team. The initial results are very encouraging.

Adding new journals and updating journals involves very few simple steps. The user counts with tools to first verify that both the publisher and the journals are not already registered with JournalTOCs. After this, the journal, and if necessary, the publisher too, can be added to the database. Crowdselection only adds journals that meet the following Selection Criteria:

  • The journal is a scientific or academic journal that publishes peer-reviewed research papers.
  • The journal must have an editor, an editorial board and a verifiable peer-review system in place.
  • The journal must publish TOC RSS feeds for its most recent issues.
  • The journal can be a magazine provided that it has a proven record of publishing only technical and professional reviewed material that is relevant to industry, government and research (e.g. Harvard Business Review Magazine)
  • The journal is an active journal that has published different issues in this year and the previous year. Brand new journals with only one issue published cannot be added to JournalTOCs. In particular we are carefully with new Open Access journals published by dubious houses.

Crowdselection includes an automated system that verifies new journals and the user who has created the journal is contacted if we notice that further guidance is needed.

A positive consequence of using crowdsourcing to maintain the entire database would be the possibility of making all the features of JournalTOCS Premium, that do not require institutional customisation, freely available to anyone, starting with the users that have helped to maintain the database of journals.

Written by Santiago Chumbe

September 29th, 2014 at 12:26 pm