Friday, January 21, 2011

The Ideal Scientific Journal

In the world of science, a journal's quality is typically measured with what's called an Impact Factor. The impact factor is really a measurement of how often any given article published in a particular journal gets cited in other scientific contributions. Journals with high impact factors tend to generate a lot of citations for each of their articles and tend to be quite prestigious.

Whether a given article has scientific importance is not necessarily reflected in its citations over a few years (which is how the impact factor is measured - Spotlight-on-Science has recently published another article on the Impact factor that you can check out here).

Although I have succeeded in acquiring a solid set of scientific publications for a young scientist I have often found that the peer-reviewed journal paper publication process has many flaws. In an ideal world, scientists wouldn't care about issues related to impact factors and would publish all of their work in some sort of central repository, however, this is not an ideal world. Anyone who sets up such a thing will face the fact that without an impact factor, scientists will readily see it as a low quality place for publication. Given that there is a pre-existing bias among scientists towards wanting their work published in journals with high impact factors, I propose the following 'ideal' journal structure that I believe would greatly benefit the world today.

One of the biggest problems with journal publications is that they reject the majority of their work even though the majority of the work deserves to be published somewhere. Most decent journals have acceptance rates below 20% and so most of the work submitted is getting rejected. Many decent studies never get properly published because of the high rejection rates and these issues are also connected with the related problem of how the peer review system breaks down in its standard form. One of the biggest problems with the peer review process is that authors are often not given the right of reply. After a scientist submits their paper to a journal, the journal gets reviewers to critique the study at which point the paper is very often rejected without allowing the study's authors to reply to the negative commentary they received. When a study author is denied the ability to reply to a fair-minded editor then the peer-review process has just failed. It is not a complete review process unless the author gets a chance to have their say as well. To that end the ideal journal would always provide a study's author the opportunity to reply to the negative commentary provided on their work. Although most studies need revisions, most of them have involved a large amount of work on the part of the contributing authors and so studies should not be casually rejected. The journal should publish work from any field, without bias against the many peculiar areas of research that some scientists choose to study. It is critical that the journal publication process not reject papers unless there is a very serious problem in the science of the study. Small contributions and unusual contributions should be able to be published as well. To that end, creating a functional structure for the journal requires a few unusual features. First, the journal would not have a fixed number of studies it publishes in a year. With limits on the number of studies published, as the number of submissions to the journal increases, the rejection rate will increase as well. Not fixing the number of studies to publish avoids this issue.

More importantly, accepting the existence of a significant bias in favour of good impact factors, the journal would be forced (like most others) to try and produce good impact factor scores in order to achieve respectability. As such the ideal journal (as I would imagine it given the present environment) would be general interest, would discourage the use of field-specific jargon and would have 3 tiers in order to facilitate producing respectable impact factors. The top tier of the journal would try to emulate the successes of high profile journals such as Science and Nature by publishing the most generally interesting and broadly relevant studies that produce the highest number of citations. Solid studies that don't meet the editorial staff's high expectations for the top tier journal would get published in the second tier with a lower requirement in terms of being written in a general interest style. Studies whose results are of small significance or whose results are somewhat questionable but not disproved would be published in the third tier. The journal would get 3 different impact factors - one for each tier. The associate editors in charge of each manuscript not only accept or reject the study but make a recommendation regarding which journal tier should publish that paper. Papers are only rejected if there is a serious problem in the science and study authors are not refused the ability to reply.

Another problem with journal publications is money. Some journals charge institutions large amounts of money so that their scientists can access the relevant research. Other journals avoid this by being "open access" but it is common that they request or require that the scientists contribute money in order to guarantee that their research document will be made available on the internet indefinitely. It occurred to me that the costs of storing all of the scientific literature out there are small enough for some modern big corporations. It is plausible that a new journal as I've described could enter into a partnership with a company like Google (who've already produced the useful Google Scholar software). Google could host all of the journal's studies and make money by providing the reader with the kinds of relevant ads they show within programs like Gmail - using a small unobtrusive ad that is selling something similar to what you are reading about. There would be no print publication, all published studies would be made available online. Such a journal would have no physical overhead, it would simply need a legion of volunteer associate editors to arrange for peer review of each article they receive.

As a general interest publication the top tier of the journal would need a good general interest name like: Advances in Research. The second tier of the journal presenting solid scientific contributions could be called Advances in Research A, and the third tier of the journal could be called Advances in Research B. If a scientist is unhappy with which tier their paper was assigned to they have the opportunity to withdraw their paper and submit it to another journal. Additionally, it would surely be impossible for the journal's editorial staff to assign research papers to the different tiers perfectly. As such the journal could also provide special incentive awards for papers in the lower tiers that generate enough citations as to have warranted being included in the top tier.

Such a model might constitute a huge contribution to science and I would love to help make it happen one day.
Jacob Levman