The American Historical Review was the most cited journal in history in 2010, garnering one in every eight citations to a history journal in 2010, according to a Journal Citation Reports analysis of references to 1,000 articles from 43 history journals. But does “most cited” equal “best journal”?  The underlying data highlights some of the problems inherent in using this sort of information as a measure of history scholarship.

Perhaps most significantly, the database used to make the calculation includes articles from only a limited number of history journals in its analysis (from 43 journals, as compared to 406 peer-reviewed English-language journals in our Directory of History Journals database). A significant number of the citations used in the analysis came from journals far outside the history field (though 65 percent of the Review’s citations did come from other history journals), which gives greater weight to publications that attract interest across disciplinary lines.

By the measure that tends to get the most attention in this report, the Review had an “impact factor” of 1.907. Even though that is down a bit from last year (see Figure 1, above), it is still twice as high as the next closest journal (0.957 for the journal Cliometrica). Internally, we can celebrate this as evidence that Review articles are read and cited relatively quickly.

But the number itself is calculated by simply taking the number of citations to articles in the journal over the previous two years (43 in 2009 and 2008) and dividing it by the number of citations to those articles in 2010 (in this case, 82). While this makes sense for the science journals, as anyone who reads history scholarship knows, that is not a very accurate measure of the way articles in our discipline are typically read or used.

The report itself acknowledges this, noting that more than half of the citations to history journal articles were published ten years ago or longer. For instance, there were 1,156 citations to articles in the AHR in 2010 (out of 8,217 citations tabulated), but 770 of citations were to articles published before 2001.

To address the immediacy issue, the company that creates the reports recently added a “five-year impact factor,” but the results are essentially the same (a 2.165 rating for the AHR, for instance, while Cliometrica slips to 0.939).

However positive these findings are for the Review, the limitations of these measures are quite worrisome for the discipline as a whole. As administrators at colleges, libraries, and funding agencies in the U.S. and abroad seek objective measures of all forms of scholarship, they are increasingly looking to these sorts of objective measures to weigh not just the journals, but also the importance of articles contained within them. The narrow measure of the Journal Citation Reports makes many specialized and distinctive works of history invisible to the larger academic world, at considerable cost to the rest of us.

This post first appeared on AHA Today.