Special journal issues differ in citation impact and prestige compared to regular issues
the verdict
CONTESTED
contested - the weight sits with the supporting side
refutedsupported
the weight of evidence
3 sources for · 0 against
While studies investigate citation patterns across different publication tracks and journals, the provided sources do not universally support the broader claim regarding special journal issues differing in citation impact and prestige.
This paper starts with a brief description of the introduction of the likelihood approach in econometrics as presented in Cowles Foundation Monographs 10 and 14. A sketch is given of the criticisms on this approach mainly from the first group of Bayesian econometricians. Publication and citation patterns of Bayesian econometric papers are analyzed in ten major econometric journals from the late 1970s until the first few months of 2014. Results indicate a cluster of journals with theoretical and applied papers, mainly consisting of Journal of Econometrics, Journal of Business and Economic Statistics and Journal of Applied Econometrics which contains the large majority of high quality Bayesian econometric papers. A second cluster of theoretical journals, mainly consisting of Econometrica and Review of Economic Studies contains few Bayesian econometric papers. The scientific impact, however, of these few papers on Bayesian econometric research is substantial. Special issues from the journals Econometric Reviews, Journal of Econometrics and Econometric Theory received wide attention. Marketing Science shows an ever increasing number of Bayesian papers since the middle nineties. The International Economic Review and the Review of Economics and Statistics show a moderate time varying increase. An upward movement in publication patterns in most journals occurs in the early 1990s due to the effect of the ‘Computational Revolution’. The paper continues using a visualization technique
Results indicate a cluster of journals with theoretical and applied papers, mainly consisting of Journal of Econometrics, Journal of Business and Economic Statistics and Journal of Applied Econometrics which contains the large majority of high quality Bayesian econometric papers. A second cluster of theoretical journals, mainly consisting of Econometrica and Review of Economic Studies contains few Bayesian econometric papers. The scientific impact, however, of these few papers on Bayesian econometric research is substantial. Special issues from the journals Econometric Reviews, Journal of Econometrics and Econometric Theory received wide attention.
The number of pages written on Bayesian econometrics have been recorded as a percentage of the total number of pages for each year for all journals. There appear four prominent results: The existence of two clusters of journals, one with a high number and one with a low number of Bayesian econometric papers; a substantial scientific impact of a few papers in the more theoretical journals; a large number of citations from papers published in special issues and, fourthly, an increase in the number of published Bayesian econometric papers since the early nineties due to the ‘Computational Revolution’.
More specifically, journals which contain both theoretical and applied papers, such as Journal of Econometrics, Journal of Business and Economic Statistics and Journal of Applied Econometrics, publish the large majority of high quality Bayesian econometric papers in contrast to theoretical journals like Econometrica and the Review of Economic Studies. These latter journals publish, however, some high quality papers that had a substantial impact on Bayesian research. The journals Econometric Reviews and Econometric Theory publish key invited papers and/or special issues that received wide attention, while Marketing Science shows an ever increasing number of papers since the mid nineties.
The information distilled from this analysis shows also names of authors who contribute substantially to particular subjects. 4 Influential papers in Bayesian econometrics have been earlier analyzed in Poirier (1989, 1992), where quantitative evidence is provided of the impact of the Bayesian viewpoint as measured by the percentage of pages devoted to Bayesian topics in leading journals. We contribute to this literature by extending the bibliographical data with more recent papers and additional leading journals. Our contribution differs from the literature in several ways.
27 The bottom panel in Figure 2.2 presents the average percentage of pages for Bayesian papers in each journals over 5-year intervals. These average percentages provide general publication patterns compared to the top panel of Figure 2.1 since the influence of special journal issues related to Bayesian econometrics is now more limited due to the 5-year averaging. This panel of Figure 2.2 shows that the influence of Bayesian econometrics in terms of the percentage of allocated pages is time varying and journal dependent. Journals such as Ectra, ET, RES, ReStat and IER typically have low percentages of Bayesian pages, below 3% over the whole period.
Journal abbreviations are as in Figure 2. Figure 3.4: Average citation patterns for papers in leading
Effect of special issues: Special issues of the journals like Econometric Reviews and Econometric Theory receive more citations than usual issues since they contain influential papers. Increasing trend since 1990’s: A structural break, indicating a switch to an increasing number of Bayesian papers, occurs since the early 1990s, which is due to the use of novel computational techniques. 3. Subject and Author Connectivity 3 http://www.jstor.org/ . JSTOR provides a randomly selected set of 1000 papers from their digital arc (...) 4 http://scholar.google.com/ , http://thomsonreuters.com/web-_of-_science/ .
The average h-index for 1000 such random samples is used to approximate the expected h-index for an author with J publications in leading journals. For a young Bayesian econometrician who is the author of 5 such publications, we find that the h-index is approximately 4, i.e. very high compared to the total number of publications of this author, and the expected number of citations for this author’s papers is 334. For an author, coming up for tenure, with 12 publications in leading journals the expected h-index is approximately 9 with an expected number of citations of 765.
After a brief description of the first Bayesian steps into econometrics in the 1960s and early 70s, publication and citation patterns are analyzed in ten major econometric journals until 2012. The results indicate that journals which contain both theoretical and applied papers, such as Journal of Econometrics, Journal of Business and Economic Statistics and Journal of Applied Econometrics, publish the large majority of high quality Bayesian econometric papers in contrast to theoretical journals like Econometrica and the Review of Economic Studies. These latter journals published, however, a few papers that had a substantial impact on Bayesian research. The journals Econometric Reviews and Econometric Theory published key invited papers and special issues that received wide attention, while Marketing Science shows an ever increasing number of papers since the middle n ineties. The International Economic Review and the Review of Economics and Statistics show a moderate time varying increase. The early nineties indicate an upward movement in publication patterns in most journals probably due to the effect of the ‘Computational Revolution'. Next, a visualization technique is used to connect papers and authors around important theoretical and empirical themes such as forecasting, macro models, marketing models, model uncertainty and sampling algorithms. The information distilled from this analysis shows the names of authors who contribute substantially to particular themes. This i
Background: Citation data can be used to evaluate the editorial policies and procedures of scientific journals. Here we investigate citation counts for the three different publication tracks of the Proceedings of the National Academy of Sciences of the United States of America (PNAS). This analysis explores the consequences of differences in editor and referee selection, while controlling for the prestige of the journal in which the papers appear. Methodology/Principal Findings: We find that papers authored and “Contributed” by NAS members (Track III) are on average cited less often than paper
This analysis explores the consequences of differences in editor and referee selection, while controlling for the prestige of the journal in which the papers appear. Methodology/Principal Findings We find that papers authored and “Contributed” by NAS members (Track III) are on average cited less often than papers that are “Communicated” for others by NAS members (Track I) or submitted directly via the standard peer review process (Track II). However, we also find that the variance in the citation count of Contributed papers, and to a lesser extent Communicated papers, is larger than for direct submissions.
Therefore when examining the 10% most-cited papers from each track, Contributed papers receive the most citations, followed by Communicated papers, while Direct submissions receive the least citations. Conclusion/Significance Our findings suggest that PNAS “Contributed” papers, in which NAS–member authors select their own reviewers, balance an overall lower impact with an increased probability of publishing exceptional papers. This analysis demonstrates that different editorial procedures are associated with different levels of impact, even within the same prominent journal, and raises interesting questions about the most appropriate metrics for judging an editorial policy's success.
status released display-pdf yes is-olf no is-manuscript no is-preprint no is-journal-matter no is-scanned no is-retracted no Received 2009 Sep 17; Accepted 2009 Nov 3; Collection date 2009. Introduction Citation data play an important role in the evaluation of scientific research. Citation counts can be used to characterize individual studies and researchers, scientific disciplines, journals, institutions, and entire nations [1] – [5] . At the level of scientific journals, citation data can help to investigate the effect of review policies and procedures on the subsequent impact of published papers [6] , [7] .
The possibility that impact varies systematically across track has received a great deal of recent attention, particularly in light of the decision by PNAS to discontinue Track I [9] . The citation analysis we now present provides a quantitative treatment of the quality of papers published through each track, a discussion which as hitherto been largely anecdotal in nature. Methods To empirically investigate the impact of papers published via each track, we inspect 2695 papers published between June 1, 2004 and April 26, 2005, covering PNAS Volume 101 Issue 22 through Volume 102 Issue 17.
For each paper, we examine Thomson Reuters Web of Science citation data as of October 2006 and May 2009, as well as page-view counts as of October 2006. We also note the track through which each paper was published, the topic classification of each paper, the date of publication, and whether each article was published as open access and/or as part of a special feature. For all subsequent analysis, we log10-transform citation counts, with the addition of a constant value 1 to all citation entries so as not to exclude un-cited papers.
Comparing the variance in citation counts across Tracks, however, reveals that there is significantly greater variation in the impact of Contributed papers compared to Direct submissions or Communicated papers (Levene's F-Test for homogeneity of variances, Direct vs Contributed: 2006, p = 0.0001; 2009, p<0.0001; Communicated vs Contributed: 2006, p = 0.10; 2009, p = 0.002), and marginally greater variation in Communicated papers relative to Direct submissions (Levene's F-Test for homogeneity of variances, 2006, p = 0.058; 2009, p = 0.15). To explore the consequences of this larger variance, we now compare the citation counts of the 10% least and most cited papers from each Track.
We now demonstrate that the relationships shown in Figure 1 , Figure 2 , and Figure 3 are robust to controlling for a number of additional factors which may affect citation counts. Open access publication [10] – [12] has been suggested to affect impact, as has time since publication and topic classification [13] . Therefore we control for these factors, as well as publication as part of a special feature issue. We use regression to a linear model with robust standard errors [14] using log-transformed citation rate as the dependent variable, and submission track as well as the above mentioned factors as independent variables.
Discussion The analysis presented here clearly demonstrates variation in impact among papers published using different review processes at PNAS. We find that overall, papers authored by NAS member and Contributed to PNAS are cited significantly less than papers which are Direct submissions. Strikingly, however, we find that the 10% most cited Contributed papers receive significantly more citations than the 10% most cited Direct
Further empirical and theoretical work exploring these questions is needed. One additional factor which may influence citation counts is whether a given paper was highlighted in a PNAS press release. We were unable to explore this issue because of lack of data regarding which papers received press releases. The effect of press releases, and popular press coverage more generally, on citation counts is an open question which deserves further study [2] . A related issue involves the potential for selection bias in which papers are submitted to PNAS via each track.
Everything we examined (3)
This check searched the claim as stated. It did not run a separate search for evidence against it.