(C) PLOS One [1]. This unaltered content originally appeared in journals.plosone.org.
Licensed under Creative Commons Attribution (CC BY) license.
url:https://journals.plos.org/plosone/s/licenses-and-copyright

------------



A survey of biomedical journals to detect editorial bias and nepotistic behavior

['Alexandre Scanff', 'Univ Rennes', 'Chu Rennes', 'Inserm', 'Cic', 'Centre D Investigation Clinique De Rennes', 'Rennes', 'Florian Naudet', 'Ioana A. Cristea', 'Department Of Brain']

Date: 2021-11

Alongside the growing concerns regarding predatory journal growth, other questionable editorial practices have gained visibility recently. Among them, we explored the usefulness of the Percentage of Papers by the Most Prolific author (PPMP) and the Gini index (level of inequality in the distribution of authorship among authors) as tools to identify journals that may show favoritism in accepting articles by specific authors. We examined whether the PPMP, complemented by the Gini index, could be useful for identifying cases of potential editorial bias, using all articles in a sample of 5,468 biomedical journals indexed in the National Library of Medicine. For articles published between 2015 and 2019, the median PPMP was 2.9%, and 5% of journal exhibited a PPMP of 10.6% or more. Among the journals with the highest PPMP or Gini index values, where a few authors were responsible for a disproportionate number of publications, a random sample was manually examined, revealing that the most prolific author was part of the editorial board in 60 cases (61%). The papers by the most prolific authors were more likely to be accepted for publication within 3 weeks of their submission. Results of analysis on a subset of articles, excluding nonresearch articles, were consistent with those of the principal analysis. In most journals, publications are distributed across a large number of authors. Our results reveal a subset of journals where a few authors, often members of the editorial board, were responsible for a disproportionate number of publications. To enhance trust in their practices, journals need to be transparent about their editorial and peer review practices.

We may draw parallels between the PPMP and studies on resource distribution in economics. A highly prolific author who is an outlier on the PPMP measure in effect monopolizes a large proportion of a journal’s publications. This analogy supports another, more complex measure, the Gini index [ 4 ], used in econometrics to describe resource distribution inequalities. This measure was recently applied in bibliometrics to explore inequality in authorship across authors publishing in high-impact academic medical journals [ 5 ]. Applied to our context, it could be used to quantify imbalances in the patterns of authorship within a journal: the values of the Gini index range from 0 (perfect equality in numbers of articles among authors) to 1 (major inequality).

Coincidently, one of our colleagues (DB) reported a similar analysis for the addiction subfield of psychology in a blog post [ 3 ]. This analysis focused on research articles only. She found a bimodal distribution of “the percentage by the most prolific” measure, identical to that observed for NMNI, with only 3 out of 99 journals having a score over 8%. In 2 of these journals, the high score was attributable to the same individual, who was on the editorial board of the journal, and who had published together with the editor-in-chief. Bishop also noted that the same method identified a journal editor, Johnny Matson, who had been found previously to be publishing copiously in journals edited by himself or other editors [ 3 ]. Furthermore, for many of these papers, indirect evidence of superficial or absent peer reviews was suspected because of the remarkably rapid turnaround, often within a week or less, between the dates recorded for submission and manuscript acceptance. This circumstantial evidence of unethical editorial practice can only be obtained, however, in journals that report these dates for published manuscripts.

Research integrity matters across the research ecosystem. In this process, scientific journal editors are key actors that ensure the trustworthiness of the scientific publication process. But, paraphrasing Dr. Drummond Rennie’s famous quote, who is guarding those guardians? [ 1 ] Some of our team (CL, IC, DM, and FN) had doubts that anyone does such safe guarding in the case of New Microbes and New Infections (NMNI), an Elsevier journal, whose most prolific author, Didier Raoult, coauthored 32% of its 728 published papers [ 2 ]. NMNI’s editor-in-chief and 6 additional associate editors of the journal work directly for, and report to, Raoult. Together, these editors authored 44% of the 728 papers published in the journal as of June 25, 2020. We suggested that such “self-promotion journals” were “a new type of illegitimate publishing entity, which could have certain key characteristics such as (i) a constantly high proportion of papers published by the same group of authors, (ii) relationships between the editors and these authors, and (iii) publication of low-quality research” [ 2 ]. We applied a preliminary approach to detect these “self-promotion journals” in the field of infectious disease using a measure easy to compute: the proportion of contributions published in a journal by the most prolific author, i.e., the one who published the most articles in a given time period [ 2 ]. In journals publishing more than 50 papers over 5 years, it was rare to see journals where a specific author published more than 10% of the papers, and, indeed, NMNI was a clear outlier. Note, however, this is a crude measure as it is based on all published articles, whatever their type (research, letter, editorial, etc.) and therefore may give high scores for legitimate contributions by active editors.

Overlap between indices (PPMP versus Gini index) and analyses (principal versus sensitivity analyses) is presented in S8 Fig . Among the 648 journals flagged as outliers either through the principal analyses, or through the sensitivity analyses, 299 (46.1%) are common to both analyses. Furthermore, qualitative analysis of the 100 randomly selected outlier journals identified through the sensitivity analyses is consistent with those for the principal analyses (see S3 Table ).

Because there was sometimes more than one author with the same largest number of published articles, 108 “most prolific” authors were identified in the 100 outlier journals. We identified errors in author identification for one journal, MMW-Fortschritte der Medizin (an outlier on the Gini index), where the most prolific “author” was named “Red,” which seems to be a diminutive for “Redaktion,” possibly encompassing several physical individuals. When “Red” was ruled out as a valid author name (i.e., these papers were considered has having no author), the next most prolific author was, however, a member of the editorial board, the PPMP increased very slightly from 7.5% of 4,920 authored articles to 7.6% of 4,553 authored articles, and the Gini index decreased very slightly from 0.637 to 0.616. Out of 1,978 identified authors (excepting “Red”), 1,435 (72.5%) did not publish more than one article in this journal. Among the remaining 107 individual authors, 95 were formally identified from WoS. Among the 12 remaining, identification was considered unreliable for 8 because of possible homonyms, and 4 were not indexed in WoS. For these 12 authors, manual disambiguation using PubMed and Google identified 6 journalists, 5 physicians with a consistent affiliation to the journal considered, and one author where a high risk of homonym persisted (“Wang, Y” in Journal of Clinical Otorhinolaryngology, Head, and Neck Surgery). Among the 95 other authors, the median of the H-index was 28 (IQR 13 to 50).

For 2 of the 100 journals, the full composition of the editorial boards could not be found and only the editor-in-chief was known, but was not the most prolific author. In the remaining 98 journals, at least one of the most prolific authors was a member of the editorial board in 60 journals (61%), among whom 25 (26%) were the editors-in-chief. Journals where at least one of the most prolific authors was on the editorial board tended to have a higher impact factor than the others with a median of 3.4 (IQR 2.0 to 5.3) versus a median of 1.4 (IQR 1.0 to 3.1).

Only 56 of these 100 journals were indexed in Web of Science (WoS), which enables an assessment of the journal impact factor and other citation metrics. For these 56, the median journal impact factor was 2.9 (IQR 1.5 to 4.8) with a median self-citation ratio of 0.11 (IQR 0.047 to 0.21), corresponding to a median self-citing boost according to Ioannidis and Thombs of 13% (IQR 5% to 26%) [ 6 ]. The skewness and nonarticle inflation median was 86% (IQR 55% to 144%) [ 6 ]. Calculation of this metric was not possible for 11 journals (20%) that had a median of article citation of 0. Only 5 journals were indexed in the Directory of Open Access Journals as being full open access, and the median proportion of open access articles was 2.0% (IQR 0.47% to 8.0%). None of the outliers had an open peer review policy.

The main characteristics of the 100 randomly selected outlier journals are presented in S3 Table . Of the 100 journals identified through the principal analyses, 98 were reported in English, among which 31 were also in another language (either fully multilingual journals or translation of abstracts). The most common non-English languages were German (6 journals), Chinese (5 journals), Japanese (5 journals), and French and Italian (4 journals each). These outliers were well-established journals, with a median year of start of activity in 1990 (IQR 1976 to 2001).

Distribution of the publication lag median for the subgroup of 2,725 (49.8%) journals reporting submission and publication dates. Publication lag median (in days) are presented for articles signed by the most prolific authors compared to the articles without any of the most prolific authors (with marginal density plot of distributions). The data underlying this figure may be found in https://osf.io/6e3uf/ .

Fig 3 shows the scatter plot for all articles with the marginal density curve for the median of publication lag for the most prolific author(s) versus nonprolific authors, for each journal: For articles authored by the most prolific author(s), the distribution of the publication lag was skewed toward a shorter time lag. Using a cutoff of 3 weeks for the median of publication lag, 277 (10.2%) of the journals had a median below this for articles by the most prolific author(s), 51 (1.9%) journals had a median below this for articles not by prolific author(s), and 38 (1.4%) journals had a median below this for both types of articles (i.e., authored by the most prolific author(s) or not). For the most prolific authors, publication lag decreased with the number of articles published as shown in Fig 4 , not solely in outlier journals. The results of the sensitivity analyses based on “research articles” alone were consistent with those for all articles ( S6 and S7 Figs ).

Because of failures to report submission or acceptance dates, publication lag was not calculable for 2,743 journals (50.2%). Compared to journals that did report submission and acceptance dates, these journals had fewer authored articles (369 (IQR 200 to 712) versus 637 (IQR 355 to 1,186)). There were no differences for the Gini index but a higher PPMP (3.4% (IQR 2.0% to 5.9%) versus 2.4% (IQR 1.5% to 4.0%)). For the 2,725 journals with data on submission and publication (49.8%), the median of publication lag for all authored articles over the 5 years was 85 days (IQR 53 to 123) for articles published by the most prolific author(s) versus 107 days (IQR 80 to 141) for articles not published by the most prolific author(s).

Distribution of PPMP author(s) (A) and Gini index (B) in relation to journal size, and comparison between the PPMP and the Gini index (C), across all articles published by all journals in the United States NLM catalog having at least one Broad Subject Term and having published at least 50 authored articles between 2015 and 2019. The data underlying this figure may be found in https://osf.io/6e3uf/ . NLM, National Library of Medicine; PPMP, Percentage of Papers by the Most Prolific author.

For the principal analyses based on all articles published during the 2015 to 2019 period, the PPMP median was 2.88% (IQR 1.71% to 4.91%), and the 95th percentile of the PPMP value was 10.6%. Fig 2A details the PPMP and numbers of published outputs for all journals. It also shows that the number of publications by a prolific author in a given journal can result in different PPMP depending on the number of papers published in this journal. The most prolific author(s) in each journal published a median of 14 articles (IQR 8 to 25). For 1,022 journals (19%), there was more than one author with the same largest number of published articles. To assess authorship disparity from not only a single highly prolific author but from a group of authors, we also computed the Gini index, a measure of the degree of unequal distribution of authorship within a journal ( S2 Fig ). Gini index range from 0 to 1, with smaller values indicating a more equal distribution of articles across authors and higher values representing greater inequality. Over the 2015 to 2019 period, the Gini index median was 0.183 (IQR 0.131 to 0.246), and the 95th percentile was 0.355. Fig 2B details the Gini indices and numbers of published outputs for all journals. Fig 2C presents the correlation between the PPMP and Gini indices. This correlation was 0.35 (95% CI 0.33 to 0.37).

These journals published a total of 4,986,335 articles of which 4,183,917 were considered research articles (i.e., original articles, case reports, and reviews; S1 Fig ). The main characteristics of the journals analyzed are described in Table 1 . Briefly, they published a median of 500 articles (IQR 262 to 964) for the 2015 to 2019 period, of which 426 (IQR 232 to 798) were considered research articles. Two “mega-journals” published more than 25,000 articles over the 5-year period (Scientific Reports with 95,900 articles, and PLOS ONE with 108,990 articles). For 3,668 journals (67%), there was at least one article without any author, and a median percentage of 0.9% of articles (IQR 0.4% to 2.1%) with no named author in these journals. The author with the largest number of articles in a given journal is a journalist (N = 767), and the author with the largest number of research articles is an academic (N = 252). Both authors publish in established journals (The BMJ and Physical Review Letters, respectively) and are members of their respective editorial boards.

Discussion

In this comprehensive survey of 5,468 biomedical journals, we describe several features of editor–author relationships among which were the following: (i) an article output was sometimes dominated by the prolific contribution of one author or a group of authors; (ii) with time lags to publication that were in some instances shorter for these prolific authors (when this information was available); and (iii) more than half of those prolific authors were typically members of the journal’s editorial board.

In our study, the threshold to define the 95th percentile was 10.6% of articles published by the most prolific author. In absolute terms, we believe that it is reasonable to closely inspect these journals as it may question the judgment of an editor where more than 10% of the published papers are authored by the same person. To better characterize the lack of heterogeneity in authorship, we also computed the Gini index, for which the corresponding 95th percentile was 0.355 over 5 years and 0.20 when computed annually. One possible explanation is that a broader time frame allowed for more occasional authors to be recruited while maintaining the regular authors, revealing the latent heterogeneity. One other explanation is that some recent journals are relying on a group of authors on their first year before allowing these authors to move on to other publishing duties.

The PPMP and the Gini index explore complementary patterns of asymmetry in publishing patterns. The PPMP reflects author practices, while the Gini index is more sensitive to groups of highly prolific authors. The Gini index has one advantage over the PPMP, in that it is less constrained by the total number of articles published. If a journal publishes a very large number of papers (i.e., 1,000 articles over 12 months), it becomes increasingly implausible that a single prolific author could account for 10% or more of them, as there is a natural upper limit to how many papers any one individual can author. Conversely, for a journal that publishes very few papers, an author could be identified as above the 95th percentile with a relatively modest number of publications. In other words, there might be a masking effect in journals with higher rates of publishing causing lower publishing rate journals to show a greater number of PPMPs. Further developments may consider breaking the journals into groups based on rates of publishing and/or using crude number of publications by the most prolific author.

For the subgroup of 2,725 (49.8%) journals reporting submission and publication dates, the time lag for the most prolific author(s) was shorter, suggesting that for certain journals, which are outliers on PPMP and/or Gini index, peer reviews may have been absent or only superficial for prolific authors. However, for the most prolific authors, the publication lag decreases with the number of articles published across all journals and not solely in the subsample of outliers. This suggests that our description of these outliers based on PPMP could be only the tip of the iceberg, capturing solely the most extreme cases of hyperprolific publication in a given journal. Because not all journals report publication lag data, this finding is necessarily based on a subgroup. This could introduce bias if journals reporting publication dates differ from those that do not in terms of factors such as size or monitoring of the publication process [7].

Our findings persisted when all articles were considered as well as when only “research articles” were considered (i.e., excluding articles explicitly referenced as editorial, correspondence, or news articles), suggesting that editorials, correspondence, and news are not the only drivers of the indicators we explored. Conversely, it is possible that the definition we used to identify research articles is not perfect and carries a risk of misclassification bias. However, analyzing both the overlap of PPMP and Gini and the overlap between both analysis populations cautions against making simple binary distinctions. When based on all articles, these metrics carry a risk of false positives, represented by active editors and/or professional writers. When based on research articles, it may miss problematic behaviors by certain editors. In other words, there is surely a gray zone; the proposed metrics are useful to delineate the big picture of medical journals behaviors, but the specific behaviors of these journals will necessarily deserve more fine-grained scrutiny in future. For instance, it will be interesting to see whether hyperprolific authors publishing in their own journals adhere to COPE policy regarding conflicts of interest disclosures [8]. This was studied for authors of Cochrane reviews who are also editorial board members: The adherence to conflicts of interest policy was low [9].

We should beware of assuming that a hyperprolific author is necessarily engaged in questionable publishing practices: Some people are highly productive, and the speed with which good research can be completed is highly variable across research fields. Furthermore, authors may be represented in many papers because they play a key role in one aspect of the research, such as the statistical analysis, and senior researchers who oversee multiple projects may end up as authors on many papers. Similarly, shorter publication lags may occur simply because it is easier to find reviewers for eminent authors, or in a particular subject area, and/or because their expertise means that their papers require less revision. Nevertheless, there is no doubt that some highly prolific authors achieve an unusual level of productivity by exploiting the system or engaging in academic misconduct [10,11]. It is important to make a distinction between hyperprolific authors who publish a lot in a range of different journals and those who are exploiting a select pool of a few journals in which they appear as prominent authors (as we explored here). It is also very important to complement the PPMP and the Gini index with the absolute numbers of papers authored by the most prolific authors, because some problematic journal behaviors could pass unnoticed when only these 2 indices are used.

On a random sample of outlier journals identified using the PPMP and/or the Gini index, we found that the prolific authors can be “established” scientists with a relatively high H-Index (for instance, a median of 28), and that 60% of these most prolific authors were editors-in-chief or members of the journal’s editorial board. About half of these journals had a median journal impact factor of 2.9 (IQR 1.5 to 4.8). These journals generally presented a large self-citation ratio, meaning that some of them may have questionable practices by manipulating their impact factor. The other half of these journals did not have an impact factor, possibly indicating they were new journals joining the WoS. Even though WoS uses an extensive list of eligibility criteria, it is also possible that some of the new journals are predatory, journals that are known to have “leaked into” trusted sources [7]. Importantly, while NLM is skewed toward English language journals, several outlier journals were in other languages. These journals are likely to represent smaller communities with the possibility of closer interactions between researchers.

Our results underscore possible problematic relationships between authors who sit on editorial boards and decision-making editors. Typically, publishers promote independence between authors and journals. Hyperpublished authors may see such relationships as a way to more easily reach publication thresholds for hiring, promotion, and tenure. There may be defensible reasons for members of the editorial board of a journal to hyperpublish in a journal [3]. There are, for instance, certain research fields that are research niches, where the contributing authors are part of a very small community of specialists and are therefore the most likely authors.

Although our findings are based solely on a subsample of journals, they provide crucial evidence that editorial decisions were not only unusually, but also selectively, fast for the favored subset of prolific authors. This pattern was also found by Sarigöl and colleagues when exploring favoritism toward collaborators and coeditors, which persists even after taking into account individual article quality, measured as citation and download numbers [7]. This phenomenon could have an impact on productivity-based metrics and suggests a risk of instrumentalization, if not corruption, of the scientific enterprise, by using journals as a “publication laundry” for “vanity publication” [12] by authors closely related to the editorial board. Exploiting productivity metrics has been widely described, in the form of self-citation, honorary authorship, and/or ghost writing. Manipulation of individual metrics by resorting to a dedicated “nepotistic” journal appears to be a little studied way of exploiting the system.

[END]

[1] Url: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001133

(C) Plos One. "Accelerating the publication of peer-reviewed science."
Licensed under Creative Commons Attribution (CC BY 4.0)
URL: https://creativecommons.org/licenses/by/4.0/


via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.