Scientific Communication and Education

eLife assessment

This convincing study, which is based on a survey of researchers, finds that women are less likely than men to submit articles to elite journals. It also finds that there is no relation between gender and reported desk rejection. The study is an important contribution to work on gender bias in the scientific literature.

eLife assessment

This study used ChatGPT to assess certain linguistic characteristics (sentiment and politeness) of 500 peer reviews for 200 neuroscience papers published in Nature Communications. The vast majority of reviews were polite, but papers with female first authors received less polite reviews than papers with male first authors, whereas papers with a female senior author received more favourable reviews than papers with a male senior author. Overall, the study is an important contribution to work on gender bias, but the evidence that generative AI programs like ChatGPT have the potential to be useful in meta-research is incomplete, especially given the lack of a comparison to the many existing and already-validated tools for sentiment analysis based on natural language processing.

eLife assessment

This is a descriptive paper in the field of metascience, which documents levels of accessibility and reproducible research practices in the field of cardiovascular science. As such, it does not make a theoretical contribution, but it argues, first, that there is a problem for this field, and second, it provides a baseline against which the impact of future initiatives to improve reproducibility can be assessed. The study was pre-registered and the methods and data are clearly documented. This kind of study is extremely labour-intensive and represents a great deal of work.

eLife assessment

This study follows the career trajectories of the winners of an early-career funding award in the United States, and finds that researchers with greater mobility, men, and those hired at well-funded institutions experience greater subsequent funding success. Using data on K99/R00 awards from the National Institutes of Health's grants management database, the authors provide convincing evidence documenting the inequalities that shape faculty funding opportunities and career pathways, and show that these inequalities disproportionately impact women and faculty working at particular institutions, including historically black colleges and universities. Overall, the article is an important addition to the literature examining inequality in biomedical research in the United States.

eLife assessment

This work shows that higher socioeconomic status is associated with a higher risk of obesity, which should inform China's obesity public health programs and policies, and also be of interest to other countries and communities. The evidence supporting the conclusions is strong, but the data analysis is incomplete and would benefit from more rigorous approaches.

eLife assessment

This important bibliometric analysis shows that authors of scientific papers whose names suggest they are female or East Asian get quoted less often in news stories about their work. While caveats are inevitable in this type of study, the evidence for the authors' claims is convincing, with a rigorous, and importantly, reproducible analysis of over 20,000 articles from across 15 years. This paper should be of interest to all scientists and science journalists, as well as to those who study science communication.

eLife assessment

This is an important manuscript that provides support for the hypothesis that the relative increase in NIH funding matches the rate of overall inflation. The level of evidence is solid, and a clearer description of the analysis will further strengthen the manuscript. This paper should be of relevance to funders, investigators who are currently funded, and those who are seeking federal support.

Evaluation Summary:

This manuscript presents a descriptive audit on statistically treatment, reporting and interpretation of the effects of sex as a biological variable (SABV) on the studied outcomes in articles published across nine scholarly disciplines. The manuscript highlights and provides data on prevalence of several inconsistencies and inaccuracies in the literature regarding treatment of SABV as an important moderator of the effects of an intervention on a considered outcome and how such inconsistencies could lead to biased conclusions regarding the effects of SABV. As such, the manuscript may inform not only funding agencies and grant reviewers, but also researchers in most scientific disciplines regarding the importance of adhering to rigorous methodological standards when examining the effects of SABV.

(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #3 agreed to share their names with the authors.)

Evaluation Summary:

This manuscript by Nakamura et al provides additional information on the impact of bias in the scientific review process. The authors find that on average applications from White scientists scored better than those from Black scientists. However, blinding reviewers to race worsened the score of the White scientists with no impact on the score for Black scientists. In view of the recognized value of increased diversity in science, the additional information provided in this manuscript adds new data to this discussion. Potential solutions to this bias though are complex.

(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)

Evaluation Summary:

This paper provides the basis for further discussion about the perceived inequities in NIH funding based on race. The strengths of this manuscript are the detailed breakdown of the available data in order to evaluate for biases, the availability of data for multiple years (2011-2015) and the consideration of alternate explanations (e.g. new applications vs resubmissions; single vs multi PI). With that said, given their conclusion that Institute (IC) assignment was the main determinant of funding rates, the approach for IC assignment should have been discussed. Other issues relate to the complexity of statistical analyses and a lack of clarity on confounding issues towards firm conclusions.

(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #2 agreed to share their name with the authors.)