National Institutes of Health research project grant inflation 1998 to 2021
Curation statements for this article:-
Curated by eLife
eLife assessment
This is an important manuscript that provides support for the hypothesis that the relative increase in NIH funding matches the rate of overall inflation. The level of evidence is solid, and a clearer description of the analysis will further strengthen the manuscript. This paper should be of relevance to funders, investigators who are currently funded, and those who are seeking federal support.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
We analyzed changes in total costs for National Institutes of Health (NIH) awarded Research Project Grants (RPGs) issued from fiscal years (FYs) 1998 to 2021 . Costs are measured in ‘nominal’ terms, meaning exactly as stated, or in ‘real’ terms, meaning after adjustment for inflation. The NIH uses a data-driven price index – the Biomedical Research and Development Price Index (BRDPI) – to account for inflation, enabling assessment of changes in real (that is, BRDPI-adjusted) costs over time. The BRDPI was higher than the general inflation rate from FY1998 until FY2012; since then the BRDPI has been similar to the general inflation rate likely due to caps on senior faculty salary support. Despite increases in nominal costs, recent years have seen increases in the absolute numbers of RPG and R01 awards. Real average and median RPG costs increased during the NIH-doubling (FY1998 to FY2003), decreased after the doubling and have remained relatively stable since. Of note, though, the degree of variation of RPG costs has changed over time, with more marked extremes observed on both higher and lower levels of cost. On both ends of the cost spectrum, the agency is funding a greater proportion of solicited projects, with nearly half of RPG money going toward solicited projects. After adjusting for confounders, we find no independent association of time with BRDPI-adjusted costs; in other words, changes in real costs are largely explained by changes in the composition of the NIH-grant portfolio.
Article activity feed
-
-
Author Response
Reviewer #2 (Public Review):
- The manuscript assumes an understanding of both economic terminology and statistical approaches that will not be familiar to most of the audience, if I am a representative example. This begins in the abstract, much of which I found incomprehensible. I still am not sure about the definition of "nominal costs ", and I certainly have no idea what they mean by a "wholly non-parametric machine learning regression". This continues throughout-presenting much of the data as Log10-transformed costs means that many of the graphs become impossible for a normal mortal like me to interpret.
We agree with the reviewer. We provide definitions of terms in the Introduction (lines 29-41) and explain the regression methods in greater detail in the text (lines 173-182) and appendix (Tables 1 and 2).
- The …
Author Response
Reviewer #2 (Public Review):
- The manuscript assumes an understanding of both economic terminology and statistical approaches that will not be familiar to most of the audience, if I am a representative example. This begins in the abstract, much of which I found incomprehensible. I still am not sure about the definition of "nominal costs ", and I certainly have no idea what they mean by a "wholly non-parametric machine learning regression". This continues throughout-presenting much of the data as Log10-transformed costs means that many of the graphs become impossible for a normal mortal like me to interpret.
We agree with the reviewer. We provide definitions of terms in the Introduction (lines 29-41) and explain the regression methods in greater detail in the text (lines 173-182) and appendix (Tables 1 and 2).
- The version presented is written like some early outline draft. Rather than using narrative to guide the reader through the data, it reads like a series of Figure legends. For example, I literally thought the text on page 4 were the Figure legends, but they are not. "Figure 2 shows...." "Table 1 shows...". The Discussion is similarly difficult to follow. Given the complexity and importance of the data they present, this is a major missed opportunity/
We agree with the reviewer. We have extensively rewritten the text as recommended by the reviewer.
- What will most interest my own part of the NIH-community is the assertion that "real dollar adjusted" grant funding has not decreased, but has instead remained flat. Few people I know will believe this. The authors address in a less-than-clear fashion some of the reasons for this-solicited versus non-solicited awards, clinical trials, etc, but do not dig into their own data to identify what are likely to be other issues. I doubt any one of the 20+ NIH-funded researchers in my Department (predominantly NIGMS funded) has a grant that reaches the "median level"-I do not after 32 years of continuous NIH-funding. Most new NIGMS-funded researchers, including many in my Department, are coming in funded by MIRA grants, which at $250K are half the median grant size. They do spend a few moments on disparities in Figure 7, but much more could be pulled out of this data set. Digging into issues like this-distributions in different NIH Institutes, at different career levels, etc, would make this work much more impactful.
We agree with the reviewer. We provide additional data on R01-equivalent awards (as previously noted) and on the $250K and $500 nominal values. See new Tables 2 and 4. We acknowledge that our analysis is based on NIH as an agency, not on individual Institutes and Centers (lines 259-260).
-
eLife assessment
This is an important manuscript that provides support for the hypothesis that the relative increase in NIH funding matches the rate of overall inflation. The level of evidence is solid, and a clearer description of the analysis will further strengthen the manuscript. This paper should be of relevance to funders, investigators who are currently funded, and those who are seeking federal support.
-
Reviewer #1 (Public Review):
This paper provides the first comprehensive analysis since the doubling of the NIH budget, on how the institute is able to keep up with inflationary pressures and fully support investigators. Through a series of descriptive graphs and regression analyses as well as modeling and transformations, the authors demonstrate the relative similarities between inflation trends and NIH support over time. Interestingly larger, more solicited projects, including greater number of clinical studies, are now driving a greater proportion of the costs for NIH. The modeling is relevant but the limitations need to be recognized and these include: the issue of personnel costs, not well captured by their approach, and productivity; i.e. is the increase in spending matched by an increase in traditional metrics (manuscripts, other …
Reviewer #1 (Public Review):
This paper provides the first comprehensive analysis since the doubling of the NIH budget, on how the institute is able to keep up with inflationary pressures and fully support investigators. Through a series of descriptive graphs and regression analyses as well as modeling and transformations, the authors demonstrate the relative similarities between inflation trends and NIH support over time. Interestingly larger, more solicited projects, including greater number of clinical studies, are now driving a greater proportion of the costs for NIH. The modeling is relevant but the limitations need to be recognized and these include: the issue of personnel costs, not well captured by their approach, and productivity; i.e. is the increase in spending matched by an increase in traditional metrics (manuscripts, other grants, policy change, etc. Nevertheless, the bottom line, of interest to funders, investigators, and institutions, is that NIH has been able to maintain support at a level commensurate with inflation.
-
Reviewer #2 (Public Review):
When I was asked to review this paper, I was quite excited, as the analysis seemed very timely. Many of us in biomedical science feel like we are at an inflection point in our field. The combined impact of the pandemic on both people's outlook and on the supply chain, the sharply rising costs of living in major metropolitan areas, and the increasing gap in potential salaries between industry and stipends for graduate students and postdocs are shaking our field. The need to increase salaries for PhD students and postdocs is colliding with a 20+ year stagnation in the size of a non-modular R01, creating major challenges for many basic science labs.
However, having read the piece, I am quite disappointed at what seems to be a missed opportunity. The scientific community at large, and particularly the basic …
Reviewer #2 (Public Review):
When I was asked to review this paper, I was quite excited, as the analysis seemed very timely. Many of us in biomedical science feel like we are at an inflection point in our field. The combined impact of the pandemic on both people's outlook and on the supply chain, the sharply rising costs of living in major metropolitan areas, and the increasing gap in potential salaries between industry and stipends for graduate students and postdocs are shaking our field. The need to increase salaries for PhD students and postdocs is colliding with a 20+ year stagnation in the size of a non-modular R01, creating major challenges for many basic science labs.
However, having read the piece, I am quite disappointed at what seems to be a missed opportunity. The scientific community at large, and particularly the basic science community is hungry for data like this, to use to try and convince Congress and NIH as a whole to address the issues above. However, as far as I can tell, the authors are not writing for this audience-in fact I was puzzled about their view of for what audience this was intended. I will note several major issues-all could be addressed but would require an effort to tell this story in a much more comprehensible and complete way
1. The manuscript assumes an understanding of both economic terminology and statistical approaches that will not be familiar to most of the audience, if I am a representative example. This begins in the abstract, much of which I found incomprehensible. I still am not sure about the definition of "nominal costs ", and I certainly have no idea what they mean by a "wholly non-parametric machine learning regression". This continues throughout-presenting much of the data as Log10-transformed costs means that many of the graphs become impossible for a normal mortal like me to interpret.
2. The version presented is written like some early outline draft. Rather than using narrative to guide the reader through the data, it reads like a series of Figure legends. For example, I literally thought the text on page 4 were the Figure legends, but they are not. "Figure 2 shows...." "Table 1 shows...". The Discussion is similarly difficult to follow. Given the complexity and importance of the data they present, this is a major missed opportunity
3. What will most interest my own part of the NIH-community is the assertion that "real dollar adjusted" grant funding has not decreased, but has instead remained flat. Few people I know will believe this. The authors address in a less-than-clear fashion some of the reasons for this-solicited versus non-solicited awards, clinical trials, etc, but do not dig into their own data to identify what are likely to be other issues. I doubt any one of the 20+ NIH-funded researchers in my Department (predominantly NIGMS funded) has a grant that reaches the "median level"-I do not after 32 years of continuous NIH-funding. Most new NIGMS-funded researchers, including many in my Department, are coming in funded by MIRA grants, which at $250K are half the median grant size. They do spend a few moments on disparities in Figure 7, but much more could be pulled out of this data set. Digging into issues like this-distributions in different NIH Institutes, at different career levels, etc, would make this work much more impactful.
As one example, this analysis from NIGMS suggests the median grant was likely under $225K, a year when their data suggest the median grant overall was $400k
https://loop.nigms.nih.gov/2016/05/distribution-of-nigms-r01-award-sizes/My bottom line-this study addresses a key question, but as currently written does so in a way that will minimize its impact
-
Reviewer #3 (Public Review):
The issues raised in this review are more conceptual in nature and my suggestions are designed to sharpen the focus of the paper. The paper does a good job of explaining how prices of NIH project have changed over time but leaves the reader wanting a clearer understanding as to why this has happened. The paper raises the issue of price effects compared with compositional effects at the beginning and the very end of the paper. It would have been helpful for the paper to be more explicit about examining price changes and composition changes in the organizing structure of the paper (e.g. the solicited v. unsolicited is a compositional change and should be highlighted as such). The authors conclude that changes in NIH prices are associated with changes in the composition of NIH funding, and the evidence supports …
Reviewer #3 (Public Review):
The issues raised in this review are more conceptual in nature and my suggestions are designed to sharpen the focus of the paper. The paper does a good job of explaining how prices of NIH project have changed over time but leaves the reader wanting a clearer understanding as to why this has happened. The paper raises the issue of price effects compared with compositional effects at the beginning and the very end of the paper. It would have been helpful for the paper to be more explicit about examining price changes and composition changes in the organizing structure of the paper (e.g. the solicited v. unsolicited is a compositional change and should be highlighted as such). The authors conclude that changes in NIH prices are associated with changes in the composition of NIH funding, and the evidence supports that. However, the NIH has inordinate control over prices because of the salary cap imposed in 2012. It would be helpful to see the relative weights of the various components of the BRDPI index in the paper graphed over time. I suspect the personnel salaries receive the highest weight. Figure 1B indicates BRDPI dropped by over 1.5 percentage points once the salary cap was put into place. When the NIH mechanically caps the price increases in salaries, they will hold research inflation (BRDPI) in check.
In addition, many of the notable trends in the data deserve further discussion. For example, in Figure 1A, awards are much higher than awardees, indicating that there are many PIs with multiple awards. This difference narrowed after 2013, but by 2021, there are ~5,000 multiple RPG awardees. This deserves some discussion. Furthermore, in Figures 2 through 4, the real value of NIH funding per project has fallen since the NIH doubling. This is a hugely important point and deserves more discussion. Eyeballing the real drop in value in Figures 3 and 4, it's approximately ~$50,000 (about 10%) close to the cost of one postdoc on an RPG. Clearly, by keeping the real costs of funding per project down, NIH is able to fund more projects. But what are the tradeoffs of this kind of policy? This may be beyond the scope of the paper, but it would be helpful for the authors to discuss the possibility that imposing the salary cap may have had some unintended consequences.
On Page 9 the authors state: "From 2012 through 2021 whisker ranges increased, exceeding levels for the doubling for untransformed costs, and not quite reaching doubling levels for logtransformed costs." Later the authors argue that this is the result of changes in the composition of research grants-that solicited grants are a larger share and cost more. However, it may be possible that the variance of funding costs is a by-product of the salary cap in 2012. When PIs could no longer charge full personnel costs, they may have developed different approaches to maximizing funding from NIH. This should be commented on in the paper. For example, are certain institutions (perhaps those that receive a lot of NIH funding in the first place) better at this kind of budget request than others.
While the authors attribute much of the change in the variance of costs to composition effects (solicited vs. unsolicited projects), the timing of the variance changes is interesting. It's very telling that during the doubling, the variance in grants was higher and then when NIH funding fell in real terms, the variance in funding narrowed (Figure 6). After the salary cap and the 2015 budget increases, the variance in funding increased again. This suggests that when money is tight the variation in funding narrows. I know the authors ran a regression on the time effects of actual funding costs (Figure 13) but not on the variance. Again, the time series of the variance in funding begs for further explanation.
Since much of the change in the composition of NIH grants is between solicited vs. unsolicited projects, it would be helpful to provide more information on the nature of solicited proposals and why NIH has shifted to funding more of them. For example, are these one-time solicitations? Are these U-mechanisms? Some combination of both? How would COVID-related funding appear in the NIH portfolio? A paragraph describing this change in emphasis and the types of projects being solicited would be very helpful.
In the conclusion, it would be helpful to mention the NIH salary cap during the discussion of the Baumol cost disease. While it is true that services will cost more overtime relative to goods (since robots can replace production workers in manufacturing but not postdocs in laboratories), the NIH effectively has its thumb on the price level with the salary cap. Cost disease is not going to be as problematic as long as the salary cap remains in place. However, there is growing evidence that the effective price cap that NIH has in place on NRSA stipend levels is generating shortages of postdocs (see https://www.science.org/doi/pdf/10.1126/science.add6184 and https://www.statnews.com/2022/11/10/tipping-point-is-coming-unprecedented-exodus-of-young-life-scientists-shaking-up-academia/). The authors should comment on the growing reports of labor shortages and consider how NIH may have to respond to this in the coming years.
-
