Patient-reported outcomes measures (PROMs) profile patient health status, have been found to be helpful in identifying high health care utilizers, and may be useful in providing targeted interventions to decrease health care costs. In 2013 the University of Utah Health (UU Health) began collecting mental and physical health PROMs using Patient Reported Outcomes Measurement Information System (PROMIS) instruments through a tool called My Evaluation (mEVAL). In 2012 UU Health began cataloguing inpatient and outpatient healthcare-associated costs. The objective of this study was to identify association of poor PROMIS physical function and depression scores with (1) likelihood of inpatient hospitalization and (2) overall inpatient healthcare costs.

This study was a retrospective observational cohort study including patients seen at UU Health between 1/2013 and 12/2017 who completed PROMIS instruments at an outpatient visit using the mEVAL platform. PROMIS instruments were completed prior to outpatient visits. The primary outcome was time to incident hospitalization modeled by using the Cox proportional hazards approach. For cost analysis, raw inpatient healthcare costs were fitted using a median regression model. Both results were adjusted.

Of 92,383 people, the average age was 48 (SD 18.6); 57% were female; and 87% identified as non-Hispanic white. A total of 11,909 patients who completed one or both of the mEVAL PROMIS instruments were admitted. The average PROMIS physical function and depression scores were 44.9 and 51.1, respectively. Those with worse physical function scores and worse depression scores were more likely to be hospitalized [HR = 1.77, 95% confidence interval (CI) (1.678, 1.872); HR (95% CI) = 1.149 (1.059, 1.246), respectively]. A physical function score 1.5 SD below the mean was associated with an increased median hospitalization cost of $2496; there was no statistically significant association between depression score 1.5 SD above mean and hospitalization costs.

Poor physical function scores were associated with an increased risk of hospitalization and higher inpatient health costs, while poor depression scores were only associated with increased risk of hospitalization. Future work should examine if improvement in these PROMs alters these metrics.

Patient reported outcome measures (PROMs) assess a patient’s view of his or her health status. PROMs have many applications ranging from evaluating success of medical treatments to improving health care costs [1, 2]. PROMs have been used to assess stability or progression of chronic disease [3], predict mortality in the setting of new diagnosis [4], and provide a more comprehensive evaluation of responses to medical interventions [5]. PROMs have the potential to identify patient populations that are currently, or more likely to become, high health care utilizers as poor physical function has been associated with increased emergency department visits and hospitalizations [6] and worsening of both mental and physical health has been associated with an increased odds of hospitalization [7]. In this study we sought to determine whether PROMs obtained at outpatient visits could predict likelihood of hospitalization and costs associated with hospitalization to provide a potentially intervenable assessment of hospitalization risk.

In 2004, the National Institutes of Health (NIH) initiated the Patient-Reported Outcomes Measurement Information System (PROMIS) Roadmap project, which incorporated item response theory and computerized adaptive testing to develop modern instruments for assessing PROMs [8]. The PROMIS initiative aimed to create functional, patient-centered, easily comprehended, standardized instruments, absent of rhetorical question items, for improved data collection. As a result, these measures allow more accurate assessment of an individual with fewer questions. The PROMIS instruments are well validated and provide consistent scores within its multiple domains of measurement, such as physical and mental health function [9]. Their brevity and accuracy make them attractive for use in routine clinical care.

In 2013, the University of Utah Health began to implement the standard collection of mental and physical health using PROMIS instruments through a tool called My Evaluation, or mEVAL [5, 10]. Patients complete mEVAL prior to outpatient clinic visits and the scores are integrated into the electronic health record in near real time, providing information regarding the patient health status to the clinician, and contributing to a larger pool of data with the potential for population-based health interventions. Of note, these efforts were suspended during the COVID-19 pandemic due to redeployment of handheld electronic devices towards increasing access to virtual care but are now in the process of resuming. Hospitalization data was only collected for the two years following 2017 as this prevented pandemic related hospitalizations from being included in our analysis.

In these analyses, our objective was to examine the association between PROMs and health care utilization in the inpatient setting. We test the hypotheses that patient’s self-reported physical and mental health are associated with (1) incident hospitalization and (2) overall costs of those hospitalizations.

Patients at the University of Utah who completed a mEVAL PROMIS physical function v1.2 and/or depression v1.0 instrument (https://www.healthmeasures.net/explore-measurement-systems/promis) between January 2013 and December 2017 were included in our analyses. Surveys that were not completed in conjunction with a clinical visit, such as visits with physical therapy, occupational therapy, or speech therapy, and those where the patient was not seen by any provider, were excluded. Data regarding inpatient admissions, patient comorbidities, and patient demographics were extracted from the University of Utah’s Electronic Data Warehouse (EDW), which serves as the data repository for all health system clinical data. This study was deemed non-human subjects by the University of Utah institutional review board (IRB) and exempt from IRB review.

The mEVAL tool was developed by University of Utah Health in 2013 to collect PROMs for use in clinical care. The mEVAL survey employs PROMIS’ computer-adaptive testing to decrease response burden while maintaining sensitivity. Patients can complete mEVAL from home up to one week prior to clinic appointments after receiving an email notification or in the office prior to their visit. There is no incentive for completion of mEVAL surveys.

Our analysis examined the association between the PROMIS physical function and depression scores and (1) time to first hospitalization within two years of index mEVAL completion and (2) costs of these hospitalizations. We defined the index mEVAL completion as the first time a patient completed a mEVAL survey. Subsequent mEVAL surveys completed were not included in our analysis.

PROMIS scores represent standardized T scores with a mean of 50 for the average of the general population in which they were first studied with a standard deviation of 10. A higher depression PROMIS score indicates greater symptoms of depression reported by the patient while a higher physical function PROMIS score represents higher (or better) physical function.

Costs of inpatient hospitalization were obtained from the University of Utah’s Value Driven Outcomes system (VDO). VDO was developed in 2012 at the University of Utah to determine costs associated with patient care. It provides users with direct clinical care costs associated with each individual clinical encounter, including components such as cost of facilities, healthcare providers, imaging, laboratory, medication, and adjunctive therapy services. The VDO also has the capacity to provide process and outcome measures with the ultimate goal of supporting value-based care [11].

Other covariates included in these analyses were chosen because of their relationship to either the primary predictors (physical function or depression) or the outcome (incident hospitalization and overall inpatient costs). These included age; sex (male or female); marital status (married, widowed, divorced, single, or other); race/ethnicity (Caucasian/White, American Indian/Alaska Native, Asian, Black/African American, Native Hawaiian and Other Pacific Islander, Hispanic/Latino, or Others); and presence of anxiety, substance, or mood disorder. All confounders were limited to what could be easily extractable from the EDW and are listed in Table 1. As these were secondary analyses of existing data, variables not included in the EDW were not available to include in these analyses. Presence of an anxiety, substance, or mood disorder was evaluated by searching for a pre-determined list of ICD-9 and ICD-10 codes documented within the patient chart. A co-morbidity score was calculated using the van Walraven Elixhauser Score, a modified form of the Elixhauser score [12, 13] with higher scores representing a higher medical complexity. In order to calculate this score diagnostic ICD-9 and ICD-10 codes were pulled for a full 365-day period prior to index mEVAL date for each individual [14].

Incident hospitalization

We used the Cox proportional hazard model [15] to estimate the association of PROMIS physical function and depression scores with time to incident hospitalization up to two years after completion of index mEVAL instrument. Risk of incident hospitalization was reported as an adjusted hazard ratio.

Our primary analysis was performed using mEVAL survey results as a dichotomous independent variable with the cut off 1.5 SD below and above the mean in our sample, for physical function and depression, respectively, with low physical function and high depression scores considered as impaired. The cutoff PROMIS score was 29 for physical function and 65 for depression. These cut off points were chosen based on their ability to separate those with poor physical function and more severe depression from those with only mild or moderate self reports of poor physical function or depression and were based off of the mean scores found in the patient population originally used to develop the PROMIS instrument (https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/promis-score-cut-points). Additional sensitivity analyses were performed using mEVAL survey results as a continuous independent variable.

Costs of care

For patients who had an initial hospitalization during the two years after index mEVAL date and up to current date of data extraction, the costs of hospitalization were extracted from the VDO database. Only costs from the first hospitalization following mEVAL completion were included. Costs from any subsequent hospitalizations or hospitalizations which occurred prior to mEVAL completion were not included in the analysis.

For the cost analysis a median regression model [16] was used with raw costs of hospitalization reported as the outcome. Costs included direct patient costs associated with first hospitalization to occur within 365 days from index mEVAL date. Similar to the analysis performed for hospitalization incidence, mEVAL survey results were used as a dichotomous independent variable with the same cut offs as noted above.

Effect modification

An additional analysis was performed to determine effect modification of age and sex on both the incidence of hospitalization and cost outcomes. To differentiate between older and younger patients, the median age was used. This differed between those who completed the physical function mEVAL (48 years) versus the depression mEVAL (56 years).

Analyses were conducted in R Statistical Software version 4.0.3. All data were de-identified by the EDW prior to analysis. All statistical tests were performed at the significant level of 0.05.

A total of 93,687 mEVAL surveys were completed between January 2013 and December 2017. Of these, 657 individuals were removed from subsequent analyses due to lack of documented clinical contact associated with the mEVAL completion (i.e., the patient completed the mEVAL survey and then did not attend the associated appointment or the instrument was completed in association with an ancillary service such as physical therapy) resulting in 93,030 individuals included in the analyses. Of these, 92,383 patients completed at least the Physical Function mEVAL survey and 63,344 completed at least the Depression mEVAL survey; 62,040 patients completed both (Fig. 1).

Diagram outlining patient inclusion in analysis for patients seen at University of Utah Health (UUHC)

Full size image

The mean PROMIS Physical Function v1.2 score was 44.9 (standard deviation (SD): 10.5, median 45); 6148 (7%) had low physical function (1.5 SD below the mean, which is a physical function score of 29). Those with low physical function were older than those with high physical function (mean age 55.1 and 47.4, respectively, Table 1). The mean PROMIS Depression v1.0 score was 51 (SD: 9.3; median 51); 4237 (7%) had a high (worse) depression score (1.5 SD above the mean, which is a depression score of 65). Those with higher depression scores were younger than those with lower depression scores (mean age 45.1 and 48.4 respectively, Table 2).

Of the 93,030 individuals who completed a mEVAL survey and were seen in clinic at UU Health, 12,049 (13%) had at least one hospital admission (Table 3). The mean age of patients with a hospitalization was 53 years compared to 48 years for the population overall; 61% of hospitalized patients were female (vs. 57% overall), and 87% were non-Hispanic White (vs. 87% overall). Of those with an incident hospitalization, 42% had a documented history of anxiety, 30% a history of substance use disorder, and 48% a history of a mood disorder; average van Walraven Elixhauser Score was 2.2 for those with a hospitalization compared to 0.8 of those without.

Scores for physical function were lower for those hospitalized compared to those not hospitalized (mean: 40 vs. 46, p < 0.001), and scores for depression were higher for those hospitalized compared to those not (mean: 53 vs. 51, p < 0.001). A higher percentage of those with low physical function scores (26% vs. 12%, p < 0.001) and a higher percentage of those with high depression scores (16% vs. 12%, p < 0.001) were admitted to the hospital.

After controlling for multiple confounders, we found that both those with worse physical function and depression scores were more likely to be admitted to the hospital [HR 1.77, 95% confidence interval (CI) (1.68, 1.87) and HR 1.15, 95% CI (1.06, 1.24), respectively] (Table 4). This association remained significant in secondary analyses using PROMIS scores as continuous independent variables.

An additional analysis was performed to determine effect modification of age and sex, using PROMIS scores as a dichotomous variable. Different age cut offs were used for the effect modification for physical function versus depression due to different median ages of those who completed each survey. Both older patients, over the age of 48, and younger patients with lower self-reported physical function were more likely to be hospitalized than those with higher self-reported physical function, although the likelihood of admission was much higher in older patients vs. younger patients with low self-reported physical function [Hazard Ratio (HR) 1.90 vs. 1.52, p < 0.001]. Sex also had a significantly positive effect modification on the association between low physical function and likelihood of hospitalization, although more so in male patients than in female patients (HR 2.06 vs. 1.60, p < 0.001).

Older patients, defined as greater than age 56, with high self-reported depression were more likely to be admitted to the hospital than older patients with lower self-reported depression (HR 1.41, p < 0.001). Male patients with high depression scores were more likely to be admitted than those with low depression scores (HR 1.35, p = 0.003). The effect modification of female sex was not significant on the association between depression scores and likelihood of admission (Table 5).

Among those who were hospitalized, having a PROMIS physical function score of 1.5 SD below the mean was associated with an increased median hospitalization cost of $2502 (p < 0.001). PROMIS physical function scores were also significantly associated with hospitalization costs when viewed as a continuous variable ($130, p < 0.001). Secondary analysis using a linear regression for transformed direct hospitalization costs showed similar results (data not shown).

Among those who were hospitalized, having a PROMIS depression score of 1.5 SD above mean versus below was associated with a non-significant increase in costs of $833 (p < 0.12, Table 4). However, when PROMIS scores were evaluated as a continuous variable, a 1-point increase in the PROMIS depression score was associated with an increased median hospitalization cost of $34 (p < 0.001).

Additional analyses were performed to determine effect modification of age and sex on the association between PROMIS physical function scores and inpatient costs. Age and sex were both found to have significant effect modification on the association between dichotomous PROMIS physical function scores and hospitalization costs. Patients older than 48 who reported low physical function cost on average $1864 more than those with higher physical function while younger patients with lower physical function had costs on average $3914 more than those with higher physical function (p = 0.009). Female patients with lower reported physical function on average cost $3525 more per hospitalization than female patients with higher physical function (p < 0.001). Male sex had no significant effect modification on the association between reported physical function and hospitalization costs.

For depression scores, age did not have a significant effect modification on the association between depression scores and inpatient costs, again when primarily looking at scores as a dichotomous variable. Sex was found to have a significant effect modification on the association between depression scores and inpatient costs (p = 0.003). Hospitalizations of female patients with high depression scores cost on average $1340 more than those for female patients with low depression scores. There was no significant effect modification of male sex on the association between depression scores and costs (Table 5).

In these analyses of patients from a single health system, patients with poorer PROMIS physical function and depression scores are more likely to be admitted to the hospital. Additionally, patient reports of physical function are strongly associated with inpatient health costs when controlling for other sociodemographic and medical conditions including substance use, depression, and anxiety. While many studies have evaluated the association of patient reported outcomes in the context of specialty specific procedures, such as orthopedic procedures [5], and individual disease states, such as cancer [4, 17], few have looked at whether patient reports of health from a heterogenous mix of healthcare encounters can predict health care utilization.

In the orthopedic patient population PROMs are more frequently used to assess outcomes from surgical procedures. In the oncology patient population PROMs serve as a prognostic indicator or to assess recovery from treatment such as stem cell transplant. Our study is unique in that it evaluates PROMs ability to predict healthcare utilization for patients of an entire healthcare system, rather than just those receiving treatment for a specific subset of conditions.

Our results add to the available evidence showing PROMs association with a patient’s future healthcare utilization and costs. PROMIS depression and physical function scores could potentially be used to create models to predict a patient’s risk of admission. Similar models have been used in the UK due to the large amount of healthcare data accessible through the NHS [18, 19]. Models with the best predictive value utilize both administrative and clinical data, such as data regarding previous hospitalizations.

Early recognition of these individuals in the outpatient setting may help target interventions aimed at helping to reduce hospitalization and higher health care costs. One prior study attempted to implement this type of intervention by providing patients with real time feedback regarding potential areas of intervention during clinic visits based on their responses to survey data asking about personal health behaviors, medical history, and their responses to a patient reported outcome measure, the health-related quality of life (HRQoL). These surveys were completed prior to the clinic visit. Frequency of patient-initiated discussions regarding these lifestyle risk factors were then measured [20]. Although this study did not assess impact on health care utilization it does represent one mechanism by which patient reported outcomes can be utilized to assess need for medical intervention.

There are several limitations to our analyses. We assume that PROMs are relatively stable and only subject to variation over long periods of time. Although multiple PROMIS scores were collected per individual patient we chose to use only the index, or initial, mEval completed to eliminate any outcome effect from interventions administered in response to PROMIS scores. We also do not factor in potential influence from clinic visits following the initial mEval collection, such as treatments prescribed or interventions performed. Another major limitation is the source of our data. This study only evaluated patients within University of Utah Health and only incorporated hospitalizations and costs occurring at University of Utah Health. Though our sample size is large, it is quite homogenous and younger than the average hospital population, which means our findings are not necessarily translatable to the population of the United States as a whole.

One strength of our study was the collection of mEVALs in both specialty and non-specialty healthcare settings allowing us to broaden the scope from a specialty specific lens. The patients included in this study received health care in both community and academic practices and drew from a large geographic region.

While our study examined only incidence of admission and admission costs associated with inpatient admissions future work could consider whether an association exists between patient reported outcomes and utilization in the outpatient setting. We are also unable to disaggregate the types of health care services that contribute to these higher costs (e.g., length of stay, testing volume).

Download references

RK served as primary author. YZ and HL performed statistical analysis of data. All authors read and approved the final manuscript.

Correspondence to Rachel Kroencke.

This study was deemed non-human subjects by the University of Utah institutional review board (IRB) and exempt from IRB review.

Not applicable.

The authors declare that they have no competing interests.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions