Results Summary
What was the project about?
Many research studies seek to learn how treatments affect patients’ quality of life. Quality of life includes mood and energy. It also includes how people view their roles in their families or communities and whether they can perform those roles. Researchers use surveys to ask about patients’ quality of life. But patients may answer the same question differently depending on their age, gender, or where they live. Some patients may think about their work roles while others may think about their families or social lives. How patients think about quality of life can affect what researchers learn about the effects of treatment.
In this study, the research team tested two surveys they created to measure differences in how patients think about their quality of life. The first, long survey had 74 questions, and the second, short survey had 23 questions.
What did the research team do?
The research team used statistical methods to compare how well the two surveys measured patients’ reports of quality of life. The team compared patients’ answers to questions in the two surveys. They also checked if the short survey worked as well as the long one.
The study included 110 patients treated for bladder cancer at three hospitals. Of these, 72 percent were white, 17 percent were black, and 16 percent were multiracial. The average age was 71, and 70 percent were men. Patients filled out the two surveys at the start of the study and again three months later. They also filled out other surveys about mood and life experiences.
Patients, doctors, and patient advocates gave input on the study design.
What were the results?
The research team found that both surveys accurately measured how different patients think about their quality of life.
The research team found little overlap between the two surveys. The long survey captured more information than the short survey.
Both surveys predicted how quality of life responses change with changes in mood and life experiences. However, the long survey did better at these predictions than the short survey.
What were the limits of the project?
The study included a small number of patients with bladder cancer. Results may differ if the study included more patients or patients with other health problems. Also, the study lasted three months. Three months may not be enough time to study long-term changes in quality of life.
Future studies could test the surveys with patients in other locations or with other health problems.
How can people use the results?
Researchers can use these surveys when considering ways to look at how treatments affect patients’ quality of life.
Professional Abstract
Background
Quality of life (QOL) assessments can help researchers measure how treatments affect patients. However, a patient’s relationships, illness, and treatment can affect how patients assess QOL. These factors can change over time, and they can change how a patient assesses QOL, known as response shift. Measuring differences in patients’ QOL assessments requires instruments that address these factors.
In this study, researchers tested two such instruments: the 74-item QOL Appraisal Profile Version 2 (QOLAPv2) and the 23-item Brief Appraisal Inventory (BAI).
Objective
To test the validity and ease of administration of the QOLAPv2 and BAI for measuring differences in how patients assess QOL
Study Design
| Design Element | Description |
|---|---|
| Design | Prospective cohort study |
| Data Sources and Data Sets | Patient electronic health records, surveys, and interviews at 3 medical centers |
| Analytic Approach |
|
| Outcomes |
|
Methods
Researchers tested the QOLAPv2 and BAI in a prospective cohort study that included 110 patients treated for bladder cancer. Of these, 72% were white, 17% were black, and 16% were multiracial. The average age was 71, and 70% were male. Patients filled out the instruments at the beginning of the study and again three months later. They also completed other surveys that measured mood and life experiences.
To test construct validity, researchers examined relationships between the QOL assessment items and patients’ age, gender, and socioeconomic characteristics, using correlations and regressions.
Researchers conducted a redundancy analysis to test whether the BAI could serve in place of the longer QOLAPv2, using correlations, principal component analysis, and regression models to compare the two instruments.
To examine response shift, researchers used regression models to assess whether the instruments accounted for changes in QOL due to changes in mood and life experiences.
Patients, other researchers, patient advocates, and healthcare providers helped develop the study.
Results
Construct validity. Demographic variables accounted for differences across patients’ QOL responses. For example, in the BAI and QOLAPv2, younger patients were more likely than older patients to emphasize work roles and independence. In the QOLAPv2, older patients were more likely than younger patients to prioritize staying in their current home as long as possible.
Redundancy. Correlations between the QOLAPv2 and BAI were statistically significant. However, the shared variance ranged between 9% and 28%, indicating low redundancy and low overlap between the two instruments.
Response shift. Both instruments accurately measured response shift of QOL outcomes with changes in mood and life experiences. Compared with the BAI, the QOLAPv2 instrument performed better in predicting changes in QOL and capturing response shift effects.
Limitations
Researchers tested the instruments with a small sample of patients with a specific health condition. Results may differ if the study included more patients or patients with other health problems. The study’s three-month follow-up period may have been too short to capture additional changes in QOL over time.
Conclusions and Relevance
Researchers can use the QOLAPv2 and BAI to improve interpretation of QOL outcomes and changes in QOL outcomes across patients. The BAI may be a useful alternative when the QOLAPv2’s length makes it impractical to use.
Future Research Needs
Future research could further test the validity of both instruments with populations who have other health conditions. Researchers could also examine how to improve the redundancy between the BAI and QOLAPv2.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers asked that the researchers expand the background section of the report to describe the years of work that occurred before PCORI funding. They also asked for early clarification of the distinction between the broad construct of quality of life and the more specific health-related quality of life (HR-QOL) construct. The researchers edited the background section to better justify the practical need for this research and tried to clarify the theoretical arguments behind the importance of cognitive appraisal in making evaluations such as HR-QOL. The researchers also noted that all of the measures discussed in this report related specifically to HR-QOL but might overlap with non-health-related measures of well-being.
- The reviewers cautioned that drawing strong conclusions about the two instruments tested may not be warranted at this point. In particular, the reviewers were not convinced that both instruments were practical and interchangeable given the difference in the length of the instruments. The researchers edited the discussion section to soften some of their recommendations and provide a clearer rationale for their conclusions. They also expanded their comparison of the two instruments’ usability, noting especially the trade-offs between the longer and shorter measures. They said that further validation is needed to make a stronger recommendation about the two instruments and is underway.
- The reviewers noted that the stakeholder panel, including those stakeholders with experience as patients, was drawn from highly trained professional fields and not representative of patients generally. The researchers said they brought together stakeholders who had expertise as clinicians, policy makers, or researchers because they expected the discussions to be technical. The fact that some of the stakeholders also had experience as patients was not considered when the researchers recruited the stakeholders. The researchers also based most of their consideration of patient needs on a decade of earlier research that involved asking patients questions about appraisal assessment.
- The reviewers reminded the researchers that the Draft Final Research Report’s instructions ask for numerical results to include 95 percent confidence intervals that will convey the magnitude of the effect rather than only p-values to address statistical significance. The researchers explained that this information would not change the interpretability of the report, since they chose only statistically significant predictors to include in their regression models, so the magnitude of the relationship is not important. The 95 percent confidence intervals would also not fit well in the results tables, which are already quite large. In the discussion, the researchers went on to say that they would need to complete several more studies and a meta-analysis of appraisal and response shift differences in order to calculate the magnitude of the effect on appraisal of study and sample characteristics.