Results Summary
What was the research about?
Medicare Part D is the part of Medicare that covers prescription medicines. People who enroll in Medicare Part D coverage must choose a plan. But choosing among plans can be hard, and people may not always make an informed choice.
In this study, the research team created a decision aid called CHOICE. Decision aids help people choose among healthcare options based on what is most important to them. The CHOICE website gave people personalized information to help them choose a plan.
The research team compared three ways of helping people choose:
- CHOICE decision aid only. For this group, the CHOICE website showed a list of plans, including each person’s current plan. CHOICE pulled information about each person’s prescriptions from their electronic health records. It then showed each person’s likely spending based on these prescriptions. CHOICE also included a star rating based on quality ratings from consumers.
- CHOICE decision aid with expert advice. For this group, the website labeled the top three plans as “recommended for you” for each person. The team based these recommendations on plan cost and quality.
- Publicly available plan information. People in this group received a reminder to enroll in a plan and information on how to do so. They could also use public resources such as the Medicare.gov plan finder to review plan options.
What were the results?
Users of CHOICE with expert advice switched plans more often and were happier with the process than people who got only publicly available plan information. People in the CHOICE only and public information groups did not differ in plan switching or happiness with the process. Users of CHOICE both with and without expert advice spent more time choosing a plan than people who got publicly available plan information.
The three groups did not differ in
- How conflicted they felt about their decision after they enrolled
- How much they knew about Medicare Part D
Who was in the study?
The study included 928 people ages 66–85 with Medicare who received care from a health system in California. Of these, 80 percent were white, and 8 percent were Asian. The average age was 72, and 50 percent were women. The median household income in the areas where people in the study lived was a little over $113,000.
What did the research team do?
The study took place in Fall 2016 when people were choosing a plan for 2017. After enrolling people in the study, the research team assigned them by chance to one of the three ways to help people pick a plan.
People took surveys at the start and end of the study. The surveys asked
- Whether people switched plans
- How happy they were with the choice process
- How much time they spent choosing a plan
- How conflicted they felt about their decisions
- How much they knew about Medicare Part D
Older adults with Medicare, a caregiver, a pharmacist, and medical foundation staff gave input on the CHOICE website. They also helped with recruitment.
What were the limits of the study?
People in the study lived in areas where many people have high incomes. Results may differ for people on Medicare who live in low-income areas.
Future research could study people’s satisfaction with plans chosen using the CHOICE website over the long term.
How can people use the results?
Groups that help people enroll in Medicare could use these results when considering whether to offer a decision aid like CHOICE to help people choose a Part D plan.
Professional Abstract
Objective
To determine the effect of providing an online decision aid (CHOICE), with or without expert recommendations, on Medicare beneficiaries’ choice of a prescription drug plan compared with access to publicly available information
Study Design
| Design Elements | Description |
|---|---|
| Design | Randomized controlled trial |
| Population | 928 Medicare beneficiaries ages 66–85 with active medication orders receiving care from a large, multispecialty provider group in northern California |
| Interventions/ Comparators |
|
| Outcomes |
Primary: plan switching, satisfaction with the CHOICE process, decisional conflict Secondary: time spent choosing a plan, Medicare Part D knowledge |
| Timeframe | Up to 43-day follow-up for primary outcomes |
This randomized controlled trial compared two versions of CHOICE, an online decision aid personalized for Medicare beneficiaries to help them choose among prescription drug plans, with a control arm consisting of access to publicly available information.
Researchers recruited people with Medicare prescription drug plans before and during the 2017 Medicare open enrollment period between October and December 2016. Then, researchers randomized participants to one of three groups: CHOICE only, CHOICE with expert recommendations, or control.
Once open enrollment began, participants in both CHOICE groups received an email with a link to the CHOICE website. On this website, participants could review and update their prescription information. They could then see a list of plans, including their current plan. Next to each plan, the website showed participants how much they were likely to spend on their prescriptions and a star rating based on consumer quality ratings.
In addition to this information, participants in the CHOICE with expert recommendations group saw the top three expert-rated plans under a “recommended for you” label at the top of the screen. The CHOICE website generated expert recommendations using algorithms that combined information on individuals’ prescription drug use; cost estimates, including individuals’ premium and expected out-of-pocket prescription drug spending; and a measure of plan quality from the Centers for Medicare and Medicaid Services.
Participants in the control group received information on plan enrollment, including a reminder about the open enrollment period; information about the advantages of reviewing coverage; and links to publicly available information to review plan options.
The study included 928 Medicare beneficiaries receiving care at a large, multispecialty provider group in northern California. Of these, 80% were white, and 8% were Asian. The average age was 72, and 50% were female. The median household income in the areas where participants lived was $113,119.
Researchers compared participants’ plan choices and Medicare Part D knowledge at baseline and again after open enrollment ended. In addition, researchers surveyed participants about whether they switched plans, satisfaction with the choice process, and decisional conflict after open enrollment ended.
Medicare beneficiaries, a caregiver, a pharmacist, and medical foundation representatives helped researchers develop CHOICE and provided input on the study.
Results
More participants in the CHOICE with expert recommendations group switched plans (p=0.04) and reported that they were very satisfied with the decision process (p=0.05) compared with control. The CHOICE alone and control groups did not differ in plan switching or satisfaction with the decision process. Participants in both CHOICE groups were more likely to report spending more than an hour choosing their plans compared with participants in the control group (both p<0.05).
Researchers found no significant differences among groups in Medicare Part D knowledge or decisional conflict.
Limitations
The study disproportionately included Medicare beneficiaries who lived in high-income areas. Results may differ among Medicare beneficiaries who live in lower income areas.
Conclusions and Relevance
CHOICE with expert recommendations increased rates of plan switching and satisfaction with the choice process among people enrolled in Medicare Part D plans. CHOICE did not affect other aspects of decision making, such as decisional conflict.
Future Research Needs
Future research could focus on understanding Medicare beneficiaries’ satisfaction and healthcare use with plans selected using the CHOICE website over the longer term.
Final Research Report
View this project's final research report.
Journal Citations
Article Highlight: People who enroll in Medicare Part D, which covers prescription medicines, must choose a plan, but many people struggle to make an informed choice. Researchers on this study created a decision aid to help people choose among the options based on what is most important to them. They then tested whether the decision aid, with or without machine-based, personalized expert recommendations, influenced outcomes for Medicare Part D enrollees. Results published in Health Affairs show that patients who used the decision aid reported increased satisfaction with the process of choosing a prescription drug plan and the amount of time they spent choosing a plan.
Results of This Project
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- Reviewers asked for additional information about the selection of 1,185 people in the original study sample from the 29,000 originally eligible for the study. They noted that there were limitations that should be noted in the report regarding the use of passive recruitment strategies that required participants to contact investigators rather than researchers actively recruiting participants. In particular, this recruitment strategy may have led to sampling bias, where the participants were not truly representative of the population. The researchers explained that given their short study enrollment period, they believed this passive enrollment strategy to be the most effective way of achieving an adequate study sample. They also believed that a more active recruitment strategy would not have been feasible.
- Reviewers repeatedly expressed concern about the study’s lack of generalizability to a wide segment of the Medicare population since patients in all phases of this study were predominantly Caucasian and highly educated. The researchers responded, indicating the demographic characteristics of study participants more prominently in the report. They also expanded their discussion of the potential sampling bias in relation to study generalizability as well as study limitations.
- Reviewers asked whether the study attempted to assess whether family members assisted participants in using the project’s web-based tool. The researchers said they neither encouraged nor discouraged participants from seeking help in using the website. However, they found in a final survey that 15 percent of participants reported receiving some assistance from family members or friends. There were no significant differences between study arms in the percent of participants receiving such assistance.
- Reviewers suggested that it would have been interesting to stratify focus group and ethnographic results based on a number of key themes and participant characteristics. The researchers agreed that this would be interesting but explained that they did not link exit survey data with qualitative results. Therefore, the researchers stated that they could not stratify results by patient characteristics because of the way they collected data.