Results Summary
What was the project about?
Clinical practice guidelines help doctors decide on treatments to recommend for their patients. Guidelines are based on research that looks at the benefits and harms of different treatments. Patient and caregiver input can improve the usefulness of guidelines. But guideline developers often rely on the input of only a few patients and caregivers.
In this study, the research team created a process for getting feedback on guidelines from larger groups of patients and caregivers. This process is called the RAND/PPMD Patient-Centeredness Method, or RPM. The team tested RPM with guidelines for Duchenne muscular dystrophy, or DMD. DMD is a severe form of muscle loss that mostly affects young boys.
What did the research team do?
The research team recruited 24 patients and 71 caregivers from an online Duchenne Registry and assigned them by chance to one of two online panels. Ninety-one percent of panel members were white and 9 percent were other races. Also, 62 percent were women.
Each panel completed the same three-round process. In round one, panel members rated the importance and acceptability of 19 recommendations included in the DMD guidelines. Panel members explained each of their ratings and the factors that affected them.
In round two, panel members reviewed charts that compared their round one responses with those of the full panel. The charts also showed whether the panel agreed on ratings. Panel members then discussed their feedback in a moderated online forum.
In round three, panel members could change their original ratings based on round two feedback and discussions.
The research team looked at
- How much panel member agreement on ratings changed between rounds one and three
- How similar the two panels’ final ratings were
- How many recommendations were patient-centered, which the team defined as two panels rating a recommendation both important and acceptable
Patients, caregivers, doctors, and guideline developers gave feedback on the study.
What were the results?
Agreement. Between rounds one and three, panel member agreement on ratings increased from 95 percent to 100 percent.
Panel decision similarity. The panels reached the same decision on ratings for 89 percent of the recommendations.
Patient-centeredness. The research team deemed 12 of the 19 recommendations patient centered.
What were the limits of the project?
This study focused on one set of guidelines. Results may differ for other health problems. The research team tested RPM on finalized guidelines. Ideally, patients and caregivers would give input on draft guidelines. Then guideline developers could make changes based on their input.
Future research could test RPM on other guidelines. Studies could also look at how guidelines and their impact change when patients and caregivers are involved from the start.
How can people use the results?
Guideline developers can use the results when considering how to get input from large groups of patients and caregivers.
Professional Abstract
Background
Patients and caregivers can enhance the quality and legitimacy of clinical practice guidelines, which may promote increased guideline adherence among patients and clinicians. However, guideline developers typically include only a few patients or their representatives on review panels, and their inclusion is not systematic.
Objective
To develop and test a scalable process for systematically eliciting patient and caregiver input on clinical practice guidelines
Study Design
| Design Element | Description |
|---|---|
| Design | Mixed methods |
| Data Sources and Data Sets | Ratings of importance and acceptability of 19 recommendations from the DMD care considerations made by 95 members of the Duchenne Registry |
| Analytic Approach | Quantitative analysis of importance and acceptability ratings and qualitative analysis of participant comments |
| Outcomes | Participant agreement, process replicability, patient-centeredness of guideline recommendations |
Methods
This mixed-methods study developed an online modified-Delphi approach for eliciting feedback on clinical practice guidelines from large numbers of patients and caregivers, called the RAND/PPMD Patient-Centeredness Method (RPM). Researchers tested RPM by asking patients with Duchenne muscular dystrophy (DMD) and their caregivers to rate the patient-centeredness, which the researchers operationalized as importance and acceptability, of recommendations for the management of DMD included in the 2018 DMD care considerations.
Researchers recruited 24 patients and 71 caregivers from the Duchenne Registry to provide input. Of these participants, 91% were white and 9% were other races; 62% were female.
Participants were randomly assigned to one of two panels. In round one, participants in each panel used nine-point Likert scales to rate the importance and acceptability of 19 recommendations. Participants also explained their ratings and the factors that influenced their responses.
In round two, participants reviewed bar charts showing how their round one responses compared with the responses of other panel members and whether the panel reached consensus on each recommendation’s importance and acceptability. Participants then discussed their feedback in a moderated online forum.
In round three, participants could revise their original ratings based on round two feedback and discussions.
Researchers assessed the extent to which agreement among participants on importance and acceptability ratings changed between rounds one and three, the replicability between the two panels’ final ratings, and the patient-centeredness of DMD care considerations, which the researchers defined as two panels agreeing that a recommendation is both important and acceptable.
Patients, caregivers, clinicians, and guideline developers helped design and provide feedback on the study.
Results
Agreement. Participant agreement on ratings assigned to recommendations increased from 95% in round one to 100% in round three.
Panel decision replicability. The two panels reached identical decisions on 89% of the ratings, yielding a kappa of 0.69, which indicates a high level of inter-panel agreement.
Patient-centeredness. The researchers deemed 12 of the 19 recommendations patient centered.
Limitations
Researchers tested RPM for a single clinical condition, and these recommendations were in final form. Results may differ for different guidelines. Ideally, patients and caregivers would provide feedback on draft guidelines, so developers could incorporate their input during the guideline development process.
Conclusions and Relevance
This study developed a scalable, online process for eliciting feedback from patients and caregivers on clinical practice guidelines.
Future Research Needs
Future research could test the process with different guidelines or evaluate the value added of patient engagement by working with patients and their representatives early in clinical practice guideline development.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers asked for more justification for why it is necessary or helpful to include a large number of patients and caregivers in developing clinical practice guidelines. The reviewers asked if there is any previous research on how much patient input is needed. The researchers replied that they were not aware of literature that recommends input from a particular number of patients but said one or two patients are typically involved. The researchers added that patients with less education, income, mobility, and with cognitive impairments are typically not included. A larger, more diverse patient group can reflect a broader range of healthcare experiences, the researchers explained.
- The reviewers suggested that key findings from the report be made more accessible to patients and patient advocates, for example, in a how-to manual or other report that would be easier to digest and use. The researchers responded that they had shared their results with the Duchenne muscular dystrophy community and planned to continue their efforts to disseminate their work to other rare disease communities. The researchers also addressed this in the future research section of the report’s discussion, stating that although their methods were meant to be agnostic to disease or condition, it would be helpful to test the methods in groups that differ by age, disease area, and other characteristics. They went on to recommend that the emphasis in these future opportunities should be in making sure the methodology is accessible to patients with different levels of education.
- The reviewers noted that one consideration in online patient engagement is that it is difficult to confirm who might be responding to the request for input and therefore, how the respondents are viewing their own experiences or those of their loved ones. The researchers agreed this is a concern, which is why they relied in this study on the Duchenne Registry to recruit study participants. The researchers also created a recruitment survey to ensure that their participants had the necessary expertise and experience to respond. Furthermore, the researchers recommended in the report that groups using these methods should strive to recruit participants with diverse perspectives, and to confirm participants’ eligibility to participate.
- The reviewers suggested consistently referring to patients and caregivers rather than patients and representatives. The researchers said they deliberately used the different terms, using patients and caregivers when discussing their own data collection. However, they chose to use patients and their representatives when referring to literature that uses those terms or discussing the implications of their work to the guideline development process. The researchers explained that patient representatives include family members, patient advocates, and others who are not necessarily caregivers.