Small sample sizes are a problem in O&P research; these can lead to underpowered studies and poorly rated evidence. Low recruitment numbers are attributed to lack of funding, strict inclusion criteria and researchers’ inability to directly contact patients. According to sources interviewed for this Cover Story, clinician involvement is essential in order to address these issues.
‘Vital to the future’
Unless one is a clinician-researcher, it can be challenging to recruit subjects without the partnership of clinicians who are willing to give their valuable time, Mark D. Geil, PhD, professor and chair of the Department of Kinesiology and Health at Georgia State University, told O&P News. “[The Health Insurance Portability and Accountability Act] HIPAA guidelines often add a barrier to recruiting. The clinicians have to call the patients, explain the study and then have the families contact the researcher. It takes a lot of the clinicians’ time.”
Historically, O&P research has focused on new prosthetic componentry, but more emphasis is being placed on evaluating validated outcome measures to better understand the efficacy of current and new technology, according to Laura Miller, PhD, CP, associate professor at Northwestern University and research prosthetist at Rehabilitation Institute of Chicago.
She said the O&P profession is getting pressure from payers to justify decisions and recommendations, and without solid research to quantify and show efficacy of new technology, payers could deny it as not medically necessary. “There is a big endgame that people need to care about to get quality research done,” she said.
Geil agreed, adding that research is vital to the future of O&P. “It is clear that we need evidence to justify patient care,” he said. “Anecdotal observation of good results is just not enough anymore.”
Evolution of research
Compared with other allied health professions, O&P is a relatively new research field. “It is still going through a rapid evolution, and many of the techniques used by practitioners are based on experience and skills they have acquired through education or on the job,” Matthew Major, PhD, research health scientist at the Jesse Brown VA Medical Center and assistant professor in the Department of Physical Medicine and Rehabilitation at the Northwestern University Feinberg School of Medicine, said. “Now, research is beginning to focus on ways that we can start to standardize and quantify techniques that are being used to develop evidence-based practice going forward.”
Image: ©Shutterstock
Because of these changes, sources said, the quality of research is improving.
“Not too many years ago, people would just do a little observational study and get it published to share with practitioners. Now, the randomized controlled trial is the standard, as researchers have adopted many of the standard practices in medical trials to try and improve quality,” Glenn Klute, PhD, research career scientist in the Rehabilitation Research and Development Service at the Department of Veterans Affairs (VA) and affiliate professor in the Department of Mechanical Engineering at the University of Washington, told O&P News.
Many research questions in O&P are now focused on how techniques benefit patients and lead to optimal rehabilitation outcomes. “We can begin to understand which techniques are best for our patients, by enhancing clinical outcomes and then utilize those techniques to develop a standard of practice,” Major said.
In addition, more researchers are focusing their careers on orthotics and prosthetics, according to Sara Morgan, CPO, PhD, acting assistant professor in the Department of Rehabilitation Medicine at the University of Washington. “The pool of research is rapidly growing,” she said. “We are also seeing more of systematic and narrative reviews of the literature, which can help focus researchers on areas where evidence is lacking.”
“With the advancements in technology, the field has certainly come a long way over the past 10 years or so,” Major added. “It is an exciting time.”
Funding
However, despite the need for clinically relevant research that addresses problems users and practitioners see on a daily basis, “it is difficult to get funding to address these issues, some of which have existed for decades,” Steven A. Gard, PhD, executive director of the Northwestern University Prosthetics-Orthotics Center, director of the Jesse Brown VA Medical Center Motion Analysis Research Laboratory and associate professor in the Department of Physical Medicine and Rehabilitation in the Feinberg School of Medicine, told O&P News.
In many cases, O&P researchers find it difficult to obtain funding because there are only a few projects in their discipline competing among many other disciplines. “Without a large body of research competing for major federal funding, we are sometimes not the best fit in a grant review panel,” he said.
However, there are organizations that offer support. The NIH, Department of Defense (DOD), Veterans Administration and National Science Foundation offer funding for specific types of research. For example, the DOD has provided grants for studies that look at outcome measures and component development.
Major said he receives much of his funding from the VA. “We can develop a more complete picture of a patient’s mobility capabilities, motor control, quality of life and statistics with devices,” he said. “Once we can obtain this comprehensive characterization of the patient, then we can start to apply more personalized patient care.”
Geil said researchers should be creative in seeking funding sources. For example, he currently has a 3-year grant from the Gerber Foundation to study the outcomes of two different prosthetic prescription protocols in young children with limb loss.
A look at the numbers
Funding is just one of many contributors to low patient numbers in research studies, according to Morgan. He said low enrollment can lead to underpowered studies, making it difficult to detect clinically substantial differences. “What we see is that many O&P studies have 10 or fewer participants, which is often insufficient for most designs and outcome measures to actually show differences if a true difference exists,” he said.
In a recent editorial in the Journal of Prosthetics and Orthotics (JPO), Geil provided a brief review of the sample sizes in studies published in JPO in 2015. He found the two largest primary research studies involved 29 patients with trigger thumb and 41 patients with idiopathic scoliosis.
Sample sizes ranged from three to 41, with an average of 14.1. These numbers are in stark contrast to other scientific studies. Geil discussed a recent review published in Nature of meta-analyses in neuroscience that documented an average sample size of 185.9.
At his institution, Major said it is not uncommon to have sample sizes of 10 patients to 20 patients for biomechanical studies. “Oftentimes, the sample size is much smaller than we would desire,” he said. “If you want to adequately power the study in terms of [statistics], you need to often have at minimum 20 [patients] to 30 patients for [these types of] studies.”
Gard and colleagues typically enroll 15 to 20 patients in trials – and sometimes, ten patients. “But the rehabilitation field in general typically involves smaller numbers of subjects,” he said. “So it is not unusual to find studies being published with only five to 10 subjects. There is still good information that can come out of that, assuming the study is well-structured and the protocol is well-designed.”
The fact that researchers are recruiting from a small population to begin with is a main reason for the low recruitment numbers, Morgan said. “This is especially true for researchers who are doing single-site studies and are limited to the population of people in their immediate area.”
O&P research interventions are typically more expensive than those in pharmaceutical drug trials, sources said. “Our studies tend to cost quite a bit of money because we are fitting patients and we need lots of people to help train the patients to use the new device and build the device,” according to Klute.
Strict inclusion criteria required for many O&P studies also contributes to low numbers. In addition, researchers typically have a limited number of components.
“We might have one or two of something, so the recruitment might be limited based on the person’s size or side,” Miller said. “For example, [if] we have a right hand to test, we need someone who is wearing a gel liner with a certain brand. All [of] those limitations to try to get that homogenous sample can make it challenging to find a good sample size.”
Major added that studies often enroll convenience samples of patients in the community who are ambulatory and motivated to participate. “Those individuals tend to be more physically able, and as a result, it is not uncommon for results to be biased toward that population.”
In other cases, samples may be biased toward one gender or age group. “I need [to enroll] more women,” Klute said. “But women do not tend to be victims of trauma compared with the number of men who are. Likewise, in the military, there are not nearly as many women in the age bracket of 60- [years] to 70-years who are facing amputation because of diabetic or vascular problems.”
Enrollment in research studies also requires the participant’s time. The experiments often take 2 hours to 8 hours to conduct and participants have to be evaluated multiple times during the course of the study, according to Gard. Patients may not be willing to commit to research projects that require them to alter their normal routines.
Study requirements also may eliminate individuals with other conditions. “For example, for walking, we need someone to test a foot or socket under several configurations, speeds and environments, and that might exclude individuals with comorbidities that would limit their ability to walk for an extended period of time,” Miller said.
Retaining participants also can be a challenge. Patients may be happy with their current prosthesis and not interested in experimenting with another. “Some of our prototypes can be a little on the rough side and not as finely polished as a commercial product,” Klute said. “The patient has to wear the prototype day in and day out. That is one of the reasons it is hard to both recruit and retain.”
Clinician involvement
Communication between researchers and practitioners is imperative to increasing enrollment numbers for research in O&P, according to Gard. “Researchers need to reach out to practitioners, not only for recruitment, but also to identify problems to address through research,” he said.
“Practitioners play an important role, which is to keep the research at the ground level,” Laurent Frossard, PhD, adjunct professor at Queensland University of Technology and University of Sunshine Coast and director and chief scientific officer at YourResearchProject in Australia, added.
“It is important to have practitioners so we do not have a gap between what is happening at the ground level from a practitioner’s view versus what is happening in an experimental laboratory that could be interesting but is not necessarily practical for patients.”
Gard and colleagues maintain relationships with local clinics. When looking for particular types of patients, they send fliers to clinics to request practitioners assist them in the research.
“The practitioners will keep an eye out for the types of individuals we are looking to include in our studies,” Gard said. “We can even write some of their effort into grant proposals, and if those studies get funded, we can perhaps compensate the practitioners for their time.”
Major said developing professional relationships with practitioners allows them to communicate the importance of research, the clinical application and how it could impact overall practice. He and colleagues reach out to practitioners by presenting at local chapter meetings for the American Academy of Orthotists and Prosthetists (AAOP). “We make a strong effort to disseminate our work, and have people become aware of what we are doing and what types of patients we need to recruit,” he said.
Major and Morgan said they advertise their studies on the Amputee Coalition website, the O&P listserv and through the AAOP. Morgan has also advertised in magazines.
Involvement at all levels
Clinicians can become involved in research and recruitment in several ways. One is to offer services to a local research group. “Find out what they are doing, and invite them [to] come and spend time in the office,” Miller said.
If clinicians are not able to meet with the researchers locally, Miller suggested attending national and international meetings. He also suggested clinicians sign up to be journal reviewers, which will allow them to understand and see the publications and provide clinical critique.
Clinicians could also conduct their own case studies as a starting point for becoming more involved. If an intervention is successful with one patient, then the clinician could then contact researchers to conduct a randomized, controlled trial and obtain funding. “Starting small and working up is the place to start,” Klute said.
Frossard suggested practitioners collect data in their own practice, with the ultimate goal of creating a national registry. He stressed the importance of correctly designing a data collection system, which requires the assistance of researchers.
“We will have a far more accurate picture of the actual benefit of an intervention from the sense of what ‘Practitioner A’ is doing and what ‘Practitioner B’ is doing in two different parts of the world,” he said. “[We] will be able to look at real implications of an intervention. In order to do that, every practitioner needs to be on board.”
Major includes someone with a clinical background in every project to ensure the results are clinically meaningful and can be applied to practice. “It is the idea of translating from theory to practice,” he said. “If there is no direct connection, the research message can get lost.”
According to Major, practitioners should be involved at all levels: from developing experimental protocols to screening, recruiting and collecting data, to interpreting results and preparing them for publication.
“I have research ideas percolating in my head all the time, but I do not know if those ideas are worthwhile until I bounce them off a clinician who knows what the field and patients need,” Geil said. “Alternatively, I have partnered with clinicians who have ideas percolating in their heads all the time, too.”
“I like to think that ultimately, as researchers, we are helping the practitioner make better decisions about treatment options,” Gard added. “We are trying to improve their procedures or introduce new technology that will increase quality of life for the end user. We rely on the practitioner to implement the findings from the research and development projects.”
Klute agreed. He said practitioners got into the field because they like helping people. “In clinical practice, they help one patient at a time. In research, if your work is making progress, you can help thousands of people at a time.” – by Tina DiMarcantonio
- Reference:
- Geil MD. J Prosthet Orthot. 28(3):93-94.
Disclosures: Frossard, Gard, Geil, Klute, Major, Miller and Morgan report no relevant financial disclosures.
