When Clinical Experience and Research Collide: A Recent Controversy in Lower-limb Orthotics
November 2018 Issue
Clinicians and researchers with a strong desire to know the truth often hold strong opinions, even though few areas of O&P practice are supported by an equally strong base of research evidence. Different conclusions reached in clinical and research contexts can cause a collision of sorts when clinicians must make decisions about specific cases but are unsure how to integrate conflicting evidence. The importance of outcomes assessment and documentation has been stressed for decades but only in recent years become mainstream. Critically assessing the impact of the care we provide by systematically evaluating the data we're collecting will require an even higher level of commitment. Are we ready for the challenges that come with conflicting evidence about the efficacy of our treatment?
In September, the Canadian organization that credentials O&P clinicians, Orthotics Prosthetics Canada (OPC), published a statement expressing concerns regarding the provision of AFOs by untrained providers.1 The document lists concerns regarding incorrect angles, inappropriate pressure resulting in skin breakdown, and damage to other joints, and provides information regarding the training that credentialed orthotic practitioners receive. OPC also published a "Custom Ankle Foot Orthotic Fact Sheet," describing these concerns in greater detail.2 In making the case that the provision of AFOs by unqualified individuals constitutes negligence, OPC President Dan Mead, CPO(c), points out that "even professions with legitimate credentials are stepping outside of their knowledge and training and providing over-the-counter AFOs, when that may be an inadequate treatment solution." Many practitioners in the United States share these concerns and would expand this criticism to other areas of O&P practice. Five days prior to the release of the OPC's statement, an article by Tyson et. al. published in Clinical Rehabilitation reached a different conclusion about lower-limb orthotic management. The article reported on a randomized controlled trial, called AFOOT, that was designed to determine whether the recommendations in the United Kingdom's National Clinical Guidelines for Stroke regarding the provision of custom AFOs were justified.3 The authors of the study concluded that "no differences between off-the-shelf and bespoke ankle-foot orthoses were found except that participants in the off-the-shelf group had less fear of falling at short term follow-up." In other words, the custom AFOs were not found to be more beneficial, and by one measure (fear of falling), the off-the-shelf (OTS) AFOs could be considered superior to custom orthoses. These two publications are an example of a common tension between clinical care and research within O&P. Clinicians and researchers often come to different conclusions in the pursuit of clinical truth and tend to value more highly the type of evidence more common in their disciplines. This article will use the AFOOT research study as an opportunity to discuss some of these tensions in the search for clinical truth.
Description of the Study
In the introduction to their paper, the authors distinguish their study from most other AFO studies, which "have used a paired subject design (a randomized comparison with and without the ankle-foot orthosis) and only looked at the immediate effects of the ankle-foot orthosis."4 The authors reference the 2016 "National Clinical Guidelines for Stroke" which state: "People with stroke who have compromised ankle/foot stability and/or reduced ability to dorsiflex the foot (‘foot-drop') that impedes safe and efficient walking should be offered an ankle-foot orthosis to improve walking and balance. The orthosis should be evaluated and individually fitted before long-term use."5 Given the greater expense, time, and resources involved in providing custom AFOs, the aim of the authors of AFOOT was to "establish whether differences in their acceptability, safety, and effectiveness would justify the current guideline."4 Their study involved two different groups of stroke survivors in a randomized comparison between custom ("bespoke") and OTS AFOs. Subjects were included if they reported limited mobility, had impaired dorsiflexion, had no assessed ankle contractures, and could walk at least five meters without assistance. Individuals who had discontinued use of an AFO were allowed to participate, but those who currently used one were not. Subjects were referred to a local provider of orthotic services, and subjects selected for a custom AFO received a design that the orthotist believed "would best meet the patient's needs in line with their usual clinical practice."4 Those assigned to the OTS group were provided with "a light-weight flexible posterior leaf spring" AFO.4 A baseline assessment was performed, and each patient was evaluated again after six and 12 weeks. Outcomes measured included patient satisfaction, self-report of adverse events, functional mobility, fear of falling, gait speed, and step length. A total of 139 subjects began the study, 126 completed the six-week assessment, and 123 completed the 12-week assessment.
A higher percentage of patients wearing the OTS orthosis were satisfied or very satisfied at the six- and 12-week assessments. Secondary outcomes related to comfort, rubbing, donning/doffing, fitting the AFO in shoes, and appearance all favored the OTS AFO. "The off-the-shelf ankle-foot orthosis group more frequently reported a beneficial effect on foot posture and alignment…."4 Based on all clinical measures, participants in both groups improved from baseline to short- and long-term follow-up.
However, "there were no substantive differences between allocated groups in functional mobility, gait speed, step length, or longer-term falls efficacy….4 This led the authors to conclude: "…we did not find any benefits of a bespoke ankle-foot orthosis over off-the-shelf ankle-foot orthosis in terms of patient satisfaction, adverse events or clinical effects…" and "there is an obvious cost advantage to an off-the-shelf orthosis with no evidence to support the more expensive option."4 The authors recommend that "stroke survivors with limited mobility (plus dorsiflexor weakness and a plantargrade position) should be offered a light-weight, flexible off-the-shelf ankle-foot orthosis" and a custom orthosis should be provided if the OTS is not successful.4
Published Criticisms of the Study
Clinical Rehabilitation has published two letters to the editor raising questions about the article, as well as the responses of the primary author, Sarah Tyson, FCSP, MSc, PhD.5 Stephen Kirker, MB, BCh, BAO, MD (Dublin), FRCP, FRCPI, a physiatrist, raised concerns that the study overlooked the stance phase benefits of AFOs, and that "the exclusion criteria in this study excluded any patients who might benefit from a molded AFO, having no spasticity and normal range of movement."5 He compared using a custom AFO in mild cases to using an expensive antibiotic in cases that only require a less expensive one. In both cases, the mildness of the symptoms renders the more expensive treatment inappropriate. He questioned whether the plastic leaf spring OTS AFO was the best option even for relatively uninvolved cases, pointing out the benefits of carbon fiber AFOs with greater control, more energy return, and reduced bulk and weight. He also addressed the importance of considering the stance phase effects of an orthosis, rather than just swing phase control, which he describes (perhaps overstating it) as "coincidental."5 Tyson responded that "the treating orthotist had the final say in the AFO supplied. If they felt that the AFO to which the patient was randomized was going to do the patient harm then, ethically they could not supply it and could choose another one."5 Based on information included in the original article, the orthotists provided custom AFOs to six subjects who were originally assigned to the OTS group. OTS AFOs were provided to 16 subjects who were originally assigned to the custom group, although "five were fitted in error and no explanation was given for the remaining six."4 Tyson also states, "Our criteria were as broad as possible—community-dwelling, adult stroke survivors, with self-reported limited mobility, impaired dorsiflexion and no fixed ankle contractures. We had no criteria regarding spasticity and patients did not have to have full range of movement (just no fixed contractures which would prevent the patient from wearing an AFO of any description)."5
Joshua Young, an orthotist with research experience, objected that "the AFOs used are not described with enough detail to confidently apply the results clinically...."5 After listing features of AFO design such as plastic type and thickness, ankle design, straps, and padding, Young states that "the process of shape capture, positive model rectification and manufacture is not described, despite the fact that this will have implications for the fit and comfort of the device."5 In response, Tyson recalled, "Several designs were used, although I was a little surprised by the uniformity of the choices made. We did attempt to record the details of the AFOs supplied but the return was poor. The AFOs for which we received details were all made of thick polypropylene; none were padded; half enclosed the malleoli and half did not; most had a stiff full-length sole plate; all had a strap at the calf but not the ankle; only one was hinged; none had additional ankle reinforcements."5 Unfortunately, the omission of crucial information about device design is common in O&P research; discounting studies because of this omission significantly reduces the pool of clinically useful research and is likely to exclude research that demonstrates the benefits of orthotic management. It appears in the AFOOT study the participating orthotists were responsible for this lack of information. Additionally, the quality of O&P care is highly dependent on the skill of individual practitioners in shape capture, modification, and fabrication. This skill is difficult to measure, and there is little agreement on what determines optimal fit and function. Perhaps it is partly due to these reasons that this important information is generally not considered in research studies.
Clinicians and researchers often have very different perspectives on clinical questions and can come to conflicting conclusions about what represents the truth about clinical care. It is difficult to do good work in either discipline independently and combining them presents an even more arduous challenge. I was able to communicate directly with Tyson via email and a Skype call, and raised my own questions about the study. It was clear in the discussion that Tyson is an experienced researcher whose intent was to assist clinicians in making a clinical decision that we are faced with frequently. I offer these perspectives for consideration. First, as a clinician, I generally have more confidence in clinical experience and the results of consensus processes than in any individual study. The National Health Service's 2009 Best Practice Statement states that custom AFOs "are indicated in the presence of the complex gait abnormalities often associated with stroke," a recommendation that matches the experience of many clinicians.6 Clinicians must consider the complexities of each case and select designs that their experience tells them are appropriate. Second, the results of the AFOOT study may not be sufficient to overturn previous recommendations, but they certainly must be considered in future consensus processes. Third, clinicians must see participation in research as part of their professional responsibility. A meaningful search for the truth about O&P management requires the active participation of clinicians in study design and implementation. Researchers must be able to count on our contributions. Fourth, a little humility goes a long way. Clinical research and clinical care are both performed in the real world, with no perfect environment or structure for either one. Responsible professionals must acknowledge the limitations of their respective disciplines, be willing to critically examine their own work, and welcome collaboration. Finally, most of our opinions should be held more loosely. It is worthwhile remembering Niccolo Machiavelli's observation that "men are more apt to be mistaken in their generalizations than in their particular observations."
In his 1923 book The Prophet, Kahlil Gibran wrote, "Say not, ‘I have found the truth,' but rather, ‘I have found a truth.'" This advice seems particularly relevant for O&P clinical practice and research, given the high degree of diversity in clinical approaches, variations in device designs, and the limited evidence base. As much as we and our patients would like absolute answers and solutions, they are rarely available in the complex world of clinical care. When we have found clinical practices that produce positive results, it would be better to understand our conclusions as a truth rather than the truth. When confronted with research evidence that collides with that clinical experience, we should seek out opportunities to collaborate and make contributions to deepen the knowledge base of our profession.
John T. Brinkmann, MA, CPO/L, FAAOP(D), is an assistant professor at Northwestern University Prosthetics-Orthotics Center. He has more than 20 years of experience treating a wide variety of patients.
3. National clinical guideline for stroke: Intercollegiate stroke working party. 5th ed. 2016. London: Royal College of Physicians, https://www.strokeaudit.org/SupportFiles/Documents/Guidelines/2016-National-Clinical-Guideline-for-Stroke-5t-(1).aspx
4. Tyson, S. F., A. Vail, N. Thomas, K. Woodward-Nutt, S. Plant, P. J. Tyrrell. 2018. Bespoke versus off-the-shelf ankle-foot orthosis for people with stroke: randomized controlled trial. Clinical Rehabilitation 32(3):367-76.
5. Kirker S, A. Tyler. 2018. Bespoke versus off the shelf ankle-foot orthosis for people with stroke: randomized controlled trial. Clinical Rehabilitation 32(10):1418-25, doi:0269215518781195.
6.NHS Quality Improvement Scotland. 2009. Use of ankle-foot orthoses following stroke. Edinburgh.