Outcome Measures: Are We There Yet?

How do you measure the quality of the prosthetic or orthotic devices you provide to your patients? Many practitioners say a successful outcome is a happy, comfortable patient who returns to you when it's time for a new fitting. In today's reimbursement climate, however, a happy patient isn't enough. O&P practitioners are now being asked to justify their experience-based clinical decisions with evidence-based outcome measures. Insurers want objective, quantifiable data to support the clinician's choice of components for a given device. After decades of researching and developing meaningful measures of O&P outcomes, clinicians and academicians agree on one thing: There is no single appropriate measure. What makes a good device-or makes one device better than another for a particular patient-depends on a wide array of factors, many of which are highly subjective. That doesn't lessen the demand for objective, universally accepted, scientifically validated, easy-to-administer, and even easier-to-understand measurement tools. Such tools are being developed in both academic and clinical settings, supported by companies and foundations, as well as with government funding, although experts decline to set any sort of timeline for arriving at a "gold standard." "O&P training is rooted in apprentice-oriented approaches," explains Christopher Hovorka, MS, CPO, LPO, FAAOP , codirector of the master of science in prosthetics and orthotics (MSPO) program at the Georgia Institute of Technology (Georgia Tech), Atlanta. "In the past, new practitioners learned from local experts in a culture of performance, and all that mattered was if the orthotic or prosthetic device made the patient happy. There was little reason for referral sources to ask additional questions." With the rise of health maintenance organizations (HMOs), however, the O&P practitioner's role began to change from being primarily a fabricator/fitter of devices to being a member of the patient care team. Increasingly, these organizations began to look for justification for the cost of O&P care as part of a treatment plan. Why did you fit patient A with a microprocessor knee but patient B with a traditional, mechanical knee-and why should we pay the additional cost? What were you thinking? With no formal background in chronicling their work, O&P practitioners borrowed documentation processes from other professions, some of which did not adequately show evidence for their clinical decisions. Since then, most individual facilities have developed their own internal checklists to satisfy payer information requests, but not many of these have been scientifically validated. Fortunately, there is a growing awareness within the O&P profession of the importance of using validated outcome measures. "A core group of people is moving the issue [of validation] forward," says D. Scott Williamson, MBA, CAE , president of Quality Outcomes, Fredericksburg, Virginia. "A lot more are waiting for the testing to be complete, but everybody has their ears up. I'd say 90 percent of practitioners have at least heard of outcome measures, up from about 40 percent four years ago." Right Direction, Wrong Reason Industry experts say, however, that providing outcomes documentation just to get paid places the wrong emphasis on the reason for measuring outcomes. Correctly used, such data can help O&P practitioners make more informed decisions when fitting patients as well as when designing the next generation of componentry. It can also help to improve communication among all of the professionals caring for the patient, including physicians, therapists, and hospital staff, and raise the profile of O&P practitioners as integral members of the healthcare team. "Physicians don't understand the clinical importance of what we do because we've not done a good job of communicating that aspect," Williamson says. "With standardized, scientifically validated data on outcomes, we can build credibility and give the doctors a comfort level in dealing with us." The biggest challenge to establishing and validating O&P outcome measures, according to Brian Hafner, PhD , assistant professor in the Department of Rehabilitation Medicine at the University of Washington (UW), Seattle, is finding enough representative people to test a measure while it is being developed. "It's rare that outcome measures [in O&P] are developed and tested across hundreds of people," he says. "Testing in small samples may be one reason why we don't know how outcome measures are supposed to perform when used [in] everyday fittings in normal clinical settings." Another reason, Hovorka says, is that "science takes time." And when dealing with the Byzantine reimbursement processes, Williamson adds, "time is our enemy." Tools to Open Windows Many of the tools currently used to measure O&P outcomes have been adapted from those used to measure a patient's progress in neurological rehabilitation. The American Academy of Orthotists and Prosthetists (the Academy) suggests a dozen tests that can be used in the clinical O&P setting, from simple timed walking and functional mobility tests to the Amputee Mobility Predictor (AMP), a 20-item scale originally designed to help assign Centers for Medicare & Medicaid Services (CMS) K-levels, and the Socket Comfort Score (SCS), which was developed in an attempt to quantify socket discomfort and pain. 1 The variety of available tests alone underscores the challenge of finding that gold standard of measurement. O&P care is multidimensional, and outcomes can be measured with many different yardsticks. To get a complete picture of the efficacy of any orthosis or prosthesis, it's important to measure the user's response more than once and understand why no one measurement of the wearer can be "the best." "It's complicated," Hovorka says. "There are ten physiological organ systems-nervous, muscle, bone, skin, etc.-in the human body, and generally at least four and many times more of these systems are affected by traumatic limb loss; perhaps at least five or more of these systems are affected if the limb loss is the result of chronic or systemic disease. Because physiological organ systems have interrelated and overlapping functions, there's no one single outcome measurement of physiological function that targets the performance of a person wearing an orthosis or prosthesis. Instead, a number of outcome measurements are commonly implemented to characterize a person's performance." David Boone, PhD, MPH, CP, LP , chief technology officer at Orthocare Innovations, Oklahoma City, Oklahoma, recalls that during his training nearly 30 years ago, the instructor asked students to measure one outcome—how their devices made a difference in a patient's life. "To determine that, you have to look at it from the patient's point of view," he says. "And then you have to understand that each patient has many points of view." These various points of view may be captured by patient self-reporting questionnaires such as the Orthotic and Prosthetic User Survey (OPUS), developed more than a decade ago at the Northwestern University Prosthetics-Orthotics Center, Chicago, Illinois. Its series of 91 questions assesses functional status, health-related quality of life, satisfaction with the device, and satisfaction with services provided. "We've only made a positive difference if the patients feel they've been helped," Boone says. Subjective evaluation is important, but it is not the only window through which O&P outcomes can be viewed, adds Boone, who also serves as editor of the Journal of Prosthetics and Orthotics . It is also important to know objectively if the device gives the patient back as much function as possible. "This is a very important outcome measure, but sometimes we get caught up in the technology of building things and lose sight of the goal of returning function," he says. "You can convince yourself you are providing the best biomechanical device possible, but if the patient never uses it, it's not a good outcome." The sheer number of options available for each patient further complicates documenting how a particular combination of components affects biomechanical function and how the patient interfaces with the device. Do the patient's daily physical activities require top-of-the-line components-and their associated expense? Could the patient actually do more with a less-sophisticated device? This opens another window on outcome measures, one that reveals how complex determining the most appropriate device can be. What are the patient's actual abilities and motivations, and how do they change over time? "As practitioners, we need to know our patient and what a reasonable outcome is for this patient at this time," Boone explains. "Someone may have the physical potential to become a Paralympian, but if that's not what they want to do, it's still our job to create something that will allow them to achieve their goals." Function can be both easy and difficult to measure. An experienced practitioner can evaluate the ambulation of lower-limb patients by administering walking and timed-up-and-go tests in a long hallway. A complete assessment of how well patients can walk in different environments as well as their balance and confidence in their own mobility could require expensive visits to a gait lab—not usually a practical option, especially since neither patients nor practitioners are reimbursed for the time spent on measurement and documentation. There's an App for That The convergence of microprocessors, wireless communications, and ever-present smartphones is now bringing the gait lab to the patient. Sensors that monitor the patient's everyday activity can now be built into lower-limb prostheses to collect real data. For example, Orthocare's europa system, previously called the Smart Pyramid, installs on the pylon of a transfemoral or transtibial prosthesis and measures and stores dynamic data on gait, alignment, and socket torque in real time. When the data is downloaded to a computer, the prosthetist can use it to modify the socket as needed. The galileo connection technology allows a wireless connection from a smartphone, desktop computer, or laptop computer to the data collected and generates documentation on performance for payers. The europa system has been online since 2009 and has collected information on thousands of sockets, according to Boone. Drawing on this database gives clinicians access to a computer model that can recommend changes to components to optimize fit based on years of outcomes data. Researchers at Georgia Tech are also working on adapting small, inexpensive wearable sensors that can be attached to a person's existing prosthesis or orthosis to record and quantify movements of the person's limb segments. The data gathered can be downloaded to a server and accessed by both the practitioner and the patient via smartphone technology. While the clinician examines the data for biomechanical analysis, the patient could also examine portions of the data as a form of biofeedback, perhaps to improve his or her own gait pattern. Hovorka says the system, which is now in beta testing, could be available within the next one to two years. Hafner sees this type of Internet-based administration and testing as a way to tap into larger test populations to provide validation for outcome measures. "Such a strategy can enhance our ability to develop and test both new and existing outcome measures," he adds. "It can also help us to make better recommendations about which measures work best for which situations." No Time to Apply While there are a growing number of tools that are or will become available to O&P professionals that measure and record patient data, clinicians have not yet fully embraced using the information gathered to improve patient care and service—largely because they are still primarily focused on satisfying insurers' demands. For example, although the OPUS psychometric questionnaire has been completed by thousands of patients, there is little information about how these measures are being incorporated into clinical practice, according to a study titled "Clinical Administration of a Standardized Patient Satisfaction Measure," by Halsne et al. presented at last year's Academy Annual Meeting & Scientific Symposium. 2 Williamson says this is where outcome measures can have the greatest impact on O&P practices. He came to the O&P industry from the hospital side of healthcare, first working with the American Board for Certification in Orthotics, Prosthetics & Pedorthics (ABC), then purchasing Quality Outcomes from its founder three years ago. He saw the need for data collection and documentation similar to hospitals' quality outcomes efforts, not only to meet reimbursement needs but also to improve clinical care. Despite the barriers to applying outcomes data to O&P patient care, Williamson says that practitioners need to complete the feedback loop by integrating the information collected into daily practice. They must be willing to break down the silos between O&P and referral sources and develop more collaborative working relationships based on mutual trust and providing the best possible patient outcomes. "For example, I think every O&P office should collect a basic medical history from every patient, even though the physician has it in his records," he says. "It can help make sense of other, O&P-specific data we collect later." To illustrate his point, he uses an example of a patient who develops problems on his residual limb that aren't resolving. "If we see that the patient reported being a smoker on the medical history, that can help solve that mystery," he says. "It also helps the physician understand the skills and knowledge that O&P professionals bring to caring for the whole patient." But there is no reason to devote limited staff resources to interviewing patients when patients can complete a health history form online, Williamson says. In fact, Quality Outcomes is working to provide patient satisfaction and outcome measurement tools that practitioners can use in their clinical practice with little to no extra staff time required. The company recently announced technology partnerships with both OPIE Software, Gainesville, Florida, and Futura International, Clearwater, Florida, the practice management systems in use at more than half of independent O&P practices in the United States. The partnership will allow users to create complete, electronic patient medical records as well as document outcomes for reimbursement purposes. "Integrating with Futura and OPIE in this unique manner creates a dramatic shift in the discussion about measuring patient satisfaction in the O&P clinical care arena," Williamson says. "As the data from all of these patient events [are] tracked with our...tools, the O&P profession will have a credible database of information from which to create professionwide benchmarks and standards for patient satisfaction." The integration allows O&P practices to initiate patient satisfaction surveys automatically as a normal part of daily activity. "Taken individually, or even across a typical O&P practice, patient satisfaction surveys provide anecdotal evidence," Williamson says. "But when administered consistently and collectively across a wide range of care settings and nationwide, they provide a unique body of information that stimulates scientific investigation." Are There Funds for That? Another aspect of scientific investigation that could use some stimulation is funding. Hafner, who is also the research committee chair for the Orthotic and Prosthetic Education and Research Foundation (OPERF), says there has been little funding available for the development of O&P-specific standardized outcome measures. Funding sources have historically included institutional and private foundations and federal agencies, but the largest initiatives have been designed for persons with a variety of health conditions. OPERF awards grants between $1,000 and $25,000 for pilot projects in O&P-related research although the foundation does not have a specific focus on outcome measures. The Eunice Kennedy Shriver National Institute of Child and Human Development (NICHD) is supporting a five-year project Hafner is leading at UW to create a standardized measurement tool specifically for use in O&P, the Prosthetic Limb Users Survey of Mobility (PLUS-M). Working with individual practices, hospitals, and university centers across the country, his team has recruited more than 50 clinical sites to help identify more than 1,100 prosthesis users to complete a survey that will aid in the development of the PLUS-M. "Our long-term goal is to create a suite of instruments that measure important aspects of prosthetic and orthotic users' health and do so in a way that is not only valid, reliable, and meaningful, but also flexible enough to be integrated into routine O&P clinical operations," he explains. Hovorka says he is skeptical that clinicians, who are already pressed for time and working with limited resources, would be eager to implement a suite of outcome measures in their practices any time soon. They may be more likely to select a small number of measures that are simple, inexpensive, and quick to administer. Some of these outcome measurements and clinical tools have already been developed, but until the tests and instruments are scientifically validated, the information remains anecdotal, he stresses. "The profession is looking to the universities and elsewhere for the most effective performance measures that can be integrated into their practices," Hovorka says. "It would be ideal if we could be sure that whatever tests or measurements are implemented to quantify and characterize the patient's clinical outcomes...are evaluated with reasonably rigorous scientific methodology. This would then confirm reliability and validity before introduction into the clinic." Incorporating measurement as part of clinical practice has been a struggle for some practitioners because many may not have been required to implement science-based methods in their clinical training, Hovorka says. "I believe that the addition of clinically relevant scientific approaches are as important to the future of O&P professional practice as hand skills or the desire to help patients." To help fill this gap in O&P clinician training, students in Georgia Tech's MSPO program are paired with clinical- and tenure-track science faculty members to examine clinically relevant research questions as part of their education. Not only does this bring clinician students to the science arena but it also brings meaningful research to the clinical arena, Hovorka says. He sees the master's-level education of O&P practitioners as an integral part of moving the profession toward more rigorous measures of the efficacy of O&P devices as well as the development of more specific device use dosing (i.e., frequency, intensity, and duration) that expands the continuum of care the prosthetist or orthotist provides. "Our graduates are capable of providing a clinical practice with value-added care and evidence-based documentation, an approach that referral sources such as physicians understand and prefer," Hovorka says. "They understand what is needed to be compliant with insurance requirements to expedite reimbursement, so the O&P practice can take the lead and reduce the workload for the referral source to improve the likelihood that the patient is receiving appropriate care with effective outcomes." However you approach the issue, evidence-based practice is the future of O&P. It will take time for the slow-moving train of science to reach the station of universal standardization, but more and more practitioners are getting on board with the idea. Kate Hawthorne is a freelance writer living and working in Fort Collins, Colorado. She can be reached at kate@wordsforhire.net References Stevens, P., N. Fross, and S. Kapp. 2009. Clinically relevant outcome measures in orthotics and prosthetics . The Academy TODAY 5(1):A4-11. Halsne, E. G., and A. D. Peaco. 2012. Clinical administration of a standardized patient satisfaction measure. Paper presented at the Annual Meeting & Scientific Symposium of the American Academy of Orthotists and Prosthetists.

Outcome Measures: Are We There Yet?

Medicare Audits, Overpayments, and Appeals

Walk This Way: Gait Assessment Technology Steps Forward

PFA Call for Abstracts Deadline Extended

O&P JOBS

Certified Prosthetist / Prosthetist-Orthotist (CP/CPO) Gates Prosthetics + Mobility Clinic — Mountain Home, AR

Clinic Manager – Certified Prosthetist Orthotist (CPO) – Rogers, AR

CPO

Get unlimited access!

EDGE DIRECT

Welcome Back!

Retrieve your password

Not enough quota to unlock this post

Outcome Measures: Are We There Yet?

How do you measure the quality of the prosthetic or orthotic devices you provide to your patients?

Support authors and subscribe to content.

Subscribe

Related posts:

Medicare Audits, Overpayments, and Appeals

Walk This Way: Gait Assessment Technology Steps Forward

PFA Call for Abstracts Deadline Extended

O&P JOBS

Certified Prosthetist / Prosthetist-Orthotist (CP/CPO) Gates Prosthetics + Mobility Clinic — Mountain Home, AR

Clinic Manager – Certified Prosthetist Orthotist (CPO) – Rogers, AR

CPO

Get unlimited access!

EDGE DIRECT

Welcome Back!

Retrieve your password

Not enough quota to unlock this post