Providing Evidence-based Care in Low-, Middle-income Countries

Several years ago I attended a meeting where a young practitioner reported on a medical service trip (MST) involving prosthetic care in a low-income country (LIC) that lacked such services. Many features of the industrialized world’s infrastructure, such as transportation, housing, plumbing, etc., were not readily available in the area served by that venture. As the presenter reported on the details of the trip, it became apparent that little thought had been given to the long-term sustainability of the services provided. For example, at the most basic level, the components provided were not suitable for the patients’ living conditions.

A discussion with the speaker following the presentation confirmed that she had no prior experience with that type of project and had not informed herself by reading literature or speaking with other prosthetists who had provided services in a similar situation. The trip had been a one-time undertaking, providing exactly the same types of prostheses that would have been provided to patients in the United States, and there were no plans to provide follow-up care.

P&O Services in Low-, Middle-income Countries (LMICs)

When implementing evidence-based medicine, a practitioner combines clinical experience, the values and needs of the patient, and the best available research evidence to make the most appropriate care decisions for the patient. Practitioners participating in charitable activities have a professional responsibility to integrate those same considerations into their services. However, research evidence related to humanitarian efforts is limited.

Harkins et al. summarize their 2013 review of P&O services in LICs by saying that “the lack of and quality of available research made efficacy of methods used to provide services in low-income countries difficult to determine.”¹ They also report that “a substantial body of research is needed to prove the efficacy of the methods of service provision; maximise the effective methods; and to develop evidence-based, sustainable services…. [M]ore specific data are necessary to create effective prosthetic and orthotic services.”¹ Ikeda et al.’s two scoping reviews on the provision of P&O services in resource-limited environments (RLEs) published in 2014 provide thorough descriptions of considerations for success and research and outcomes.^2,3 While these articles provide a helpful evidence base of available literature and outcomes reported by P&O service providers working in RLEs, they conclude that their research highlights “the need for standard, valid, and reliable methods of data collection and reporting.”^2,3

Lessons From Surgical Literature

While surgical and P&O services differ in many obvious ways, reviewing research related to surgical MSTs can provide helpful perspective on P&O efforts in similar contexts. Several systematic reviews of articles related to surgical MSTs have been completed in recent years and can be used to promote professionalism in P&O MSTs.

Models for providing medical services to LMICs involve either self-contained operations (e.g. hospital ships), established local facilities that are staffed and supplied in part by high-income countries (HICs), or ad hoc trips in which personnel and supplies from HICs are brought in to provide services on a short-term basis. Short-term missions can span from one week to two months. Shrime et al. reviewed articles related to MSTs to determine the effectiveness of each of these platforms for delivering medical assistance in LMICs.⁴ They report that “self-contained temporary platforms and specialized surgical centers appear to provide more effective and cost-effective care than short-term surgical mission trips, except when no other delivery platform exists,” concluding that “the short-term temporary surgical mission’s role should be limited to areas and conditions for which no other surgical delivery platform is available.”⁴

This recommendation is made regarding MSTs that focus on medical conditions with obvious potential for surgical resolution. Ongoing, local, and sustainable services are even more important when providing P&O care, since the disability is ongoing and the intervention requires adjustment and repairs over time. In these cases, short-term, ad hoc P&O MSTs may be even less effective than their surgical counterparts.

In 2012, Martiniuk et al. published a review of articles to determine advantages and disadvantages of medical missions.⁵ They report that only 5 percent of articles they reviewed included theoretical or conceptual analysis, and that “there was little quantitative data to analyse.”⁵ The authors advise organizations and individuals participating in MSTs to adopt “a more precise approach to mission planning, implementation, and reporting….”⁵ The 2015 review by Caldron et al. focused on empirical studies on the social, economic, and diplomatic aspects of short-term medical missions (STMMs), and the authors report that “scant analyses exist in the current literature wherein normative concepts of volunteerism, altruism, and philanthropy is applied specifically to the praxis of STMMs.”⁶ This research demonstrates that the difficulty of implementing evidence-based practice is not unique to P&O.

A Systematic Review of Empirical Evidence

Multiple researchers report that the available literature on MSTs has been primarily descriptive and presented little objective evidence of cost effectiveness or patient outcomes. Most notably, in 2015, Sykes published a review of articles that included empirical results of intentional data collection in areas including treatment interventions, costs, cost effectiveness, quality assessment, or surveys of perspectives of involved parties.⁷ He found 67 such studies published from 1993-2013, which accounted for only 6 percent of the articles published on the topic in that timeframe.⁷ The majority of the 67 articles (80 percent) reported on surgical MSTs.⁷

Sykes classified the articles based on their study designs, and found that almost half “employed a retrospective study design or simple descriptive statistics to report their findings,” with nearly all representing low-level quality evidence.⁷ Thirteen percent were quality improvement or quality assurance projects, and 10 percent involved surveys of individuals who volunteered on MSTs and organizations that operated them.⁷

Output Versus Outcomes

While there is an increased emphasis on collecting outcomes data on medical services provided in HICs, this is not a priority during most MSTs. Sykes concludes that most MSTs report “outputs rather than outcomes,” a criterion that “falls short of the measures used to identify high-quality evidence-based medicine.”⁷

Nearly three-quarters (74 percent) of the articles Sykes reviewed did not report on outcomes or limited those reports to outcomes within seven days of the service provided.⁷ Since these were reports primarily of surgical MSTs, the focus was on the immediate outcomes of the surgical procedure or other aspects of the mission, and not on long-term effects of the treatment. Sykes reports that most of the publications lack significant data collection and information about changes in quality of life resulting from medical interventions.⁷

Intention Versus Impact

The desire to help others is a common reason for choosing a medical profession, and volunteering for an MST may spring from genuine altruism. MSTs are complex and multifaceted undertakings, with many opportunities for good intentions to result in poor outcomes. Eddy Fuentes, CPO, provides valuable insight into one such unintended negative consequence in his article “The Effect of Good Missions Overseas.”⁸ Concerns about the negative effects of well-intentioned but poorly implemented charitable efforts resulted in the development of the International Society for Prosthetics and Orthotics (ISPO) Code of Conduct for Humanitarian Organizations, which can be accessed at www.usispo.org/code.asp.

Martiniuk et al. report that very few of the articles they reviewed “discussed the ethics, policies, standards, or evaluations of short-term medical missions.”⁵ These are important topics to discuss prior to, during, and following an MST and should be a standard part of volunteer recruitment and training. Caldron et al. point out that participants in MSTs may assume that because these missions are charitable and altruistic they are valid in their own right, and therefore the collection of objective outcomes data is not mandated.⁶ Sykes asserts, however, that “delivering care without understanding the impact or the outcome of that care presents ethical challenges…. [T]here should be no assumed ethical immunity solely based on the altruistic nature of these efforts.”⁷ He adds that to measure quality, organizations should incorporate outcomes evaluations in their activities.

Which Outcomes?

Collecting outcomes data during MSTs may be perceived as unnecessary by U.S. practitioners, since the relatively recent (and by no means universal) practice of collecting this data can be closely tied to reimbursement pressure. The freedom from the financial and administrative pressures of everyday practice is one of the appeals of MSTs, so it is understandable that the collection of outcomes data is not a high priority. Additionally, surgical procedures represent a much greater risk than most P&O interventions, and the often immediate and obvious beneficial impact of P&O services may appear to obviate the need for outcomes measurement in LMICs. However, measurements of the treatment’s direct effect on the patient’s function is only one aspect of outcomes measurement. There are many ways to measure the impact of a particular medical service, and P&O MST organizers can and should use a variety of different measures to demonstrate their effectiveness. Martiniuk et al. suggest that MST reports should include the “number of people treated, follow-up needed and how this will occur, cost per beneficiary, training of local counterparts conducted, and challenges faced.”⁵ Sykes recommends that organizations collect patient demographics including socioeconomic status, the availability of regular care in the patients’ communities, and the MST’s cost of delivering care.⁷ He reports that outcomes in the literature he reviewed involved “the impact on the volunteer, on the patient relative to cost, and on the local health care providers.”⁷

The literature reviews by Ikeda et al., mentioned earlier, reported that outcomes collected as part of P&O provision in RLEs included “durability, cost, satisfaction, use/nonuse of device, amount of utilization, walking speed, discomfort, pain, fit, misalignment, capacity for service provision, number of devices produced or delivered, and number of graduates from training programs…. [M]easurements of inclusion, participation, or [quality of life] were rare….”^2,3 Organizers of P&O MSTs should be familiar with ISPO’s protocol and assessment, as well as benchmarks developed by researchers and clinicians providing services in RLEs. Evaluating and reporting on all of these factors using valid methods will improve the level of service provided on MSTs.

Conclusion

Providing care in any context without adequate training and preparation is irresponsible, and the majority of P&O practitioners participating in MSTs are doing so more responsibly than the efforts described in the scenario at the beginning of this article. Those considering participating in charitable efforts have a professional and ethical responsibility to develop a coherent philosophy and inform themselves of the unique challenges of providing care in LMICs, including being familiar with the P&O literature on this topic and the output of national and international organizations that have been providing this type of service for many years.

P&O-focused articles and reviews such as those by Ikeda et al. and Harkin et al. should inform the development of outcomes programs by MST organizers. The best available research evidence may be limited, but as MST organizers adopt and expand an evidence-based approach to their activities, the level of credibility and professionalism will increase.

John Brinkmann, MA, CPO/L, FAAOP, is an assistant professor at Northwestern University Prosthetics-Orthotics Center. He has more than 20 years of experience treating a wide variety of patients.

References

1. Harkins, C. S., A. McGarry, and A. Buis. 2013. Provision of prosthetic and orthotic services in low-income countries: A review of the literature. Prosthetics and Orthotics International 37 (5):353-61.
2. Ikeda, A. J., A. M. Grabowski, A. Lindsley, E. Sadeghi-Demneh, and K. D. Reisinger. 2014. A scoping literature review of the provision of orthoses and prostheses in resource-limited environments 2000-2010. Part one: Considerations for success. Prosthetics and Orthotics International 38 (4):269-86.
3. Ikeda, A. J., A. M. Grabowski, A. Lindsley, E. Sadeghi-Demneh, and K. D. Reisinger. 2014. A scoping literature review of the provision of orthoses and prostheses in resource-limited environments 2000-2010. Part two: Research and outcomes. Prosthetics and Orthotics International 38 (5):343-62.
4. Shrime, M. G., A. Sleemi, and T. D. Ravilla. 2015. Charitable platforms in global surgery: A systematic review of their effectiveness, cost-effectiveness, sustainability, and role training. World Journal of Surgery 39 (1):10-20.
5. Martiniuk, A. L., M. Manouchehrian, J. A. Negin, and A. B. Zwi. 2012. Brain gains: A literature review of medical missions to low and middle-income countries. BMC Health Services Research 12 (1):134.
6. Caldron, P. H., A. Impens, M. Pavlova, and W. Groot. 2015. A systematic review of social, economic and diplomatic aspects of short-term medical missions. BMC Health Services Research 15:380.
7. Sykes, K. J. 2014. Short-term medical service trips: A systematic review of the evidence. American Journal of Public Health 104 (7):e38-48.
8. Fuentes, E. The effect of food missions overseas. 2009. E-Highlights, International Society for Prosthetics and Orthotics. www.usispo.org/news-missions_effect.asp.
9. Welling, D. R., J. M. Ryan, D. G. Burris, and N. M. Rich. 2010. Seven sins of humanitarian medicine. World Journal of Surgery 34 (3):466-70.

Recommended Reading

• Day, H. J. B. 1996. A review of the consensus conference on appropriate prosthetic technology in developing countries. Prosthetics and Orthotics International 20 (1):15-23.
• Marino, M., S. Pattni, M. Greenberg, A. Miller, E. Hocker, S. Ritter, and K. Mehta. 2015. Access to prosthetic devices in developing countries: Pathways and challenges, in Global Humanitarian Technology Conference (GHTC), 2015 IEEE, 45-51. Seattle.
• Staats, T. B. 1996. The rehabilitation of the amputee in the developing world: A review of the literature. Prosthetics and Orthotics International 20 (1):45-50.
• Stevens, P. 2015. Prosthetics in resource-limited countries. The O&P EDGE 14 (6):40-50.