Game On: The Developing Realm of Myoelectric Training Through Game Play
October 2020 Issue
Unlike other aspects of rehabilitation, where patients are trying to relearn previous motor patterns and skillsets, users of myoelectric prostheses are called upon to develop an entirely novel skillset in which unfamiliar muscle contractions are used to regulate non-intuitive movements of a prosthesis. For traditional dual-site myoelectric control, these muscle activation patterns must be accurate and distinct for effective control of the prosthesis.
Basic gaming platforms have existed for many years, including the Ottobock car game that provides basic biofeedback as users visualize their raw EMG signal strength while navigating cars through a rolling series of staggered gaps in sequential walls. However, such systems can be difficult for the novice user to succeed in, are limited in their capacity to engage the end user, and have historically been tethered to the use of extremely expensive equipment that practically limits training to brief periods of professional interaction with treating clinicians.
In the past decade, creative engineers have piloted attempts to use myoelectric inputs to participate in existing popular video games such as Guitar Hero1 and Pong.2 However, these reports have been confined to feasibility reports among able-bodied test subjects and, while entertaining, may not be directly transferable to prosthetic control.
Until recently, designers of novel, game-based EMG training tools have failed to incorporate feedback from end-users into their games.3 A series of recent publications, however, have highlighted current efforts to develop novel training resources that are more accessible to the emerging capacities associated with the early rehabilitation phase of myoelectric prosthetic rehabilitation. This article provides an overview of several of these developing resources.
The Falling of Momo
In creating "The Falling of Momo," the game developers interviewed clinical experts to understand the limitations associated with currently available myoelectric training resources.3 The replies they received suggested that current training approaches were perceived as boring and unengaging, that they were generally more difficult than using a real prosthesis, and that they are not accessible outside of the clinic.3 The developers sought to address these identified shortcomings in several ways. In an effort to make their system more viable and affordable, they elected to work with consumer-grade electrodes, rather than medical-grade, using the previously commercially available Myoband from Thalmic Labs.3 At the time of development, the Myoband was an off-the-shelf EMG input device that cost less than $200. While less elegant and sensitive than medical-grade EMG input options, the fidelity of the signals from the Myoband was sufficiently accurate to facilitate user training.3
The game is described by its creators as "a casual, survival-style game" where points are earned as the user's character, Momo, descends through gaps in a series of rising platforms to avoid being squished by spikes across the top of the screen. The left and right movements of the character are mapped to the opposing flexion and extension signals generated by the end user.3 By sliding Momo across the platforms, successful navigation of the gaps is somewhat easier to attain than the navigation of the sequential gates in the Ottobock car game.3
In addition to facilitating more early success in game play, the developers attempted to enhance user engagement by providing them the ability to collect coins along the ascending platforms that could be used later to purchase characters, themes, and unlockables.3 In addition, variable speed was enabled by allowing movements to be proportional, with greater EMG intensity yielding faster speeds along the platform. In addition, the user could jump to avoid obstacles on the platform by performing a targeted co-contraction.3 Challenge and variety were enhanced with the addition of icy (low-friction) and sticky (high-friction) platforms to the game so that players would need to use the full range of proportional control to navigate increasingly difficult levels of game play.3
The developers of "The Falling of Momo" describe a process of iterative improvement using a set of collected observations, feedback, and suggestions from playtest interview sessions. These subsequent improvements ultimately included integrating new features such as disappearing coins to encourage quick collection, bonus levels for additional game depth, and overexertion warnings to indicate when muscle contractions were stronger than necessary.3 The developers also added a second screen with raw EMG visualization to allow clinicians to better access muscle signals during live game play.3
The end result was described as an engaging training game suitable to the emerging abilities of individuals learning the novel control signals associated with myoelectric control strategies.
Winslow et al. from Design Interactive described a second approach to game-based training of myoelectric control.4 Their system is named the Auto-Diagnostic Adaptive Precision Trainer for Myoelectric Prosthesis Users or ADAPT-MP. As with "The Falling of Momo," the control inputs for the system came from the Myoband. These games were described as app-based, allowing for play across the range of operating systems (Android, iOS, and Windows). A suite of four dinosaur-themed games was described, with each training a different aspect of myoelectric control.4
In "Volcanic Crush," players learn to control flexion, extension, and co-contraction myosignals by crushing boulders as they are erupted from a volcano.4 Two to five seconds after their eruption, the boulders adopt a targeted color that is assigned to the required EMG signal. When the correct muscle is activated, the boulder explodes. Incorrect muscle activation yields an error, with five such errors ending the game. The more boulders the user explodes, the higher the game score.4
In "Dino Sprint," the contractions trained in "Volcanic Crush" are used in various sequences as the dinosaur avatar runs along an infinite pathway trying to avoid obstacles. Different sequences are required to avoid different obstacles. Incorrect sequencing or sequencing that takes too long results in an error with five errors ending game play. As correct sequences accrue, the game score goes up.4
"Dino Feast" was designed to train both EMG isolation and proportional control. In this game, the strength of a given contraction is illustrated on the screen along with targeted threshold ranges. Points are earned when the user sustains the appropriate target contraction within the target intensity range. If muscle signals exceed or drop below the threshold ranges, the contraction is counted as an error. As with the other games, five errors conclude the game.4
"Dino Claw," the final game in the suite, adds the input from the onboard accelerometer of the Myoband to track the gross motor movement of the limb. This movement, coupled with the EMG signals, allows the users to pick up virtual objects, move them to a new location, and drop them. Constrained by a 40-second time frame, users move as many objects as possible within the time limit, earning points with each successfully moved object.4
The tested efficacy of the game in eliciting the desired training of control EMG signals was confined to a group of able-bodied participants as they trained the strength and isolation of the wrist flexors and extensors of the non-dominant extremity. Following a week of daily practice, the scores on all games increased, suggesting improved EMG control.4
The Austrian Approach
Still another approach to game-based training was recently described by researchers in Vienna, Austria. Their system comprised three separate games, each of which trained EMG control in unique ways.5 In the dexterity game "Pospos," patients mimic the myoelectric control of four movement patterns through dual site control by moving their avatars through a maze. In addition to the rather native movements of left and right using flexor and extensor signals, the additional movements of up and down are elicited by intentional co-contractions that transfer the directionality of the input.5
The elements of both quick and sustained contractions are trained in a racing game, "Super TuxKart." As might be assumed, in this game players race against computer-generated adversaries, moving left and right as needed.5 In a final rhythm game, "Step Mania 5," players are invited to match arrows on the screen pointing left, right, or both ways, thus training quick, sustained, and simultaneous contractions.5
The efficacy of these systems was evaluated by seven subjects with either transradial or transhumeral amputations, six of whom lacked prior experience in myoelectric prosthetic control.5 Participants sat before computer screens where they were fitted with two Ottobock surface electrodes, either held in place by a wrist band or positioned within a prosthetic socket. Following an initial EMG assessment, they played each of the three games for ten minutes before completing a post-gaming assessment. A third EMG assessment was performed two days later. These assessments were composed of tests of precision, separation, and endurance, all of which improved following the gaming experience.5
Additional questionnaires confirmed that, compared to training EMG signal control using the standard Ottobock Myoboy, in which signal intensity is visualized in the form of two moving LED bars, patients found the game-based training to be more enjoyable, expressed more competence in their performance, and they put forth increased effort.5
Contributions From Singapore
A final manuscript describes the approach developed by researchers in Singapore.6 Recognizing the need for matching the difficulty of the game with the limited strength and control of novice EMG users, as well as the need to improve upon the monotonous training regimens in standard practice, the researchers selected two open-source games for their training exercises, "Flappy Bird," and "Space Invaders."6 These games were selected, in part, in recognition of the constraints of a number of EMG inputs generated in most prosthetic control strategies. Both of the games required two-directional movements.
In "Flappy Bird," the bird character is moved up by flexion EMG and down by extension EMG to avoid incoming obstacles.6 A third input of co-contraction is used to navigate the initial screen menu and to speed forward. For "Space Invaders," the spaceship is moved left by flexion and right by extension to avoid getting hit.6 Co-contractions are used to shoot lasers at aliens.6
In this system, EMG signals are harvested from a custom-made arm sleeve with built-in pattern recognition capabilities. This appears to be an odd choice of control as the eight channels of the arm sleeve are well beyond those necessary to capture the targeted input signals of flexion, extension, and co-contraction. The complexity of the input system was evident in the authors' reports of misclassification of certain contractions during game play.6
Unfortunately, while the system has been developed for use in individuals with transradial amputations, feedback on these games is currently limited to that collected from 16 able-bodied subjects. However, these individuals described the games as both engaging and fun to play.
The standard approaches to myoelectric control training face several limitations. Training tends to require access to expensive equipment, thus limiting patient access to the training resources. In addition, the training can be perceived as both monotonous and boring for the end user. In response, a number of research teams have sought to develop engaging video game-based training resources to allow the end user to train to this novel skill set in a more absorbing fashion. From raising platforms to exploding volcanoes, navigating mazes to auto racing, flapping birds to spaceships and belligerent aliens, a number of creative training solutions appear to be in the works to better engage new learners in the novel skill sets of myoelectric prosthetic control.
Phil Stevens, MEd, CPO, FAAOP, is a director with Hanger Clinic's Department of Clinical and Scientific Affairs. He can be contacted at email@example.com.
1. Armiger, R. S., and R. J. Vogelstein. 2008. Air-Guitar Hero: A real-time video game interface for training and evaluation of dexterous upper-extremity neuroprosthetic control algorithms. Biomedical. Circuits and Systems Conference. 121–124.
2. la Rosa, R., A. Alonso, S. de la Rosa, and D. Abasolo. 2008. Myo-Pong: A neuromuscular game for the UVa-Neuromuscular training system platform. 2008 Virtual Rehabilitation, 15: 61.
3. Tabor A., S. Bateman, E. Scheme, D. R. Flatla, K. Gerling. 2017. Designing game-based myoelectric prosthesis training. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 1352-63.
4. Winslow B. D., M. Ruble M, Z. Huber. 2018. Mobile, game-based training for myoelectric prosthesis control. Frontiers in Bioengineering and Biotechnology,11(6):94.
5. Prahm C., I. Vujaklija, F. Kayali, et al. 2017. Increasing motivation, effort and performance through game-based rehabilitation for upper limb myoelectric prosthesis control. In 2017 International Conference on Virtual Rehabilitation (ICVR), IEEE Xplore, 19:1-6.
6. Radhakrishnan M., A. Smailagic, B. French, et al. 2019. Design and Assessment of myoelectric games for prosthesis training of upper limb amputees. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), 11:151-7.