Pattern Recognition

THE CONCEPT OF PATTERN RECOGNITION AS A CONTROL SYSTEM FOR UPPER-LIMB PROSTHESES HAS BEEN AROUND SINCE THE LATE 1960S AND EARLY 1970S.¹ HOWEVER, WITH THE RECENT COMMERCIAL EMERGENCE OF COAPT’S COMPLETE CONTROL™ SYSTEM, IT HAS MOVED FROM THE RESEARCH LAB INTO CLINICAL PRACTICE.

This article introduces pattern recognition as a control scheme for externally powered prostheses and describes its basic processing requirements. It also addresses the additional implications of pattern recognition on redefining the number of necessary signal inputs at more distal amputation levels, as well as its potential to further exploit the benefits of targeted muscle reinnervation (TMR) techniques at more proximal amputation levels. Its role in facilitating simultaneous myoelectric control will also be discussed.

Defining the Problem

One of the biggest limitations of externally powered upper-limb prostheses is the insufficiency of available myoelectric control signals in the residual limb. The problem is encountered at the wrist disarticulation level and becomes progressively more limiting at increasingly proximal amputation levels. When limb loss occurs at the wrist, the lost physiologic degrees of freedom include the complex variations of hand prehension, wrist flexion and extension, and both ulnar and radial deviations. Additional physiological movements of pronation and supination, elbow flexion and extension, and glenohumeral rotation, flexion, and extension are progressively lost at more proximal amputation levels.

With traditional myoelectric control schemes, the number of input signals, or muscles that could generate a meaningful contraction to govern movement of the prosthesis, are limited to larger muscle groups that can fire independently from their antagonist partners. As a result, even at more distal levels of upper-limb amputation, there are more movements to control than there are muscle signals to control them, a problem that is compounded at more proximal amputation levels. In addition, the controlling muscle groups may be physiologically inappropriate. Thus, wrist flexors and extensors may be called upon to regulate the opening and closing of a terminal device along with pronation and supination. Similarly, biceps, triceps, and thoracic muscle groups may be called upon to regulate these same motions in addition to elbow flexion and extension.

When prosthetic joint movements are controlled by substituting muscle bellies that may have to alternate their control over multiple joint segments, the control is far from intuitive. In addition, myoelectric control of the less foundational upper-limb movements of wrist flexion and extension, along with ulnar and radial deviations and humeral rotation, have largely been neglected in the absence of a means to provide their meaningful control.

How It Works

Pattern recognition is one method under development that attempts to address these limitations of an inadequate number of muscles attempting to control prosthetic movement at counterintuitive joint segments. Unlike traditional myoelectric control strategies, pattern recognition does not require completely independent muscle signals free from co-contraction, nor does it require the user to control multiple joints with the same signal sites by cycling between joint modes. Rather than requiring the user to learn counterintuitive, physiologically artificial control strategies, pattern recognition is based on the idea that the learning is done by the classification software. As a result, the individual can generate the muscle contractions that he or she associates with a given movement. The software is programmed to identify this muscle pattern and pair it with the targeted movement. It will then recognize the pattern the next time it is generated and create the intended prosthetic movement.

Figure 1. Processing stages of pattern recognition as defined by Scheme and Englehart.¹

In a recent review article, Scheme and Englehart provide an overview of how pattern recognition works once the processor has been trained to the individual user (Figure 1).¹ The process begins with the generation of an electromyography (EMG) signal. However, as articulated earlier, the signal need not be as isolated and artificial as those required by traditional myoelectric control. The user simply generates the contraction patterns that he or she associates with a given movement.

As with traditional myoelectric technologies, the original EMG signal is subject to an initial preprocessing to remove the more obvious disruption that may occur as a result of environmental interference such as power lines and overhead lighting. From here the process becomes more complex.

The next step in processing the EMG signal requires that it be examined in a temporal window on the order of 150-300 milliseconds. As might be anticipated, longer windows provide a more complete view of the intended signal generated by the user and, as such, tend to be more accurate. However, the longer the window, the longer the inherent delay before the signal pattern is recognized and the processor can act upon it. As these delays become longer, they are perceived by the user and negatively affect the responsiveness of the prosthesis. These delays can be reduced by shortening the temporal window, but doing so provides the processor with shorter glimpses of the intended pattern and can compromise the accuracy of the recognition. Scheme and Englehart cite an optimum window length between 150 and 250 milliseconds.¹

Once a signal has been windowed, the feature extraction stage occurs in which the processor identifies defined patterns within the EMG signal while discarding irrelevant EMG noise. The various feature extraction schemes that have been developed can become complex and go well beyond the purposes of this article. However, one reliable approach that will be familiar to experienced upperlimb prosthetists are the metrics of time-domain considerations first proposed by Hudgins et al. in the early 1990s. These consist of the mean absolute values of the EMG signals (intensity) and the associated slopes (speed) along with the frequency of the signal’s zero crossings and the number of slope sign changes.² These feature sets can be further refined with additional metrics that may better define the pattern, but doing so requires additional processing capacity that may not be necessary.

If feature extraction is overwhelming, the next stage of signal classification is even more so, dredging up such terms as linear discriminant analysis, Gaussian mixture models, and hidden Markov models. For the purposes of this article, it is sufficient to say that once the necessary features of the EMG signals have been extracted by the processor, they must then be classified to affect movement of the prosthesis. While a number of complex classification strategies have been attempted, data from Hargrove et al. suggest that classification accuracy is more affected by the feature extraction strategy than by the classification strategy.³ Once classified, the processor affects the actual movement behavior of the prosthesis.

Additional Considerations: Input Signals

As stated earlier, one objective of pattern recognition is to move away from the paradigm of forcing the user to learn to generate artificial muscle contractions by having processors that are sophisticated enough to recognize the differences in the muscle patterns produced by a user when, for example, attempting to open and close a hand or pronate and supinate at the wrist. It stands to reason that these more sophisticated processors benefit from more elaborate input signals beyond the dualsite control schemes frequently seen in traditional myoelectric applications. Scheme and Englehart cite a number of sources who have demonstrated that additional inputs improve the performance of pattern recognition.

Among these, the work of Hargrove et al., cited earlier, provides insight into both the ideal number of input signals needed and the need (or lack thereof) to optimize the location of these inputs.³ Their study included a number of able-bodied individuals who were asked to perform ten different wrist and hand motions with differing numbers of electrode inputs spaced circumferentially around the forearm. In addition to varying the number of inputs, they varied the location from naïve placement (simply placing the electrodes in a symmetric fashion around the arm) and optimal site selection, in which location was refined by signal testing.

The authors report 99 percent accuracy with only four input signals when using optimal electrode location. However, even with naïve site selection, an accuracy rate of 95 percent was observed with four input signals. This accuracy progressively improved to 97 percent with the addition of more input signals. The data collectively suggest that at the transradial level, optimizing site location may not be necessary if there are enough signal inputs. Further, four input electrodes appear to be sufficient to accurately characterize the user’s intended muscle patterns.³

Additional Considerations: TMR

To be effective, pattern recognition requires adequate signal input. While native, physiologic signals can be generated at more distal amputation levels, this is not the case at the transhumeral and shoulder disarticulation levels where proximal muscle groups are traditionally used to control the functions of the prosthetic wrist and terminal device. This paradigm was altered with the introduction of TMR.

In this increasingly familiar procedure, the major nerves of the brachial plexus are isolated and transferred to new muscle sites on either the pectoral wall in the case of a shoulder disarticulation, or to distinct muscle bellies on the upper arm in the case of a transhumeral amputation. The resultant physiology not only provides a more intuitive, natural control of distal prosthetic componentry, it also provides additional signal inputs to facilitate the control of additional prosthetic joint segments.

While subjects who have undergone TMR have experienced enhanced prosthetic control with conventional myoelectric control strategies, the nuanced signals created in the newly innervated muscle groups are ideal signal sources for pattern recognition. Thus, while an individual with a shoulder-level amputation can produce four distinct control signals using conventional myoelectric control, Zhou et al. report on a small series of subjects with shoulder disarticulations and transhumeral amputations who were able to achieve highly accurate classifications of 16 intended arm, hand, and finger/thumb movements with pattern recognition.⁴ Thus the enhanced myoelectric control facilitated by TMR may be further exploited with the use of pattern recognition.

Sequential versus Simultaneous Control

Able-bodied upper-limb function is characterized by simultaneous, coordinated movements at multiple joint segments. To the extent that simultaneous control has been attained with conventional control schemes, it has been accomplished with the combination of myoelectrodes and another input source. This is commonly encountered at proximal amputation levels where myoelectrodes may govern wrist and hand function while elbow position is regulated by a linear transducer. Excluding patients who have undergone TMR, the lack of multiple electrode control sites has historically prevented simultaneous myoelectric control of upper-limb prostheses. Instead, actions at the wrist and hand are generally produced sequentially, with the user switching back and forth from wrist to hand mode to obtain the desired movements.

While pattern recognition has historically been confined to sequential control strategies, a 2014 publication describes a transition to simultaneous pattern recognition controls.⁵ To evaluate the efficacy of this approach, a small cohort of individuals who had undergone TMR was recruited. TMR patients were needed for the study as these are the only individuals capable of performing simultaneous control of their prostheses using conventional myoelectric strategies. The individuals were asked to perform a series of movements in a virtual reality setting that included both discrete (controlling a single motion) and combined (controlling two movement sets) tasks. They did so using simultaneous conventional myoelectric control, sequential pattern recognition control, and simultaneous pattern recognition control.

The authors observed that for discrete tasks, the subjects were able to perform sequential pattern recognition control the best, with the lowest average completion times and rates. For combined tasks, the lowest average completion times and rates were observed when the subjects performed simultaneous pattern recognition controls.⁵

Acknowledging that the ability to perform simultaneous control does not require its ultimate use, the authors also monitored the extent to which users elected to perform simultaneous control during combined tasks with both conventional and pattern recognition control systems. They observed that subjects chose to use simultaneous control 64 percent of the time with conventional control, compared to 78 percent of the time with pattern recognition controls.⁵

Among the implications of this study is that pattern recognition could be used to facilitate simultaneous myoelectric control in the absence of TMR. Further, it appears that when simultaneous control is available to the user, it will generally be used.

Conclusion

Traditional myoelectric control strategies are limited by the number of available signal sites and the physiologic appropriateness of these sites. Pattern recognition represents one method of addressing these limitations, facilitating a more elegant extraction of intent from residual muscle bellies. While the processing of such systems is complex and is enhanced by additional signal inputs, the strategy allows a more intuitive, physiologic control.

Phil Stevens, MEd, CPO, FAAOP, is in clinical practice with Hanger Clinic, Salt Lake City, Utah. He can be reached at [email protected]

References

1. Scheme, E., and K. Englehart. 2011. Electrogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. Journal of Rehabilitation Research and Development48 (6):643-60.
2. Hudgins, B., P. Parker, and R. N. Scott. 1993. A new strategy for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering 40 (1):82-94.
3. Hargrove, L. J., K. Englehart, and B. Hudgins. 2007. A comparison of surface and intramuscular myoelectric signal classification. IEEE Transactions on Biomedical Engineering 54 (5):847-53.
4. Zhou, P., M. N. Lowery, K. B. Englehart, H. Huang, G. Li, L. Hargrove, J. P. Dewald, and T. A. Kuiken. 2007. Decoding a new neural machine interface for control of artificial limbs. Journal of Neurophysiology 98 (5):2974-82.
5. Young, A. J., L. H. Smith, E. J. Rouse , and L. J. Hargrove. 2014. A comparison of the real-time controllability of pattern recognition and conventional myoelectric control for discrete and simultaneous movements. Journal of Neuroengineering and Rehabilitation 11:5.