Researchers at the Skoltech Institute of Science and Technology, Russia, devised an eye-tracking system to objectively gauge the comfort, embodiment, and cognitive load of upper-limb prosthesis users. The system monitored how often the user looked at the prosthesis and established an objective measurement of the level of additional mental workload.
The study presented the initial results of the fourth stage of research on prosthetic sensorization in which prosthesis users participated over four months to accommodate extended post-surgery rehabilitation and a one-month period of adaptation to invasive neuroprosthetics. Prosthetics manufacturer Motorika and the joint Center for Cybernetic Medicine and Neuroprosthetics of Motorika and the Federal Medical and Biological Agency of Russia participated in the study.
“Sometimes, test subjects may report biased experiences because of their mood or eagerness. But at home, where subjects were wearing a camera, they might not fully trust in the device to rely on it as much as they seem to in the controlled setting of the lab,” said doctoral student Mikhail Knyshenko, the lead author of the study. “Our approach allows extracting additional objective proxy measures of comfort and mental load with the potential to collect these data outside the lab.”
The new portable system encompasses eye-tracking glasses with two cameras capturing eye movement and a third camera tracking the environment, sensors incorporated into the prosthesis, and software that processes the data. Computer vision algorithms are used to recognize the key objects of interest in the videos: the prosthesis, the manipulated object, and the target area for depositing the object.
In the experiment reported in the study, a test subject seated at a table repeatedly completed a task that involved grasping the object, moving it to an illuminated target area on the table, and releasing the object. In half of the trials, the subject received sensory feedback from the prosthesis in the form of electrical stimulation delivered via an electrode implanted in the arm. The stimulation evoked sensations in the phantom limb that appeared to match the parts of the prosthetic hand that were in contact with the object. Over the course of the study, the team accumulated about 250 hours of video.
The sensory feedback provided to the test subject by the prosthesis was fine-tuned to him based on prior sessions in which the feasible range of stimulation was determined. The preparatory sessions also involved collecting the test subject’s self-reported feedback concerning the precise nature of the sensations produced by electrical stimulation, allowing the researchers to better understand stimulation map and where the sensation seemed to originate.
In the experiments, when the test subject received feedback from the prosthesis in the form of electrical stimulation, he was less likely to look at the limb and therefore did not concentrate as much of their attention on the device.
“We’ve been developing invasive stimulation and prosthetic restoration methods for five years,” said Gurgen Soghoyan, the principal investigator of the study from Skoltech Neuro. “Our goal is to steadily increase the independence and mobility of these technologies while keeping them safe and convenient for the user. In this study, for the first time, we enabled a participant to move freely and interact with real-world objects. With this added freedom in our bidirectional system, we aim to objectively evaluate how users’ relationships with their prosthetics evolve.”
The system can be used by researchers and prosthesis manufacturers to collect data complementing conventional self-reports and guiding the development of smart prosthetics, the researchers said.
Editor’s note: This story was adapted from materials provided by Skolkovo Institute of Science and Technology.
The study, “Learning to operate a sensorized prosthetic hand assessed with a portable system based on computer vision, eye tracking and prosthetic sensors,” was published in OSF Preprints.
