A new AI robotic glove has been created to help stroke patients relearn how to play the piano.
The device is the first of its kind to be able to feel the difference between correct and incorrect versions of the same song.
For people who have suffered neurotrauma such as a stroke, everyday tasks can be extremely challenging thanks to a drop in coordination and strength in one or both upper limbs.
These problems have spurred the development of robotic devices to help enhance their abilities.
However, the rigid nature of these assistive devices can be problematic, especially for more complex tasks like playing a musical instrument.
The new device avoids these issues by combining a soft robotic hand exoskeleton and AI to improve hand dexterity.
Study author, Dr, Erik Engeberg, a professor at Florida Atlantic University’s (FAU) Department of Ocean and Mechanical Engineering, said: “Playing the piano requires complex and highly skilled movements, and relearning tasks involves the restoration and retraining of specific movements or skills.
“Our robotic glove is composed of soft, flexible materials and sensors that provide gentle support and assistance to individuals to relearn and regain their motor abilities.”
Researchers integrated special sensor arrays into each fingertip of the robotic glove.
Unlike prior exoskeletons, this new technology provides precise force and guidance in recovering the fine finger movements required for piano playing.
By monitoring and responding to users’ movements, the glove offers real-time feedback and adjustments, making it easier for them to grasp the correct movement techniques.
To demonstrate the robotic glove’s abilities, researchers programmed it to feel the difference between correct and incorrect versions of the well-known tune “Mary Had a Little Lamb,” played on the piano.
To introduce variations in the performance, they created a pool of 12 different types of errors that could happen at the beginning or end of a note.
They also added in timing errors that were either premature or delayed, persisting of either 0.1, 0.2 or 0.3 seconds.
Ten different song variations consisted of three groups of three variations each, plus the correct song played with no errors.
To classify the song variations, Random Forest (RF), K-Nearest Neighbor (KNN) and Artificial Neural Network (ANN) algorithms were trained with data from the tactile sensors in the fingertips.
Feeling the differences between correct and incorrect versions of the song was done with the robotic glove independently and while worn by a person.
The accuracy of these algorithms was compared to classify the correct and incorrect song variations with and without the human subject.
The results showed that the ANN algorithm had the highest accuracy of just over 97 per cent with the human subject and just under 95 per cent without human help.
The algorithm successfully determined the percentage error of a certain song as well as identified key presses that were out of time.
These findings show that AI has the potential to help disabled people relearn dexterous tasks like playing musical instruments.
Researchers designed the robotic glove using 3D-printed polyvinyl acid stents and hydrogel casting to integrate five actuators into a single wearable device that conforms to the user’s hand.
The glove can be customized to the unique anatomy of each individual with the use of 3D scanning technology or CT scans.
Dr. Engeberg said: “Our design is significantly simpler than most designs as all the actuators and sensors are combined into a single molding process.
“Importantly, although this study’s application was for playing a song, the approach could be applied to myriad tasks of daily life and the device could facilitate intricate rehabilitation programs customized for each patient.”
Clinicians could use the data to develop personalized action plans to pinpoint patients’ weaknesses and could be used to determine which motor functions require improvement.
As patients progress, more challenging songs could be prescribed by the rehabilitation team in a game-like progression to provide a customizable path to improvement.
Dr. Stella Batalama the dean of the FAU College of Engineering and Computer Science, added: “The technology developed by professor Engeberg and the research team is truly a gamechanger for individuals with neuromuscular disorders and reduced limb functionality.
“Although other soft robotic actuators have been used to play the piano; our robotic glove is the only one that has demonstrated the capability to ‘feel’ the difference between correct and incorrect versions of the same song.”
Produced in association with SWNS Talker
Edited by Saba Fatima and Asad Ali