Learning artificial intelligence withusing trial and error, when the computer “fed” the records of a huge number of games that had already been played, on the basis of which he hones his skills, proved that the machine is able to surpass a person in such classic board games as chess and logic game go. Of the most recent examples, we can consider the recent crushing victory of a computer over a person in the strategic computer game StarCraft II, where the machine was trained on the same principle. However, another group of specialists in the field of artificial intelligence showed that this training method can also be used for more practical tasks, for example, for training robotic prostheses.
Reinforcement machine learning method(reinforcement learning), in which the test system is trained by interacting with a certain environment, showed promising results in a small experiment involving a couple of volunteers — one completely healthy person and one with a leg amputated above the knee.
When using traditional techniques to techniciansIt usually takes several hours to properly set up a robotic prosthesis, manually adjusting each artificial joint and adjusting it to the specific walking style to which the person is used. Experiments from specialists at the University of North Carolina showed that the machine learning method with reinforcements allows you to do this much faster — within 10 minutes after fully automatic tuning, a person can walk smoothly.
“Before the actual use of this technology,very far away. We only showed that it is possible. The result delighted us, ”commented Helen Huang, professor of bioengineering at North Carolina State University.
Huang and his colleagues published their findings inIEEE Transactions on Cybernetics. The results of their work can be an important first step in the direction of automating the typical processes of manual adjustment of robotic limbs, which usually takes a lot of time and requires patients to visit specialists whenever dentures require adjustment. In the future, all these settings, people will be able to perform at home on their own, without the help of technicians.
The setting of the robotic prosthesis itselfis a complex process of fitting various parameters that determine the levels of interaction between the limb and the prosthesis required to perform certain tasks. For example, some parameters determine the stiffness of a robotic knee joint or the range of deviations allowed when the leg is rolling back and forth. In the case under discussion, the knee of the robotic prosthesis required adjustment of 12 different parameters. With the standard approach, the end result was usually far from ideal, but nevertheless it was quite suitable so that a person could stand on the prosthesis and perform simple movements.
Robotic limb training isvery complex process of coadaptation. The prosthesis should literally learn to work in tandem with the human brain, which controls the mutual adaptation of organs in the whole organism. At the same time, it is necessary to learn to walk not only the prosthesis itself, but also the person. As a rule, the first results look rather awkward - not far from the examples with skis or skates, for which the person first stood.
"Our body can react rather strangely toforeign objects imitating its continuation. In a sense, our computer-assisted machine learning algorithm with reinforcement is learning to interact with the human body, ”says co-author of a published study, Jenny C, a professor of electronic, computer engineering and energy from Arizona State University.
The task of teaching robotic prosthesescomplicated by the very limited set of data available for learning the algorithm. For example, to train its AlphaZero and AlphaStar algorithms for playing chess, go, and StarCraft II, DeepMind used the recordings of millions of games of these games already played. In turn, a person with an amputated limb to gather the necessary data for learning the algorithm will not be able to walk for a very long amount of time. For example, those who visited the Huang laboratory could walk without stopping for only 15-20 minutes, after which they needed a little rest.
But this is not all the difficulties and limitations, notallowing to cover the whole range of educational information, the researchers note. For example, even between Xi and Huang, before the start of their project, there was some debate about whether volunteers participating in the experiment could be allowed to fall so that the algorithm could learn this information. As a result, they decided to abandon this idea, writing off the safety of volunteers.
And yet, even despite all these difficultiesThe first results were impressive. The researchers taught the algorithm to identify specific patterns in the data collected by sensors installed in the robotic knee. This, in turn, made it possible to establish a threshold for the functionality of a robotic prosthesis, thereby avoiding undesirable situations that could lead to a fall. Ultimately, the algorithm learned to rely on a certain pattern of actions, which made it possible to achieve stability, smoothness, and greater naturalness in the movement of the robotic limb.
Automated learning approachrobotic limbs is still very far from mass use. Now scientists want to teach the algorithm to smoothly manage the prosthesis when getting up, lifting (for example, from a chair) and descending (for example, along a ladder). In addition, the task is to make the system more autonomous, which will allow training and adjustment of prostheses not only in laboratory conditions.
One of the most difficult and at the same time most importanttasks, according to researchers, is to develop a method of "communication" of the algorithm and the person so that the latter can tell him which of the selected settings of the prosthesis is most convenient. Early attempts to solve this problem through ordinary buttons and other simple methods for entering information proved ineffective. Perhaps partly because this version of the computer-to-computer interface does not allow for transferring the entire picture of the perception of human coordination.
"This method did not work because we are not up toend understand all the features of the human body. First of all, it is necessary to fill in some gaps in the fundamental knowledge of psychology and physiology, ”concludes Huang.
Prospects for the development of robotic prostheses based on artificial intelligence can be discussed in our Telegram-chat.