Although robots cannot replace human caregivers, they can provide support so that caregivers have more time to provide the personal, human touch. The German Aerospace Center (Deutsches Zentrum f&uuml;r Luft- und Raumfahrt; DLR) is testing various robotic care assistants in a series of projects. Although these robots were developed for spaceflight, they can also help with essential tasks on Earth.What activities will robots be allowed to take on in care in the future? How will it be ensured that people and their needs are always at the heart of technological advances? How can robotic assistance systems be used in retirement homes, private households and hospitals? Researchers from the <a href="https://www.dlr.de/content/en/institutes/institute-of-robotics-and-mechatronics.html" target="_blank" title="Institute of Robotics and Mechatronics">DLR Institute of Robotics and Mechatronics</a> in <a href="https://www.dlr.de/content/en/sites/oberpfaffenhofen.html" target="_blank" title="Oberpfaffenhofen">Oberpfaffenhofen</a> are investigating these questions in the SMiLE (Service Robotics for People in Living Situations with Limitations; Servicerobotik f&uuml;r Menschen in Lebenssituationen mit Einschr&auml;nkungen) Project. &quot;Robotic care assistants are intended, on the one hand, to relieve the burden on care staff and, on the other, to provide those affected with a greater degree of independence in their everyday lives. In this way, the robots can make a decisive contribution to improving the quality of life and communication with relatives and caregivers,&quot; explains project manager J&ouml;rn Vogel.<h2>Three assistance systems: Rollin&#39; Justin, EDAN and HUG</h2>A humanoid service robot can serve as a helping hand for an elderly person. A person with severe mobility impairments is more likely to use a robotic chair. There is also an interface between the systems. Several systems are being prepared for optimal human assistance.<ul><li><a href="https://www.dlr.de/rm/en/desktopdefault.aspx/tabid-11427/#gallery/35411" target="_blank" title="External link Robot Rollin Justin">Rollin&lsquo; Justin</a> is a humanoid, two-armed, mobile home assistance robot. It monitors its environment via sensors and cameras and evaluates this information. Its lightweight arms enable sensitive interaction with the environment. Rollin&#39; Justin uses artificial intelligence (AI) to plan its workflows autonomously.</li><li><a href="http://www.dlr.de/rm/en/desktopdefault.aspx/tabid-11670/#gallery/28208" target="_blank" title="External link EDAN">EDAN</a> is a wheelchair with a lightweight robotic arm and hand. It is moved with a joystick or via muscle signals measured directly on the surface of the person&#39;s skin. EDAN and Rollin&#39; Justin can also be moved by relatives via smartphones or tablets. Remote control from a care control centre is possible.</li><li><a href="https://www.dlr.de/rm/desktopdefault.aspx/tabid-11704/#gallery/34276" target="_blank" title="External link Roboter HUG">HUG</a> is a haptic input station with two lightweight arms for remote robot control. HUG measures human movements, uses them as signals, and relays them in this way. At the same time, the user feels the exact forces that the robot senses. In this way, EDAN and Rollin&#39; Justin can also be controlled easily and intuitively.</li></ul>In everyday life, for example, EDAN with its robotic arm could help to significantly increase the independence of people with severe motor impairments. Rollin&#39; Justin could perform pick-up and drop-off services in a care facility. For unusual or difficult tasks, trained caregivers from a control centre would be able to quickly assist those in need of care via HUG.<hr><h3 id="isPasted">Focus on safe human-robot interaction </h3>Robotic technologies have long been part of our everyday lives. Simple systems include vacuum cleaner robots or autonomous lawn mowing robots. The use of mobile service robots represents a next step. Equipped with arms and hands, these robots can react to their environment. These developments are possible because modern lightweight robot technology offers safe human-robot interaction. Classic industrial robot arms, such as those used in the automotive industry, must always be operated behind barriers for safety reasons. Modern lightweight robots, on the other hand, are sensitive to physical contact with humans or the environment.The <a href="https://www.dlr.de/rm/en/desktopdefault.aspx/tabid-12464/21732_read-44586/" target="_blank" title="External link Story of the Lightweight Robot LBR">LBR Light-weight Robot</a> developed at DLR almost 20 years ago represented an important milestone in this field. It can sense and control its interaction with the environment through joint torque sensors. This additional information makes it possible for the robot to behave in an actively compliant and thus safe manner, similar to a human arm &ndash; an essential requirement for use in the direct environment of humans.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1661775337601-success-story-lightweight-robot-lbr.jpg" style="width: 766px;" class="fr-fic fr-dib">The first prototype (left) of the LBR was developed in the mid-1990s. The technology of the third generation from 2003 (right) was licensed to the company KUKA. Originally intended as a lightweight robotic arm for space applications, DLR robotics technology forms the basis for many other robotic systems and also serves as a pioneer for safe human-robot interaction. Credit: DLR (CC BY-NC-ND 3.0)

Although robots cannot replace human caregivers, they can provide support so that caregivers have more time to provide the personal, human touch.

Robots for people in need of care

While our facial expressions play a huge role in building trust, most robots still sport the blank and static visage of a professional poker player. With the increasing use of robots in locations where robots and humans need to work closely together, from nursing homes to warehouses and factories, the need for a more responsive, facially realistic robot is growing more urgent.Long interested in the interactions between robots and humans, researchers in the <a href="https://www.creativemachineslab.com/" target="_blank">Creative Machines Lab</a> at Columbia Engineering have been working for five years to create <a href="http://www.cs.columbia.edu/~bchen/aiface/" target="_blank">EVA</a>, a new autonomous robot with a soft and expressive face that responds to match the expressions of nearby humans. The <a href="http://www.cs.columbia.edu/~bchen/aiface/" target="_blank">research</a> will be presented at the <a href="https://www.ieee-icra.org/" target="_blank">ICRA conference</a> on May 30, 2021, and the <a href="https://www.sciencedirect.com/science/article/pii/S2468067220300262" target="_blank">robot blueprints</a> are open-sourced on Hardware-X (April 2021).“The idea for EVA took shape a few years ago, when my students and I began to notice that the robots in our lab were staring back at us through plastic, googly eyes,” said <a href="https://engineering.columbia.edu/faculty/hod-lipson" target="_blank">Hod Lipson</a>, James and Sally Scapa Professor of Innovation (<a href="https://me.columbia.edu/" target="_blank">Mechanical Engineering</a>) and director of the Creative Machines Lab.<div class="fr-img-space-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1622243314643-robotsmile_template.jpg" style="width: 788px;" class="fr-fic fr-dib">The Robot Smiles Back: Eva mimics human facial expressions in real-time from a living stream camera. The entire system is learned without human labels. Eva learns two essential capabilities: 1) anticipating what itself would look like if it were making an observed facial expression, known as self-image; 2) map its imagined face to physical actions. <style type="text/css"></style><style type="text/css"></style>&nbsp;Image And Video Credit: Creative Machines Lab/Columbia Engineering&nbsp;Lipson observed a similar trend in the grocery store, where he encountered restocking robots wearing name badges, and in one case, decked out in a cozy, hand-knit cap. “People seemed to be humanizing their robotic colleagues by giving them eyes, an identity, or a name,” he said. “This made us wonder, if eyes and clothing work, why not make a robot that has a super-expressive and responsive human face?”</div>While this sounds simple, creating a convincing robotic face has been a formidable challenge for roboticists. For decades, robotic body parts have been made of metal or hard plastic, materials that were too stiff to flow and move the way human tissue does. Robotic hardware has been similarly crude and difficult to work with—circuits, sensors, and motors are heavy, power-intensive, and bulky.The first phase of the project began in Lipson’s lab several years ago when undergraduate student Zanwar Faraj led a team of students in building the robot’s physical “machinery.” They constructed EVA as a disembodied bust that bears a strong resemblance to the silent but facially animated performers of the Blue Man Group. EVA can express the six basic emotions of anger, disgust, fear, joy, sadness, and surprise, as well as an array of more nuanced emotions, by using artificial “muscles” (i.e. cables and motors) that pull on specific points on EVA’s face, mimicking the movements of the more than 42 tiny muscles attached at various points to the skin and bones of human faces.<iframe src="https://www.youtube.com/embed/1vBLI-q04kM?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 788px; height: 447px;"></iframe>Overview video of the presented research, with narration.&nbsp;“The greatest challenge in creating EVA was designing a system that was compact enough to fit inside the confines of a human skull while still being functional enough to produce a wide range of facial expressions,” Faraj noted.To overcome this challenge, the team relied heavily on 3D printing to manufacture parts with complex shapes that integrated seamlessly and efficiently with EVA’s skull. After weeks of tugging cables to make EVA smile, frown, or look upset, the team noticed that EVA’s blue, disembodied face could elicit emotional responses from their lab mates. “I was minding my own business one day when EVA suddenly gave me a big, friendly smile,” Lipson recalled. “I knew it was purely mechanical, but I found myself reflexively smiling back.”Once the team was satisfied with EVA’s “mechanics,” they began to address the project’s second major phase: programming the artificial intelligence that would guide EVA’s facial movements. While lifelike animatronic robots have been in use at theme parks and in movie studios for years, Lipson’s team made two technological advances. EVA uses deep learning artificial intelligence to “read” and then mirror the expressions on nearby human faces. And EVA’s ability to mimic a wide range of different human facial expressions is learned by trial and error from watching videos of itself.The most difficult human activities to automate involve non-repetitive physical movements that take place in complicated social settings. <a href="http://www.cs.columbia.edu/~bchen/" target="_blank">Boyuan Chen</a>, Lipson’s PhD student who led the software phase of the project, quickly realized that EVA’s facial movements were too complex a process to be governed by pre-defined sets of rules. To tackle this challenge, Chen and a second team of students created EVA’s brain using several Deep Learning neural networks. The robot’s brain needed to master two capabilities: First, to learn to use its own complex system of mechanical muscles to generate any particular facial expression, and, second, to know which faces to make by “reading” the faces of humans.To teach EVA what its own face looked like, Chen and team filmed hours of footage of EVA making a series of random faces. Then, like a human watching herself on Zoom, EVA’s internal neural networks learned to pair muscle motion with the video footage of its own face. Now that EVA had a primitive sense of how its own face worked (known as a “self-image”), it used a second network to match its own self-image with the image of a human face captured on its video camera. After several refinements and iterations, EVA acquired the ability to read human face gestures from a camera, and to respond by mirroring that human’s facial expression.The researchers note that EVA is a laboratory experiment, and mimicry alone is still a far cry from the complex ways in which humans communicate using facial expressions. But such enabling technologies could someday have beneficial, real-world applications. For example, robots capable of responding to a wide variety of human body language would be useful in workplaces, hospitals, schools, and homes.“There is a limit to how much we humans can engage emotionally with cloud-based chatbots or disembodied smart-home speakers,” said Lipson. “Our brains seem to respond well to robots that have some kind of recognizable physical presence.”Added Chen, “Robots are intertwined in our lives in a growing number of ways, so building trust between humans and machines is increasingly important.”

Data Collection Process: Eva is practicing random facial expressions by recording what it looks like from the front camera.  Image And Video Credit: Creative Machines Lab/Columbia Engineering

Columbia Engineering researchers use AI to teach robots to make appropriate reactive human facial expressions, an ability that could build trust between humans and their robotic co-workers and care-givers

The Robot Smiled Back

Humanoid robots have been a topic of deep interest amongst engineers due to their ability to perform in environments primarily designed for humans. The field of robotics in general is quite unique as it is made of a fusion of many disciplines from mechanical design to biology. However, the humanoid is particularly fascinating to humankind as it mimics the human form. &nbsp;Generally speaking, a robot can perform a variety of tasks autonomously. Robots can fall into different categories such as industrial, service, social, humanoid, etc..&nbsp;Industrial robots, for instance, as the name suggests are designed with the intention of performing a specific task within an industrial context i.e. welding (commonly deployed in automotive plants), commercial vacuum cleaning, and so on. Humanoid robots on the other hand can be merely the design of a particular body part or a complete imitation of the human biomechanics. Through developing humanoids not only we can build machines that are able to operate in human environments but also gain a better understanding of the movements of the human body. This also enables us to design and create more optimal protheses for injured people.&nbsp;<h1 dir="ltr">Overall Design Considerations&nbsp;</h1>The key consideration associated with the outer appearance of the humanoid robot is how much it will emulate human features. The cornerstone of developing humanoid robots is that they can interact seamlessly with humans. So, understanding how much verbal understanding or response it should be able to perform is paramount. This also determines to what extent these robots can navigate environments natural to humans and operate as such.&nbsp;Suffice it to say, when designing a robot, important factors include size, strength and weight. In addition, application plays a crucial role in the development and design process. For example, the infamous <a href="https://www.wevolver.com/wevolver.staff/atlas.robot/" target="_blank">ATLAS robot</a> from Boston Dynamics is designed for physical performances whilst doing motion generation. To perform tasks at a fast pace, humanoid robots should account for stiffness and power output, light weight and more vigilant real-time information processing capabilities. All in all, The design principles can be divided into four following categories:<ul><li dir="ltr">Proportions of the body:&nbsp;In order to achieve human-like kinematics and dynamics, similar mass distributions and link lengths are required. Furthermore, humanoids are able to fit better within human environments when accounting for strong body proportion similarities.</li><li dir="ltr">Skeletal structure:&nbsp;One important consideration for mimicking the human body is a high degree of fidelity in skeletal structure. Human joints are made up of both single-axis rotation and rolling-sliding joints. This combination allows for sliding and rotation movements between the bones. To emulate flexible upper body movement, spine joints with multiple vertebrae are helpful.</li><li dir="ltr">Joint performance:&nbsp;Joint performance determines how the humanoid performs when it comes to the whole-body motion. Two factors impacting joint performance are joint range of motion and power output.</li><li dir="ltr">Arrangement of muscles: Muscle arrangement is essential for identifying the muscle-joint-operational mappins, which provide muscle contribution in adequate tendency during whole-body movements. To design a humanoid with human-like muscle arrangement, the humanoid should possess as many muscle actuators as the design space allows for</li></ul><iframe src="https://www.youtube.com/embed/_sBBaNYex3E?&amp;wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 753px; height: 411px;"></iframe> <h2 dir="ltr">Mechanical Structure of Humanoid Robots &nbsp; &nbsp; &nbsp; &nbsp;</h2>Human structure has always been the source of inspiration for the humanoid robot's kinematic structure. Thus, most humanoids resemble human features such as head, legs, hands, feet, etc. Moreover, these robots typically include similar Degrees of Freedom (DOFs) to a human being. Evidently, such robots are designed to be human-like. Thus, the center of motion of the robot should be located in the vicinity of its ‘belly button’. The human musculoskeletal system is essentially made up of bones, cartilage, muscles, tendons and skin. It consists of more than 206 bones and 600 muscles.&nbsp;Through biomechanics, the musculoskeletal system can be regarded as a mechanical system: the bones can be the rigid links, and the cartilage connects these links. The lower body is generally composed of a waist, upper legs, lower legs and feet in a redundant and human-like kinematic configuration, similar Range of Motion (ROM) and similar joint velocities and accelerations. The ‘legs’ consist of 3 rotation joints with one joint allowing for knee flexion-extension and two other for the flexion-extension and prono-supination of ankles. The type of application determines the grippers used in humanoid robots.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1603454172956-robots_0+%281%29.jpg">Image: Boston DynamicsFeet are perhaps the most important aspect and they must handle impacts of landing and prevent slippage. Humanoid robots should possess soles with materials and shapes that provide enough friction. For instance, the <a href="https://www.wevolver.com/wevolver.staff/lola/" target="_blank">LOLA robot</a> (a project built with the goal of creating a machine that is capable of stable and fast walking) deploys Sylomer (a special elastomer material) with an elastic modulus that is higher at the heel.&nbsp;There’s a wide variety of tasks a humanoid robot can overtake, from extreme or dangerous environments to healthcare. The technology pioneering the progress of these robots continues to grow by leaps and bounds and so does with it their application.<h1 dir="ltr">Sources &amp; Further Reading&nbsp;</h1><a href="https://www.thelancet.com/journals/ebiom/article/PIIS2352-3964(19)30534-1/fulltext" target="_blank">Human–robotic interfaces to shape the future of prosthetics</a><a href="http://ai.stanford.edu/~rxzhang/Mechanical_Design_And_Control_System.pdf" target="_blank">Mechanical Design And Control System Configuration of a Humanoid Robot</a><a href="https://www.sciencedirect.com/science/article/pii/S0928425709000400" target="_blank">Humanoid robot Lola: Design and walking control</a>

Chair of Applied Mechanics, Technical University of Munich (TUM)

Humanoid robots require deeper consideration than just mimicking humans.

Design Considerations for Humanoid Robots

Festo pneumatic muscle actuator promotional image.

Advancements in pressurized fluid power systems may hold the key to the next generation of Biomimetic Robots.

Biomechanical Marvel-Fluidic Muscle Technology

You finally have it. The most awesome robot, with all the sensors and actuators you’ve ever dreamed of. LIDAR, RADAR, vision, far-field microphone array, pressure-sensitive touch, precision GPS, and odometry. Speakers, screens, legs, arms, hands, and maybe even a flamethrower. This robot has everything. Maybe it’s something you’ve put together in your garage in your spare time. Whichever, many years of work, research, and rework have gone into building your dream robot, which you’re confident can sense and do anything.So now the real work begins — you need to somehow get this robot to do what you want.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343304603-1+u2VI1R_l3BbOS0F8E0Tn5A.jpeg" style="width: 766px;" class="fr-fic fr-dib">Let’s say you’ve got a task that you want this robot to perform. It could be anything, from juggling 6 flaming chainsaws to helping a stroke victim do their exercises to baking an apple pie. It could be the key to your money-making robot company concept, or just a useful behavior to help you around the house. Whatever it is, inside your brain, latent in your mind, is a policy that, if transferred to the robot, would make it perform the task just the way you want it. The problem of getting the task from your mind to the robot is what we call Human-Robot Policy Transfer (HRPT). And there are a few different ways to approach it.Teleoperation The first, and generally simplest, way to get a robot to do something is direct teleoperation. That is, having a human directly control the robot to perform the task. You’ll need an appropriate method of controlling the robot, which could be anything from a set of joysticks to a full-body telemetry suit. You’ll also need a human, possibly you (or a paid expert), to control the robot, or maybe even a whole team of humans, each controlling a different part of the robot in different ways. Whatever the control system, the idea is that humans implement the control policy themselves. That is, a human observes the robot’s state either directly or through sensors, then tells the robot how to use its actuators (such as manipulators, cutters, lifters, etc.) to achieve the task. With teleoperation, even though some low-level systems (for example, balance) might be automated, all high-level decision-making is done by a human.There are many advantages to teleoperation. First and foremost, it will save you the trouble of figuring out how to state your goals explicitly; that is, the policy can remain latent in your mind. This feature is a big help if the policy is fairly complex. When you consider the effort it could take to program a robot to perform a complicated, multi-step task, it can be clear that simply controlling the robot yourself might get the job done more quickly.Further, with a human in the loop, adapting to a changing environment can be done immediately. Additionally, since a human is making the decisions, legal and ethical questions of liability and responsibility are easier to answer. An excellent example of teleoperation for HRPT is the use of surgical robots, where the precision and scale of the robot are well-used by the complex — and unexpressed — control policy inside the human surgeon’s head.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343339019-1+Pa9G1zlq4JA9ysd6driv0g.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Intuitive Surgical’s DaVinci robotic surgery system.</figcaption>There are, however, also many drawbacks to teleoperation as an HRPT approach. The main one is that, since humans have to be involved with teleoperated robots at every moment, it doesn’t really achieve the dream of developing autonomous robots. A teleoperated robot doesn’t free humans from doing work or enable one human to multiply their effectiveness and become as productive as 10 people. (It does, however, allow humans to remove themselves from dangerous situations, which is sometimes the entire point.)Similarly, since a human is, in effect, still doing the work with teleoperation, all the faults and issues of humans come through. For example, teleoperators get tired and distracted, show up late, or just make mistakes — all things that we expect (perhaps incorrectly) our robots not to do.As a first-pass tool, teleoperation is great, in that it proves that the task you want your robot to perform is actually performable by the robot. It’s good to know that the robot’s hardware is physically capable of doing the task you want it to do, before you try and develop an autonomous control policy for it.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343368819-1+PTPiPsT1Umcvd3Qf7d_DVA.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Real-time teleoperation of the Mahru robot developed at the Korea Institute of Science and Technology.</figcaption>Further, if you limit yourself to only using the robot’s sensor information during teleoperation, you can prove that the task is decideable by the robot. That is, that the robot has access to enough information to actually determine what to do in response to its sensor stream. Again, this is highly useful to know before you pour hours/weeks/months/years into trying to get the robot to do the task on its own.So let’s say you’ve proved that the task you want your robot to accomplish is actually decideable and performable by this awesome robot body, and now you want to make the robot perform the task by itself. Great! How’re you going to do that? Probably with the second major approach to HRPT.Explicit ProgrammingThat right. You’re going to sit down and try to — explicitly and in great detail, in your favorite programming language — write out exactly what the robot needs to do, how it needs to react to its environment, in order to accomplish the task.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343408817-1+sPNDae_0Ib0kK4TjnwbFpQ.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Programming the LEGO(R) MINDSTORMS(R) EV3.</figcaption>Honestly, to consider every possible situation that could potentially happen while your robot goes about its business would take too long (there are an infinity of them, after all, and even people paid to come up with odd situations miss a bunch). So, you’ll likely make some assumptions about what situations are unlikely to arise and ignore them in your code. Do you really need to consider what should happen if your robot encounters a <a data-href="http://www.siliconbeat.com/2016/04/05/google-self-driving-car-brakes-for-wheelchair-lady-chasing-duck-with-broom/" href="http://www.siliconbeat.com/2016/04/05/google-self-driving-car-brakes-for-wheelchair-lady-chasing-duck-with-broom/" rel="nofollow noopener" target="_blank">wheelchair/duck/broom combo</a>?Even ignoring the more unlikely scenarios, there’s a lot of work required to program a robot to perform anything other than the most basic tasks in a wide variety of different contexts. In fact, for a given level of programming effort (say, a month), as the complexity of the task goes up, the variance of the environment that it can be programmed to operate in goes down.Sure, it’s easy to code a simple task (drive in a square) to be performed in a wide variety of environments (different floor surfaces/rooms). It’s also just about as easy to program a complex task (set the table) in one particular environment (all plates/utensils and tables/chairs in fixed, known locations). But programming a complex task for varying environments (setting the table in any kitchen) is much, MUCH harder. All of this effort (designing, architecting, coding, debugging, and testing) can in fact be a lot more work than just doing the task yourself.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343435763-1+sifPS1Kt67_XzFI5hdnezw.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Simulation and offline programming software by Staubli Robotics.</figcaption>So why bother trying to get a robot to do complex tasks autonomously? Because once your programming works — if it works — the robot can perform the task over and over and over again. With luck, you’ll recoup your effort in the long run.Almost all autonomous robots you’re likely to have interacted with up to now (outside of research lab demos) have gone through this process. They’ve been painstakingly, explicitly coded by a team of developers to perform specific tasks in a set of constrained environments. Change one parameter (the height of a step, for example) beyond the constrained range, and the whole system comes tumbling down like, well, a robot trying to walk.LearningThe third main approach to HRPT is to use learning: having a robot figure out how to perform the task by analyzing data.Now, there are many different types of machine learning using many different technologies, but they all try and get at the same goal: to find patterns in data that are useful for accomplishing a desired task. The task might be identifying a cat in an image, predicting the stock market, or determining a control policy for a robot. When it comes to HRPT, this last one is exactly what we need.A conceptually simple way to have a robot learn to perform a task is to tell the robot what the goal of the task is, and let the robot experiment to figure out how to achieve this goal. This is reinforcement learning (RL), where the robot gets a reward when the task is accomplished (or pays a penalty when the task is NOT accomplished) and tries to figure out how to use its capabilities to get to the goal. The reward itself is generated by a reward function that maps the robot’s state (or robot state-action pairs) to how good (or bad) it is for the robot to be in that state (or perform that action in that state).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343607363-1+FawajzcHQaLKocy5aNYDTQ.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Rewarding robots does not usually look like this.</figcaption>On one hand, RL can be really easy to use, as the end-state, or desired goal, is often simple to describe. When a simple end-state is all that matters, the reward function is easy to write: If there is an omelette on my plate, +1000, else 0.On the other hand, to find its way to the desired end-state, a robot might have to explore all of the possible configurations it and the world around it could be in (its state space) and all of the possible actions it could perform (its action space) in each of those states. If the state and action spaces are large, it can take a very, very long time for the robot to find ANY path to the goal, let alone an optimal one. But, for restricted situations with small state-spaces and limited action-spaces (such as simulated systems in <a data-href="http://www.prismmodelchecker.org/casestudies/robot.php" href="http://www.prismmodelchecker.org/casestudies/robot.php" rel="nofollow noopener" target="_blank">a grid world</a>), RL can often lead to robust and optimal control policies.One way to limit RL issues in larger spaces is to supply additional rewards, say, for when the robot accomplishes intermediate states on the way to the goal. Or, a developer can define heuristics to guide the robot toward more task-appropriate behavior. Unfortunately, providing these types of information make the reward function more complicated and difficult to write down and can approach the complexity of explicit programming.Learning… from DemonstrationAnother learning approach bypasses the tricky parts of RL by just letting humans provide the robot with helpful examples, or demonstrations, of what it should do to accomplish the desired task. This approach is called Learning from Demonstration (LfD) or, sometimes, Programming by Demonstration (PbD).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343653832-1+fLwb1fb_wDXc43LttIfhig.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Part of the Learning from Demonstration Challenge at the Association for the Advancement of Artificial Intelligence 2011 conference.</figcaption>LfD only assumes that you, the human, can perform the task you want the robot to do. Interestingly enough, one of the most common forms that demonstrations take are teleoperations of the robot through the task. Using teleoperation in service of robotic learning nicely leverages the benefits of that method of HRPT, namely proving the performability of the task by the robot.Another form of generating demonstrations is direct kinesthetic manipulation, where the robot is physically guided through the task, again proving performability. Unfortunately, this approach (and teleoperation as well), can prove difficult when a robot has many degrees-of-freedom (think 2 arms, each with 7 joints, as well as legs, a neck, etc).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343688526-1+EdrY_RpaRODV9RHiy428Fg.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>When Rethink Robotics was still developing robots, they were very clear — their robots are “trained, not programmed”, in this case by direct kinesthetic manipulation.</figcaption>A much-sought-after LfD approach is to have the robot merely observe the human perform the task themselves. If you can solve the correspondence problem (which parts of the human correspond to which parts of the robot), observation can be an extremely powerful way to collect demonstration data.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343743988-1+IwXJOsXnrS4Ts0z1VZxSSg.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Sylvain Calinon demonstrates robotic learning <a data-href="http://calinon.ch/research_prev.php" href="http://calinon.ch/research_prev.php" rel="nofollow noopener" target="_blank">via observation</a>.</figcaption>So let’s say you’ve got some nice demonstrations of how the task should go. What’s next?Simply having the robot play back the demonstration is not enough, as the world is not generally guaranteed to behave exactly as it did before. So, instead, many LfD techniques build a model of the demonstration data, capturing both the expected behavior and allowable variance around it. One common approach is to take the demonstrations, warp them all to be the same length in time, and then build a dynamical model of how the robot’s state evolves over time. You can even guarantee stability of the system, by imposing different constraints on it. These systems can be used to do some pretty cool things, like <a data-href="https://www.researchgate.net/publication/228975860_Learning_to_Play_Mini-Golf_from_Human_Demonstration_using_Autonomous_Dynamical_Systems" href="https://www.researchgate.net/publication/228975860_Learning_to_Play_Mini-Golf_from_Human_Demonstration_using_Autonomous_Dynamical_Systems" rel="nofollow noopener" target="_blank">play mini-golf</a>.Still, a common issue with LfD is that the learned control policy is often limited to being as good as the human demonstrator. A different approach, termed Inverse Reinforcement Learning (IRL) is designed to combine the demonstration-based approach of LfD with the optimization powers of RL.In this family of techniques, instead of relying just on a control policy derived directly from demonstrations, the system first tries to estimate the reward function underlying them. By using both the reward function and an initial control policy based on the demonstrations, IRL systems can optimize for task performance. Eventually, it’s even possible for such a system to perform a task better than the humans themselves! A classic example of this is the <a data-href="http://heli.stanford.edu/" href="http://heli.stanford.edu/" rel="noopener nofollow" target="_blank">Stanford Autonomous Helicopter project</a>.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343771925-1+wY-4vOkvEwnjRyS_WJTGYQ.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>The goal of the Stanford University project was to push the state of the art in autonomous helicopter flight: extreme aerobatics under computer control.</figcaption>I could go on and on, as there is so much interesting work in the field of LfD. There’s work that looks at discovering reusable sub-skills from demonstrations, so new, different tasks can be accomplished. And there’s work that looks at giving feedback to the human during the training process so they can generate more useful training data. And then there’s my own work that has looked at how to determine when there are multiple correct ways to achieve a goal, or how to extract useful information from failed demonstration attempts, instead of assuming the human can always perform the task correctly.As exciting as it is, learning as a solution to HRPT still has drawbacks. The major issue, as with any learning system, is how to guarantee limits on the robot’s behavior. With direct teleoperation, or explicit programming, you can say with some degree of confidence that the robot butler will never put mustard in your coffee (barring perceptual issues, or a very tired or malicious human teleoperator). However, with a learned system, you’re never quite certain what the robot will do in any given situation.And that’s where the real fun begins.

Capio Upper Body Exoskeleton for Teleoperation by the DFKI GmbH Robotics Innovation Center.

Because your robot’s not going to program itself (yet)

How to Get a Robot to Do What You Want

This is the first article, in a series, exploring the vast landscape of artificial muscle actuators. In the subsequent articles, the quest to create and harness artificial muscle technology will be explored.Until recently, conventional robots have utilized rigid materials and fabrication methods. Ironically, traditional robots are relatively weak and brittle, when compared with biological organisms, which utilize soft adaptive tissues to produce movement. A new approach to robotics has taken shape in the form of soft robotics and biomimicry.So, what does “soft” robotics mean? If you ever stayed up late as a child and remember watching old science fiction movies, as I did, your immediate concept of a “Robot” is typically a large, metal creature, lumbering about. In sharp contrast to this classic robot concept, soft robots are flexible and agile; think of a ballerina or an athlete. Obviously, modern robots have advanced from this stereotype, however our best robot is still not capable of the nuanced movements and physical strength of a human.A key component to producing a robot that more closely replicates the movement of a human, starts with the actuation system. An actuator is any mechanism that can convert some form of energy into physical movement, similar to how our muscles work. Typically, servo motors are used to produce movement in most robotic systems. However, servo motors are bulky, have a low power to weight ratio(when compared to muscle tissue) and are non-compliant. Essentially, compliance of a material means that it has some give to it, like a rubber band. You may not realize it, but the human body is capable of withstanding incredible forces.As demonstrated at the 2015 DARPA Robotics Challenge, a humanoid robot struggled to accomplish physical tasks that any average human would be able to do with ease. Common bumps and bruises that humans experience, would be catastrophic to most modern robots. Our ability to adapt to different physical environments is the template that will be used to create future robots.Artificial Muscle Research Engineers have sought a multitude of materials and fabrication methods in order to replicate the attributes of biological muscle tissue. I have also spent close to 20 years attempting to create an artificial muscle actuator, with limited success. So, what makes muscle tissue so special and replication of its attributes so elusive? Well, for starters, nature has had a 560 million year head start. Additionally, each of our muscles is a compact and powerful actuator that is made of soft, self-healing tissue. Our muscle tissue affords us dynamic movement and uses an efficient chemical fuel as a power source, glucose. Muscle tissue is composed of thousands of contractile fibers-within each fiber are up to 100,000 molecular units that synchronously shorten in length with a tremendous amount of force and velocity. This force and velocity enables basic movement and locomotion.Extensive research has been conducted to create an artificial muscle actuator which can replicate or exceed biological muscle tissue, with varying levels of success. Several types of technologies that have been explored include: Electroactive polymers (EAPs), Shape-memory metal alloy wires/liquid crystalline elastomers (SMAs), the Mckibben air muscle (Pneumatic artificial muscles), nylon fiber (torsional actuators) and Carbon fiber (PDMS rubber/torsional actuators). While all of these types of artificial muscle actuators, possess some of the sought after attributes of biological muscle, none of them are a complete solution in and of themselves. In order to fully appreciate the vast landscape of artificial muscle research and current innovations-each of these technologies will be further discussed; including history, fabrication techniques, advantages, drawbacks and future applications.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300296387-materiability2.jpg" style="width: 752px;" class="fr-fic fr-dib">EAP(Electroactive Polymer) ActuatorElectroactive polymers (EAPs) EAP technology is based upon the principle of generating a motive force in polymeric materials via an applied electrostatic field. One of the basic principles behind this technology is static electricity, if you have ever heard the crackle when you remove laundry from the dryer or accidentally, “shocked” your friend when walking, you have experienced static electricity. The electroactive effect occurs when a high voltage is applied to a flexible material, the electric static field that is generated is strong enough to produce movement in the material.Another technique used for actuation is based on elastomeric, meta-materials that exhibit the piezoelectric effect. Piezoelectric materials are actually very common, they are the technology that enables, haptic feedback in your cell phone. Typically, Piezoelectric materials are in the form of quartz crystals, which have the incredible ability to convert physical stress into electricity and vice versa. Moreover, certain polymer compounds can be, “doped”, with certain metals to produce the piezoelectric effect.&nbsp;While EAPs can embody, several permutations, the main focus of this discussion will be on the most commercially viable configurations which include, dielectric elastomers and ionic polymer-metal composites. EAP research started as early as the 1800s, however it wasn't until the turn of the century that EAP technology was realized for practical applications.Physicist, Dr. Yoseph Bar-Cohen, of the Jet Propulsion Laboratory, further advanced EAP technology by inspiring a global research initiative which challenged research groups around the world to compete in the world's first, “Arm wrestling Match of EAP Robotic Arm against Human challenge”. In 2002, Japanese tech company Eamex, released one of the first commercial products-a realistic robotic fish toy, utilizing a viable EAP, artificial muscle. Continued research, funded by DARPA, resulted in the tech start-up; Artificial Muscle, Inc-which specializes in development and manufacture of products based on EAP technology.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300403609-image022.jpg" style="width: 748px;" class="fr-fic fr-dib">Arm wrestling Match of EAP Robotic Arm against Human challenge(human opponent, Panna Felsen)Dielectric Elastomeric Actuator Dielectric elastomeric architecture involves an elastomeric film or cross sectional area of polymeric material, coated with an electrically conductive layer on either side. This configuration yields a compliant capacitor, where in, the conductive coating serves as a pair of electrodes. When an electrical field is applied to the DEA, changes in the electric field distribution of the electrodes generate a mechanical strain via Coulombic charge interaction of the compliant electrodes. Essentially, the conductive electrodes become electrostatically attracted and repelled in specific regions, causing the material to compress and expand. Additionally, the strain deformation state can be held in a static position via a constant DC voltage; analogous to an isometric muscle contraction.Composition of a DEA is relegated to a suitable elastomeric material with high elastic modulus; typically silicone or acrylic elastomers are used. During the fabrication process, elastomeric film of a predetermined thickness, is stretched upon a rigid frame in order to provide bidirectional static prestrain(stretch) of the elastomeric film. This prestrain fabrication technique has been shown to significantly increase the &nbsp;mechanical efficiency and responsiveness of DEAs. The electrically conductive coating, typically comprises conductive carbon grease or paste, applied to either side of the elastomeric film to provide diametrically opposed electrodes. This fabrication process can also be repeated several times, with each elastomeric film/electrode layer, “stacked” onto the subsequent layer in order to provide an actuator capable of exerting much larger forces.Ionic Polymer-Metal CompositesIPMC actuators are characterized by synthetic polymers engineered with positively charged metal ions, which are randomly arranged in an aqueous, polymer matrix. Typically, a noble metal coating is applied to the polymer-metal matrix in order to provide positive/negative electrodes(orientation of cathode/anode is based upon the terminal electrical wires attachment). If you remember from Chemistry class, noble elements are those that do not play well with other elements or resist chemical interaction. The positive metal ions partially entangle with water molecules and any inherent, negatively charged ions within the polymer matrix. When an electrical field is applied, the conjugated metal ions are attracted to the negatively charged metal electrode. This ionic migration of entangled water molecules provides an osmotic pressure shift resulting in mechanical strain across the polymer matrix membrane. Conversely, mechanically bending or deformation of the polymer matrix can produce electrical potential and output voltage, which can be advantageous for energy harvesting and biomedical sensors.Edge Over Other ActuatorsPossessing many biomimetic attributes has warranted continued research into EAPs development. Similar to biological muscle tissue, Dielectric Elastomers have an inherent ability to store large amounts of mechanical energy and contract with a large amount of usable force. Another appealing attribute is measured by how well an actuator can operate under continued stress and adaptability under varied conditions. A crucial element when considering building a humanoid robot that can work along side humans. With this in mind, DEAs have been shown to operate within standard room conditions, without the need for additional considerations. A crucial element for any biomimetic actuator is the ability to respond quickly when activation stimuli is applied, DEAs continue to excel with rapid response activation times measuring in the milliseconds.Additionally, Ionic Polymer-Metal Composites bring a unique set of properties to the table, including bi-directional actuation. Dependent upon the polarity of the activation voltage applied, IPMCs can reverse actuation direction, contributing further to adaptable mobility. Due to the unique chemistry of IPMCs, the mechanism of movement is attributed mostly to molecular motion, which allows for low activation voltages. An attribute that is extremely appealing in the quest to create a sustainable, autonomous robot. Configuration of IPMCs is not only constrained to simple membranes. There complaint nature allows the material to be rolled up into compact cylinders, with the ability to bend or extend. A quality that is important in order to approximate various muscle shapes and sizes, inherent in biological organisms.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300566769-a-Optical-image-of-the-bending-actuation-of-the-i-EAP-actuator-under-AC-electric.jpg" style="width: 300px;" class="fr-fic fr-dib">(a) Optical image of the bending actuation of the EAP actuator under AC electric signals.(b) Schematic of bending actuation mechanism.Challenges In The FieldWhile EAPs do posses several remarkable properties, we must also evaluate the engineering challenges faced in the development of EAPs for practical applications. One such hurdle to overcome for any robotic system is power consumption, especially within the context of realizing a self-contained humanoid robot. Unfortunately, this is a major con for DEAs, which require thousands of volts of electricity to operate. With the current state of battery technology this is far from ideal. EAP longevity is also limited due to the high voltage field, often exceeding the breakdown threshold of the polymer material. While IPMCs require minimal activation voltage, additional considerations are required for typical usage. Ironically, not unlike a biological muscle, IPMCs also require encapsulation in an impermeable membrane to prevent dehydration of the polymer matrix. In addition to IPMCs low actuation force, the most critical disadvantage remains the current inability to commercially, mass produce, a consistent ionic polymer. Until these challenges can be overcome, a viable EAP artificial muscle actuator remains elusive.Looking AheadWhat's in store for EAP technology? While an EAP based artificial muscle actuator seems unlikely in the immediate future. EAP technology is currently in development towards, biomechanical energy harvesting. Specifically, IPMC films can be attached to key flexing areas of the human body in order to harvest kinetic energy. So, next time you go to the gym, you can burn calories and charge your cell phone. This technology could also turn out to be invaluable for Soldiers in the midst of combat with little to no recharge options. The structure and mechanism of DEAs has found a practical application within the realm of virtual reality systems. An array of actuator “pixels” can be utilized for graphic displays for the visually impaired and possibly for theme park applications, such as virtual-haptic, hybrid environments. The National Science Foundation is continuing research towards an artificial muscle application, however. With the aid of 3-D printing technology, IPMC technology may prove to one day be a suitable artificial muscle solution.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300675742-3D_printed_IPMC_hand.jpg" style="width: 750px;" class="fr-fic fr-dib">A small 3-D printed ionic polymer-metal composite soft robotic hand. National Science Foundation researchers are working to transform this material into artificial muscles. This is the first article in a series&nbsp;about artificial muscles, created by&nbsp;<a href="https://www.wevolver.com/profile/samuel.manriquez" rel="noopener noreferrer" target="_blank">Samuel Manriquez</a>.

Skeleton with an artificial bicep. Credit: Aslan Miriyev - Columbia Engineering

The current state of robotic actuators and mechanical control systems are failing to keep pace with advancing AI software, exploring novel actuator technologies may bridge the gap between man and machine.

humanoids

Latest Posts

Robots for people in need of care

The Robot Smiled Back

Design Considerations for Humanoid Robots

Biomechanical Marvel-Fluidic Muscle Technology

How to Get a Robot to Do What You Want

Biomechanical marvel-creating an artificial muscle actuator