This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/deaf3bb7-ff61-48d1-ad11-30ae0112d53e?dark=false" class="fr-draggable" data-gtm-yt-inspected-23="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>“I'll have you eating out of the palm of my hand” is an unlikely utterance you'll hear from a robot. Why? Most of them don't have palms.If you have kept up with the protean field, gripping and grasping more like humans has been an ongoing Herculean effort. Now, a new robotic hand design developed in MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) has rethought the oft-overlooked palm. The new design uses advanced sensors for a highly sensitive touch, helping the “extremity” handle objects with more detailed and delicate precision.<a href="https://arxiv.org/pdf/2404.08227" target="_blank">GelPalm</a> has a gel-based, flexible sensor embedded in the palm, drawing inspiration from the soft, deformable nature of human hands. The sensor uses a special color illumination tech that uses red, green, and blue LEDs to light an object, and a camera to capture reflections. This mixture generates detailed 3D surface models for precise robotic interactions.<iframe width="640" height="360" src="https://www.youtube.com/embed/5S9dZQv5qAc?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-23="true" id="894047451" title="Robotic palm mimics human touch">&nbsp;</iframe>GelPalm. Video: MIT CSAILAnd what would the palm be without its facilitative fingers? The team also developed some robotic phalanges, called ROMEO (“RObotic Modular Endoskeleton Optical”), with flexible materials and similar sensing technology as the palm. The fingers have something called “passive compliance,” which is when a robot can adjust to forces naturally, without needing motors or extra control. This in turn helps with the larger objective: increasing the surface area in contact with objects so they can be fully enveloped. Manufactured as single, monolithic structures via 3D printing, the finger designs are a cost-effective production.Beyond improved dexterity, GelPalm offers safer interaction with objects, something that’s especially handy for potential applications like human-robot collaboration, prosthetics, or robotic hands with human-like sensing for biomedical uses.Many previous robotic designs have typically focused on enhancing finger dexterity. Liu's approach shifts the focus to create a more human-like, versatile end effector that interacts more naturally with objects and performs a broader range of tasks.&nbsp;“We draw inspiration from human hands, which have rigid bones surrounded by soft, compliant tissue,” says recent MIT graduate Sandra Q. Liu SM ’20, PhD ’24, the lead designer of GelPalm, who developed the system as a CSAIL affiliate and PhD student in mechanical engineering. “By combining rigid structures with deformable, compliant materials, we can better achieve that same adaptive talent as our skillful hands. A major advantage is that we don't need extra motors or mechanisms to actuate the palm's deformation — the inherent compliance allows it to automatically conform around objects, just like our human palms do so dexterously.”The researchers put the palm design to the test. Liu compared the tactile sensing performance of two different illumination systems — blue LEDs versus white LEDs — integrated into the ROMEO fingers. “Both yielded similar high-quality 3D tactile reconstructions when pressing objects into the gel surfaces,” says Liu.But the critical experiment, she says, was to examine how well the different palm configurations could envelop and stably grasp objects. The team got hands-on, literally slathering plastic shapes in paint and pressing them against four palm types: rigid, structurally compliant, gel compliant, and their dual compliant design. “Visually, and by analyzing the painted surface area contacts, it was clear having both structural and material compliance in the palm provided significantly more grip than the others,” says Liu. “It's an elegant way to maximize the palm's role in achieving stable grasps.”One notable limitation is the challenge of integrating sufficient sensory technology within the palm without making it bulky or overly complex. The use of camera-based tactile sensors introduces issues with size and flexibility, the team says, as the current tech doesn't easily allow for extensive coverage without trade-offs in design and functionality. Addressing this could mean developing more flexible materials for mirrors, and enhancing sensor integration to maintain functionality, without compromising practical usability.“The palm is almost completely overlooked in the development of most robotic hands,” says Columbia University Associate Professor Matei Ciocarlie, who wasn’t involved in the paper. “This work is remarkable because it introduces a purposefully designed, useful palm that combines two key features, articulation and sensing, whereas most robot palms lack either. The human palm is both subtly articulated and highly sensitive, and this work is a relevant innovation in this direction.”“I hope we're moving toward more advanced robotic hands that blend soft and rigid elements with tactile sensitivity, ideally within the next five to 10 years. It's a complex field without a clear consensus on the best hand design, which makes this work especially thrilling,” says Liu. “In developing GelPalm and the ROMEO fingers, I focused on modularity and transferability to encourage a wide range of designs. Making this technology low-cost and easy to manufacture allows more people to innovate and explore. As just one lab and one person in this vast field, my dream is that sharing this knowledge could spark advancements and inspire others.”Ted Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences and CSAIL member, is the senior author on a <a href="https://arxiv.org/pdf/2404.08227" target="_blank">paper describing the work</a>. The research was supported, in part, by the Toyota Research Institute, Amazon Science Hub, and the SINTEF BIFROST project. Liu presented the research at the International Conference on Robotics and Automation (ICRA) earlier this month.

MIT CSAIL student Sandra Q. Liu displays her innovative GelPalm robotic design in her lab workspace. Photo: Michael Grimmett/MIT CSAIL

MIT CSAIL researchers enhance robotic precision with sophisticated tactile sensors in the palm and agile fingers, setting the stage for improvements in human-robot interaction and prosthetic technology.

Robotic palm mimics human touch

At Voltera, we love our test engineering interns. They bring fresh new ideas into our workspace and have a ton of fun exploring with our machines. It’s a privilege to provide mentorship, equipment, and resources to nurture brilliant young minds, and see what they dream up.&nbsp;At the beginning of his work term, Dawson Martian — a second-year undergrad studying Mechatronics, Robotics, and Automation Engineering at the University of Waterloo — came up with a project idea that would put both the <a href="http://voltera.io/v-one" target="_blank">V-One</a> and <a href="http://voltera.io/nova" target="_blank">NOVA</a> to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.Dawson had been working on a different project at the time, designing a light-up glove using stretchable ink and thermoplastic polyurethane&nbsp;(TPU). “I noticed when the glove was put on and powered, the brightness of the LEDs would vary depending on whether the user’s hand was opened or closed,” he recalled. “This got me thinking about how the lights were dimming due to the resistance of the traces increasing whenever they were stretched, essentially the traces were acting as strain gauges,” he said. “From there I realized I could take advantage of this property and use this varying resistance to control a machine or program…what better way to show this unique property than by having a robotic hand mimic my movements?”So, Dawson got to work. He started by building a robotic hand with movable fingers. The hand components were 3D-printed using non-flexible filament, the joints were made with TPU, and then assembled. He ran the wire up the fingers and connected these to servos. These would operate as tendons, so the hand could move. He connected the servos to an Arduino™ with a custom breakout board — which he made using the V-One.&nbsp;For the control glove, Dawson printed strain gauges onto TPU using NOVA and heat-pressed the strain gauges onto a fabric glove using a t-shirt press. Dawson wired a second Arduino and custom breakout board to the traces on the control glove using a conductive adhesive and mounted the board to a wrist strap.&nbsp;The two boards communicate via Bluetooth® transceivers, so if all worked as it should, the board on the control glove would measure resistance in the strain gauges, then transmit movement instructions to the second board. This information would control the actuators of the robotic hand.&nbsp;Of course, nothing goes exactly&nbsp;as planned, and Dawson hit a bit of a snag when he was designing the strain gauges. But, let’s rewind for a minute.&nbsp;<h2>What is a strain gauge, anyway?</h2>“A strange gauge is essentially just a resistor whose resistance value varies with changes in strain — being stretched, bent, or compressed,” Dawson explained. “You can think of it as a stretchable pipe with water running through it. As you stretch the pipe, it becomes longer but also narrower, restricting the flow”.&nbsp;Now, apply this same principle to the traces in the control glove, which are made with stretchable conductive ink. As the traces stretch, they become longer and narrower, restricting the flow of electricity — it changes the resistance. The accompanying transmitter board interprets this change in resistance, and that information can then be used to control other devices — like the robotic hand.&nbsp;Dawson’s biggest hurdle was designing the strain gauges for the control glove — finding the optimal procedure that would allow the gauges to fully stretch without losing conductivity completely, but still have a measurable change in resistance. In the end, he found that thicker traces resulted in more consistent continuity while stretching.&nbsp; <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1704362901155-6373badbe9c8bd5af2c5faf5_Robot+Glove_1_optimized.gif" style="width: 300px;" class="fr-fic fr-dib"> <h2 id="isPasted">Imagine the possibilities</h2>Stretchable conductive ink gives you the ability to create a strain gauge on almost any stretchable material, and that has incredible potential to drive innovation. Industries across the board, from manufacturing to healthcare to wearable technology, are on the threshold of major change, thanks to the emerging field of&nbsp;<a href="https://www.idtechex.com/en/research-article/flexible-hybrid-electronics-future-of-flexible-printed-circuit-boards/20603" target="_blank">flexible hybrid electronics</a>.&nbsp;Over the course of just&nbsp;two months, Dawson was able to successfully design and build a fully-functional robotic hand that mimics the movements of an operator wearing a control glove. Imagine what could be accomplished by seasoned researchers and engineers.When asked about the potential applications for this kind of technology, Dawson lit up. “Why stop at just a hand? Why not use this technology to have a whole arm that mimics my own? Possibly even a full robot that mirrors my every move, and the majority of that could be achieved using similar techniques that were used for just the hand,” Dawson exclaimed. “The most exciting thing for me was the realization of the true possibilities that NOVA brings to the world of stretchable and flexible electronics”.This technology could have life-altering and transformative potential in industries across the board — from biomedical applications and wearables to prosthetics. “I could see this technology being used in workplaces that present a high risk of injury, but still require the delicate handling of a human. For instance, if some work needs to be done on a machine but the area is radioactive or unsafe, the worker could remotely control this hand to perform the task, while staying in a safe location.” Dawson said. “This project could also be scaled up or down in size to meet specific requirements, for instance, you could have a small robotic hand do work inside a machine, in spaces where a human could not fit, being controlled by an operator outside the machine”.&nbsp;You can read the finer details of Dawson’s work with&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6362829f808f8037c5b5e9d9_NOVA-0094%20-%20Robot%20Glove%20Case%20Study_FINAL%20(4).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">NOVA</a> and the&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/636282bb3c1c464ccd273de1_VONE-0001%20-%20Robot%20Glove%20Case%20Study_FINAL%20(3).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">V-One</a> by checking out&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6554cb7c0b836d9be99ef8fa_2022_NOVA%20robot%20glove%20case%20study.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">case studies</a> on this project.

Dawson Martian — a second-year undergrad at the University of Waterloo — came up with a project idea that would put both the V-One and NOVA to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.

Building a Remote Controlled Robotic Hand with Strain Gauges

The recent advances in Large Language Models (LLM) have made a lot of noise recently – everybody is using ChatGPT. Robotics is no exception; there has been a lot of talk about how to integrate them in various kind of robotic use cases; in this article, we will talk about their applications to social robots like NAO and Pepper.&nbsp;<h2>Social Robots? &nbsp;</h2>Social robots are designed to be used in a places, interacting with people, and as such are a natural fit for conversation-oriented language models. But they have been used for years before that, and the field of human-robot interaction is well-developed.&nbsp;Compared to a GUI chat interface like ChatGPT, social robots have to deal with extra uncertainty linked to speech recognition (the input the langauge model received might not be what the user intended to say), and handling several users speaking, some of which might not even intend to speak to the robot.&nbsp;<h2>Improving what the robot understands&nbsp;</h2>One advantage LLMs have is in understanding open-ended instructions. There are several challenges in understanding voice commands:&nbsp;<ol><li>Speech recognition: turning the raw sounds into text; how effective this is is going to depend of how the user speaks (accents, slurred speech, volume, clarity of enunciation), the characteristics of the microphone, surrounding sound, etc.&nbsp;</li><li>Variety of expressions: In practice, people have a huge variety of ways of expressing even simple things (For example: "I'm looking for an ATM", "Would you show me the nearest ATM?", "Is there an ATM around here?", "Where's the nearest ATM?","Do you know where I can withdraw some money?", "Where can I withdraw cash around here?", "Is there an ATM in this building?" etc.); and handling all of those properly usually requires some bespoke customisation via a dialogue engine like dialogflow or qichat.&nbsp;</li><li>Context: In addition to the explicit information contains in a question like the ATM example above, some information comes from what was previously discussed ("How much does one cost?" "How can I get there?"), or from gestures such as pointing ("Is it that way?" "What's that?"); keeping track of that information usually requires more bespoke logic related to keeping track of users.&nbsp;</li></ol>These are all challenges that could be handled previously with some work; but LLMs introduced a great increase in quality: they can be very good at handling a wide variety of expressions out of the box. They are also very good at taking into account any context included in the conversation log (typically sent alongside the prompt); this usually includes previous conversation but (with a bit more work) can include any information that can be infered from the sensors (for example, by using an object recognition model to turn what the robot sees into a text description).&nbsp;One more surprising improvement when using LLMs is on speech recognitions – more specifically, even if there are mistakes in the speech recognition (that can be misheard words, but also homophones like their/there/they're), the LLM will often use context and common sense to handle the input appropriately anyway, giving the impression of a robot that hears better.&nbsp;<h2>Improving how the robot answers&nbsp;</h2>One obvious output is Speech, and plugged into a speech-to-text engine, a LLM can easily create a conversational robot.&nbsp;However, this brings a few new issues:&nbsp;<ul><li>One notorious ones is hallucinations, a problem that's far from solved for LLMs. The robot can understand a wider variety of inputs, his answers to those might be confident but highly inaccurate – if there's a word he doesn't understand, he might decide to invent a fanciful product description; if an unfamiliar name is given, he might make someone up. This can be mitigated, but at the risk of making the robot repetitive or boring – it's a fine balance, and it might depend on the use case.&nbsp;</li><li>A less obvious one is speech that's too verbose. If ChatGPT talks too much, the user can just skim the paragraphs; but a speaking robot takes time and can be much more annoying. This can be improved with better prompting, with pretraining etc. as well as UX tweaks (e.g. a way of interrupting the robot) but requires careful attention. More generally, the LLM's output might sometimes not be as good quality as hand-crafted one.&nbsp;</li></ul>However, a robot is more than a smart speaker – it might have arms, wheels, etc. and the LLM can be made to control those, either directly or from a library of presets (e.g. navigation waypoints) - in effect, the output of the LLM is not just speech but actions.&nbsp;<h2>How to deal with it the best&nbsp;</h2>How to deal with hallucinations and other problems may depend of the use case:&nbsp;<ul><li>For entertainment purposes, the hallucinations may not be a problem, and can even be part of the fun, as long as that is well-presented to the user.&nbsp;</li><li>For information purposes, a choice needs to be made between using LLM output and risking hallucinations, or using fixed, well-prepared outputs.&nbsp;</li><li>One should also keep in mind other non-speech actions and see how many of those can be controlled by the LLM.&nbsp;</li></ul>Apart from the technical performance on input and output, the largest strengths of LLMs is flexibility – adapting the robot to a new use case, or to a specific context for a new use case, can be as simple as editing a prompt, and produces a behavior of decent quality.&nbsp;This allows for much quicker iterations. &nbsp;<h2>Further directions&nbsp;</h2>In robotics (social or not) and elsewhere, engineers, researchers and tinkerers all over the world are experimenting with interesting ways to use LLMs. We've covered a few we have experience with here, but there are many more directions that could be applicable, such as:&nbsp;<ul><li>integrating vision models to plug it with what the robot sees, to make it even more aware of the context&nbsp;</li><li>making the robot proactively decide what to do (not just respond to what he heard, but decide to e.g. walk up to such-and-such a person and engage conversation)&nbsp;</li><li>giving the robot a long-time memory&nbsp;</li></ul>LLMs offer great possibilities for social robots if used well, and it will be interesting to observe how the field evolves in the coming years – in the coming months even.&nbsp;

A Pepper robot showing a ChatGPt-type dialogue

Lessons learnt from integrating ChatGPT in Pepper and NAO

Bringing the LLM revolution to social robots

On November 3, Unitree Robotics released the new industrial quadruped robot B2, continuing to lead the global quadruped robot industry application.

Unitree Robotics Releases Industrial Quadruped Robot B2, Breaking Through Limits with Hyper Evolution!

On 12 July, Unitree unveiled its latest quadruped robot - the Unitree Go2, which features embodied AI technology, standard 4D ultra-wide LiDAR and GPT empowerment. This product is expected to revolutionize the consumer-grade quadruped robots industry, leading to a transformation of the sector.  

Revolution: A New Quadruped Robot of Embodied AI

<h2 id="isPasted">How Forward AM and OECHSLER Enable Robots and Humans to Collaborate Safely at Profitable Speed</h2>In today&rsquo;s modern factories around the globe, many production steps are automated and carried out by robots.<div data-parent="true" data-section="2"><div data-imgready="true"><div data-imgready="true"><hr>Humans sharing a workspace and collaborating with robots has become a reality that companies and their smart factories just can&rsquo;t do without, as it allows flexible manufacturing in variable batch sizes with maximum efficiency. In short, successful human-machine interaction is key for future-proof manufacturing.</div>With Additive Manufacturing (AM), it is now possible to create new functionalities for robots and to unlock novel possibilities. The AM technology is perfectly suited for the production of small, customized batches for robotic parts like grippers &ndash; and it is ideally capable to integrate internal ducts, as air pressure and vacuums play an important role in this industry.</div></div><div data-parent="true" data-section="3"><div data-imgready="true"><h2>The challenge &ndash; safe human-robot collaboration at profitable speed</h2>Collaborative robots (short: cobots) become increasingly present in manufacturing. Designed to work alongside humans in shared areas, they work together &lsquo;hand in hand&rsquo; &nbsp;&ndash; which makes it crucial to ensure safety by avoiding collisions of cobot and human co-worker.This is prevalently achieved by inherent optical sensors which trigger an immediate stop of the cobot in case of unexpected contact: whenever a human co-worker enters the defined warning zone, the system lowers its speed to a minimum and if the person keeps approaching, the system stops &ndash; only after the individuum departs, the system accelerates back to full speed. It&rsquo;s obvious: this results in highly limited processing speeds of the cobot and thus higher cost and lower productivity for the manufacturing company. These interruptions in the workflow may lead to unstable cycle times as well, and foremost &ndash; it is absolutely contrary to the intention of cobots being deployed flexibly and easy to integrate as a plug-and-play system: To achieve an appropriate reaction to an approaching human co-worker, cobots have to be programmed accordingly; always depending on the cobot&rsquo;s task, its exact form of cooperation with the human worker, and its physical position in the factory. Whenever the cobot is deployed at another workstation on the assembly line, or performs different tasks and movements, its programming has to be adapted. This limited flexibility only allows pre-defined reactions of the cobot.</div></div><div data-parent="true" data-section="4"><div data-imgready="true"><h2>The solution &ndash; a protective lattice skin for cobots made of Forward AM&rsquo;s Ultrasint&reg; TPU01</h2><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg2MTI0MTYzMjMxLWZvcndhcmQtYW0tdXNlLWNhc2UtY29ib3RpY3Mtcm9ib3NraW4tMy0xLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><div id="isPasted"><div>The cobot Kuka iiwa with mounted RoboSkin made from lattice structures printed with Ultrasint&reg; TPU01. (Source: OECHSLER).&nbsp;</div></div> A new approach by Forward AM and OECHSLER, a leading German polymer processing company, enables humans and cobots to share a workspace without any safety risks and manufacturing companies to keep productivity constantly high.How? By adding a new layer to the current solution &ndash; literally. A thin, high cushioning second &lsquo;skin layer&rsquo; is wrapped around the cobot&rsquo;s joints. Using Forward AM&rsquo;s <a href="https://forward-am.com/find-material/powder-bed-fusion/ultrasint-tpu-line/tpu01/" target="_blank">Ultrasint&reg; TPU01</a> to 3D print the RoboSkin enables a flexible and individualized lattice structure. <a href="https://forward-am.com/find-material/powder-bed-fusion/ultrasint-tpu-line/tpu01/" target="_blank">Ultrasint&reg; TPU01</a> is a thermoplastic polyurethane powder specifically designed for HP 5200 series Multi Jet Fusion printers. This flexible material is ideal for producing parts requiring high shock absorption at excellent flexibility depending on the functional requirements of the application, making it the perfect match for the cobot&rsquo;s protective skin and its cushioning function.The benefits of the lattice RoboSkin are manifold. Putting safety first, it reduces the risk of injuries to human workers interacting with the cobot. At the same time, production speed can be increased strikingly: Since the collision forces and pressures are highly damped by the RoboSkin, the cobot&rsquo;s speed can accordingly be higher up to 150 percent&ndash; thus enabling increased productivity for the manufacturer.The overall handling of the skin is easy: Because the RoboSkin works as a mechanical collision buffer, it requires no additional sensors, just the TPU layer that takes up only minimal installation space and is mounted with an easy system. Moreover, the lattice design of the skin is easily, rapidly and cost-effectively adaptable to every cobot type, thanks to the advantages of 3D printing. In production halls, cobots are often deployed at different workstations. The RoboSkin eliminates the need for dismantling, re-assembly and re-calibration, thus saving time and cost.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg2MTI0MTAzOTE3LWZvcndhcmQtYW0tdXNlLWNhc2UtY29ib3RpY3Mtcm9ib3NraW4tMS0xLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><div id="isPasted"><div>Thanks to Additive Manufacturing and its design freedom, the RoboSkin is easily adaptable to different shapes and functionalities of each cobot. This image shows a RoboSkin for the axes 2 and 3 of the cobot Kuka iiwa created with a lattice structure printed with Ultrasint&reg; TPU01. (Source: OECHSLER).&nbsp;</div></div> </div></div><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg2MTI0MTE4OTI5LWZvcndhcmQtYW0tdXNlLWNhc2UtY29ib3RpY3Mtcm9ib3NraW4tMi0xLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Different axis, different RoboSkin &ndash; this image shows a cover for the axis 4 of the cobot Kuka iiwa (Source: OECHSLER).&nbsp;<blockquote><div data-imgready="true" id="isPasted">&ldquo;The RoboSkin solves a complex problem with a seemingly simple solution, enabling the end user to optimize the productivity of the human-robot-collaboration application. With AM, we can offer a configurable soft skin according to the safety needs at reasonable pricing &ndash; and which is reliable, easy to handle, lightweight, space-saving and permeable to ensure profitable performance.&rdquo;Meike Wischgoll, Program Manager, OECHSLER</div></blockquote><div data-parent="true" data-section="4"><div data-imgready="true"><div data-parent="true" data-section="6"><div data-imgready="true">Together, OECHSLER and Forward AM are enabling a solution that delivers a significant increase in safety and productivity. Harnessing the advantages of AM, human workers and cobots can work together at minimum risk of injury while maximizing productivity.</div></div></div></div>

Humans sharing a workspace and collaborating with robots has become a reality that companies and their smart factories just can’t do without, as it allows flexible manufacturing in variable batch sizes with maximum efficiency. In short, successful human-machine interaction is key for future-proof manufacturing.

Protective Lattice Layer for Cobots

Although robots cannot replace human caregivers, they can provide support so that caregivers have more time to provide the personal, human touch. The German Aerospace Center (Deutsches Zentrum f&uuml;r Luft- und Raumfahrt; DLR) is testing various robotic care assistants in a series of projects. Although these robots were developed for spaceflight, they can also help with essential tasks on Earth.What activities will robots be allowed to take on in care in the future? How will it be ensured that people and their needs are always at the heart of technological advances? How can robotic assistance systems be used in retirement homes, private households and hospitals? Researchers from the <a href="https://www.dlr.de/content/en/institutes/institute-of-robotics-and-mechatronics.html" target="_blank" title="Institute of Robotics and Mechatronics">DLR Institute of Robotics and Mechatronics</a> in <a href="https://www.dlr.de/content/en/sites/oberpfaffenhofen.html" target="_blank" title="Oberpfaffenhofen">Oberpfaffenhofen</a> are investigating these questions in the SMiLE (Service Robotics for People in Living Situations with Limitations; Servicerobotik f&uuml;r Menschen in Lebenssituationen mit Einschr&auml;nkungen) Project. &quot;Robotic care assistants are intended, on the one hand, to relieve the burden on care staff and, on the other, to provide those affected with a greater degree of independence in their everyday lives. In this way, the robots can make a decisive contribution to improving the quality of life and communication with relatives and caregivers,&quot; explains project manager J&ouml;rn Vogel.<h2>Three assistance systems: Rollin&#39; Justin, EDAN and HUG</h2>A humanoid service robot can serve as a helping hand for an elderly person. A person with severe mobility impairments is more likely to use a robotic chair. There is also an interface between the systems. Several systems are being prepared for optimal human assistance.<ul><li><a href="https://www.dlr.de/rm/en/desktopdefault.aspx/tabid-11427/#gallery/35411" target="_blank" title="External link Robot Rollin Justin">Rollin&lsquo; Justin</a> is a humanoid, two-armed, mobile home assistance robot. It monitors its environment via sensors and cameras and evaluates this information. Its lightweight arms enable sensitive interaction with the environment. Rollin&#39; Justin uses artificial intelligence (AI) to plan its workflows autonomously.</li><li><a href="http://www.dlr.de/rm/en/desktopdefault.aspx/tabid-11670/#gallery/28208" target="_blank" title="External link EDAN">EDAN</a> is a wheelchair with a lightweight robotic arm and hand. It is moved with a joystick or via muscle signals measured directly on the surface of the person&#39;s skin. EDAN and Rollin&#39; Justin can also be moved by relatives via smartphones or tablets. Remote control from a care control centre is possible.</li><li><a href="https://www.dlr.de/rm/desktopdefault.aspx/tabid-11704/#gallery/34276" target="_blank" title="External link Roboter HUG">HUG</a> is a haptic input station with two lightweight arms for remote robot control. HUG measures human movements, uses them as signals, and relays them in this way. At the same time, the user feels the exact forces that the robot senses. In this way, EDAN and Rollin&#39; Justin can also be controlled easily and intuitively.</li></ul>In everyday life, for example, EDAN with its robotic arm could help to significantly increase the independence of people with severe motor impairments. Rollin&#39; Justin could perform pick-up and drop-off services in a care facility. For unusual or difficult tasks, trained caregivers from a control centre would be able to quickly assist those in need of care via HUG.<hr><h3 id="isPasted">Focus on safe human-robot interaction </h3>Robotic technologies have long been part of our everyday lives. Simple systems include vacuum cleaner robots or autonomous lawn mowing robots. The use of mobile service robots represents a next step. Equipped with arms and hands, these robots can react to their environment. These developments are possible because modern lightweight robot technology offers safe human-robot interaction. Classic industrial robot arms, such as those used in the automotive industry, must always be operated behind barriers for safety reasons. Modern lightweight robots, on the other hand, are sensitive to physical contact with humans or the environment.The <a href="https://www.dlr.de/rm/en/desktopdefault.aspx/tabid-12464/21732_read-44586/" target="_blank" title="External link Story of the Lightweight Robot LBR">LBR Light-weight Robot</a> developed at DLR almost 20 years ago represented an important milestone in this field. It can sense and control its interaction with the environment through joint torque sensors. This additional information makes it possible for the robot to behave in an actively compliant and thus safe manner, similar to a human arm &ndash; an essential requirement for use in the direct environment of humans.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1661775337601-success-story-lightweight-robot-lbr.jpg" style="width: 766px;" class="fr-fic fr-dib">The first prototype (left) of the LBR was developed in the mid-1990s. The technology of the third generation from 2003 (right) was licensed to the company KUKA. Originally intended as a lightweight robotic arm for space applications, DLR robotics technology forms the basis for many other robotic systems and also serves as a pioneer for safe human-robot interaction. Credit: DLR (CC BY-NC-ND 3.0)

Although robots cannot replace human caregivers, they can provide support so that caregivers have more time to provide the personal, human touch.

Robots for people in need of care

The retail world is changing. In today&rsquo;s highly competitive environment, retailers are leveraging cutting-edge technologies to boost efficiency and run their businesses more effectively.The emergence of artificial intelligence (<a href="https://venturebeat.com/2022/06/15/what-is-artificial-intelligence/" target="_blank">AI</a>) in the retail industry has <a href="https://www.forbes.com/sites/forbestechcouncil/2020/08/21/seven-ways-artificial-intelligence-is-disrupting-the-retail-industry/" target="_blank">transformed&nbsp;</a>how businesses engage customers, tailor marketing campaigns, and manage their inventory. This approach also helps reduce waste and operational inefficiencies.&nbsp;According to <a href="https://www.marketwatch.com/press-release/retail-automation-market-2022-increase-in-demand-expected-in-forecasted-year-to-2031-product-specification-consumption-status-industry-share-top-manufacture-with-types-and-applications-2022-06-08?mod=search_headline" target="_blank">MarketWatch</a>, the global retail automation market was valued at $8.96 billion in 2016 and is forecasted to be worth as much as $23.1 billion by the end of 2024. Furthermore, the <a href="https://www.prnewswire.com/news-releases/artificial-intelligence-ai-robot-market-is-predicted-to-rise-at-a-cagr-of-20-5-during-the-forecast-period-observes-tmr-study-301574923.html" target="_blank">AI robot market</a> is expected to be worth $ 52.63 billion by 2031.As such, AI-powered automation and robotics are a business imperative for retailers who want to maintain a competitive advantage in the marketplace.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjU5OTgxNjIyNzcxLTE2NTk5ODE2MjI3NzEucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">What problems does an AI robot solve in the retail sector?</h2>Retail automation has the potential to optimize customer experiences and drive revenue. However, this doesn&rsquo;t mean that <a href="https://hbr.org/2021/11/automation-doesnt-just-create-or-destroy-jobs-it-transforms-them" target="_blank">robots will replace humans</a>. Instead, they will work together to maximum effect.&nbsp;Humans and retail robots together solve four key issues in the retail industry:<h3>1. Data analytics challenges</h3>Smart algorithms are much better at collecting, processing, and analyzing vast amounts of data from disparate sources in real-time (a feat forever unmatched by human counterparts). It&rsquo;s also the only way to collect and process data in real-time for analytics. As human and machine collaboration becomes the norm, people that work together with robots will earn more (than those who don&rsquo;t). Similarly, businesses that leverage this approach will receive a maximum return on investment.<h3>2. Pricing (human) errors</h3>More often than not, digital and physical commerce channels have diverse agendas and follow different methods. While it may seem easier to operate each of them as a separate business unit, this approach often creates more complexity, including on-shelf pricing errors. Trying to match up the online shopping experience is becoming more challenging for brick-and-mortar retailers. &nbsp;Price discrepancies between the online and physical stores often occur because of human error, data entry and update errors, unapplied promotions or discounts, and various variables specific to the business, location, and so on. Even if online retail grows exponentially, brick-and-mortar retailers are here to stay. However, to remain competitive, they must strive to eliminate potential mistakes caused by human error, considering that as many as 73% of customers stated that <a href="https://www.pwc.com/us/en/services/consulting/library/consumer-intelligence-series/future-of-customer-experience.html" target="_blank">customer experiences drive purchasing behavior</a>.Retail automation can help quickly resolve this issue by alerting staff about price discrepancies. This approach ensures that offline customers have smooth and enhanced shopping experiences. By scanning shelves in real-time, AI-powered robots quickly alert staff to misplaced items and products missing price tags.&nbsp;<h3>3. Inventory management and supply chain challenges&nbsp;</h3>Retail automation keeps pricing up to date and helps avert potential <a href="https://www.magestore.com/stockout-in-retail-stores/" target="_blank">stockout</a> situations. Stockout situations lead to lower customer loyalty, higher operational costs, and higher labor costs (as more staff spend time on inventory activities).Retail automation guarantees that shelves are always stocked, and customers aren&#39;t left disappointed. By analyzing in-store data in real-time while considering external factors and past purchasing patterns, retailers can rethink their old supply chains and quickly adjust their stocks to meet consumer demand. &nbsp;<h3>4. Brand compliance</h3>Some retailer-vendor agreements come with special requests, especially when it comes to where certain products are displayed within a store. In this scenario, retail automation can help quickly detect inconsistencies in product placement to ensure brand compliance.<div id="_com_1" language="JavaScript"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjU5OTg0NzgzMjgzLURTQ18wNTU2LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 610px;" class="fr-fic fr-dib">ERIS quickly alerts staff to misplaced items &amp; price tag errors.</div><h2 id="isPasted">Why is there a need for an AI-powered robot?</h2>Robots like the Effective Retail Intelligent Scanner or <a href="https://www.adaptarobotics.com/robots/eris/" rel="noopener noreferrer" target="_blank">ERIS</a> are necessary to scan all the products within stores and warehouses quickly and accurately. Physically attempting to take stock, check price tags for errors, and look for misplaced items, it could take days or even weeks to complete the task.In contrast, ERIS with human operators can effectively scan the whole premises to detect errors and anomalies within minutes (depending on the size of the establishment and the number of products available on-premises).This approach helps optimize employee efficiency and enables data-driven decisions by collecting accurate pricing and inventory data that marketing and supply chain management teams can use.&nbsp;With solutions like ERIS<a href="#_msocom_3" id="_anchor_3" language="JavaScript" name="_msoanchor_3" target="_blank"></a>, potential sales aren&#39;t lost to pricing errors or stockouts. Return on investment will also be complemented by customer satisfaction, customer loyalty, and much more. As a human operator is always present, they can solve potential errors on the spot. This approach helps mitigate the risk of miscommunications and provides opportunities to prioritize and escalate issues as needed. &nbsp;ERIS can also help improve staff morale by taking over the tedious and time-intensive task of detecting pricing errors and managing stock effectively. Although a human staff member will work in collaboration with the smart robot, ERIS is doing most of the heavy lifting.However, preconceived notions about AI replacing humans continue to derail the adoption of retail automation technologies. But, just like chatbots help customer service representatives by quickly processing simple queries, ERIS makes life a little easier for human colleagues.&nbsp;Retail automation is the new frontier. It will help businesses better engage their customers, optimize inventory, avoid pricing errors, and maintain business relevance.Adapta Robotics developed ERIS to solve an age-old retail problem concerning human error and inventory management. ERIS is an innovative and powerful machine, a fast shelf auditing solution that smooths out the transition towards retail automation.<div id="_com_4" language="JavaScript"> </div>

Human & retail robots together solving key issues in the retail industry

In a highly competitive environment, retailers are leveraging cutting-edge technologies to boost efficiency and run their businesses more effectively. AI-powered robots are transforming the retail sector by optimizing pricing strategies, inventory management, and more.

How AI-powered robots are transforming the retail industry

Virtual human experience has been on the rise to train real-human interpersonal scenarios for doctor-patient interactions. Visual feedback on involuntary pain facial expressions in response to physical inputs is an essential source of information for skilled physicians and medical students. Standardized patients have the highest level of fidelity for simulating patient behavior to enable self-learning and improve training efficiency. But this methodology is time-consuming and requires diverse patient identities in terms of ethnicity and gender to achieve cultural competence. Even though robotic simulators offer lower fidelity than standardized patients, they can simulate a greater variety of medical conditions and respond to several physical stimuli such as movement, haptic, visual, and auditory feedback.&nbsp;Some commercially available medical robotic simulators, such as Pediatric HAL, are currently the most advanced pediatric patient simulators capable of simulating emotions through dynamic facial expression, movement, and speech. They offer gender and skin color variations, but the options are limited and cannot be easily changed in the training session. Other virtual simulations, such as Virtual Patients, offer different demographics and medical history with clinical scenarios but are often only verbal or dialogue-based. Combining these features with virtual and physical robotic simulators will enhance the training experience for clinicians.&nbsp;A team of researchers affiliated with Dyson School of Design Engineering, Imperial College London, and the School of Psychology and Neuroscience, University of Glasgow, have collaborated to model dynamic pain facial expressions using a data-driven perception-based psychophysical method combined with visual-haptic inputs [1].&nbsp;<h2 dir="ltr">Robotic system for simulating dynamic facial expressions</h2><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651253396196-Schematics+of+the+Robotic+Simulator.png" style="width: 800px;" class="fr-fic fr-dib" alt="Schematics of the Robotic Simulator">Schematics of the Robotic Simulator [Image Credit: Research Paper]<div is-sapling-overlay="true" style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 383px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>In 2021, the team developed a robotic face MorphFace [2], a 3D physical-virtual hybrid face to represent the pain expressions of patients, based on the MakeHuman and FACSHuman systems that can simulate six human avatars. The system can be integrated with capturing devices such as force sensors for real-time facial expressions. The modifications to the existing system to address the limitation of pain facial expressions from the various demographic backgrounds are presented in the paper titled &ldquo;Simulating dynamic facial expressions of pain from visuo‑haptic interactions with a robotic patient.&rdquo;&nbsp;&ldquo;We refine the paradigm by asking participants to perform actions (abdominal palpation) that would directly affect the exhibition of such facial expressions, thus enabling the modeling of facial expressions of pain exhibited within a given medical context and specific user interactions: pain received by the patient and subsequently the facial expressions of pain arise from palpation of the abdomen performed by the participant,&rdquo; the team explains. &ldquo;Thus, we anticipate that our system and procedure can be used to create dynamic pain facial expression simulation models of interactive physical examination procedures for diverse patient identities, ultimately providing bespoke medical training solutions to reduce erroneous judgments in individual medical students due to perceptual biases.&rdquo;The participants performed palpation action on a block of silicon abdomen phantom placed on a load cell. The applied force is then captured by the load cell and sent to Unity3D through an embedded device, Arduino Uno, at a baud rate of 9600. The pain expression generation function is implemented in C# language the describes the relation between palpation force and the action unit (AU) activity intensity. The force triggered a real-time display of six pain expressions on the robotic face, MorphFace, controlled by the rate of change &beta; and activation delay &tau;. Each participant then rated the appropriateness of the facial expression on a 4-point scale from &ldquo;strongly disagree&rdquo; to &ldquo;strongly agree&rdquo; within a time limit of 6.5 seconds.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651253426257-Experimental+Plan+for+a+Robotic+Simulator.png" style="width: 600px;" class="fr-fic fr-dib" alt="Experimental Plan for a Robotic Simulator">Experimental Plan for the Robotic Simulator [Image Credit: Research Paper]<div is-sapling-overlay="true" style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 480px; height: 39px; width: 602px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div><h2 dir="ltr">What do the results have to say?</h2>The results showed that all the participants comprised a higher rate of change and shorter delay for upper face AUs than those in the lower face. The results suggest that gender and ethnicity biases affect palpation strategies and the perception of pain-related expressions displayed on MorphFace. &ldquo;We anticipate that our approach will be used to generate physical examination models with diverse patient demographics to provide focused training and address errors,&rdquo; the team concludes.&rdquo; These findings highlight the usefulness of visual-haptic interactions with a robotic patient as a method to quantify differences in behavioral variables relating to a medical examination with diverse participant groups and to assess the efficacy of future bespoke interventions aimed at mitigating the effects of such differences.&rdquo;The research paper is published under Scientific Reports in <a href="https://www.nature.com/articles/s41598-022-08115-1" target="_blank">open-access terms</a>. The datasets for the research paper are published for public viewing in the <a href="https://osf.io/q64x2/?view_only=025d443f148d41778c82761088ac0c1e" target="_blank">OSF repository</a>.<h2 dir="ltr">References</h2>[1] Tan, Y., R&eacute;rolle, S., Lalitharatne, T.D. et al. Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient. Sci Rep 12, 4200 (2022). <a href="https://doi.org/10.1038/s41598-022-08115-1" target="_blank">https://doi.org/10.1038/s41598-022-08115-1</a>&nbsp;[2] T. D. Lalitharatne et al., &quot;MorphFace: A Hybrid Morphable Face for a Robopatient,&quot; in IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 643-650, April 2021, DOI: 10.1109/LRA.2020.3048670.

Student receives real-time facial expressions from the robotic platform [Left– virtual];  Experiment setup [Right– actual]

Researchers modeled dynamic facial expressions of pain using a data-driven perception-based psychophysical method with visual-haptic interaction of users.


Human-Robot Interaction

Latest Posts