Robotic Arms

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/96f146e3-d84e-4bc9-8fa4-af61e7ce87ed?dark=false" class="fr-draggable" data-gtm-yt-inspected-9="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>The full article will continue below.<hr>Pursuing a viable alternative to invasive brain-computer interfaces (BCIs) has been a continued research focus of Carnegie Mellon University’s <a href="https://www.cmu.edu/bme/helab/index.html" rel="noopener" target="_blank">He Lab</a>. In 2019, the group used a noninvasive BCI to successfully demonstrate, for the first time, that a mind-controlled robotic arm had the ability to continuously track and follow a computer cursor. As technology has improved, their AI-powered deep learning approach has become more robust and effective. In new work published in PNAS Nexus, the group demonstrates that humans can control continuous tracking of a moving object all by thinking about it, with unmatched performance.Noninvasive BCIs bring a host of advantages, in contrast to their invasive counterparts (think Neuralink or Synchron). These include increased safety, cost-effectiveness, and an ability to be used by numerous patients, as well as the general population. Amid these benefits, however, noninvasive BCIs face challenges because their recordings are less accurate and difficult to interpret.<iframe width="640" height="360" src="https://www.youtube.com/embed/EoKHw6bdoZs?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="917213689" title="Bin He: Neuroengineering and Dynamic Brain Mapping">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</iframe>In He’s <a href="https://academic.oup.com/pnasnexus/article-lookup/doi/10.1093/pnasnexus/pgae145" rel="noopener" target="_blank" id="isPasted">recent study</a>, a group of 28 human participants were given a complex BCI task to track an object in a two-dimensional space all by thinking about it. During the task, an electroencephalography (EEG) method recorded their activity, from outside the brain. Using AI to train a deep neural network, the He group then directly decoded and interpreted human intentions for continuous object movement using the BCI sensor data. Overall, the work demonstrates the excellent performance of non-invasive BCI for a brain-controlled computerized device.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE0NTE3ODg4OTAzLTA0MzAtZW0tbm9uaW52YXNpdmUtYmNpLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib">Source:&nbsp;He Lab“The innovation in AI technology has enabled us to greatly improve the performance versus conventional techniques, and shed light for wide human application in the future,” expressed <a href="https://engineering.cmu.edu/directory/bios/he-bin.html" rel="noopener" target="_blank" id="isPasted">Bin He</a>, professor of <a href="https://www.cmu.edu/bme/" rel="noopener" target="_blank">biomedical engineering</a> at Carnegie Mellon University.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1714517922793-0430-em-2-noninvasive-bci.gif" style="width: 100%px;" class="fr-fic fr-dib">This video shows the cursor and target trajectories throughout a single 60-second trial with lag adjusted. The randomly moving target is represented by a yellow circle, while the cursor that the subject controlled using the BCI system is shown as a blue circle. A human subject controlled the blue cursor using the deep learning-based BCI decoder just by thinking about it. Source:&nbsp;He LabMoreover, the capability of the group’s AI-powered BCI suggests a direct application to continuously controlling a robotic device.“We are currently testing this AI-powered noninvasive BCI technology to control sophisticated tasks of a robotic arm,” said He. “Also, we are further testing its applicability to not only able-body subjects, but also stroke patients suffering motor impairments.” In a few years, this may lead to AI-powered assistive robots becoming available to a broad range of potential users.To this end, motor-impaired patients who are suffering from spinal cord injury, stroke, or other movement impairment, but do not want to receive an implant, stand to benefit immensely from research in this vein. “We keep pushing noninvasive neuroengineering solutions that can help everybody,” added He.This work was supported in part by the National Institute of Neurological Disorders and Stroke, National Center for Complementary and Integrative Health, and National Institute of Biomedical Imaging and Bioengineering.Other collaborators on the PNAS Nexus paper include the first authors Dylan Forenzo, BME Ph.D. student, and Hao Zhu, former BME Masters student; Jenn Shanahan, former lab technician; and Jaehyun Lim, BME undergraduate student.

Achieving a noteworthy milestone to advance noninvasive brain-controlled interfaces, researchers used AI technology to improve the decoding of human intention and control a continuously moving virtual object all by thinking about it, with unmatched performance.

Refined AI approach improves noninvasive BCI performance

In this episode, we talk about how a fashion company spun-off from MIT has developed the world’s first truly one-size-fits-all garment.

Podcast: Novel Fabrics + Robots -- Sustainable One-size-fits-all Fashion Revolution

A pick and place robot in an industrial setup

Understanding the inception, evolution, composition, and types of pick and place robots and their impact on various industrial sectors 

Pick and Place Robots: An In-Depth Guide to Their Functionality and Applications

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/7a6ed573-f46a-40bb-b396-d38d1207aa2e?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Until recently, bespoke tailoring — clothing made to a customer’s individual specifications — was the only way to have garments that provided the perfect fit for your physique. For most people, the cost of custom tailoring is prohibitive. But the invention of active fibers and innovative knitting processes is changing the textile industry.“We all wear clothes and shoes,” says Sasha MicKinlay MArch ’23, a recent graduate of the MIT Department of Architecture. “It’s a human need. But there’s also the human need to express oneself. I like the idea of customizing clothes in a sustainable way. This dress promises to be more sustainable than traditional fashion to both the consumer and the producer.”McKinlay is a textile designer and researcher at the Self-Assembly Lab who designed the <a href="https://selfassemblylab.mit.edu/4d-knit-dress" target="_blank">4D Knit Dress</a> with Ministry of Supply, a fashion company specializing in high-tech apparel. The dress combines several technologies to create personalized fit and style. Heat-activated yarns, computerized knitting, and robotic activation around each garment generates the sculpted fit. A team at Ministry of Supply led the decisions on the stable yarns, color, original size, and overall design.“Everyone’s body is different,” says Skylar Tibbits, associate professor in the Department of Architecture and founder of the Self-Assembly Lab. “Even if you wear the same size as another person, you're not actually the same.”<iframe width="640" height="360" src="https://player.vimeo.com/video/893959788" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;&nbsp;&nbsp;&nbsp;</iframe>4D Knit Dress: Transforming Style. Video: Self-Assembly Lab<h2 id="isPasted">Active textiles</h2>Students in the Self-Assembly Lab have been working with dynamic textiles for several years. The yarns they create can change shape, change property, change insulation, or become breathable. Previous applications to tailor garments include <a href="https://selfassemblylab.mit.edu/active-textile-tailoring" target="_blank">making sweaters</a> and <a href="https://news.mit.edu/2022/form-fitting-face-mask-lavender-tessmer-0718" target="_blank">face masks</a>. Tibbits says the 4D Knit Dress is a culmination of everything the students have learned from working with active textiles.McKinlay helped produce the active yarns, created the concept design, developed the knitting technique, and programmed the lab’s industrial knitting machine. Once the garment design is programmed into the machine, it can quickly produce multiple dresses. Where the active yarns are placed in the design allows for the dress to take on a variety of styles such as pintucks, pleats, an empire waist, or a cinched waist.“The styling is important,” McKinlay says. “Most people focus on the size, but I think styling is what sets clothes apart. We’re all evolving as people, and I think our style evolves as well. After fit, people focus on personal expression.”<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEwMTM1NjY2MDgwLU1pbmlzdHJ5X0ZJTkFMX2RyZXNzZXMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 100%px;" class="fr-fic fr-dib">Designers explored this series of possible styles for the robotic heat activation of the 4D Knit Dress. Image courtesy of the Self-Assembly Lab.Danny Griffin MArch ’22, a current graduate student in architectural design, doesn’t have a background in garment making or the fashion industry. Tibbits asked Griffin to join the team due to his experience with robotics projects in construction. Griffin translated the heat activation process into a programmable robotic procedure that would precisely control its application.“When we apply heat, the fibers shorten, causing the textile to bunch up in a specific zone, effectively tightening the shape as if we’re tailoring the garment,” says Griffin. “There was a lot of trial and error to figure out how to orient the robot and the heat gun. The heat needs to be applied in precise locations to activate the fibers on each garment. Another challenge was setting the temperature and the timing for the heat to be applied.”It took a while to determine how the robot could&nbsp;reach all areas of the dress.“We couldn’t use a commercial heat gun — which is like a handheld hair dryer — because they’re too large,” says Griffin. “We needed a more compact design. Once we figured it out, it was a lot of fun to write the script for the robot to follow.”A dress can begin with one design — pintucks across the chest, for example — and be worn for months before having heat re-applied to alter its look. Subsequent applications of heat can tailor the dress further.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEwMTM1NjgyOTQyLVJvYm90aWNfYXJtX2F0X3dvcmsuanBlZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib">Robotic heat activation can be seen during a demonstration at Ministry of Supply in Boston. Photo: Olivia Mintz<h2>Beyond fit and fashion</h2>Efficiently producing garments is a “big challenge” in the fashion industry, according to Gihan Amarasiriwardena ’11, the co-founder and president of Ministry of Supply.“A lot of times you'll be guessing what a season's style is,” he says. “Sometimes the style doesn't do well, or some sizes don’t sell out. They may get discounted very heavily or eventually they end up going to a landfill.”“Fast fashion” is a term that describes clothes that are inexpensive, trendy, and easily disposed of by the consumer. They are designed and produced quickly to keep pace with current trends. The 4D Knit Dress, says Tibbits, is the opposite of fast fashion. Unlike the traditional “cut-and-sew” process in the fashion industry, the 4D Knit Dress is made entirely in one piece, which virtually eliminates waste.“From a global standpoint, you don’t have tons of excess inventory because the dress is customized to your size,” says Tibbits.McKinlay says she hopes use of this new technology will reduce the amount of waste in inventory that retailers usually have at the end of each season.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEwMTM1NzEwMDIxLU1pbmlzdHJ5X0ZJTkFMX0RyYWZ0MDUgY29weS5qcGVnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 100%px;" class="fr-fic fr-dib">A complete 4D Knit Dress comes out of a Stoll industrial knitting machine after being produced in one piece, without any cutting or sewing. Photo courtesy of the Self-Assembly Lab.“The dress could be tailored in order to adapt to these changes in styles and tastes,” she says. “It may also be able to absorb some of the size variations that retailers need to stock. Instead of extra-small, small, medium, large, and extra-large sizes, retailers may be able to have one dress for the smaller sizes and one for the larger sizes. Of course, these are the same sustainability points that would benefit the consumer.”The Self-Assembly Lab has collaborated with Ministry of Supply on projects with active textiles for several years. Late last year, the team debuted the <a href="https://selfassemblylab.mit.edu/4d-knit-dress" target="_blank">4D Knit Dress</a> at the company’s flagship store in Boston, complete with a robotic arm working its way around a dress as customers watched. For Amarasiriwardena, it was an opportunity to gauge interest and receive feedback from customers interested in trying the dress on.“If the demand is there, this is something we can create quickly” unlike the usual design and manufacturing process, which can take years, says Amarasiriwardena.Griffin and McKinlay were on hand for the demonstration and pleased with the results. For Griffin, with the “technical barriers” overcome, he sees many different avenues for the project.“This experience leaves me wanting to try more,” he says.McKinlay too would love to work on more styles.“I&nbsp;hope this research project helps people rethink or reevaluate their relationship with clothes,” says McKinlay. “Right now when people purchase a piece of clothing it has only one ‘look.’ But, how exciting would it be to purchase one garment and reinvent it to change and evolve as you change or as the seasons or styles change? I'm hoping that's the takeaway that people will have.”

Caption:In late 2023 the MIT Self Assembly Lab and the high-tech fashion company Ministry of Supply debuted their 4D Knit Dress at the latter's store Boston, complete with a robotic arm working its way around a dress as customers watched. Credits:Image courtesy of MIT Self Assembly Lab

Developed by the Self-Assembly Lab, the 4D Knit Dress uses several technologies to create a custom design and a custom fit, while addressing sustainability concerns.

Is this the future of fashion?

In this episode, we delve into the cognitive strategies employed by researchers at EPFL to augment the human body with an additional robotic arm and learn about the profound impact of cognitive enhancements on the integration of advanced robotic limbs

Podcast: Why Robotic Limbs Are Critical To The Human Race

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/85f51311-32bd-4a6e-953c-0d6777ee40fc?dark=false" class="fr-draggable" data-gtm-yt-inspected-9="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Neuroengineer Silvestro Micera develops advanced technological solutions to help people regain sensory and motor functions that have been lost due to traumatic events or neurological disorders. Until now, he had never before worked on enhancing the human body and cognition with the help of technology.<iframe width="640" height="360" src="https://www.youtube.com/embed/w78rRbaH7Wg?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-8="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="720627293" title="Cognitive strategies for augmenting the body with a wearable, robotic arm">&nbsp;</iframe>Now in a study published in Science Robotics, Micera and his team report on how diaphragm movement can be monitored for successful control of an extra arm, essentially augmenting a healthy individual with a third – robotic – arm.“This study opens up new and exciting opportunities, showing that extra arms can be extensively controlled and that simultaneous control with both natural arms is possible,” says Micera, Bertarelli Foundation Chair in Translational Neuroengineering at EPFL, and professor of Bioelectronics at Scuola Superiore Sant’Anna.The study is part of the <a href="https://nccr-robotics.ch/research/wearable-robotics/third-arm/" target="_blank" rel="noopener">Third-Arm project</a>, previously funded by the Swiss National Science Foundation (NCCR Robotics), that aims to provide a wearable robotic arm to assist in daily tasks or to help in search and rescue. Micera believes that exploring the cognitive limitations of third-arm control may actually provide gateways towards better understanding of the human brain.Micera continues, “The main motivation of this third arm control is to understand the nervous system. If you challenge the brain to do something that is completely new, you can learn if the brain has the capacity to do it and if it’s possible to facilitate this learning. We can then transfer this knowledge to develop, for example, assistive devices for people with disabilities, or rehabilitation protocols after stroke.”“We want to understand if our brains are hardwired to control what nature has given us, and we’ve shown that the human brain can adapt to coordinate new limbs in tandem with our biological ones,” explains Solaiman Shokur, co-PI of the study and EPFL Senior Scientist at the Neuro-X Institute. “It’s about acquiring new motor functions, enhancement beyond the existing functions of a given user, be it a healthy individual or a disabled one. From a nervous system perspective, it’s a continuum between rehabilitation and augmentation.”To explore the cognitive constraints of augmentation, the researchers first built a virtual environment to test a healthy user’s capacity to control a virtual arm using movement of his or her diaphragm. They found that diaphragm control does not interfere with actions like controlling one’s physiological arms, one’s speech or gaze.In this virtual reality setup, the user is equipped with a belt that measures diaphragm movement. Wearing a virtual reality headset, the user sees three arms: the right arm and hand, the left arm and hand, and a third arm between the two with a symmetric, six-fingered hand.“We made this hand symmetric to avoid any bias towards either the left or the right hand,” explains Giulia Dominijanni, PhD student at EPFL’s Neuro-X Institute.In the virtual environment, the user is then prompted to reach out with either the left hand, the right hand, or in the middle with the symmetric hand. In the real environment, the user holds onto an exoskeleton with both arms, which allows for control of the virtual left and right arms. Movement detected by the belt around the diaphragm is used for controlling the virtual middle, symmetric arm. The setup was tested on 61 healthy subjects in over 150 sessions.“Diaphragm control of the third arm is actually very intuitive, with participants learning to control the extra limb very quickly,” explains Dominijanni. “Moreover, our control strategy is inherently independent from the biological limbs and we show that diaphragm control does not impact a user’s ability to speak coherently.”The researchers also successfully tested diaphragm control with an actual robotic arm, a simplified one that consists of a rod that can be extended out, and back in. When the user contracts the diaphragm, the rod is extended out. In an experiment similar to the VR environment, the user is asked to reach and hover over target circles with her left or right hand, or with the robotic rod.Besides the diaphragm, but not reported in the study, vestigial ear muscles have also been tested for feasibility in performing new tasks. In this approach, a user is equipped with ear sensors and trained to use fine ear muscle movement to control the displacement of a computer mouse.“Users could potentially use these ear muscles to control an extra limb,” says Shokur, emphasizing that these alternative control strategies may help one day for the development of rehabilitation protocols for people with motor deficiencies.Part of the third arm project, previous studies regarding the <a href="https://actu.epfl.ch/news/a-smart-artificial-hand-for-amputees-merges-user-a/" target="_blank">control of robotic arms</a> have been focused on helping amputees. The latest Science Robotics study is a step beyond repairing the human body towards augmentation.“Our next step is to explore the use of more complex robotic devices using our various control strategies, to perform real-life tasks, both inside and outside of the laboratory. Only then will we be able to grasp the real potential of this approach,” concludes Micera.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyNTY4NDIyNzU0LTgxOTJ4NTQ2NC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Martina Gini controls a simplified robotic arm with breathing. © 2023 EPFL / Alain Herzog, CC-BY-SA<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyNTY4NTQ4Mjg5LWV6Z2lmLTUtNDVjNDI4M2Y2Zi5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Silvestro Micera (left) and Leonardo Pollina (right) prepare for the third arm virtual reality experiment. © 2023 EPFL / Alain Herzog, CC-BY-SA

EPFL scientists show that breathing may be used to control a wearable extra robotic arm in healthy individuals, without hindering control of other parts of the body.

Cognitive strategies to augment the body with an extra robotic arm

Imagine that dinner could be prepared by a robot in your own home with just the push of a button. It sounds like something straight out of a far-flung future. Now, researchers at CMU are laying the groundwork for this technology with artificial intelligence.&nbsp;Alison Bartch, Atharva Dikshit, and Abraham George, under the guidance of <a href="https://engineering.cmu.edu/directory/bios/barati-farimani-amir.html" target="_blank">Amir Barati Farimani</a>, assistant professor of <a href="https://www.meche.engineering.cmu.edu/index.html" rel="noopener" target="_blank">mechanical engineeringOpens in new window</a>, trained a robot arm on different fruit slices and shapes to help improve its accuracy with cutting and moving real pieces of food on a cutting board.&nbsp;“So I’m cooking and have a cluttered cutting board. I’m reasoning about how to chop and how to select the objects that I want to cut, without cutting things that I don’t want,” said Bartch, a Ph.D. candidate in mechanical engineering. “It’s something that humans are pretty good at. And it’s a surprisingly difficult problem for robots.”<blockquote>It’s something that humans are pretty good at. And it’s a surprisingly difficult problem for robots.<cite>Alison Bartch<cite id="isPasted">,&nbsp;Ph.D. student<cite id="isPasted">, Mechanical Engineering</cite></cite> </cite></blockquote>The researchers combined two vision foundational models—models trained on large visual data sets—to help the robot arm recognize the shape and the type of fruit and vegetable slices. The AI would then determine what motion to use on these slices to produce the correct results with 70% accuracy.One of the models was only trained on whole fruits and vegetables, so the researchers needed to fine-tune the AI to more accurately recognize slices with a relatively small amount of newly collected data. According to Bartch, their fine-tuned framework can potentially be used for other related research; the combined foundational models, however, can be used for a variety of other AI projects in the future.“That’s just one of the things in a lot of robotics and machine learning—a lot of the data sets are already collected,” said George, also a Ph.D. candidate in mechanical engineering.&nbsp;Bartch said that the research focuses more on the “practicalities” of cutting food, such as producing the correct number of slices and moving pieces out of the way, making this research more applicable to a real kitchen than other research.“For this work, we focused more on the pipeline surrounding it, as opposed to the actual chop action,” Bartch said. “You want to pick which object to cut. You want to clear the scene before you cut. After you cut, detecting where the objects are then and continuing to plan. A lot of existing work focused on cutting assumed the position of the vegetable and that there is nothing else in the environment.”&nbsp;However, it will still be a while before AI will be chopping vegetables in homes. According to Bartch, there is still a lot of research and fine-tuning needed, and their robot still needs some human input. It’s difficult to replicate human reasoning in machines, even with tasks many humans find simple.“I think there is definitely a really long way to go before it’s in a person’s house.” Bartch said. “Because reasoning about cooking is really difficult. When you want to go into more practical situations, you need the system to be able to reason when the food is done.Atharva Dikshit, a master’s student in mechanical engineering, also contributed to this research. This research was submitted to the Journal of Intelligent and Robotic Systems.

Researchers at CMU combined two vision foundational models—models trained on large visual data sets—to help a robot arm recognize the shape and the type of fruit and vegetable slices.

Chop, chop: Improving food prep with the power of AI

Engineers at Princeton University and Google have come up with a new way to teach robots to know when they don’t know. The technique involves quantifying the fuzziness of human language and using that measurement to tell robots when to ask for further directions. Telling a robot to pick up a bowl from a table with only one bowl is fairly clear. But telling a robot to pick up a bowl when there are five bowls on the table generates a much higher degree of uncertainty — and triggers the robot to ask for clarification.Because tasks are typically more complex than a simple “pick up a bowl” command, the engineers use large language models (LLMs) — the technology behind tools such as ChatGPT — to gauge uncertainty in complex environments. LLMs are bringing robots powerful capabilities to follow human language, but LLM outputs are still frequently unreliable, said <a href="https://engineering.princeton.edu/faculty/anirudha-majumdar" target="_blank">Anirudha Majumdar</a>, an assistant professor of <a href="https://mae.princeton.edu/" target="_blank">mechanical and aerospace engineering</a> at Princeton and the senior author of a <a href="https://openreview.net/forum?id=4ZK8ODNyFXx" target="_blank">study</a> outlining the new method.“Blindly following plans generated by an LLM could cause robots to act in an unsafe or untrustworthy manner, and so we need our LLM-based robots to know when they don’t know,” said Majumdar.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1701898003603-knowno-ambiguities_WEB-2048x871.jpg" style="width: 766px;" class="fr-fic fr-dib">The researchers tested their method on a simulated robotic arm and on two types of robot hardware. The most complex experiments (left) involved a robotic arm mounted on a wheeled platform and placed in an office kitchen with a microwave and a set of recycling, compost and trash bins. Another set of experiments used a tabletop robotic arm (right) tasked with sorting a set of toy food items into two different categories; a setup with a left and right arm (center) added an additional layer of ambiguity. Images by the researchersThe system also allows a robot’s user to set a target degree of success, which is tied to a particular uncertainty threshold that will lead a robot to ask for help. For example, a user would set a surgical robot to have a much lower error tolerance than a robot that’s cleaning up a living room.“We want the robot to ask for enough help such that we reach the level of success that the user wants. But meanwhile, we want to minimize the overall amount of help that the robot needs,” said&nbsp;<a href="https://allenzren.github.io/" target="_blank">Allen Ren</a>, a graduate student in mechanical and aerospace engineering at Princeton and the study’s lead author. Ren received a&nbsp;<a href="https://www.corl2023.org/awards" target="_blank">best student paper award</a> for his Nov. 8 presentation at the Conference on Robot Learning in Atlanta. The new method produces high accuracy while reducing the amount of help required by a robot compared to other methods of tackling this issue.The researchers tested their method on a simulated robotic arm and on two types of robots at Google facilities in New York City and Mountain View, California, where Ren was working as a student research intern. One set of hardware experiments used a tabletop robotic arm tasked with sorting a set of toy food items into two different categories; a setup with a left and right arm added an additional layer of ambiguity.The most complex experiments involved a robotic arm mounted on a wheeled platform and placed in an office kitchen with a microwave and a set of recycling, compost and trash bins. In one example, a human asks the robot to “place the bowl in the microwave,” but there are two bowls on the counter — a metal one and a plastic one.The robot’s LLM-based planner generates four possible actions to carry out based on this instruction, like multiple-choice answers, and each option is assigned a probability. Using a statistical approach called conformal prediction and a user-specified guaranteed success rate, the researchers designed their algorithm to trigger a request for human help when the options meet a certain probability threshold. In this case, the top two options — place the plastic bowl in the microwave or place the metal bowl in the microwave — meet this threshold, and the robot asks the human which bowl to place in the microwave.In another example, a person tells the robot, “There is an apple and a dirty sponge … It is rotten. Can you dispose of it?” This does not trigger a question from the robot, since the action “put the apple in the compost” has a sufficiently higher probability of being correct than any other option.“Using the technique of conformal prediction, which quantifies the language model’s uncertainty in a more rigorous way than prior methods, allows us to get to a higher level of success” while minimizing the frequency of triggering help, said the study’s senior author&nbsp;<a href="https://engineering.princeton.edu/faculty/anirudha-majumdar" target="_blank">Anirudha Majumdar</a>, an assistant professor of mechanical and aerospace engineering at Princeton.Robots’ physical limitations often give designers insights not readily available from abstract systems. Large language models “might talk their way out of a conversation, but they can’t skip gravity,” said coauthor&nbsp;<a href="https://andyzeng.github.io/" target="_blank">Andy Zeng</a>, a research scientist at Google DeepMind. “I’m always keen on seeing what we can do on robots first, because it often sheds light on the core challenges behind building generally intelligent machines.”Ren and Majumdar began collaborating with Zeng after he gave a talk as part of the&nbsp;<a href="https://robo.princeton.edu/seminar" target="_blank">Princeton Robotics Seminar</a> series, said Majumdar. Zeng, who earned a computer science Ph.D. from Princeton in 2019, outlined Google’s efforts in using LLMs for robotics, and brought up some open challenges. Ren’s enthusiasm for the problem of calibrating the level of help a robot should ask for led to his internship and the creation of the new method.“We enjoyed being able to leverage the scale that Google has” in terms of access to large language models and different hardware platforms, said Majumdar.Ren is now extending this work to problems of active perception for robots: For instance, a robot may need to use predictions to determine the location of a television, table or chair within a house, when the robot itself is in a different part of the house. This requires a planner based on a model that combines vision and language information, bringing up a new set of challenges in estimating uncertainty and determining when to trigger help, said Ren.The paper, “<a href="https://openreview.net/forum?id=4ZK8ODNyFXx" target="_blank">Robots That Ask for Help: Uncertainty Alignment for Large Language Model Planners</a>,” was presented Nov. 8 at the Conference on Robot Learning. In addition to Ren, Zeng and Majumdar, coauthors include Anushri Dixit and Alexandra Bodrova of Princeton; and Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu and Dorsa Sadigh of Google DeepMind. The research was supported in part by the U.S. National Science Foundation and the Office of Naval Research.

Engineers at Princeton University and Google have come up with a new way to teach robots to know when they don’t know and ask for clarification from a human. Photo by the researchers

Modern robots know how to sense their environment and respond to language, but what they don’t know is often more important than what they do know. Teaching robots to ask for help is key to making them safer and more efficient.

How do you make a robot smarter? Program it to know what it doesn't know.

Imagine you’re visiting a friend abroad, and you look inside their fridge to see what would make for a great breakfast. Many of the items initially appear foreign to you, with each one encased in unfamiliar packaging and containers. Despite these visual distinctions, you begin to understand what each one is used for and pick them up as needed.Inspired by humans' ability to handle unfamiliar objects, a group from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) designed Feature Fields for Robotic Manipulation (<a href="https://arxiv.org/abs/2308.07931" target="_blank">F3RM</a>), a system that blends 2D images with foundation model features into 3D scenes to help robots identify and grasp nearby items. F3RM can interpret open-ended language prompts from humans, making the method helpful in real-world environments that contain thousands of objects, like warehouses and households.F3RM offers robots the ability to interpret open-ended text prompts using natural language, helping the machines manipulate objects. As a result, the machines can understand less-specific requests from humans and still complete the desired task. For example, if a user asks the robot to “pick up a tall mug,” the robot can locate and grab the item that best fits that description.“Making robots that can actually generalize in the real world is incredibly hard,” says Ge Yang, postdoc at the National Science Foundation AI Institute for Artificial Intelligence and Fundamental Interactions and MIT CSAIL. “We really want to figure out how to do that, so with this project, we try to push for an aggressive level of generalization, from just three or four objects to anything we find in MIT’s Stata Center. We wanted to learn how to make robots as flexible as ourselves, since we can grasp and place objects even though we've never seen them before.”<h2>Learning “what’s where by looking”</h2>The method could assist robots with picking items in large fulfillment centers with inevitable clutter and unpredictability. In these warehouses, robots are often given a description of the inventory that they're required to identify. The robots must match the text provided to an object, regardless of variations in packaging, so that customers’ orders are shipped correctly. For example, the fulfillment centers of major online retailers can contain millions of items, many of which a robot will have never encountered before. To operate at such a scale, robots need to understand the geometry and semantics of different items, with some being in tight spaces. With F3RM’s advanced spatial and semantic perception abilities, a robot could become more effective at locating an object, placing it in a bin, and then sending it along for packaging. Ultimately, this would help factory workers ship customers’ orders more efficiently.“One thing that often surprises people with F3RM is that the same system also works on a room and building scale, and can be used to build simulation environments for robot learning and large maps,” says Yang. “But before we scale up this work further, we want to first make this system work really fast. This way, we can use this type of representation for more dynamic robotic control tasks, hopefully in real-time, so that robots that handle more dynamic tasks can use it for perception.”The MIT team notes that F3RM’s ability to understand different scenes could make it useful in urban and household environments. For example, the approach could help personalized robots identify and pick up specific items. The system aids robots in grasping their surroundings — both physically and perceptively.“Visual perception was defined by David Marr as the problem of knowing ‘what is where by looking,’” says senior author Phillip Isola, MIT associate professor of electrical engineering and computer science and CSAIL principal investigator. “Recent foundation models have gotten really good at knowing what they are looking at; they can recognize thousands of object categories and provide detailed text descriptions of images. At the same time, radiance fields have gotten really good at representing where stuff is in a scene. The combination of these two approaches can create a representation of what is where in 3D, and what our work shows is that this combination is especially useful for robotic tasks, which require manipulating objects in 3D.”<h2>Creating a “digital twin”</h2>F3RM begins to understand its surroundings by taking pictures on a selfie stick. The mounted camera snaps 50 images at different poses, enabling it to build a <a href="https://www.matthewtancik.com/nerf" target="_blank">neural radiance field</a> (NeRF), a deep learning method that takes 2D images to construct a 3D scene. This collage of RGB photos creates a “digital twin” of its surroundings in the form of a 360-degree representation of what’s nearby.In addition to a highly detailed neural radiance field, F3RM also builds a feature field to augment geometry with semantic information. The system uses&nbsp;<a href="https://openai.com/research/clip" target="_blank">CLIP</a>, a vision foundation model trained on hundreds of millions of images to efficiently learn visual concepts. By reconstructing the 2D CLIP features for the images taken by the selfie stick, F3RM effectively lifts the 2D features into a 3D representation.<h2>Keeping things open-ended</h2>After receiving a few demonstrations, the robot applies what it knows about geometry and semantics to grasp objects it has never encountered before. Once a user submits a text query, the robot searches through the space of possible grasps to identify those most likely to succeed in picking up the object requested by the user. Each potential option is scored based on its relevance to the prompt, similarity to the demonstrations the robot has been trained on, and if it causes any collisions. The highest-scored grasp is then chosen and executed. To demonstrate the system’s ability to interpret open-ended requests from humans, the researchers prompted the robot to pick up Baymax, a character from Disney’s “Big Hero 6.” While F3RM had never been directly trained to pick up a toy of the cartoon superhero, the robot used its spatial awareness and vision-language features from the foundation models to decide which object to grasp and how to pick it up. F3RM also enables users to specify which object they want the robot to handle at different levels of linguistic detail. For example, if there is a metal mug and a glass mug, the user can ask the robot for the “glass mug.” If the bot sees two glass mugs and one of them is filled with coffee and the other with juice, the user can ask for the “glass mug with coffee.” The foundation model features embedded within the feature field enable this level of open-ended understanding.“If I showed a person how to pick up a mug by the lip, they could easily transfer that knowledge to pick up objects with similar geometries such as bowls, measuring beakers, or even rolls of tape. For robots, achieving this level of adaptability has been quite challenging,” says MIT PhD student, CSAIL affiliate, and co-lead author William Shen. “F3RM combines geometric understanding with semantics from foundation models trained on internet-scale data to enable this level of aggressive generalization from just a small number of demonstrations.” Shen and Yang wrote the paper under the supervision of Isola, with MIT professor and CSAIL principal investigator Leslie Pack Kaelbling and undergraduate students Alan Yu and Jansen Wong as co-authors. The team was supported, in part, by Amazon.com Services, the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research’s Multidisciplinary University Initiative, the Army Research Office, the MIT-IBM Watson AI Lab, and the MIT Quest for Intelligence. Their work will be presented at the 2023 Conference on Robot Learning.

Feature Fields for Robotic Manipulation (F3RM) enables robots to interpret open-ended text prompts using natural language, helping the machines manipulate unfamiliar objects. The system’s 3D feature fields could be helpful in environments that contain thousands of objects, such as warehouses. Images courtesy of the researchers.

By blending 2D images with foundation models to build 3D feature fields, a new MIT method helps robots understand and manipulate nearby objects with open-ended language prompts.

Using language to give robots a better grasp of an open-ended world

&nbsp; Achieving precision is&nbsp;a must&nbsp;in ultrasonic inspections, but&nbsp;it's&nbsp;not always straightforward due to&nbsp;some&nbsp;challenges like interference and noise. These issues can arise from various sources, including surface rust and environmental dust. To address these challenges,&nbsp;there's&nbsp;an exciting development on the horizon: the introduction of ultrasonic testing (UT) UAVs, often referred to as UT drones or UT flying robots. These innovative robots come equipped with&nbsp;state-of-the-art&nbsp;technology and, importantly, enhance safety by&nbsp;eliminating&nbsp;the need for human inspectors to enter high-risk areas.&nbsp;What's more, they have the capability to reach elevated inspection sites, effectively reducing the risks associated with working at heights while ensuring highly accurate results.&nbsp; &nbsp; &nbsp;One of the standout players in this field is SKYRON,&nbsp;the&nbsp;intelligent tethered&nbsp;flying&nbsp;robot equipped with a customized 38DL PLUS ultrasonic thickness gauge from Olympus. This technology not only enhances the safety of inspections but also elevates precision. The 38DL PLUS gauge is designed to tackle challenging conditions, whether that involves a rusty surface, a dusty environment,&nbsp;high temperatures, or access to difficult spaces. SKYRON can conduct meticulous inspections even in challenging locations.&nbsp;&nbsp; &nbsp; <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699488071311-1699488071311.png" class="fr-fic fr-dii">&nbsp;Moreover, what sets SKYRON apart is its ingenious built-in brush system, integrated into its rotary arm. This system is engineered to smooth rough surfaces and remove debris before an inspection, ensuring clean and&nbsp;accurate&nbsp;readings.&nbsp;It's&nbsp;a critical feature that ensures the reliability of the collected data. Furthermore,&nbsp;there's&nbsp;no need to interrupt the inspection process to change the brush head since&nbsp;it's&nbsp;seamlessly integrated into SKYRON's rotary arm. Additionally, before each inspection, SKYRON&nbsp;applies&nbsp;couplant&nbsp;gel, further streamlining the process.&nbsp;SKYRON brings even more improvements to the table. It boasts a continuous power supply, allowing it to fly for an unlimited duration, which is particularly&nbsp;advantageous&nbsp;for projects&nbsp;where&nbsp;relying on batteries would be inconvenient. The constant power supply also enables a higher payload capacity,&nbsp;facilitating&nbsp;the integration of various sensors, including advanced NDT,&nbsp;DFT,&nbsp;PAUT,&nbsp;and an array of robotic features.&nbsp;SKYRON is not just a UAV;&nbsp;it's&nbsp;a versatile platform that can seamlessly incorporate various sensors and features. It ensures safe and secure operations, minimizing the risk of unexpected issues during flights. Moreover, it guarantees&nbsp;accurate&nbsp;contact-based inspections of critical structures. SKYRON&nbsp;can fly,&nbsp;attach&nbsp;and measure multiple assets, capturing precise thickness measurements and conducting corrosion checks. What truly sets it apart is its adaptability, making it highly effective in challenging environments, and demanding conditions where&nbsp;maintaining&nbsp;a clear line of sight can be a challenge. In summary, SKYRON stands as a valuable&nbsp;tool&nbsp;for addressing a variety of inspection challenges and spearheading a new era of ultrasonic precision.&nbsp;In conclusion, SKYRON, equipped with its remarkable 38DL PLUS ultrasonic thickness gauge and ingenious built-in brush system,&nbsp;embodies&nbsp;the future of ultrasonic inspections. The seamless integration of these advanced features not only enhances the safety of inspections but also ensures the highest level of precision, regardless of the conditions. SKYRON&nbsp;doesn't&nbsp;just&nbsp;prepares&nbsp;a new era of ultrasonic precision; it serves as a shining example of what's possible when technology and innovation join forces. With its adaptability and versatility,&nbsp;it&nbsp;is ready&nbsp;to tackle a wide array of inspection challenges and&nbsp;help to&nbsp;redefine the standards of excellence in the field. As the skies&nbsp;open up&nbsp;to these remarkable flying robots.&nbsp;&nbsp;

Achieving precision is a must in ultrasonic inspections, but it's not always straightforward

Advancing Ultrasonic Inspections

At EPFL's&nbsp;<a href="https://www.epfl.ch/labs/create/" target="_blank">CREATE lab</a>, under the guidance of Josie Hughes, a breakthrough has been made in the realm of soft robotics. Drawing inspiration from the versatile movement of elephant trunks and octopus tentacles, the team introduced the trimmed helicoid — a novel robotic structure that promises greater compliance and control in robotic designs. With a blend of keen biological observation and computational modeling, the researchers have now unveiled a soft robot arm capable of intricate tasks, ensuring safer human-robot interactions. The findings, detailing both the structure and methodology, are a collaboration with the Department of Cognitive Robotics at TU Delft and were published in Nature's new journal, npj Robotics.Professor Hughes highlighted the importance of this development: "Through the invention of a new architectured structure, the trimmed helicoid, we've designed a robot arm that excels in control, range of motion, and safety. When the novel architecture is combined with distributed actuation— where multiple actuators are placed throughout a structure or device—this robot arm has a vast range of motion, high precision, and is inherently safe for human interaction."Whereas traditional robots are rigid, often making them unsuitable for delicate tasks or close human interactions, CREATE's soft robot arm is designed for safer interactions with humans and adaptability to a wider range of tasks. With an unprecedented combination of flexibility and precision, the soft and compliant nature of the arm reduces potential risks during human-robot interactions. This opens doors for its application in healthcare, elderly care, and more. Unlike their rigid counterparts, the soft robot arm can adapt to different shapes and surfaces, making it an ideal tool for intricate tasks like picking fruits or handling fragile items. In industry, it might become the go-to solution for delicate assembly lines, working alongside humans, augmenting their capacity instead of replacing them. The agricultural industry could also benefit from its gentle touch in handling crops, accompanying workers to lessen their workload during intense harvesting periods.The crux of the research lies in the robotic arm’s novel architecture. The researchers have creatively modified a spring-like spiral, which they call a 'helicoid', by trimming parts of it to give it diverse functionalities. This seemingly simple act has allowed them to precisely control how flexible or stiff the spiral becomes in different directions. By adjusting its shape, they can make the inner part resistant to being squashed and the outer part flexible enough to bend easily. With this special design, they've created a soft robot that can move and act in ways previously unseen, showing the kind of dexterity and soft touch found in nature, like in an elephant's trunk or an octopus's tentacle."By observing these animals and developing a novel architectured structure, we aim to mimic this range of motion and control found in Nature," Josie Hughesnoted. To do so, the team employed advanced computer modeling to turn observations into tangible results. Using these models, they iteratively tested their innovative spiral designs — into a final trimmed helicoid shape. Qinghua Guan and Francesco Stella, who spearheaded the actual creation of the robot, gave insights into the design and optimization process: "We introduce a specific surface into the computer model, then trim and adjust. Computational methods guide us, helping assess the optimal geometric structure for maximum workspace and compliance." The result? A robotic creation, drawing from nature, but refined with precise human ingenuity and computer modeling. “In the end, our computer models were accurate to the point where we only had to build one version of the arm.”The advancements at EPFL's CREATE lab symbolize a pivotal shift in the robotics field. Traditional robotic applications, dominated by rigid mechanics, could see a shift towards this softer, more human-friendly counterpart. A patent has been filed for this first commercial soft manipulator, and a EPFL and TU Delft joint start-up has been launched under the name Helix Robotics. As Professor Hughes aptly summarized, "Merging keen observations from nature with precise computational modeling has unveiled the potential of soft robotics for future commercial applications. As we move forward, our aim is to bring robots closer to humans, not just in proximity but in understanding and collaboration. We hope that this soft robotic arm exemplifies a future where machines assist, complement, and understand human needs more deeply than ever before."<hr>Professor Josie Hughes will be showing the Helix robot along with several other of her creations at the&nbsp;<a href="https://swissroboticsday.ch/" target="_blank" rel="noopener">Swiss Robotics Day</a> at ETHZ on November 3rd. Many other EPFL professors will be showcasing their work. The day is dedicated to the best of Swiss robotics across industry, startups, universities, research labs, nonprofits, and government. The event features high-profile speakers (Boston Dynamics, Disney) and panelists, along with startup pitches and 50+ exhibition booths with live demos.ReferencesGuan, Q., Stella, F., Della Santina, C. et al. Trimmed helicoids: an architectured soft structure yielding soft robots with high precision, large workspace, and compliant interactions. npj Robot 1, 4 (2023). https://doi.org/10.1038/s44182-023-00004-7

EPFL researchers have designed a bio-inspired robot with a novel trimmed helicoid structure that allows for a wide range of motion and safe interaction with humans.

Soft, elephant trunk-like robot for close interaction with humans

<h3 id="isPasted">Let’s explore some methods on how to achieve this.</h3>First and foremost, there exists the “Straightforward Programming” approach. This means: getting a skilled person to meticulously describe a task, to such an extent that one can write either a simple script or an intricate algorithm that can guide the robot to do the task. This method is great and highly effective when the task is well-defined and specific, something often done in industrial automation. However, the complexity can escalate rapidly, making it challenging. Thus, if the approach is successful, that’s excellent – but if not, we must ponder – what alternative strategies can we employ?Secondly, an alternative approach to consider is “Programming by Demonstration“. This method involves guiding the robot in performing the task. By equipping the robot with advanced force sensing capabilities and vision algorithms, this technique can prove to be quite effective. However, it’s crucial to note that its effectiveness is reliant on the robot’s abilities. Usually, robots are equipped with rudimentary grippers rather than adept, dexterous hands, which can limit the complexity of tasks they can perform.&nbsp;Okay, so how can we effectively record human actions for a robot to mimic?Ideally, we’d guide a human through the task, document it in detail, and then replay it to the robot. We might even consider feeding this information into a machine-learning system, enabling it to learn from our human-guided demonstration. Alternatively, at the very least, we can analyze these human activities, deconstruct them into simpler parts, and use this information to develop more efficient algorithms in the way that a song is deconstructed in a recording studio.&nbsp;OK, so how about, filming the person doing the task and using “machine vision” to track what they are doing?It’s certainly a good starting point. This approach provides valuable insights into the nuances of human behaviour – such as hand-switching moments, object-holding techniques, and tool usage – But our experience is that it’s really hard to track human movement&nbsp;using cameras. You need clear lines of sight, and you’ll have lots of trouble tracking the fine movements of the fingers and hands.&nbsp;Even the most advanced image-based tracking systems, while capable of monitoring hands with minimal occlusion, fail to consistently track all joint movements. Consequently, the resultant data tends to be somewhat inconsistent and noisy.&nbsp;Alright, what about using motion capture?Motion capture systems, commonly employed in film production and sports science, operate by attaching easily visible markers to the individual, such as passive elements like white dots or active elements like pulsed LEDs. This method certainly helps with the accuracy of tracking. However, it doesn’t entirely overcome the previously mentioned issues related to occlusion. Additionally, calibrating the optical systems to an error margin of less than a few millimetres is quite challenging – which means that if the task involves intricate movements, such as rolling a pen between fingers, it’s likely it’ll miss a lot.&nbsp;&nbsp;Okay. I need it to be accurate, what’s my best option then?For precise human movement tracking, your best bet is to use a tight-fitting movement-tracking&nbsp;<a href="https://www.shadowrobot.com/shadow-hand-glove/" target="_blank">glove</a>. This glove can incorporate a multitude of techniques to achieve such a feat, some even providing additional features to assist the operator (and some that make it more difficult for the operator). For instance, some gloves can relay a sense of weight, touch, or stiffness, all functionalities grouped under the term ‘haptics.’ The realm of haptics is broad, including diverse technologies from ultrasound devices transmitting signals to fingers, electrical skin stimulation, vibrating piezoelectric elements, to inflating air balloons, miniature motors, or linear actuators, among many others.Gloves that measure human hand movement can do it by utilising:<ul><li dir="ltr">Sensors that fit around the joints and track the specific movement of the joint</li><li dir="ltr">Mechanical linkages that go from one end of the finger to the other and track the whole movement of the finger</li><li dir="ltr">Inertial sensors (that clever combination of accelerometer, gyroscope and possibly magnetometer) - worn at various points on the hand to follow the movement</li><li dir="ltr">Wireless measuring sensors - detecting the location of the fingers relative to a source worn on the hand somewhere.</li><li dir="ltr">Cameras that look away from the hand and track the movement of the world. (I’m not sure I’ve seen this one used in anger but I’m sure it’s possible!)</li></ul> Even then several variations of these technologies exist. For instance, some sensors measure the flex of a glove, correlating it to joint movement. Other types of sensors stretch and track their own deformation to gauge distance. Occasionally, mechanical linkages are placed over individual joints for precise motion tracking.&nbsp;Each of these methods presents a unique approach to capturing and interpreting human movement<h3 id="isPasted">Our experience:</h3>From our extensive experience, we have found that accurately measuring the mechanical properties of the human hand can be challenging, as it often leads to significant inaccuracies. The placement of the measurement sensor is crucial, and unfortunately, the potential for slippage is high – gloves and similar apparatus tend to shift excessively, complicating the data collection. Relating the measurements to the actual rotation of a finger joint’s axis, especially the thumb, is another daunting task!<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3NTM2Nzk3ODE3LWxvbmcgLSBQb2xoZW11cyAmIFNoYWRvdyAoMTkyMCB4IDEwODAgcHgpLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Mechanical linkages, although valuable, can quickly become cumbersome. There is also a risk that they could discomfort the operator to the point where task completion becomes impossible (too heavy, too rigid etc). Inertial sensors present a viable option; however, (once again) preventing them from drifting is generally difficult, and the precision level they offer often falls short of what we needWe have found that wireless measurement sensors, particularly those from&nbsp;<a href="https://www.shadowrobot.com/shadow-robot-polhemus/" target="_blank">Polhemus</a>, give us excellent results. These sensors entail a transmitting source worn on the palm and receivers located at the fingertips, providing us with high-quality data for hand movement tracking. However, there remains the challenge of locating the hand within the wider three-dimensional space. For this, we have found that the HTC Vive is an excellent solution and so you will often see the Vive tracker featured in our demonstrations.Now – what we want is to accurately know where to put the robot hand – which isn’t quite the same thing as knowing accurately where the human hand is. We have to take the data from the human hand and map it to the robot hand. This is a kinematic challenge – the movements of the human and the movements of the robot don’t agree entirely, and we need to convert one to the other. The best method for mapping human data to a robot hand will depend on the specific application. For example, if you need to control a robot’s hand in real-time, then direct mapping may be the best option. If you are only interested in controlling a robot’s hand for offline analysis, then inverse kinematics or hybrid mapping may be more feasible options. Needless to say, the more complex the robot the more challenging this becomes. Lucky for us complex robots are a speciality at Shadow!&nbsp;So, the human wears a set of gloves, and they control the robot to do the task. After completing a few rudimentary training tasks the operator gets the hang of the process and they are doing it pretty well. What do we do to learn from this?We can precisely monitor the instructions sent to the robot and reproduce them. This is ideal when operating in a controlled environment where all variables remain consistent – a scenario commonly found in an automation setting within a quality assurance laboratory. In the past, we employed this technique to build a functional robotic kitchen. While it allows you to execute repetitive tasks, it lacks the ability to cope with variation.We can capture and analyze data from numerous tests, assessing them for consistency. If the results are largely similar, they can be directly applied. However, if they vary a lot, we can delve into our algorithm toolbox to try and understand these discrepancies. This is where it helps to have a <a href="https://www.shadowrobot.com/artificial-intelligence-machine-learning/" target="_blank">wide range of data</a> from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">the robot</a>, from force, <a href="https://www.shadowrobot.com/shadow-robot-11-lessons-about-tactile-sensors/" target="_blank">touch</a>, and joint sensors. The question arises: Can we employ data from vision, such as cameras aimed at the work area, to discern what’s different? To be effective, this vision data needs to be synchronized with the robot sensor data, necessitating a shared framework for all data streams.We can gather data from multiple tests and use it by “Feeding it to the AI” – meaning use it to instruct a machine learning system on how to replicate similar movements from similar data.This can be done in several ways.We might train a traditional learning model on the datasets so that similar inputs generate similar outputs. Alternatively, if we have enough data, we could use a transformer model to generate movements. We can even break down the data into smaller segments and train a network to assemble these fragments into the correct sequences.Given the impressive capabilities of modern machine learning systems in generating text, images, video, and even code, this approach appears highly promising – and it’s only the beginning!So, does this sound like it could help with a problem that you face? If it does, then let’s have&nbsp;<a href="https://www.shadowrobot.com/blog/data-humans-to-robots/hand@shadowrobot.com" target="_blank">a chat to</a> see if the Shadow Robot team can help!

Humans excel at manual tasks, and ideally, robots should surpass us in these functions. We have the advantage due to our superior dexterity, and we can perform tasks that robots should, in theory, be capable of.
As we advance robotics our goal is to effectively transfer our human skills to robots.

Getting Data from Humans to Robots

This article originally appeared in <a href="https://news.itmo.ru/en/science/life_science/news/13440/" target="_blank" rel="noopener noreferrer">ITMO News</a> and has been republished with permission.<hr>A family of medical devices named implants are manufactured to replace a missing organ or its parts. Dental implants are implanted into the jaw to support replacement teeth. Apart from dentistry, implants are also used to resurface a knee, repair skull bones, as well as for other purposes.&nbsp;All medical devices are customized to each patient's unique characteristics. In the case of dental implants, doctors must be aware of a person's skull bones, gums, and roots to determine the appropriate form and size of the future implant.&nbsp;To make an implant last longer, as well as heal faster and easier, its surface is enhanced with functional features, like antibacterial properties and biocompatibility. The latter is usually accomplished by roughening up the implant's surface with sandblasting, followed by acid and anodizing to make the surface antimicrobial. So far, there is no one-stop complex to execute both procedures – except for the one currently under development at ITMO University.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MTk5NTcwODA3LXAxMzQ0MC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">The research team (from left to right): Yulia Karlagina, Galina Romanova, Andrey Lisichnikov, and Fedor Inochkin. Photo courtesy of Evgeniy Shamshin<h2 dir="ltr" id="isPasted">The laser-based complex</h2>Researchers from ITMO’s Institute of Laser Technologies and their colleagues from Laser Center designed a multiuse robotic complex for surface treatment of diverse medical materials. According to the developers, the technology is equipped with a fiber laser system and possesses several advantages:&nbsp;Versatility. The technology is well-suited for use with medical devices such as scalpels and surgical scissors, as well as a variety of implants, including teeth, skull, hip, and knee replacements.&nbsp;Turnkey treatment in one system. The team created and merged several laser treatment options that are used consecutively:<ul><li dir="ltr">The technology for antibacterial surface properties prevents bacterial complications, which lead to implant rejection. Laser heating makes oxide layers grow on the surface of a titanium implant, which demonstrates bactericidal properties when exposed to a UV source.</li><li dir="ltr">The technology for biocompatible surface property. The laser is applied to generate a relief with unique properties on the surface, followed by the laser ablation process, which creates a biomimetic micro- and nanorelief that promotes excellent adhesion of proteins in the early stages of osseointegration, cell proliferation, and differentiation, ensuring implant success.&nbsp;</li><li dir="ltr">The technology for applying non-toxic identification labels on medical devices. Legally binding product labeling makes it possible to track the life cycle of a product, from its early production to the end user.&nbsp;</li></ul><blockquote>“We came up with a versatile device that makes it possible to functionalize a wide spectrum of medical equipment without any additional consumables or tools. The single system can boast several treatment options, related to the biocompatible and antibacterial properties of products, their roughness, as well as labeling,” states&nbsp;Yulia Karlagina, a junior researcher at ITMO’s Institute of Laser Technologies.</blockquote><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697199551134-ezgif-2-348e4b0195.gif" style="width: 766px;" class="fr-fic fr-dib">The robot laser complex in action. Video courtesy of Evgeniy ShamshinCompatibility with custom devices. All it takes is to load a 3D model of the implant into the program and indicate which elements of the 3D electronic geometric model need to be processed. The software was developed by the team at ITMO.&nbsp;The complex consists of a six-axis robotic manipulator and a laser scanning system mounted on the robot's arm, which ensures high accuracy of laser positioning, achieved by optical methods, and universality of robotic systems. Its ability to focus a laser beam on a product’s surface in all six spatial coordinates makes it possible to process products of complex shapes in one technological cycle.The team incorporated a computer vision system to make it easier for the operator to determine the position of the product in reality. In addition to its position, the system reports on its orientation and geometrical parameters to ensure precision laser processing.<blockquote>“The existing robotic solutions are aimed at mass-production items. Once adjusted for a specific device, they perform a series of programmed actions again and again. This approach is not suitable for small-scale or one-of-a-kind equipment, whereas our technology can be easily customized for a new product,” explains&nbsp;Fedor Inochkin, a researcher at ITMO’s Institute of Laser Technologies<img data-fr-image-pasted="true" alt="" src="https://news.itmo.ru/en/science/life_science/news/13440/" class="fr-fic fr-dii"></blockquote><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MTk5Mzg4MjEyLTEzNDQwMDEuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">An operator positions a medical device and loads a 3D model into the system, selects the areas to be processed, and starts the program. The robot manipulator travels from one area to another with a laser beam. The procedure time varies depending on the chosen technology and item size. Photo courtesy of Evgeniy ShamshinA compact automated laser system for dental implants and their components is another device created by the team to serve exclusively dental purposes.&nbsp;<blockquote>“The systems have different functions, which affects their price and audience. If a customer needs to process a certain implant, like a dental one, they should purchase the second device, whereas the first one will be more beneficial with a wide range of items, including custom ones. For instance, a person who suffered from head injury can receive an individually designed skull implant. The multiuse system allows specialists to incorporate a desired set of properties into personalized implants in accordance to their unique shape,” notes Yulia Karlagina.&nbsp;</blockquote><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MTk5NDEwOTc4LTEzNDQwMDIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">An operator positions an implant, closes the complex’s cover, chooses the type of device under study and processing mode in the program, and, finally, starts the program. The implant is ready in 10-15 minutes. Photo courtesy of Evgeniy Shamshin<h2 dir="ltr" id="isPasted">Prospects of the technology</h2>The ITMO researchers in collaboration with their co-authors from Samara State University have carried out preliminary testing of both complexes, in particular preclinical trials of the processed medical devices. All samples were successfully tested in vitro and in vivo and are recommended for clinical trials.&nbsp;In the future, the developers plan to run acceptance tests to ensure the complexes are ready for operation and mass production. The team also intends to attract new industrial partners. In the long run, the systems are expected to be used in the production of medical equipment and implants.&nbsp;<blockquote>“We already have surface functionalization technologies for implants and soon we, with our colleagues from Laser Center, will start to look for new partners who will integrate them into their productions. If requested, we can also modify our products to the client’s needs – for instance, we can add a second robot manipulator that will upload new implants into the system instead of an operator,” notes Galina Romanova, a senior researcher at ITMO’s Institute of Laser Technologies.&nbsp;</blockquote><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MTk5NDQxMTQzLTEzNDQwMDMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">Galina Romanova. Photo courtesy of Evgeniy ShamshinThe project is implemented as part of the Government Decree No. 218 on request by Laser Center. Additionally, the developers continue to run research within ITMO’s 2030 Development Strategy.&nbsp;The research team has been working on the processing of medical titanium alloys for several years now. Together with their partners from Lenmiriot, they <a href="https://news.itmo.ru/en/science/new_materials/news/9655/" target="_blank">developed</a> a design of an implant surface that helps implants heal faster without inflammation. In 2022, Lenmiriot <a href="https://news.itmo.ru/en/startups_and_business/partnership/news/12313/" target="_blank">announced</a> the market launch of the dental implants made with the new laser technology.

The robot laser complex. Photo courtesy of Evgeniy Shamshin

Researchers from ITMO University have created a multipurpose robot complex for laser treatment of medical device surfaces, like those of dental and skull implants. The designed technology can be utilized to imbue metal implants with antibacterial and biocompatible properties, as well as mark medical items. All one needs to do is load a 3D model of an implant into a program, set a processing trajectory, and pick a surface attribute of choice.

A Multiuse Robot for Medical Applications Designed at ITMO

Effective time management is essential when conducting inspections on container ships, as it directly influences the cost-effectiveness and competitiveness of maritime cargo transportation. Inspection plays a&nbsp;crucial&nbsp;role in ensuring container ship safety, averting cargo damage,&nbsp;maintaining&nbsp;compliance with environmental rules, and preserving the vessel's value.&nbsp;Ensuring the safety and dependability of container ships involves employing fundamental methods such as contact-based Ultrasonic Thickness (UT) inspections and visual assessments.&nbsp;These inspections proactively&nbsp;identify&nbsp;potential issues before they escalate into significant concerns, thereby aiding in the prevention of maritime mishaps.&nbsp;&nbsp; Regularly inspecting cargo vessels becomes a complex task due to their significant size,&nbsp;necessitating&nbsp;careful maneuvering to access various compartments, structures, and cargo holds.&nbsp;Traditional approaches, like rope access and scaffolding, come with inherent risks and time consumption. Addressing these hurdles, innovative flying robots have&nbsp;emerged&nbsp;as&nbsp;an optimal&nbsp;solution for ship inspections.&nbsp;Avestec's&nbsp;intelligent airborne robot, SKYRON, effectively addresses the difficulties&nbsp;encountered&nbsp;in ship inspections.&nbsp;Watch the video above to see how SKYRON performed UT and visual inspections on a container ship in Vancouver, Canada. It focused on checking the back wall of the cargo hold, the guides for the columns, and the ship's side facing the water. This inspection happened while the ship was parked at the port to unload cargo, highlighting the importance of saving time. SKYRON is a highly advanced flying robot designed for various inspections, reducing risks for&nbsp;humans&nbsp;and cutting down on inspection time and costs.&nbsp;

Watch How Flying Robots Like SKYRON Are Redefining Safety Checks at Sea 

From Robo-Flyers to Smart Scans, Unveiling the Future of Vessel Safety and Performance

<img alt="A blue line drawing of a person wearing a hard hatDescription automatically generated" src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695324692147-1695324692147.png" id="isPasted" class="fr-fic fr-dii">For the management of hydropower conveyance systems within pulp and paper facilities, surge tanks stand as critical components, serving as key players for efficient operations. A robust maintenance strategy and a secure work environment are paramount for these systems to thrive. Non-destructive testing (NDT) inspections emerge as a vital practice in maintaining surge tank performance. However, circumstances can be challenging, as showcased in a recent case undertaken by Avestec.&nbsp;The situation presented a dual challenge. Initially, accessing the surge tanks proved intricate due to obstructive towering trees on one front. Subsequently, a haunting specter of potential asbestos contamination&nbsp;emerged&nbsp;on the opposing&nbsp;side, adding&nbsp;an element of danger that&nbsp;couldn&#39;t&nbsp;be ignored. The confluence of these obstacles demanded an innovative approach, one that safeguarded&nbsp;workers&rsquo;&nbsp;well-being while ensuring thorough assessments of the tanks.&nbsp;<img alt="A group of men in safety gearDescription automatically generated with medium confidence" src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695324672404-1695324672404.jpeg" class="fr-fic fr-dii">&nbsp;In response to these complex challenges,&nbsp;Avestec&nbsp;utilized&nbsp;the capabilities of SKYRON, an intelligent flying robot.&nbsp;SKYRON&nbsp;emerged&nbsp;as the perfect&nbsp;inspection&nbsp;tool, smoothly navigating the intricate&nbsp;environment&nbsp;and flawlessly conducting ultrasonic thickness tests on the tank shell.&nbsp;By doing so, it enabled inspection without exposing human personnel to the hazards.&nbsp;Operating SKYRON was a precise coordination of human&nbsp;expertise&nbsp;and&nbsp;cutting-edge&nbsp;technology.&nbsp;A skilled pilot and inspector collaborated to guide the flying robot with precision. While the robot soared over the dense canopy of trees, the team&nbsp;maintained&nbsp;a safe distance, evading the potential asbestos threat. A&nbsp;first-person&nbsp;view&nbsp;camera&nbsp;system&nbsp;interface&nbsp;facilitated&nbsp;real-time monitoring as SKYRON ventured beyond the limits of direct line of sight.&nbsp;The&nbsp;results&nbsp;of&nbsp;Avestec&#39;s&nbsp;inspection were&nbsp;truly&nbsp;remarkable.&nbsp;Within&nbsp;a single day, SKYRON&nbsp;conducted thorough&nbsp;inspections,&nbsp;collecting&nbsp;577 ultrasonic thickness readings in&nbsp;60 minutes&nbsp;flight time.&nbsp; <img alt="A large metal tower with a ladderDescription automatically generated" src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695324672418-1695324672418.jpeg" class="fr-fic fr-dii">&nbsp;This achievement marked a transformative change in the methods of inspecting surge tanks. Through overcoming challenging conditions and executing seamless inspections, SKYRON proves its advantages over conventional approaches, triumphing even in the most demanding environments.&nbsp;

SKYRON's Cutting-Edge Solution Overcomes Hazardous Obstacles and Delivers Rapid Results

Revolutionizing Tank Inspections: How Intelligent Flying Robots Ensure Safety and Efficiency

Ever wondered how robotic arms achieve precise movement in manufacturing, healthcare, or logistics? It’s all about smart design — from sturdy components and kinematics to advanced control systems. Discover how these elements work together to deliver accuracy, flexibility, and real-world efficiency.

What are manipulator robots? Understanding their Design, Types, and Applications

Modern High Tech Authentic Robot Arm Holding Contemporary Super Computer Processor

This technical article examines the mechanical engineering principles of CoWoS Packaging Technology, its mechanism, control systems, and performance metrics that define this cutting-edge technology! 

CoWoS Packaging Technology: Advanced Automation Systems in Modern Industrial Packaging

Robotic joints, which are sometimes known as axes, are the moveable parts of a robot that cause relative motion between adjacent links. These links refer to the rigid components that connect the joints to ensure their proper and straightforward operation. 

Robot Joints: An In-Depth Guide to Anatomy, Physics and Challenges in Design

Robotic arms in the industry based on PID loop 

This guide delves into the basics and components of PID control, implementation, application examples, impacts, challenges, and solutions aiming to provide readers with a thorough understanding of this integral concept.

PID Controller & Loops: A Comprehensive Guide to Understanding and Implementation

Delta arms have become an essential part of modern robotics and automation. These versatile machines are at the forefront of innovation, driving efficiency and precision in numerous industries. In this comprehensive guide, we will explore the fascinating world of delta arms, from their basic structure and components to their wide-ranging applications and future developments.<h2 dir="ltr">What is a Delta Arm?</h2>A delta arm, also known as a delta robot, is a type of parallel robot that features a unique, triangular base and lightweight, interconnected arms. These arms can move in all three dimensions and are attached to a central <a href="https://www.wevolver.com/article/end-effector" target="_blank" rel="noopener noreferrer">end effector</a>, which can be customized for various applications, such as gripping, cutting, or dispensing materials. Delta arms are known for their speed, precision, and flexibility, making them highly sought-after in industries like manufacturing, packaging, and assembly.<h2 dir="ltr">History of Delta Arms</h2>The delta arm was first invented in the early 1980s by Swiss engineer Reymond Clavel, who was inspired by the desire to create a high-speed, accurate robot for use in the watchmaking industry. Clavel's invention sparked a revolution in robotics, with companies and researchers around the world quickly adopting and refining delta arm technology. Notable companies like ABB, FANUC, and KUKA participated in the development of delta arms. Today, delta arms are used in a wide array of applications, from <a href="https://www.wevolver.com/article/understanding-industrial-automation-a-comprehensive-guide" target="_blank" rel="noopener noreferrer">industrial automation</a> to 3D printing and even medical procedures.<h2 dir="ltr">How Delta Arms Work?</h2>At the heart of the delta arm's impressive performance are its unique design and kinematics. Delta arms employ a parallel mechanism, which allows the arms to move simultaneously and in harmony with one another. This setup enables the end effector to maintain a consistent orientation while moving through its workspace, resulting in highly precise and accurate motion.To better understand delta arm kinematics, it's helpful to visualize the system as a series of interconnected links and joints. As the arms move, the joints rotate, causing the end effector to translate through space. By carefully controlling these rotations, the delta arm can achieve a wide range of positions and orientations with remarkable speed and precision.<iframe width="640" height="360" src="https://www.youtube.com/embed/3-NeyKcyfC4?&amp;ab_channel=ArturWiebe&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="840383707" title="#BottleCapChallenge" data-gtm-yt-inspected-23="true"></iframe> A prototype of a delta arm showcases its movements by precisely attaching a bottle cap and then removing it with ease.<h2 dir="ltr">Components of Delta Arms</h2><h3 dir="ltr">Base</h3>The base of a delta arm serves as the foundation for the entire system, providing stability and support for the arms and end effector. Bases come in various designs, but most commonly feature a triangular or circular shape. The material used in the base's construction can vary depending on the intended application, with common choices including aluminum, steel, and even 3D-printed materials for lighter-weight applications.<h3 dir="ltr">Arms</h3>The arms of a delta robot are crucial to its movement and overall performance. These slender, interconnected components enable the robot to move with great speed and agility. Arms can be constructed from a variety of materials, such as carbon fiber, aluminum, or steel, with the choice of material typically dependent on the required strength, weight, and cost considerations for the specific application.<h3 dir="ltr">Joints</h3>Joints are essential to the delta robot kinematics, providing the flexibility and range of motion necessary for the robot to function effectively in its working envelope while exhibiting good repeatability. There are several types of joints used in delta arms, including revolute, prismatic, and ball-and-socket joints. The selection of the appropriate joint type depends on the desired movement and performance characteristics for the specific delta arm application.<iframe width="640" height="360" src="https://www.youtube.com/embed/ih3oXigeY-U?&amp;ab_channel=KeyanGhazi-Zahedi&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="739495307" title="Prismatic Joint" data-gtm-yt-inspected-23="true"></iframe>A prismatic joint<h3 dir="ltr">End effector</h3>The end effector is the business end of a delta arm, responsible for carrying out the tasks the robot is designed to perform. End effectors can include grippers, cutters, welding tools, dispensers, vision systems, and more. Some delta arms even feature interchangeable end effectors, allowing them to be easily adapted for different tasks as needed.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgwNTQyNjMzMTMwLTE2ODA1NDI2MzMxMzAucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="720" height="540" class="fr-fic fr-dii">Fig. 1: A delta arm robot with a customizable end effectorSuggested reading:&nbsp;<a href="https://www.wevolver.com/article/what-are-end-effectors-in-robotics-types-of-end-effectors-applications-future/" rel="noopener noreferrer" target="_blank">What are End Effectors in Robotics? Types of End Effectors, Applications, Future</a><h3 dir="ltr">Actuators</h3>Actuators are the driving force behind a delta arm's movement, converting electrical or pneumatic energy into mechanical motion. There are several types of actuators used in delta arms, including stepper motors, servo motors, and pneumatic cylinders. The choice of actuator depends on factors such as the required speed, precision, and force, as well as cost considerations for the specific application.Suggested reading:&nbsp;<a href="https://www.wevolver.com/article/what-is-an-actuator-principles-classification-and-applications" rel="noopener noreferrer" target="_blank">What is an Actuator? Principles, Classification, and Applications</a><h3 dir="ltr">Controllers</h3>Controllers play a crucial role in delta arm operation, managing the movement and coordination of the robot's various components. Controllers can range from simple microcontrollers to more advanced robot control systems incorporating artificial intelligence and machine learning capabilities. The selection of a controller depends on factors such as the complexity of the delta arm's movements, the desired level of precision, and the integration requirements with other systems or devices.<h2 dir="ltr">Applications of Delta Arms</h2><h3 dir="ltr">Industrial Automation</h3>Delta arms have become a staple in production line automation, with their speed, precision, and flexibility making them ideal for applications such as manufacturing, packaging, material handling,&nbsp;and assembly. In manufacturing, delta arms are used to pick and place components, assemble products, and even perform quality control checks. In packaging, they can quickly and accurately fill containers, apply labels, and seal packages. Additionally, delta arms have found a home in assembly lines, where their speed and dexterity make them perfect for tasks such as electronic component placement and assembly.Suggested Reading:&nbsp;<a href="https://www.wevolver.com/article/what-are-robotic-assembly-lines-history-components-advantages-limitations-applications-and-future" target="_blank">What are Robotic Assembly Lines? History, Components, Advantages, Limitations, Applications, and Future</a><h3 dir="ltr">3D Printing</h3>Delta arms have made a significant impact on the world of 3D printing, with their unique design and movement capabilities making them well-suited for this application. Delta arm-based 3D printers can achieve high levels of accuracy and resolution, while their lightweight arms allow for rapid movement and reduced print times.<h3 dir="ltr">Medical and Pharmaceutical Industries</h3>Delta arms are increasingly finding applications in the medical and pharmaceutical fields, with their precision and dexterity making them ideal for tasks such as drug discovery, surgery, and patient care. In drug discovery, delta arms can be used to quickly and accurately handle and analyze samples, speeding up the research process.&nbsp;In surgery, delta arms have been employed in procedures such as laparoscopy and microsurgery, where their precision and minimal invasiveness offer significant benefits.&nbsp;In patient care, delta arms can be used for tasks such as administering medications, drawing blood samples, and even assisting with rehabilitation exercises.<h3 dir="ltr">Food and Beverage Industry</h3>The food industry has also embraced delta arms for their speed, accuracy, and hygienic properties. Delta arms are used in various aspects of food processing, from sorting and grading fruits and vegetables to handling delicate baked goods. In packaging, delta arms can rapidly and accurately fill containers, apply labels, and seal packages, ensuring product quality and safety. Additionally, delta arms have been utilized in quality control tasks, such as inspecting food items for defects or contamination.<h3 dir="ltr">Other Applications</h3>While the aforementioned industries represent some of the most common applications for delta arms, these versatile robots have also found their way into many other fields. For example, delta arms have been used in research and development, aerospace, and entertainment industries, to name a few. As technology advances and delta arms become even more adaptable and efficient, their potential applications will undoubtedly continue to grow.<h2 dir="ltr">Choosing and Implementing Delta Arms</h2>When selecting a delta arm for a specific application, there are several factors to consider, such as payload capacity, degrees of freedom, speed, accuracy, and cost. Payload capacity refers to the maximum weight the delta arm can carry or manipulate, and it is essential to choose a delta arm with the appropriate capacity for the intended task. Degrees of freedom is nothing but the number of joints present in the robot that allow it to move in the x, y, and z-axis. Speed and accuracy are also vital considerations, as they will impact the overall performance and efficiency of the delta arm in its specific application. Finally, cost is always a factor, with the selection of materials, actuators, and controllers playing a significant role in determining the overall price of the delta arm.Suggested reading:&nbsp;<a href="https://www.wevolver.com/article/types-of-robots" target="_blank">7 Types of Industrial Robots: Advantages, Disadvantages, Applications, and More</a><h3 dir="ltr">Integration with Existing Systems</h3>Integrating a delta arm into an existing system can be a complex process, requiring careful planning and consideration. Factors such as compatibility with existing hardware and software, workspace constraints, and safety protocols must all be taken into account. Additionally, training employees to operate and maintain the delta arm is another critical aspect of successful integration.<h3 dir="ltr">Maintenance and Troubleshooting</h3>Regular maintenance is essential to keep a delta arm functioning optimally and ensure a long service life. This maintenance may include tasks such as lubricating joints, checking for wear and tear on components, and calibrating the system for optimal performance. Familiarity with common troubleshooting techniques is also crucial, as it allows operators to quickly diagnose and resolve issues that may arise during operation. Potential issues can include mechanical failures, software glitches, and problems with the end effector or other components.<h2 dir="ltr">The Future of Delta Arms</h2>Recent advancements in delta arm technology have focused on improving performance, efficiency, and versatility. Innovations such as lightweight materials, advanced actuators, and more sophisticated controllers have all contributed to the ongoing evolution of delta arms. As technology continues to advance, we can expect even more impressive capabilities from these remarkable machines.<iframe width="640" height="360" src="https://www.youtube.com/embed/yXNbG4P8fTU?&amp;t=37s&amp;ab_channel=ABBRobotics&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="421892765" title="IRB 365 FlexPicker® Delta Robot - lightweight picking, reorientating and packing" data-gtm-yt-inspected-23="true"></iframe>The ABB FlexPicker: a high-throughput variant of a delta robot arm suitable for rapid pick and place applications and more<h3 dir="ltr">The Role of AI in Delta Arms</h3>Artificial intelligence (AI) is increasingly being integrated with delta arms, allowing for enhanced performance and adaptability. AI-powered delta arms can learn from experience, adapting their movements and strategies to optimize efficiency and effectiveness. As AI continues to advance, we can expect even more exciting developments in the world of delta arm robotics.<h3 dir="ltr">The Impact of Delta Arms on the Workforce</h3>The rise of delta arms in various industries has raised questions about their potential impact on the workforce and job market. While it is true that delta arms can automate many tasks that humans once performed, they can also complement human labor rather than replace it entirely. By taking over repetitive, mundane, or dangerous tasks, delta arms can free up human workers to focus on more creative, strategic, and higher-value activities. Nevertheless, the integration of delta arms in the workforce also presents challenges, such as the need for retraining and upskilling workers to adapt to new technologies and roles. Addressing these challenges through effective education and training programs will be essential for ensuring a smooth transition and maximizing the benefits of delta arm technology.<h2 dir="ltr">Conclusion</h2>In conclusion, delta arms have made a significant impact on the world of robotics and automation, offering speed, precision, and flexibility that make them invaluable in a wide range of industries. From manufacturing and packaging to medical procedures and 3D printing, these versatile robots are revolutionizing the way we work and live. As technology continues to advance, we can expect even more exciting developments in the world of delta arms, making them an essential part of our future.<h2 dir="ltr">Frequently Asked Questions (FAQs)</h2><ol><li dir="ltr" style="font-weight: bold;">What are the main benefits of using delta arms in automation?</li></ol>Delta arms offer numerous advantages, such as speed, precision, flexibility, and cost-effectiveness, making them ideal for a wide range of automation tasks.<ol start="2"><li dir="ltr" style="font-weight: bold;">How do delta arms compare to other robotic arms in terms of performance?</li></ol>Delta arms are known for their unique combination of speed, precision, and agility, which sets them apart from other types of robotic arms.<ol start="3"><li dir="ltr" style="font-weight: bold;">Can delta arms be used in hazardous environments?</li></ol>With appropriate modifications and precautions, delta arms can be used in hazardous environments, such as those involving high temperatures, radiation, or corrosive substances.<ol start="4"><li dir="ltr" style="font-weight: bold;">What are the main challenges in implementing delta arms in existing systems?</li></ol>Potential challenges include integration with existing hardware and software, workspace constraints, safety protocols, and training employees to operate and maintain the delta arm.<ol start="5"><li dir="ltr" style="font-weight: bold;">How much do delta arms typically cost?</li></ol>The cost of a delta arm can vary widely, depending on factors such as size, capabilities, and materials used. Prices can range from a few thousand dollars for basic models to tens or even hundreds of thousands of dollars for more advanced systems.<ol start="6"><li dir="ltr" style="font-weight: bold;">What is the expected lifespan of a delta arm?</li></ol>The average lifespan of a delta arm depends on factors such as usage, maintenance, and build quality. With proper care, a delta arm can typically last for many years or even decades.<ol start="7"><li dir="ltr" style="font-weight: bold;">Are delta arms safe to use alongside human workers?</li></ol>With appropriate safety measures and best practices in place, delta arms can be safely used in collaborative work environments, complementing human labor and improving overall efficiency.<h2 dir="ltr">References</h2>[1] Delta robot. (2023, March 7). In Wikipedia. <a data-fr-linked="true" href="https://en.wikipedia.org/wiki/Delta_robot" id="isPasted" rel="noopener noreferrer" target="_blank">https://en.wikipedia.org/wiki/Delta_robot</a>[2] Pierrot F, Reynaud C, Fournier A. DELTA: a simple and efficient parallel robot. Robotica. Cambridge University Press; 1990;8(2):105–9.&nbsp;

Delta arms are high-performance robotic arms that use parallel linkages and jointed construction to provide fast, precise, and flexible movement, making them ideal for various industrial and manufacturing applications.

Robotic Arms

Technical Guides

What are manipulator robots? Understanding their Design, Types, and Applications...

CoWoS Packaging Technology: Advanced Automation Systems in Modern Industrial Packaging...

Pick and Place Robots: An In-Depth Guide to Their Functionality and Applications...

Robot Joints: An In-Depth Guide to Anatomy, Physics and Challenges in Design...

PID Controller & Loops: A Comprehensive Guide to Understanding and Implementation...

The Ultimate Guide to Delta Arms: Revolutionizing Robotics and Automation