<h3 id="isPasted">Let’s explore some methods on how to achieve this.</h3>First and foremost, there exists the “Straightforward Programming” approach. This means: getting a skilled person to meticulously describe a task, to such an extent that one can write either a simple script or an intricate algorithm that can guide the robot to do the task. This method is great and highly effective when the task is well-defined and specific, something often done in industrial automation. However, the complexity can escalate rapidly, making it challenging. Thus, if the approach is successful, that’s excellent – but if not, we must ponder – what alternative strategies can we employ?Secondly, an alternative approach to consider is “Programming by Demonstration“. This method involves guiding the robot in performing the task. By equipping the robot with advanced force sensing capabilities and vision algorithms, this technique can prove to be quite effective. However, it’s crucial to note that its effectiveness is reliant on the robot’s abilities. Usually, robots are equipped with rudimentary grippers rather than adept, dexterous hands, which can limit the complexity of tasks they can perform.&nbsp;Okay, so how can we effectively record human actions for a robot to mimic?Ideally, we’d guide a human through the task, document it in detail, and then replay it to the robot. We might even consider feeding this information into a machine-learning system, enabling it to learn from our human-guided demonstration. Alternatively, at the very least, we can analyze these human activities, deconstruct them into simpler parts, and use this information to develop more efficient algorithms in the way that a song is deconstructed in a recording studio.&nbsp;OK, so how about, filming the person doing the task and using “machine vision” to track what they are doing?It’s certainly a good starting point. This approach provides valuable insights into the nuances of human behaviour – such as hand-switching moments, object-holding techniques, and tool usage – But our experience is that it’s really hard to track human movement&nbsp;using cameras. You need clear lines of sight, and you’ll have lots of trouble tracking the fine movements of the fingers and hands.&nbsp;Even the most advanced image-based tracking systems, while capable of monitoring hands with minimal occlusion, fail to consistently track all joint movements. Consequently, the resultant data tends to be somewhat inconsistent and noisy.&nbsp;Alright, what about using motion capture?Motion capture systems, commonly employed in film production and sports science, operate by attaching easily visible markers to the individual, such as passive elements like white dots or active elements like pulsed LEDs. This method certainly helps with the accuracy of tracking. However, it doesn’t entirely overcome the previously mentioned issues related to occlusion. Additionally, calibrating the optical systems to an error margin of less than a few millimetres is quite challenging – which means that if the task involves intricate movements, such as rolling a pen between fingers, it’s likely it’ll miss a lot.&nbsp;&nbsp;Okay. I need it to be accurate, what’s my best option then?For precise human movement tracking, your best bet is to use a tight-fitting movement-tracking&nbsp;<a href="https://www.shadowrobot.com/shadow-hand-glove/" target="_blank">glove</a>. This glove can incorporate a multitude of techniques to achieve such a feat, some even providing additional features to assist the operator (and some that make it more difficult for the operator). For instance, some gloves can relay a sense of weight, touch, or stiffness, all functionalities grouped under the term ‘haptics.’ The realm of haptics is broad, including diverse technologies from ultrasound devices transmitting signals to fingers, electrical skin stimulation, vibrating piezoelectric elements, to inflating air balloons, miniature motors, or linear actuators, among many others.Gloves that measure human hand movement can do it by utilising:<ul><li dir="ltr">Sensors that fit around the joints and track the specific movement of the joint</li><li dir="ltr">Mechanical linkages that go from one end of the finger to the other and track the whole movement of the finger</li><li dir="ltr">Inertial sensors (that clever combination of accelerometer, gyroscope and possibly magnetometer) - worn at various points on the hand to follow the movement</li><li dir="ltr">Wireless measuring sensors - detecting the location of the fingers relative to a source worn on the hand somewhere.</li><li dir="ltr">Cameras that look away from the hand and track the movement of the world. (I’m not sure I’ve seen this one used in anger but I’m sure it’s possible!)</li></ul> Even then several variations of these technologies exist. For instance, some sensors measure the flex of a glove, correlating it to joint movement. Other types of sensors stretch and track their own deformation to gauge distance. Occasionally, mechanical linkages are placed over individual joints for precise motion tracking.&nbsp;Each of these methods presents a unique approach to capturing and interpreting human movement<h3 id="isPasted">Our experience:</h3>From our extensive experience, we have found that accurately measuring the mechanical properties of the human hand can be challenging, as it often leads to significant inaccuracies. The placement of the measurement sensor is crucial, and unfortunately, the potential for slippage is high – gloves and similar apparatus tend to shift excessively, complicating the data collection. Relating the measurements to the actual rotation of a finger joint’s axis, especially the thumb, is another daunting task!<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3NTM2Nzk3ODE3LWxvbmcgLSBQb2xoZW11cyAmIFNoYWRvdyAoMTkyMCB4IDEwODAgcHgpLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Mechanical linkages, although valuable, can quickly become cumbersome. There is also a risk that they could discomfort the operator to the point where task completion becomes impossible (too heavy, too rigid etc). Inertial sensors present a viable option; however, (once again) preventing them from drifting is generally difficult, and the precision level they offer often falls short of what we needWe have found that wireless measurement sensors, particularly those from&nbsp;<a href="https://www.shadowrobot.com/shadow-robot-polhemus/" target="_blank">Polhemus</a>, give us excellent results. These sensors entail a transmitting source worn on the palm and receivers located at the fingertips, providing us with high-quality data for hand movement tracking. However, there remains the challenge of locating the hand within the wider three-dimensional space. For this, we have found that the HTC Vive is an excellent solution and so you will often see the Vive tracker featured in our demonstrations.Now – what we want is to accurately know where to put the robot hand – which isn’t quite the same thing as knowing accurately where the human hand is. We have to take the data from the human hand and map it to the robot hand. This is a kinematic challenge – the movements of the human and the movements of the robot don’t agree entirely, and we need to convert one to the other. The best method for mapping human data to a robot hand will depend on the specific application. For example, if you need to control a robot’s hand in real-time, then direct mapping may be the best option. If you are only interested in controlling a robot’s hand for offline analysis, then inverse kinematics or hybrid mapping may be more feasible options. Needless to say, the more complex the robot the more challenging this becomes. Lucky for us complex robots are a speciality at Shadow!&nbsp;So, the human wears a set of gloves, and they control the robot to do the task. After completing a few rudimentary training tasks the operator gets the hang of the process and they are doing it pretty well. What do we do to learn from this?We can precisely monitor the instructions sent to the robot and reproduce them. This is ideal when operating in a controlled environment where all variables remain consistent – a scenario commonly found in an automation setting within a quality assurance laboratory. In the past, we employed this technique to build a functional robotic kitchen. While it allows you to execute repetitive tasks, it lacks the ability to cope with variation.We can capture and analyze data from numerous tests, assessing them for consistency. If the results are largely similar, they can be directly applied. However, if they vary a lot, we can delve into our algorithm toolbox to try and understand these discrepancies. This is where it helps to have a <a href="https://www.shadowrobot.com/artificial-intelligence-machine-learning/" target="_blank">wide range of data</a> from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">the robot</a>, from force, <a href="https://www.shadowrobot.com/shadow-robot-11-lessons-about-tactile-sensors/" target="_blank">touch</a>, and joint sensors. The question arises: Can we employ data from vision, such as cameras aimed at the work area, to discern what’s different? To be effective, this vision data needs to be synchronized with the robot sensor data, necessitating a shared framework for all data streams.We can gather data from multiple tests and use it by “Feeding it to the AI” – meaning use it to instruct a machine learning system on how to replicate similar movements from similar data.This can be done in several ways.We might train a traditional learning model on the datasets so that similar inputs generate similar outputs. Alternatively, if we have enough data, we could use a transformer model to generate movements. We can even break down the data into smaller segments and train a network to assemble these fragments into the correct sequences.Given the impressive capabilities of modern machine learning systems in generating text, images, video, and even code, this approach appears highly promising – and it’s only the beginning!So, does this sound like it could help with a problem that you face? If it does, then let’s have&nbsp;<a href="https://www.shadowrobot.com/blog/data-humans-to-robots/hand@shadowrobot.com" target="_blank">a chat to</a> see if the Shadow Robot team can help!

Humans excel at manual tasks, and ideally, robots should surpass us in these functions. We have the advantage due to our superior dexterity, and we can perform tasks that robots should, in theory, be capable of.
As we advance robotics our goal is to effectively transfer our human skills to robots.

Getting Data from Humans to Robots

There are many robot arms on the market, and you&rsquo;re probably familiar with a few. Your modern cobot ranges in cost from &euro;25k to &euro;200k &ndash; and that is for something with 6 or 7 degrees of freedom (DoF). So you might say &ldquo;&euro;4k-&euro;14k per DoF&rdquo; and use that as a &ldquo;rule of thumb&rdquo; to price off-the-shelf commercial robots.There are plenty of people with very low-cost robot hands and arms out there &ndash; if you&rsquo;re a hobbyist or you love to prototype, then that&rsquo;s where you should go! A quick look online finds 3d printed structures for a few tens or hundreds of Euro to print, although you still have to supply valves and compressors and electronics and maybe sensing. A fun project for a hardware developer! Then there are some cool hobbyist complete systems &ndash; I can find a 6DoF robot arm on Amazon for &euro;120 today, which would be great for teaching beginners the fundamentals of robot operation.But if you&rsquo;re planning to do work with the robot, then you want a complete system. And you want something that will lift a few kilos, has the sensors and motors to move with precision and speed across a decent set of kinematics, has safety circuits and controls built in, and supports the &ldquo;mechanical engineering&rdquo; interfaces allowing you to build it into your lab properly. Suddenly you&rsquo;re back to &ldquo;it&rsquo;s going to be a 5 figure sum&rdquo;. And there are good reasons for that &ndash; you&rsquo;re getting something with mechanical strength, highly engineered sensors, motors and gearboxes, and sophisticated control systems to make it work.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NzcwMTg4NzE4LVVudGl0bGVkICgyOTcgw5cgMjEwbW0pICgyMDAgw5cgMjAwbW0pICgzMDAgw5cgMTAwbW0pLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"> And then you&rsquo;re going to want documentation, software interfaces, and support when it doesn&rsquo;t do what you want.Don&rsquo;t get me wrong &ndash; that isn&rsquo;t because you bought something that wasn&rsquo;t up to the job. Most support tickets in robotics come with one of two sentences at the start: &ldquo;I was doing &hellip; and it broke&rdquo; or &ldquo;How can I get it to do &hellip;&rdquo;. And we&rsquo;ve seen enough robot arms slam things into tables to know that breaking things is part of the game here &ndash; honestly, if you&rsquo;re not breaking the odd robot you&rsquo;re probably not doing research at the frontiers!So that&rsquo;s a real, commercial robot arm. Assume &euro;5k per DoF for a generic cobot, and maybe &euro;8-10k if you&rsquo;re getting something really amazing. Even the dinky 6DoF arm we use in one of our production processes was &euro;17k for 6DoF (to be fair to it, the precision is really impressive, but it&rsquo;s still shoebox-size).However, <a href="https://www.shadowrobot.com/" target="_blank" rel="noopener noreferrer">Shadow Robot</a> isn&#39;t in the robot arm business. We make what the marketing people politely call &ldquo;end of arm tooling&rdquo;. This space ranges from welding guns to lasers, from two jaw grippers to vacuum suckers, and right the way over there &mdash;&gt; robot hands. And according to the good people at the International Federation of Robotics, adding EoAT and the other parts needed to make it work can triple the price of a robot arm!A simple gripper that opens and closes without any fine movement control, without force sensing and without position sensing might well be included in the price of the cobot. Lots of companies do that and it works well for their customers. But we&rsquo;re here for data. We want precise control. We want to be able to move each fingertip independently and position it accurately in space with respect to the other fingertips.In short, we want a robot hand where each finger behaves like a small robot arm &ndash; and we want all that complexity and performance in something 10x smaller! What would that cost? Well, &euro;5k per DoF gets us to &euro;100k for 20DoF &hellip; and the price for the full Shadow Hand Plus today is &hellip; &euro;110k including shipping, installation, training and support!Of course, we can make simpler Hands with fewer degrees of freedom &ndash; and fewer fingers &nbsp;&ndash; and these go right down to a 2-finger 7DoF system which is more like &euro;45k including shipping, installation, training and support. You can see the full range <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">here</a>, and if you want to talk to us about getting one, then <a href="https://www.shadowrobot.com/contact/" target="_blank" rel="noopener noreferrer">get in touch</a>!&nbsp;

Spending your hard-earned budget on a robot is a step you don’t want to take lightly, Any worthwhile robot is likely to represent a significant piece of capital equipment.

So, let’s take a look at some hardware and what you might expect to pay for.

How Much does a Robot Hand Cost?

In this episode, we talk about how ostriches are inspiring a new generation of bipedal robots and the innovation enabling remote endovascular surgery. As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/e2238814-9212-4010-afb6-ac48d637140f?dark=false" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe><hr>This podcast is sponsored by <a href="https://www2.mouser.com/" id="isPasted" rel="nofollow" target="_blank">Mouser Electronics</a>.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651565842197-Mouser-sponsor-1-a.jpg" style="width: 766px;" class="fr-fic fr-dib"><hr>(0:50) -&nbsp;<a href="https://www.wevolver.com/article/birdbot-a-3d-printed-bipedal-robot-offers-major-efficiency-gains-through-avian-biomimicry" target="_blank">Using Bird Biology To Make Efficient Walking Robots</a>(20:45) -&nbsp;<a href="https://www.wevolver.com/article/joystick-operated-robot-could-help-surgeons-treat-stroke-remotely" target="_blank">Joystick-operated Robot For Treating Strokes</a>Episode 68 was brought to you by Mouser Electronics, Farbod &amp; Daniel&rsquo;s favorite electronics distributor. Click <a href="https://www.mouser.com/applications/future-robotics-singularity/" target="_blank">here</a> to read the article discussing the merger of technology and humanity.<hr>About the podcast:Every day, some of the most innovative universities, companies, and individual technology developers share their knowledge on Wevolver. To ensure we can also provide this knowledge for the growing group of podcast listeners, we started a collaboration with two young engineers, Daniel Scott Mitchell &amp; Farbod Moghaddam who discuss the most interesting content in this podcast series.&nbsp;To learn more about this show, please visit the <a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">shows page</a>. By following the page, you will get automatic updates by email when a new show is published.Be sure to give us a follow and review on <a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" rel="nofollow" target="_blank">Apple</a> podcasts, <a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" rel="nofollow" target="_blank">Spotify</a>, and most of your favorite podcast platforms!Take a few seconds to leave us a review. It really helps! <a href="https://apple.co/2RIsbZ2" rel="nofollow" target="_blank">https://apple.co/2RIsbZ2&nbsp;</a>if you do it and send us proof, we&rsquo;ll give you a shoutout on the show.

The prototype motorized BirdBot, seen here with legs in the locked position, uses a quarter the energy of its immediate predecessor.

In this episode, we talk about how ostriches are inspiring a new generation of bipedal robots and the innovation enabling remote endovascular surgery. 

Podcast: Bird-inspired Bipedal Robots & Robotic Surgeons

This article was discussed in our <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1651619086659000&usg=AOvVaw31M1Y-wsRw4c0h8-at-VxZ" href="https://www.wevolver.com/profile/the.next.byte" id="isPasted" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/e2238814-9212-4010-afb6-ac48d637140f?dark=false" class="fr-draggable" data-gtm-yt-inspected-1_19="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe>The full article will continue below.<hr>This article was first published on <a href="https://news.mit.edu/2022/robot-stroke-treatment-remote-0413" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>. &nbsp;<hr>MIT engineers have developed a telerobotic system to help surgeons quickly and remotely treat patients experiencing a stroke or aneurysm. With a modified joystick, surgeons in one hospital may control a robotic arm at another location to safely operate on a patient during a critical window of time that could save the patient&rsquo;s life and preserve their brain function.The robotic system, whose movement is controlled through magnets, is designed to remotely assist in endovascular intervention &mdash; a procedure performed in emergency situations to treat strokes caused by a blood clot. Such interventions normally require a surgeon to manually guide a thin wire to the clot, where it can physically clear the blockage or deliver drugs to break it up.One limitation of such procedures is accessibility: Neurovascular surgeons are often based at major medical institutions that are difficult to reach for patients in remote areas, particularly during the &ldquo;golden hour&rdquo; &mdash; the critical period after a stroke&rsquo;s onset, during which treatment should be administered to minimize any damage to the brain.The MIT team envisions that its robotic system could be installed at smaller hospitals and remotely guided by trained surgeons at larger medical centers. The system includes a medical-grade robotic arm with a magnet attached to its wrist. With a joystick and live imaging, an operator can adjust the magnet&rsquo;s orientation and manipulate the arm to guide a soft and thin magnetic wire through arteries and vessels.The researchers demonstrated the system in a &ldquo;phantom,&rdquo; a transparent model with vessels replicating complex arteries of the brain. With just an hour of training, neurosurgeons were able to remotely control the robot&rsquo;s arm to guide a wire through a maze of vessels to reach target locations in the model.&ldquo;We imagine, instead of transporting a patient from a rural area to a large city, they could go to a local hospital where nurses could set up this system. A neurosurgeon at a major medical center could watch live imaging of the patient and use the robot to operate in that golden hour. That&rsquo;s our future dream,&rdquo; says Xuanhe Zhao, a professor of mechanical engineering and of civil and environmental engineering at MIT.Zhao and his team have <a href="https://www.science.org/doi/10.1126/scirobotics.abg9907" target="_blank">published their findings</a> today in Science Robotics. MIT co-authors include lead author Yoonho Kim, Emily Genevriere, and Jaehun Choe, along with Pablo Harker, Robert Regenhardt, Justin Vranic, Adam Dmytriw, and Aman Patel at Massachusetts General Hospital (MGH), and Marcin Balicki of Philips Research North America.<iframe src="https://www.youtube.com/embed/IYlHl6h8lm4?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 767px; height: 427px;" data-gtm-yt-inspected-1_19="true" id="391874016"></iframe><h2 id="isPasted">Tilt and twist</h2>Endovascular surgery is a specialized, minimally invasive procedure that involves carefully twisting and guiding a thin medical wire through the body&rsquo;s arteries and vessels to a target location, in a way that avoids damaging vessel walls. The procedure typically requires years of training for a surgeon to master.Robotic systems are being explored as assistive technologies in endovascular surgery. These systems mainly involve motor drives that advance and retract a wire while twisting it through the body&rsquo;s vasculature.&ldquo;But having a robot twist with the same level of sophistication [as a surgeon] is challenging,&rdquo; Kim notes. &ldquo;Our system is based on a fundamentally different mechanism.&rdquo;The team&rsquo;s new system builds on <a href="https://news.mit.edu/2019/robot-brain-blood-vessels-0828" target="_blank">work from 2019</a>, in which they demonstrated steering a magnetically controlled thread through a life-sized silicone model of the brain&rsquo;s blood vessels. They did so at the time using a handheld magnet, about the size of a soup can, that they manually manipulated.They have since affixed the magnet to the end of a medical-grade robotic arm, which can be steered using a small joystick knob on a mouse. By tilting the joystick, researchers can tilt the magnet in an orientation that a magnetic wire can follow. Buttons on the mouse control a set of motorized linear drives, which advance and retract the wire to make it move forward and back.The wire is as thin and flexible as a conventional neurovascular guidewire, with a soft, magnetically responsive tip that follows and bends in the direction of a magnetic field.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ5OTIzNDk1NTU1LU1JVC1Sb2JvLVN0cm9rZS0wMi1wcmVzcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">The MIT team envisions that its robotic system could be installed at smaller hospitals and remotely guided by trained surgeons at larger medical centers. Courtesy of the researchers<h2>Finding a path</h2>The team tested the robotic system in MGH&rsquo;s Catheter Lab &mdash; an operating room with standard medical imaging equipment used in endovascular procedures. The researchers installed the robotic arm in the lab, along with a life-sized silicone model of blood vessels. They set the joystick, along with a monitor displaying a live video of the model, in a control room. From there, an operator watched the video while using the joystick to remotely steer the wire through the vessels.The team trained a group of neurosurgeons to use the robotic system. After just one hour of training, each surgeon was able to successfully operate the system to guide the wire through complex vessels that are difficult to navigate with a manual guidewire.The team also used the robotic system to clear simulated clots in difficult-to-reach areas in the model. They steered the guidewire through vessels, and around sharp corners and turns, to reach regions where the researchers simulated clots. Once they guided the wire to the clot, the surgeons proceeded with standard endovascular methods to thread a microcatheter along the wire to the site of the clot. They retracted the wire, leaving the catheter, which they then applied to successfully remove the clot.&ldquo;The primary purpose of the magnetic guidewire is to get to the target location quickly and safely, so that standard devices like microcatheters can be used to deliver therapeutics,&rdquo; Kim says. &ldquo;Our system is like a pathfinder.&rdquo;He hopes that the teleoperated system can help more patients receive time-critical treatment. He also sees benefits for surgeons, who typically perform such vascular procedures in the same room as the patient, while being exposed to radiation from X-ray imaging.&ldquo;The neurosurgeons can operate the robot in another room or even in another city without repeated exposure to X-rays,&rdquo; Zhao says. &ldquo;We are truly excited about the potential impact of this technology on global health, given that stroke is one of the leading causes of death and long-term disability.&rdquo;<hr>&quot;Reprinted with permission of <a href="https://news.mit.edu/2022/robot-stroke-treatment-remote-0413" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>&quot;&nbsp;

MIT engineers developed a telerobotic system to help surgeons remotely treat patients experiencing stroke or aneurysm. With a modified joystick, surgeons may control a robotic arm at another hospital to operate on a patient. Courtesy of the researchers

The system could provide teleoperated endovascular treatment to patients during the critical time window after a stroke begins.

Joystick-operated robot could help surgeons treat stroke remotely

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&amp;amp;-gas, environments.In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video will be showed during the international conference ICRA 2021, on May 31st.With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot&nbsp;combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (&gt; 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (&lt; 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.<hr>Further informationThe research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).&nbsp;Website: <a href="https://advr.iit.it/projects/inail-scc/teleoperazione" target="_blank">https://advr.iit.it/projects/inail-scc/teleoperazione</a>

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation.