A non-terrestrial network (NTN) is a type of wireless communication network used for applications such as mobile communications. The term refers to multilayered networks linking terrestrial base stations, ships at sea, high-altitude platform stations (HAPS), and communication satellites in space. These characteristics allow an NTN to overcome the limitations of land-based networks, dramatically expanding the coverage of the communication infrastructure to include areas such as mountain valleys and areas of the ocean far from land.Among non-terrestrial networks, this article focuses on wireless communication networks comprising elements at high altitudes, far above the land and the sea. R&amp;D, test beds, and demonstrations are starting to be rolled out as part of plans for the development of non-terrestrial networks in the stratosphere and outer space in preparation for the advent of the next-generation telecommunications standard known as Beyond 5G (5th Generation Mobile Communication System, B5G)/6G (6th Generation Mobile Communication System).<h2 id="isPasted">From the stratosphere to outer space. Configuration and features of non-terrestrial networks (NTNs).</h2><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1732785400292-FIGURE+1.jpg" style="width: 832px;" class="fr-fic fr-dib">Figure 1: Conceptual Diagram Showing Altitude and Coverage Area of Different Types of Transmission Equipment in a Non-Terrestrial NetworkAs shown in Figure 1, a non-terrestrial network deployed in the skies above us would be composed of different types of transmission equipment for which higher altitude corresponds to wider coverage area. Naturally, the greater the distance from the Earth’s surface, the greater the signal delay, and the different types of transmission equipment will be used for different purposes depending on their location, in outer space or the stratosphere, or at ground level. This makes it possible to build a communication infrastructure with a wider coverage range than was previously attainable, such that we will be able to say that “nowhere on Earth is out of range.” Below, we present an overview and feature description of the artificial satellites (GEO and LEO) and unmanned aircraft (HAPS) that compose the non-terrestrial network shown in Figure 1.<h3 id="isPasted">Geostationary Satellite (GEO)</h3>A geostationary satellite or geostationary Earth orbit satellite (GEO) is the component of non-terrestrial networks for B5G/6G that operates at the highest altitude. These artificial satellites are positioned approximately 36,000 km above the Earth’s equator. They orbit the Earth at a speed that matches the Earth’s rate of rotation, so they appear to be stationary above a specific spot on the ground. Generally speaking, weather satellites that predict the weather based on meteorological conditions over wide areas belong to this category of satellites.GEOs used in communications networks operate at a high altitude, allowing them to communicate with a wide area. It is said that three or four such satellites would be able to cover the entirety of the Earth’s surface. However, their distance from the ground means that they have a greater data transfer lag than other types of transmission equipment, and their data transfer speed is said generally to be on the order of several megabits per second (Mbps). Also, since sending signals to the Earth’s surface requires high output, GEOs tend to be larger than low Earth orbit satellites (LEOs), discussed below, and launching them into orbit requires larger rockets.<h3 id="isPasted">Low Earth Orbit Satellite (LEO)</h3>A low Earth orbit satellite (LEO) is a type of artificial satellite that operates in space but in a lower Earth orbit*1 than a GEO, typically at an altitude of several hundred to 2,000 km. Unlike geostationary satellites (GEOs), they are not synchronized with the rotation of the Earth. The Hubble Telescope and the International Space Station (ISS) also operate at a low altitude, using a low Earth orbit of around 400 km, at most. However, it is said that LEOs used in NTNs will be positioned at altitudes of around one thousand several hundred kilometers.LEOs operate at lower altitudes than GEOs, allowing data transfer with low delay and using low-output signals. Such satellites can also have a smaller size. LEOs support communication speeds with the Earth’s surface of several hundred megabits per second (Mbps), so it is said that they could be used for services enabling direct communication between smartphone handsets and satellites. On the other hand, since LEOs operate at lower altitudes than GEOs, the coverage area of each satellite is narrower, and satellites operating in low Earth orbits have a higher orbital speed. This mean that it is necessary to make use of “satellite constellations” consisting of multiple small LEOs operating in coordination in order to provide stable communication under these conditions. The term “constellation” is a reference to constellations of stars and is used to refer to a coordinated group of LEOs.*1 The definition of “Earth orbit” varies among different countries and organizations. For example, the European Space Agency (ESA) defines Earth orbit as orbiting the Earth at an altitude of up to 1,000 km, whereas the Japan Aerospace Exploration Agency (JAXA) specifies an altitude of up to 2,000 km.<h3 id="isPasted">High-Altitude Platform Station (HAPS)</h3>A high-altitude platform station (HAPS), sometimes called a stratospheric communication platform, is an unmanned aircraft (some are like airplanes while others take the form of balloons or airships) flying through the stratosphere at an altitude of around 20 km and that functions as an airborne communication base station. Generally speaking, passenger jetliners can reach altitudes up to around 10 km, and the portion of the stratosphere in which a HAPS operates is about twice that high. At this altitude, air currents and weather conditions are comparatively stable, and there is little air resistance. Since the air density is less than that needed for an unmanned aircraft to achieve the necessary dynamic lift, it is said that operational HAPS models will carry solar panels or batteries that will enable them to remain airborne continuously for several weeks at a time.Whereas terrestrial base stations generally have a coverage area spanning a radius of a few kilometers to a few tens of kilometers, a single HAPS is likely to have a coverage area spanning a radius of around 100 km. Also, since they are closer to the Earth’s surface than artificial satellites operating in space, they are expected to play a major role in the new communication infrastructure for B5G/6G due to their minimal data transfer lag. Nevertheless, since the stratosphere occupies the territorial airspace of different nations, unlike outer space, it can be said that the establishment of a legal framework in each country will be key to the international implementation of HAPS and the provision of communication services.<h2 id="isPasted">Advantages of and Trends in Non-Terrestrial Networks (NTNs) for B5G/6G</h2>In conclusion, we will discuss the advantages of and trends in non-terrestrial networks.<h3>Advantages of Non-Terrestrial Networks (NTNs)</h3>A big advantage of non-terrestrial networks, which unlike terrestrial networks comprise elements operating in the stratosphere and outer space, is that they can provide a communication infrastructure that is relatively unaffected by natural disasters such as earthquakes or tsunamis. In addition, they have the potential to dramatically expand the coverage area to the extent that nowhere on Earth is out of range. This will enable people to use their smartphones to communicate in emergencies even if they find themselves in locations where communication was difficult previously, such as mountain valleys or while at sea.As a result, it will be possible to maintain the communication infrastructure in time of emergencies such as natural disasters or accidents, regardless of location, and to extend the mobile communications network over ever broader areas without interruption, preventing problems from occurring when in the mountains or while traveling by sea.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1732785572794-img-0005.jpg" style="width: 100%px;" class="fr-fic fr-dib">Using a smartphone ina &nbsp;remote montain area.&nbsp;<h3 id="isPasted">Trends in Non-Terrestrial Networks (NTNs)</h3>One big feature of non-terrestrial networks is the remarkable trend toward active involvement by private companies. In the past, most space projects were national initiatives, but nowadays a wide variety of companies around the world, not only in the hardware field but in fields such as communication services as well, are teaming up and taking part in aerospace projects, including the non-terrestrial network business.Examples include the development and manufacture of artificial satellites and HAPSes, components used in them, and telecommunication equipment (payloads) loaded on them. A wide range of companies are involved in these business endeavors, including some that operate rockets that carry artificial satellites into space. In particular, the next-generation mobile communications field called B5G/6G is serving as a starting point for the rapid growth of businesses serving a market that is global in scale.Murata Manufacturing develops and manufactures a variety of wireless communication modules. As part of this effort, we are working to develop products that meet changing needs, including connectivity modules compatible with non-terrestrial networks (NTNs). For details, refer to the page linked to below.

This article highlights non-terrestrial networks at high altitudes, focusing on R&D and testing efforts to support next-generation telecommunications standards, Beyond 5G and 6G.

Non - Terrestrial Networks Employing HAPS or Artificial Satellites: Technological Trends Leading to Beyond 5G

InDro has taken the wraps off its latest innovation – the all-new Sentinel Uncrewed Ground Vehicle.Sentinel has been built for inspection, remote teleoperation and surveillance over 5G and 4G networks. Its robust build has been designed to take demanding weather and ground conditions in stride, while capturing virtually any kind of data across a broad range of use-cases.“That’s what really excites me about Sentinel,” says Head of R&amp;D Sales <a href="mailto:lcorbeth@indrorobotics.com" target="_blank">Luke Corbeth</a>. “It can be used across multiple verticals, ranging from remotely inspecting assets like electrical sub-stations, through to agriculture, mining and more. It’s a workhorse.”We could list all the sensors and processors onboard (and we will), but it’s the synergy here that counts.“Sentinel has been very purposely designed and integrated so that virtually anyone can carry out highly complex missions with ease,” explains InDro Vice President Peter King. “And with our InDro Autonomy stack onboard, Sentinel can carry out tasks completely on its own for up to four hours before returning to wirelessly recharge at its base.”That workhorse is the result of a lot of R&amp;D by our Area X.O engineering team – and also the logical step forward from our original Sentinel V1. That was our original design for remote teleoperated inspection.But, as you likely know by now, at InDro the innovation never stops.Below: The new InDro Sentinel, carrying out an autonomous mission<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzI0MDkyNDY1Mjc4LURTQ0Y3NzcwLXNjYWxlZC5qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib" alt="Sentinel Inspection and Monitoring Robot"><h3 id="isPasted">InDro Commander: Smart and Versatile</h3>What truly sets Sentinel apart from the competition is the synergy of two innovations we’re quite proud of: <a href="https://indrorobotics.ca/indro-commander-module-streamlines-robotics-rd/" target="_blank">InDro Commander</a> and <a href="https://indrorobotics.ca/new-indro-controller-a-simple-solution-to-complex-robotics-missions/" target="_blank">InDro Controller</a>. And while Sentinel takes full advantage of these products, they are both robot-agnostic.InDro Commander is a module that houses Sentinel’s brain – and allows for the seamless integration and power management of other sensors. With multiple USB slots and ROS1 and ROS2 libraries onboard, it makes modifying Sentinel a pleasure rather than a pain. You can, quite simply, plug in a new sensor and it will be detected; no complex coding or wiring required.“Having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&amp;D community,” says Engineering Lead Arron Griffiths.Commander also connects with 5G at near-zero latency for remote teleoperations or monitoring dense data during autonomous missions.“Whether it’s 4G, 5G or WiFi, it does it all,” he adds.With two state-of-the-art EDGE processors onboard, Commander also has the compute power and AI to take on anything. For autonomy, we’ve integrated the <a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/" target="_blank">NVIDIA Jetson AGX Orin</a> – a powerful 64GB developer kit. It’s capable of AI calculations at the astounding speed of 275 trillion operations per second (TOPS). Our InDro Autonomy Stack is embedded in Commander for precision autonomous missions. We’ve also outfitted Sentinel with a Realtime Kinematic (RTK) sensor for centimetre-level positioning accuracy.The other EDGE processor is the <a href="https://www.stereolabs.com/en-ca/products/zed-box" target="_blank">ZED Box</a>, which contains an NVIDIA Orin NX (100 TOPS). This carries out multiple tasks, but is specially built to work with depth-perception ZED cameras. There are two of these cameras on board Sentinel, used for AI obstacle detection and avoidance.Below: A LiDAR-equipped InDro Commander module<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzI0MDkyNTEwNzU4LUluRHJvLUNvbW1hbmRlci0xLjYtY29weS5qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Commander"><h3 id="isPasted">InDro Controller At The Helm</h3>InDro Controller is our custom-built dashboard, and it’s been designed to match perfectly with InDro Commander for an exceedingly powerful but user-friendly means of programming and monitoring missions, robot health, sensor data and more. Operators can easily customize to get precisely the layout they want.“The dashboard, the cameras, the heads-up display on the autonomous missions – those all can be customised,” says Front End Developer RJ Bundy. “We’re also adding other&nbsp;personal user customisations, like a light and dark mode, metric conversion, schedule missions – all the kinds of features you could want.”The software resides in Commander and is accessed remotely via an encrypted login system. No one will ever see your missions and data except you – and those you grant the same secure access. (Yes, multiple users can simultaneously monitor a remote operation from locations across the country or across the globe.)“We’ve created an interface that makes it very hard for a third party to intercept any of those commands or the data coming from the robot to you,” says Bundy.And while it’s a perfect fit with Sentinel, InDro Controller can be used with any robot.“It’s an all-in-one data visualization, robot management and robot control software,” he adds. “Whether&nbsp;you’re a student first learning how to use a robot or you’re a commercial giant, you’d be able to manage and maintain all of your robots.”He’s not kidding. You can intuitively control Sentinel – or any other robot – using an Xbox controller. We’ve done demonstrations more than 4,000 km away from Area X.O where we’ve simply put a controller in someone’s hands. With imperceptible lag, they’re able to operate the robot. (Plus, you can customise the buttons for specific sensor features like pan, tilt, zoom, etc.)Autonomous missions – including complex actions like stopping at a precise location, tilting a camera to a gauge and then zooming in and capturing the readout – are easily plotted using Controller. Once your mission has been set with multiple waypoints and actions, it’s a single click to repeat it. (And, of course, you can save as many different missions as required.)The InDro Autonomy stacks are pre-loaded, meaning location doesn’t really matter.“We have a GPS-based autonomy – which is better for outdoors – and then we have a SLAM- (Simultaneous Localisation and Mapping) based autonomy which is ideal for indoors,” says Head of R&amp;D Sales Corbeth.There are actually two versions of InDro Controller available; a simpler version is still full-featured but does not ship with InDro Autonomy, meaning it’s ideal for those in the R&amp;D space who want to test their own autonomy software. Several academic institutions have already been using this version, and feedback has been excellent.And the best news? Both versions will only become even more sophisticated over time.“We’re heavily invested in continuously improving the software,” says Corbeth. “So regardless of which version you’re shipped, know that this is something that InDro Robotics is constantly developing and improving with client feedback in mind to ultimately provide the best mission planning, teleoperations and development software tool in the robotics industry.”Below: A screenshot from InDro Controller, showing the path of an autonomous mission; the second image shows realtime data flowing through its customizable dashboard<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093227785-Autonomy-Ghosts.png-copy.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Controller Dashboard"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093240664-Dashboard.png-copy.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Controller Dashboard"><h3 id="isPasted">Long Duration, Repeatable Missions</h3>It’s already clear Sentinel has the brains. But it also has the brawn to match. Whether the task is inspection, surveillance or other forms of data acquisition, Sentinel has been designed from the ground up to consistently deliver. Featuring a 1500-watt dual drive system and tracks for locomotion, it’s ready for any terrain – from industrial environments through to farm fields and even snow and ice.With a 120kg load capacity and 30° climbing angle, Sentinel can power its way over pretty much anything. We’ve even designed and manufactured custom metal guards to prevent stones from getting their way between the drive gears and the treads – a common issue with tracked vehicles.That strength is supplemented by a comprehensive array of sensors for obstacle detection, Simultaneous Localisation and Mapping (SLAM), high-resolution photos and video – and more.&nbsp;<h3>Zoom, Zoom, Zoom</h3>For starters, there’s a 30x optical Point-Tilt-Zoom camera that can be manually operated via the InDro Controller UI. Every time the pilot stops and captures data, all of that information – robot position, camera angle, zoom settings etc. are stored as a Point of Interest. All of those POIs from that initial manual mission are saved, meaning the next time Sentinel is deployed the operator can simply recall the mission and launch Sentinel for an autonomous run.In fact, we recently did a demo of Sentinel for a government official. We had previously captured photos of five different objects on a path at Area X.O and had saved that mission. During the demo, the robot deployed and autonomously stopped at all five locations, replicating the original shots. This is the kind of feature that’s exceedingly helpful for spotting anomalies, change detection, etc. Of course, the built-in AI can also identify people, cars, trucks, etc. (In fact, we could set Sentinel to trigger an alert if a human is spotted somewhere they’re not supposed to be, a useful feature for both safety and security applications.)For the operator, whether they’re manually piloting or simply monitoring an autonomous mission, four pinhole cameras provide a 360° view of surroundings for situational awareness. Two ZED 3D depth cameras (one at the front, one at the rear) are on the constant lookout for obstacles. As an additional layer of safety, engineers added two 2D safety-rated LiDAR sensors specifically for detecting people or other objects that might unexpectedly move into Sentinel’s path. If that happens, the robot will automatically stop, then seek an alternate path around the person or object. (It can also be set to simply go to a dead stop.)Sentinel is also outfitted with two powerful Robosense 3D LiDAR units, enabling it to operate in GPS-denied environments, capture precision scans – and basically understand even a completely unfamiliar environment. Robosense has become our LiDAR of choice, for the reasons outlined in <a href="https://indrorobotics.ca/robosense-sets-new-bar-for-affordable-powerful-lidar-sensors/" target="_blank">this recent post</a>.Below: InDro recently built and shipped this fleet for swarm robotics research; all are equipped with Robosense LiDAR units<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093295010-Image-1-3-scaled.jpeg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Swarm Robotics Research Robots"> <h3 id="isPasted">Wireless Self-Charging</h3>Using Sentinel for remote ops? Many of our clients will be. And there’s not much point in deploying a robot at a remote location if someone has to go and plug it in to charge or run diagnostics.Sentinel returns autonomously to its home base, where a wireless charger awaits. The robot understands where it needs to position itself for optimal charging, snugging up close to (but not touching) the induction charging system. Many remotely operated robots rely on making a mechanical connection with charging equipment, which introduces wear-and-tear and additional points of failure.“Mechanical charging systems fail a lot, and we wanted to avoid that,” explains InDro VP Peter King.<a href="https://indrorobotics.ca/new-indro-controller-a-simple-solution-to-complex-robotics-missions/" target="_blank">InDro Controller</a> also carries out diagnostics remotely, reporting back on parameters such as latency, battery strength, sensor connection and other indicators of overall robot health. The operator can literally be hundreds (or even thousands) of kilometres from the robot and see its condition with a quick glance at the UI.“Sentinel can be customised for any client directly by us,” says Engineering Lead Arron Griffiths. “But having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&amp;D community.”Below: On the shoulders of giants – the original Sentinel V 1.0. InDro has incorporated new AI compute power, sensors and other learnings to take a massive leap forward<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093341924-DSCF3838-jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Original Sentinel"> <h3 id="isPasted">InDro's Take</h3>The new Sentinel is more powerful, and more suitable to broader use-cases, than our first version of this robot. Don’t get us wrong; the original Sentinel was (and still is) a great machine. But this version is equipped with more powerful EDGE processing, and AI with greater capabilities. It’s also laden with additional and newer sensors, RTK positioning, and a more powerful base platform.“The new Sentinel is a perfect fusion of industry-leading hardware and software – including InDro Commander, and the new InDro Controller and InDro Autonomy stacks,” says Indro Robotics Founder and CEO Philip Reece. “Our engineering team has put a lot of thought into making this robot capable of highly complex missions – yet easy to use and modify. Sentinel will be a perfect fit for multiple verticals and use-cases, and we look forward to our first deployments.”Want to learn more about Sentinel – or even arrange to take it for a remote test drive? You can contact Head of R&amp;D Sales Luke Corbeth<a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank">&nbsp;here</a>.

Robust robot built for demanding weather and ground conditions, while capturing virtually any kind of data across a broad range of use-cases.

Remote Inspection, Teleoperation and Surveillance over 5G and 4G Networks

At a time when the general public may not yet accept driverless taxis and ride-hailing vehicles, tele-driving could offer many of the same benefits, according to a new study led by a University of Michigan researcher. With more cars on the road, fewer drivers and fewer riderless miles, ride-sharing services could become faster and more affordable.Tele-driving typically involves a driver operating a car while sitting in front of a bank of screens that show video feeds from cameras on the car, as well as sensors and augmented reality technology. Once a passenger has been picked up, transported and dropped off, the driver can disconnect from that vehicle and is free to connect to any other that is available in an area of need. Several private tele-driving companies are already in operation, including <a href="https://halo.car/" target="_blank" rel="noreferrer noopener">halo.car</a> and <a href="https://vay.io/" target="_blank" rel="noreferrer noopener">Vay</a> in Las Vegas.One of the main advantages of tele-driving, according to researchers led by the University of Michigan’s <a href="https://benjaafar.com/" target="_blank" rel="noreferrer noopener">Saif Benjaafar</a>, a professor of industrial and operations engineering is that drivers do not need to be where the vehicles are—for instance, driving from areas of low demand to areas of high demand with no rider. In this vein, tele-driving can eliminate what ride hailing services refer to as the ‘wild goose chase’ scenario. At times when vehicle supply is low, cars need to be dispatched to customers located far away even if it’s not the most efficient pairing between rider and driver.“Tele-driving allows you to get away with far fewer drivers than vehicles without impacting the quality of service because you can still leverage the excess vehicles to get quickly to customers,” said Benjaafar, who specializes in supply chains and logistics. “A reduction of 30% to 40% in some of the test cases we considered.“There’s an opportunity to significantly increase how busy the drivers are,” he added. “One of the challenges for ride services has always been having drivers who are sitting idle. Quite a bit of that inefficiency can be eliminated.”The remaining drivers also stand to benefit as this system would shift vehicle ownership, and the cost and risk that it entails, onto the rideshare company. Tele-driving may broaden labor participation as driving becomes a desk job, the researchers suggest.Finally, the team is optimistic that the separation of drivers and riders could improve the safety of both, particularly women who have been disproportionately the targets of in-vehicle assault and other criminal behavior. However, tele-driving systems also need to guard against reckless driving in a work environment that feels more like a video game.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1719966837517-Vay_Teledrive_Station-2-featured.jpg" style="width: 100%px;" class="fr-fic fr-dib">A driver operates a vehicle remotely from a control center that features multiple video feeds. Two tele-driving services, Vayo and halo.car currently operate in Las Vegas. Credit: VayoUsing computer modeling that factors in supply, demand and road congestion over both time and space, researchers showed that a higher number of available vehicles than drivers can shorten wait times in periods of high demand, even with fewer drivers. This is because the likelihood of a driver going on a wild goose chase is reduced.“This research can be applied to the world/community by improving the efficiency and cost-effectiveness of ride-hailing and other on-demand transportation services,” said Xiaotang Yang, a postdoctoral fellow at the University of Toronto’s Rotman School of Management.&nbsp;“By using tele-driving, platforms can potentially operate with fewer drivers while maintaining or even improving service quality, which can lower operational costs and make these services more accessible. Additionally, this approach can help reduce traffic congestion and waiting times, leading to a better overall experience for users.”Benjaafar’s deep dive into the benefits and efficiencies of tele-driving come as efforts to bring autonomous ride services into the mainstream have stalled for a variety of reasons. Safety is key among them, and a regular stream of news articles chronicling crashes involving vehicles without human drivers only reinforces those concerns.“Full autonomy may take longer to become a reality,” said Benjaafar. “In the meantime, there are these technologies that can serve as a bridge toward full autonomy, including putting the human drive back into the loop.”&nbsp;The research was initiated while Benjaafar and Yang were at the University of Minnesota. They were joined by Zicheng Wang, now an assistant professor at the Chinese University of Hong Kong-Shenzhen.

If drivers could choose any available car as a starting point, ride-hailing services could become cheaper as energy and labor are used more efficiently.

Not quite ready for autonomous taxis? Tele-driving could be a bridge

The InDro Robotics engineering team, as always, has been hard at work. And we’re particularly pleased with our latest R&amp;D breakthrough: The InDro Command Module, or ICM.It’s a small but exceedingly powerful box that will immediately enable robotic platforms to reach the next level. It’s robot-agnostic, meaning you can integrate the Module with any ROS-based robot (which is pretty much all of them).And what can it do?A better question might be: What can’t it do? And the answer there is “not much.”Below: An InDro Robotics ICM mounted on a Unitree B2 quadruped. The ICM enables remote teleoperations, autonomous missions – and much more…<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE5MzIxMDU5MzE0LUlDTS1CMi1SZW5kZXJzLVYxLjIucG5nLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"> <h3 id="isPasted">Background</h3>There’s some important history here – which also tells you something about the InDro Robotics ethos.In November of 2021, we released a very popular product. <a href="https://indrorobotics.ca/the-indro-robotics-ros-in-a-box-for-ground-robots/" target="_blank">InDro Commander</a> was developed in response to the needs of clients who wanted to be able to quickly turn a platform into a smart, easily customisable robot capable of teleoperation over 4G or 5G. From developers and academics through to large technology companies, our clients had been requesting such a solution. In fact, even in the development of our own products we identified the need for an all-in-one solution to speed robot integration and enable teleoperation.“Our engineering team came up with the concept of the InDro Commander after integrating and customizing our own robots,” said Philip Reece, CEO of InDro Robotics, when the product was launched. “We realized there were hurdles common to all of them – so we designed and produced a solution. Commander vastly simplifies turning a platform into a fully functioning, teleoperated robot.”Commander, <a href="https://indrorobotics.ca/commander/" target="_blank">as this webpage explains</a>, “is designed to allow developers to quickly customize their own robotic solution. The following are common additions that can be easily added to any Commander-controlled robotic system: LiDAR (2D/3D), PTZ camera, thermal cameras, wireless charging, IMU, RTK.”Commander, quite simply, takes the pain out of integration.And how does it do that? Well, it ships with a powerful EDGE computer pre-loaded with ROS1 and ROS2 folders. It also contains multiple USB ports, allowing for the instant addition (and recognition of) pretty much any sensor you can throw at it. It allowed clients who were previously spending a ton of time on integration to simply attach Commander to the robot’s power supply, start plugging in sensors, and then remotely operate via a secure dashboard.&nbsp;When it was first released, Engineering Lead Arron Griffiths dubbed it “ROS-in-a-Box.” We later changed that to Commander, but here’s the splashy video released at the time in advance of the big IROS show, where Commander made its debut…<iframe width="640" height="360" src="https://www.youtube.com/embed/zWliBAIJvjc??si=-A9tkiZXym_kDBKa&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;</iframe><h3 id="isPasted">Smarter, Faster, Smaller</h3>At InDro, we have a motto of “Invent, Enhance, Deploy.” But deployment doesn’t mean we rest on our laurels. InDro’s engineers continuously look at ways to enhance existing products. And that’s exactly what we’ve done with Commander. We asked ourselves: How could we make this even better?“The InDro Command Module grew out of the idea of making a general purpose backpack/module that any OEM could put on their motorized product – everything from a ground robot to even a golf cart, drone, car, truck or RC platform,” says Engineering Lead Arron Griffiths. “That single module makes any platform ROS-enabled, AI-ready and 5G connected.”A lot of work went into this. One of the biggest tasks was designing and developing four new boards that pack a lot of AI compute power (and other features) into a tiny space. There are separate boards for EDGE computing, teleoperation data transfer, and sensor integration. There’s also a separate WiFi system.“And of course, it can run on either a public or private 5G network,” says InDro Vice President Peter King.&nbsp;And did we say size matters? The small size of the InDro module means it can be placed on virtually any robot – from the smallest quadruped to the largest AMR. It can even be fitted to a drone.“It’s tiny so it fits on everything. So basically we can turn any platform into a very powerful robot without all the time and hassle,” adds King.Below: Much of the magic of the InDro Control Module lies in the design, development and testing of four new circuit boards:<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE5MzIxNTI3NzI2LUlDTS1CMi1SZW5kZXJzLVYxLjQucG5nLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib" alt="Expanded View ICM"><h3 id="isPasted">Autonomy and InDro Controller&nbsp;</h3>The InDro Command Module is a powerful solution on its own. But pairing it with our new InDro Controller provides incredible synergy.We just wrote a fairly comprehensive post about InDro Controller, so if you’d like more details you can <a href="https://indrorobotics.ca/new-indro-controller-a-simple-solution-to-complex-robotics-missions/" target="_blank">check this out</a>. But in a nutshell, InDro Controller is a powerful desktop UI for operating any robot. With a highly secure data connection, InDro Controller automatically senses all key parameters of any robotic device. It will automatically detect any sensors, providing a data feed from all of them with minimal latency.“It can handle all sorts of data,” says Front-End Developer RJ Bundy. “It doesn’t matter if the robot has standard or custom sensors, InDro Controller can automatically detect and visualize them. For example, if you added a radar unit to an existing robot, the system will pick up on that immediately.”InDro Controller provides the operator with a highly customisable dashboard for carrying out complex missions. It will also keep track of any missions manually carried out, and make them repeatable as autonomous missions.And that autonomy? InDro has created two different autonomy software stacks. One is for outdoor missions that rely on GPS; the second is for indoor missions like SLAM in GPS-denied environments. That software comes with InDro Controller – and is also loaded onto the module. So the two key elements of this system – InDro Controller and the InDro Command Module – have been made to operate seamlessly together.“Anyone should be able to configure and control the new ICM for their product, and indeed, their solution,” says Griffiths.Below: A look at the powerful InDro Controller UI dashboard. The Controller software has been created to be an ideal match for the InDro Command Module and is easy to operate.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE5MzIxNjI0ODEzLURhc2hib2FyZC5wbmctY29weS5qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Controller Dashboard"> <h3 id="isPasted">InDro's Take&nbsp;</h3>We’re extraordinarily proud of the work our engineering team has put into the InDro Command Module – as well as InDro Controller. We believe these two, in combination, set a new bar for both robot integration and complex remote teleoperations.“The design and creation of the boards used in the InDro Command Module was a significant technical challenge – and a significant achievement. The same can be said of our two new autonomy stacks,” says InDro Robotics CEO Philip Reece.“Whether it’s a ground robot, a drone, or some other device that requires high processing power and minimal latency with all the features enabled by ROS, the ICM allows developers to quickly transform virtually any platform into a powerful autonomous robot with minimal effort.”Interested in learning more? Of course you are. Contact our Head of R&amp;D Sales, Luke Corbeth, right <a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank" rel="noopener noreferrer">here</a>.

The New InDro Command Module: Amazing Power in a Tiny Package

Simplify Integration with this Robot Agnostic Brain Box

InDro Robotics – as always – has been hard at work on innovative new products. And we’re particularly proud of our latest accomplishment: The InDro Controller.It’s an all-in-one solution for operating virtually any type of robot from a highly secure console. It’s completely robot-agnostic, very easy to use – and exceedingly powerful. We’ll get into the details as we go, but first let’s hear from Front End Developer RJ Bundy with an elevator pitch.“It’s an all-in-one data visualization, robot management and robot control software,” he says. “Whether&nbsp;you’re a student first learning how to use a robot or you’re a commercial giant, you’d be able to manage and maintain all of your robots.”He’s not exaggerating. We walked through a demo of this system recently, with Head of R&amp;D Sales Luke Corbeth at the controls and Bundy explaining the various features. We connected remotely with one of our InDro robots. The software immediately detected all of the sensors on the platform, offering up a display of windows showing the data they were collecting with minimal latency.“It can handle all sorts of data,” says Bundy. “It doesn’t matter if the robot has standard or custom sensors, InDro Controller can automatically detect and visualize them. For example, if you added a radar unit to an existing robot, the system will pick up on that immediately.”Of course, it has teleoperation. Missions can be run manually or autonomously (with InDro’s autonomy stack loaded onto any ROS-based robot).“We have a GPS-based autonomy – which is better for outdoors – and then we have a SLAM- (Simultaneous Localisation and Mapping) based autonomy which is ideal for indoors,” says Corbeth.And while InDro Controller has very complex capabilities, it’s a snap to use.“Once it’s set up, it’s pretty straightforward to initiate the mission. Even someone without any robotics knowledge could do it,” he adds.Below: The Pilot View mode in InDro Controller. Data from any desired sensors is displayed with minimal latency via a highly secure connection.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE4MjkyNzQ3Mzk1LVBpbG90LVZpZXcucG5nIGNvcHkuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 100%px;" class="fr-fic fr-dib" alt="Teleoperation Pilot View"><h3 id="isPasted">Highly Secure</h3>Regardless of whether you’re a startup, researcher, or a major corporation collecting sensitive data – security is important.&nbsp;InDro Controller has been built with that in mind.“We’ve created an interface that makes it very hard for a third party to intercept any of those commands or the data coming from the robot to you.”Though the person controlling the robot is the primary login, others with a secure login can also monitor the missions remotely from anywhere in the world. The software can store as many repeatable missions as you can throw at it, and you can initiate a previously stored mission with a single click.<h3> </h3><h3>Mission Planning</h3>Mission planning could not be simpler.For the first mission, the pilot would manually control the robot. InDro Controller uses an Xbox controller plugged into your computer for intuitive operation (though other options are available). All buttons on the Xbox device can be quickly mapped to carry out specific functions.InDro Controller tracks everything you’ve done – and we mean everything – and saves it as a repeatable autonomous mission.“If you manually drive the robot somewhere, it will remember it’s been there and it’ll be able to go back, follow that same path every time. It will also remember to carry out any specific actions you’ve taken at those points of interest, including camera angles, zoom, etc.” says Bundy.The mission planner also automatically loads a map to locate your robot (you also have the option of satellite view), so you can monitor exactly where it is on any given mission.And, of course, it can do this for an entire fleet of robots.Below: InDro Controller shows Points of Interest – which can be repeated with saved missions<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE4MjkyODA1NzE1LUF1dG9ub215LUdob3N0cy5wbmcgY29weS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"> <h3 id="isPasted">Highly Customizable</h3>InDro Controller has been designed to allow users to easily customize the user interface for any robot, any mission, and any dashboard view. Multiple streams of data, including upload and download speeds, battery levels and overall robot health are available at a glance. Oh, and did we mention it also works with third-party autonomy stacks?“The dashboard, the cameras, the heads up display on the autonomous missions – those all can be customized,” says Bundy. “We’re also adding other personal user customizations, like a light and dark mode, metric conversion, schedule missions – all the kinds of features you could want.”InDro Controller already works exceptionally well. But – as with all of our R&amp;D projects – it will continually be refined with additional features and functionality.“We’re heavily invested in continuously improving the software,” says Corbeth. “So regardless of which version you’re shipped, know that this is something that InDro Robotics is constantly developing and improving with client feedback in mind to ultimately provide the best mission planning, teleoperations and development software tool in the robotics industry.”Speaking of versions, there’s a simpler version of InDro Controller – which does not have the autonomy features – already being used for missions in the academic world. Feedback has been excellent.“Users tell us they find InDro Controller Lite exceedingly powerful, but also very simple to use. That was exactly our goal in developing this product,” says Corbeth.Academics and corporate innovation groups could take advantage of the Lite version, while the Industrial package is intended for solving more complex problems in an industrial environment.&nbsp;<h3>Satisfying R&amp;D</h3>For Front End Developer Bundy, who oversaw this project with support from other engineering staff, it’s been a hugely satisfying – and challenging – project.“A lot of the customization features were pretty difficult because it has to be robust and dynamic, which is always tough,” he says. “This is a relatively complicated application and I’ve managed to put together something pretty nice and functional – and it will only get better. “I’ve had a bunch of other help, but putting together the UI for InDro Controller has been, and continues to be, highly satisfying.”<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE4MjkyOTIxNTg4LUFkbWluLnBuZyBjb3B5LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib"><h3 id="isPasted">InDro's Take</h3>We’re obviously excited about InDro Controller. And we’re particularly excited because we have a forthcoming piece of hardware – the InDro Module. It’s a small box with a lot of brains that can be added to any robot to increase functionality and enable the seamless addition of sensors and other modifications (as well as pre-loaded autonomy stack and ROS drivers). We’ll have more on that soon, but it’s the perfect match for InDro Controller for users with complex requirements.For the moment, we’re looking forward to putting both the Lite and Industrial versions into the hands of clients.“When we first began remote teleoperation several years ago, we relied on third-party software as the UI,” says InDro Robotics CEO Philip Reece. “But we found it wasn’t powerful or customizable enough for our needs. It also required that we have our own autonomy stack – and we did – but many clients do not. InDro Controller comes with our proprietary autonomy stacks for both outdoor and GPS-denied locations. And, as noted previously, this is a long-term project, where even early adopters can be assured the package will be continuously refined with additional features.”Word has already been spreading in the R&amp;D and commercial fields about this product, and the feedback from those using the Lite version has been outstanding. Interested in learning more or seeing a demo? Contact Luke Corbeth <a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank">here</a>.

An all-in-one solution for operating virtually any type of robot from a highly secure console.

InDro Controller: A Simple Solution to Complex Robotics Missions

At Voltera, we love our test engineering interns. They bring fresh new ideas into our workspace and have a ton of fun exploring with our machines. It’s a privilege to provide mentorship, equipment, and resources to nurture brilliant young minds, and see what they dream up.&nbsp;At the beginning of his work term, Dawson Martian — a second-year undergrad studying Mechatronics, Robotics, and Automation Engineering at the University of Waterloo — came up with a project idea that would put both the <a href="http://voltera.io/v-one" target="_blank">V-One</a> and <a href="http://voltera.io/nova" target="_blank">NOVA</a> to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.Dawson had been working on a different project at the time, designing a light-up glove using stretchable ink and thermoplastic polyurethane&nbsp;(TPU). “I noticed when the glove was put on and powered, the brightness of the LEDs would vary depending on whether the user’s hand was opened or closed,” he recalled. “This got me thinking about how the lights were dimming due to the resistance of the traces increasing whenever they were stretched, essentially the traces were acting as strain gauges,” he said. “From there I realized I could take advantage of this property and use this varying resistance to control a machine or program…what better way to show this unique property than by having a robotic hand mimic my movements?”So, Dawson got to work. He started by building a robotic hand with movable fingers. The hand components were 3D-printed using non-flexible filament, the joints were made with TPU, and then assembled. He ran the wire up the fingers and connected these to servos. These would operate as tendons, so the hand could move. He connected the servos to an Arduino™ with a custom breakout board — which he made using the V-One.&nbsp;For the control glove, Dawson printed strain gauges onto TPU using NOVA and heat-pressed the strain gauges onto a fabric glove using a t-shirt press. Dawson wired a second Arduino and custom breakout board to the traces on the control glove using a conductive adhesive and mounted the board to a wrist strap.&nbsp;The two boards communicate via Bluetooth® transceivers, so if all worked as it should, the board on the control glove would measure resistance in the strain gauges, then transmit movement instructions to the second board. This information would control the actuators of the robotic hand.&nbsp;Of course, nothing goes exactly&nbsp;as planned, and Dawson hit a bit of a snag when he was designing the strain gauges. But, let’s rewind for a minute.&nbsp;<h2>What is a strain gauge, anyway?</h2>“A strange gauge is essentially just a resistor whose resistance value varies with changes in strain — being stretched, bent, or compressed,” Dawson explained. “You can think of it as a stretchable pipe with water running through it. As you stretch the pipe, it becomes longer but also narrower, restricting the flow”.&nbsp;Now, apply this same principle to the traces in the control glove, which are made with stretchable conductive ink. As the traces stretch, they become longer and narrower, restricting the flow of electricity — it changes the resistance. The accompanying transmitter board interprets this change in resistance, and that information can then be used to control other devices — like the robotic hand.&nbsp;Dawson’s biggest hurdle was designing the strain gauges for the control glove — finding the optimal procedure that would allow the gauges to fully stretch without losing conductivity completely, but still have a measurable change in resistance. In the end, he found that thicker traces resulted in more consistent continuity while stretching.&nbsp; <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1704362901155-6373badbe9c8bd5af2c5faf5_Robot+Glove_1_optimized.gif" style="width: 300px;" class="fr-fic fr-dib"> <h2 id="isPasted">Imagine the possibilities</h2>Stretchable conductive ink gives you the ability to create a strain gauge on almost any stretchable material, and that has incredible potential to drive innovation. Industries across the board, from manufacturing to healthcare to wearable technology, are on the threshold of major change, thanks to the emerging field of&nbsp;<a href="https://www.idtechex.com/en/research-article/flexible-hybrid-electronics-future-of-flexible-printed-circuit-boards/20603" target="_blank">flexible hybrid electronics</a>.&nbsp;Over the course of just&nbsp;two months, Dawson was able to successfully design and build a fully-functional robotic hand that mimics the movements of an operator wearing a control glove. Imagine what could be accomplished by seasoned researchers and engineers.When asked about the potential applications for this kind of technology, Dawson lit up. “Why stop at just a hand? Why not use this technology to have a whole arm that mimics my own? Possibly even a full robot that mirrors my every move, and the majority of that could be achieved using similar techniques that were used for just the hand,” Dawson exclaimed. “The most exciting thing for me was the realization of the true possibilities that NOVA brings to the world of stretchable and flexible electronics”.This technology could have life-altering and transformative potential in industries across the board — from biomedical applications and wearables to prosthetics. “I could see this technology being used in workplaces that present a high risk of injury, but still require the delicate handling of a human. For instance, if some work needs to be done on a machine but the area is radioactive or unsafe, the worker could remotely control this hand to perform the task, while staying in a safe location.” Dawson said. “This project could also be scaled up or down in size to meet specific requirements, for instance, you could have a small robotic hand do work inside a machine, in spaces where a human could not fit, being controlled by an operator outside the machine”.&nbsp;You can read the finer details of Dawson’s work with&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6362829f808f8037c5b5e9d9_NOVA-0094%20-%20Robot%20Glove%20Case%20Study_FINAL%20(4).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">NOVA</a> and the&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/636282bb3c1c464ccd273de1_VONE-0001%20-%20Robot%20Glove%20Case%20Study_FINAL%20(3).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">V-One</a> by checking out&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6554cb7c0b836d9be99ef8fa_2022_NOVA%20robot%20glove%20case%20study.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">case studies</a> on this project.

Dawson Martian — a second-year undergrad at the University of Waterloo — came up with a project idea that would put both the V-One and NOVA to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.

Building a Remote Controlled Robotic Hand with Strain Gauges

<h3 id="isPasted">Let’s explore some methods on how to achieve this.</h3>First and foremost, there exists the “Straightforward Programming” approach. This means: getting a skilled person to meticulously describe a task, to such an extent that one can write either a simple script or an intricate algorithm that can guide the robot to do the task. This method is great and highly effective when the task is well-defined and specific, something often done in industrial automation. However, the complexity can escalate rapidly, making it challenging. Thus, if the approach is successful, that’s excellent – but if not, we must ponder – what alternative strategies can we employ?Secondly, an alternative approach to consider is “Programming by Demonstration“. This method involves guiding the robot in performing the task. By equipping the robot with advanced force sensing capabilities and vision algorithms, this technique can prove to be quite effective. However, it’s crucial to note that its effectiveness is reliant on the robot’s abilities. Usually, robots are equipped with rudimentary grippers rather than adept, dexterous hands, which can limit the complexity of tasks they can perform.&nbsp;Okay, so how can we effectively record human actions for a robot to mimic?Ideally, we’d guide a human through the task, document it in detail, and then replay it to the robot. We might even consider feeding this information into a machine-learning system, enabling it to learn from our human-guided demonstration. Alternatively, at the very least, we can analyze these human activities, deconstruct them into simpler parts, and use this information to develop more efficient algorithms in the way that a song is deconstructed in a recording studio.&nbsp;OK, so how about, filming the person doing the task and using “machine vision” to track what they are doing?It’s certainly a good starting point. This approach provides valuable insights into the nuances of human behaviour – such as hand-switching moments, object-holding techniques, and tool usage – But our experience is that it’s really hard to track human movement&nbsp;using cameras. You need clear lines of sight, and you’ll have lots of trouble tracking the fine movements of the fingers and hands.&nbsp;Even the most advanced image-based tracking systems, while capable of monitoring hands with minimal occlusion, fail to consistently track all joint movements. Consequently, the resultant data tends to be somewhat inconsistent and noisy.&nbsp;Alright, what about using motion capture?Motion capture systems, commonly employed in film production and sports science, operate by attaching easily visible markers to the individual, such as passive elements like white dots or active elements like pulsed LEDs. This method certainly helps with the accuracy of tracking. However, it doesn’t entirely overcome the previously mentioned issues related to occlusion. Additionally, calibrating the optical systems to an error margin of less than a few millimetres is quite challenging – which means that if the task involves intricate movements, such as rolling a pen between fingers, it’s likely it’ll miss a lot.&nbsp;&nbsp;Okay. I need it to be accurate, what’s my best option then?For precise human movement tracking, your best bet is to use a tight-fitting movement-tracking&nbsp;<a href="https://www.shadowrobot.com/shadow-hand-glove/" target="_blank">glove</a>. This glove can incorporate a multitude of techniques to achieve such a feat, some even providing additional features to assist the operator (and some that make it more difficult for the operator). For instance, some gloves can relay a sense of weight, touch, or stiffness, all functionalities grouped under the term ‘haptics.’ The realm of haptics is broad, including diverse technologies from ultrasound devices transmitting signals to fingers, electrical skin stimulation, vibrating piezoelectric elements, to inflating air balloons, miniature motors, or linear actuators, among many others.Gloves that measure human hand movement can do it by utilising:<ul><li dir="ltr">Sensors that fit around the joints and track the specific movement of the joint</li><li dir="ltr">Mechanical linkages that go from one end of the finger to the other and track the whole movement of the finger</li><li dir="ltr">Inertial sensors (that clever combination of accelerometer, gyroscope and possibly magnetometer) - worn at various points on the hand to follow the movement</li><li dir="ltr">Wireless measuring sensors - detecting the location of the fingers relative to a source worn on the hand somewhere.</li><li dir="ltr">Cameras that look away from the hand and track the movement of the world. (I’m not sure I’ve seen this one used in anger but I’m sure it’s possible!)</li></ul> Even then several variations of these technologies exist. For instance, some sensors measure the flex of a glove, correlating it to joint movement. Other types of sensors stretch and track their own deformation to gauge distance. Occasionally, mechanical linkages are placed over individual joints for precise motion tracking.&nbsp;Each of these methods presents a unique approach to capturing and interpreting human movement<h3 id="isPasted">Our experience:</h3>From our extensive experience, we have found that accurately measuring the mechanical properties of the human hand can be challenging, as it often leads to significant inaccuracies. The placement of the measurement sensor is crucial, and unfortunately, the potential for slippage is high – gloves and similar apparatus tend to shift excessively, complicating the data collection. Relating the measurements to the actual rotation of a finger joint’s axis, especially the thumb, is another daunting task!<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3NTM2Nzk3ODE3LWxvbmcgLSBQb2xoZW11cyAmIFNoYWRvdyAoMTkyMCB4IDEwODAgcHgpLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Mechanical linkages, although valuable, can quickly become cumbersome. There is also a risk that they could discomfort the operator to the point where task completion becomes impossible (too heavy, too rigid etc). Inertial sensors present a viable option; however, (once again) preventing them from drifting is generally difficult, and the precision level they offer often falls short of what we needWe have found that wireless measurement sensors, particularly those from&nbsp;<a href="https://www.shadowrobot.com/shadow-robot-polhemus/" target="_blank">Polhemus</a>, give us excellent results. These sensors entail a transmitting source worn on the palm and receivers located at the fingertips, providing us with high-quality data for hand movement tracking. However, there remains the challenge of locating the hand within the wider three-dimensional space. For this, we have found that the HTC Vive is an excellent solution and so you will often see the Vive tracker featured in our demonstrations.Now – what we want is to accurately know where to put the robot hand – which isn’t quite the same thing as knowing accurately where the human hand is. We have to take the data from the human hand and map it to the robot hand. This is a kinematic challenge – the movements of the human and the movements of the robot don’t agree entirely, and we need to convert one to the other. The best method for mapping human data to a robot hand will depend on the specific application. For example, if you need to control a robot’s hand in real-time, then direct mapping may be the best option. If you are only interested in controlling a robot’s hand for offline analysis, then inverse kinematics or hybrid mapping may be more feasible options. Needless to say, the more complex the robot the more challenging this becomes. Lucky for us complex robots are a speciality at Shadow!&nbsp;So, the human wears a set of gloves, and they control the robot to do the task. After completing a few rudimentary training tasks the operator gets the hang of the process and they are doing it pretty well. What do we do to learn from this?We can precisely monitor the instructions sent to the robot and reproduce them. This is ideal when operating in a controlled environment where all variables remain consistent – a scenario commonly found in an automation setting within a quality assurance laboratory. In the past, we employed this technique to build a functional robotic kitchen. While it allows you to execute repetitive tasks, it lacks the ability to cope with variation.We can capture and analyze data from numerous tests, assessing them for consistency. If the results are largely similar, they can be directly applied. However, if they vary a lot, we can delve into our algorithm toolbox to try and understand these discrepancies. This is where it helps to have a <a href="https://www.shadowrobot.com/artificial-intelligence-machine-learning/" target="_blank">wide range of data</a> from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">the robot</a>, from force, <a href="https://www.shadowrobot.com/shadow-robot-11-lessons-about-tactile-sensors/" target="_blank">touch</a>, and joint sensors. The question arises: Can we employ data from vision, such as cameras aimed at the work area, to discern what’s different? To be effective, this vision data needs to be synchronized with the robot sensor data, necessitating a shared framework for all data streams.We can gather data from multiple tests and use it by “Feeding it to the AI” – meaning use it to instruct a machine learning system on how to replicate similar movements from similar data.This can be done in several ways.We might train a traditional learning model on the datasets so that similar inputs generate similar outputs. Alternatively, if we have enough data, we could use a transformer model to generate movements. We can even break down the data into smaller segments and train a network to assemble these fragments into the correct sequences.Given the impressive capabilities of modern machine learning systems in generating text, images, video, and even code, this approach appears highly promising – and it’s only the beginning!So, does this sound like it could help with a problem that you face? If it does, then let’s have&nbsp;<a href="https://www.shadowrobot.com/blog/data-humans-to-robots/hand@shadowrobot.com" target="_blank">a chat to</a> see if the Shadow Robot team can help!

Humans excel at manual tasks, and ideally, robots should surpass us in these functions. We have the advantage due to our superior dexterity, and we can perform tasks that robots should, in theory, be capable of.
As we advance robotics our goal is to effectively transfer our human skills to robots.

Getting Data from Humans to Robots

There are many robot arms on the market, and you&rsquo;re probably familiar with a few. Your modern cobot ranges in cost from &euro;25k to &euro;200k &ndash; and that is for something with 6 or 7 degrees of freedom (DoF). So you might say &ldquo;&euro;4k-&euro;14k per DoF&rdquo; and use that as a &ldquo;rule of thumb&rdquo; to price off-the-shelf commercial robots.There are plenty of people with very low-cost robot hands and arms out there &ndash; if you&rsquo;re a hobbyist or you love to prototype, then that&rsquo;s where you should go! A quick look online finds 3d printed structures for a few tens or hundreds of Euro to print, although you still have to supply valves and compressors and electronics and maybe sensing. A fun project for a hardware developer! Then there are some cool hobbyist complete systems &ndash; I can find a 6DoF robot arm on Amazon for &euro;120 today, which would be great for teaching beginners the fundamentals of robot operation.But if you&rsquo;re planning to do work with the robot, then you want a complete system. And you want something that will lift a few kilos, has the sensors and motors to move with precision and speed across a decent set of kinematics, has safety circuits and controls built in, and supports the &ldquo;mechanical engineering&rdquo; interfaces allowing you to build it into your lab properly. Suddenly you&rsquo;re back to &ldquo;it&rsquo;s going to be a 5 figure sum&rdquo;. And there are good reasons for that &ndash; you&rsquo;re getting something with mechanical strength, highly engineered sensors, motors and gearboxes, and sophisticated control systems to make it work.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NzcwMTg4NzE4LVVudGl0bGVkICgyOTcgw5cgMjEwbW0pICgyMDAgw5cgMjAwbW0pICgzMDAgw5cgMTAwbW0pLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"> And then you&rsquo;re going to want documentation, software interfaces, and support when it doesn&rsquo;t do what you want.Don&rsquo;t get me wrong &ndash; that isn&rsquo;t because you bought something that wasn&rsquo;t up to the job. Most support tickets in robotics come with one of two sentences at the start: &ldquo;I was doing &hellip; and it broke&rdquo; or &ldquo;How can I get it to do &hellip;&rdquo;. And we&rsquo;ve seen enough robot arms slam things into tables to know that breaking things is part of the game here &ndash; honestly, if you&rsquo;re not breaking the odd robot you&rsquo;re probably not doing research at the frontiers!So that&rsquo;s a real, commercial robot arm. Assume &euro;5k per DoF for a generic cobot, and maybe &euro;8-10k if you&rsquo;re getting something really amazing. Even the dinky 6DoF arm we use in one of our production processes was &euro;17k for 6DoF (to be fair to it, the precision is really impressive, but it&rsquo;s still shoebox-size).However, <a href="https://www.shadowrobot.com/" target="_blank" rel="noopener noreferrer">Shadow Robot</a> isn&#39;t in the robot arm business. We make what the marketing people politely call &ldquo;end of arm tooling&rdquo;. This space ranges from welding guns to lasers, from two jaw grippers to vacuum suckers, and right the way over there &mdash;&gt; robot hands. And according to the good people at the International Federation of Robotics, adding EoAT and the other parts needed to make it work can triple the price of a robot arm!A simple gripper that opens and closes without any fine movement control, without force sensing and without position sensing might well be included in the price of the cobot. Lots of companies do that and it works well for their customers. But we&rsquo;re here for data. We want precise control. We want to be able to move each fingertip independently and position it accurately in space with respect to the other fingertips.In short, we want a robot hand where each finger behaves like a small robot arm &ndash; and we want all that complexity and performance in something 10x smaller! What would that cost? Well, &euro;5k per DoF gets us to &euro;100k for 20DoF &hellip; and the price for the full Shadow Hand Plus today is &hellip; &euro;110k including shipping, installation, training and support!Of course, we can make simpler Hands with fewer degrees of freedom &ndash; and fewer fingers &nbsp;&ndash; and these go right down to a 2-finger 7DoF system which is more like &euro;45k including shipping, installation, training and support. You can see the full range <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">here</a>, and if you want to talk to us about getting one, then <a href="https://www.shadowrobot.com/contact/" target="_blank" rel="noopener noreferrer">get in touch</a>!&nbsp;

Spending your hard-earned budget on a robot is a step you don’t want to take lightly, Any worthwhile robot is likely to represent a significant piece of capital equipment.

So, let’s take a look at some hardware and what you might expect to pay for.

How Much does a Robot Hand Cost?

The spread of COVID-19 has changed how we live and work. With no choice but to restrict movement and reduce how often we meet others, we have moved many of our activities from the physical world to a virtual one. For many people, it seems only natural now to use remote technology to meet with friends, take classes, and have meetings. Although remote work has become more commonplace, there are still many jobs that must be done in person<h1 dir="ltr">Expanding the scope of remote work using VR/AR and 5G technologies</h1>This use of virtual environments will likely continue even after the COVID-19 pandemic is over. This is especially true in business. There are several reasons why.&nbsp;First, many people who thought online meetings would never work have realized that they in fact, do work&mdash;and can even be more productive than face-to-face meetings. Having meetings online can reduce the number of business trips taken and can also reduce cost and time. This means meetings can be held for more people.&nbsp;Second, people disinclined to make use of ICT have been forced to make use of remote technologies, and this has helped to provide them with relevant skills. Working with remote technology has helped to eliminate disparity in ICT literacy and skills.&nbsp;Finally, governments across the world continue to promote diverse working environments, and remote work is a great way to realize this goal. Remote work will also help support society and the people working within it as it becomes more established.Some companies that have observed these trends have begun to revise their HR systems and reduce the sizes of their offices, under the assumption that remote work already plays a central role in business.Although many tasks such as office work and preparing documents and materials can be performed remotely without issue, there are still many tasks that must be done on-site. Remote work is simply not an option in these cases. This is especially true at manufacturing sites.<h1 dir="ltr">Making full use of 3D CAD modeling to streamline VR/AR design work and maintenance</h1>Manufacturing companies have begun making use of technologies such as virtual reality (VR), augmented reality (AR), and IoT as new means of communicating information in situations that previously could not work remotely, such as equipment maintenance and product design.It has become possible in recent years to design all sorts of industrial products, equipment, and facilities using 3D CAD. This technology has been making tremendous progress. The individual parts of devices are now individually designed as 3D models, which can be assembled and even manipulated on a computer. This means that products can now be manufactured and tested, with no need to build a physical prototype.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjczMjAyMDA5ODAzLVZpcnR1YWxfUmVhbGl0eS5fKFVuc3BsYXNoKS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" width="720" height="453" class="fr-fic fr-dii">Fig. 1: Data from CAD tools can be shared and reviewed virtually in real-time. When developing an industrial product (such as an automobile or electronic device) or designing a building or other structure, personnel such as product planners, salespeople, designers, and production engineers will meet to review the product. Until recently, this involved gathering all these individuals in the same room to discuss a physical prototype created especially for this purpose. Now, experts in remote locations can share 3D model data and then use head-mounted displays to manipulate and disassemble these models, allowing them to review a product just as though they were in the same room with it&mdash;or, perhaps in even more detail.<img data-fr-image-pasted="true" src="https://upload.wikimedia.org/wikipedia/commons/1/16/Oil_rig_Jan_23.jpg" style="width: 300px;" class="fr-fic fr-dib">Fig. 2: Digital twin technology can be used to create a 3D model of how technologies appear and operate. Image credit: Eugene.AI.When designing manufacturing equipment or a machine tool, 3D data can be placed within a virtual production line that has also been modeled, allowing designers to verify that everything works together and operates safely&mdash;before even setting it up on-site.These 3D models can also be used after the equipment has been actually installed on-site, in order to streamline work. AR can be used when maintaining equipment or facilities at a site to display technical information from the 3D model overlaid over the actual equipment, allowing for complicated tasks to be performed without error.Sensors could also be installed on actual equipment. Data from these sensors could be imported to a 3D model on a server for use by the maintenance department or manufacturer, in order to create a digital copy that looks and operates exactly like the actual equipment on site. This so-called &quot;digital twin&quot; technology can be used to predict equipment failures or even simulate the impact of changing production conditions on factors such as throughput and durability.However, using technologies such as VR, AR, and IoT requires a low-latency network capable of transmitting a large amount of data quickly between work sites and offices, as well as homes for remote work. 5G, a fifth-generation mobile communication system, is expected to play a major role in making this possible.<h1 dir="ltr">Using robots to provide a sense of presence to remote communication</h1>As remote work becomes mainstream, it is becoming clear that this technology offers many unique benefits for office work. However, that doesn&#39;t mean that it provides the same level of communication as meeting face-to-face. Remote work technology is still in its infancy.For example, many people might feel that something is slightly off when participating in a remote meeting, brainstorming session, or product review. You might think that using a videoconferencing system would allow you to easily see how everyone reacts to a statement since you can see everyone on the screen. In reality, participants can often have trouble seeing how people react and are often not confident that their message is getting across. Online meetings have shown us that we rely on more than mere words and expressions to communicate.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjczMTk5NjkxOTM2LTE2NzMxOTk2OTE5MzYucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="720" height="480" class="fr-fic fr-dii">Fig. 3: Video conferencing has become a routine tool for work. In some situations, not being present in person can cause issues with communication.This feeling that something is somehow off during remote meetings is a shared sentiment among people all over the world. In order to resolve current issues with using remote communication tools, companies and other organizations have begun implementing advanced communication technologies utilizing ICT which is still in the development stages. We are also seeing some attempts to use the latest in ICT and robotic technology to bring a sense of presence to remote meetings and customer service situations.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjczMTk5Njg5ODA0LTE2NzMxOTk2ODk4MDQucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="720" height="959" class="fr-fic fr-dii">Fig. 4: Telepresence robots with a tablet make remote conversations feel more natural.&nbsp;In the US, simple robots called telepresence robots are already being used by large companies, hospitals, schools, and other organizations as a means of communication. Telepresence robots are very simple-looking robots consisting of a tablet installed on a mobile vehicle. The tablet is used to display the face of the person speaking through the robot. The speaker can operate and move the robot remotely, allowing them to change the robot&#39;s facing while communicating with others in video chat.It seems too simple to work, but you might be surprised to know that these robots can provide a greater sense of presence than simply using a tablet for video chatting.The individual operating the robot can control it remotely, such as moving it around or changing its facing. When a tablet alone is used for communication, it is the other person who determines what you see. With a telepresence robot, the opposite is true. When speaking with multiple people, the operator of the robot can move the robot to face whomever they want to speak with. This could be of use in a variety of situations to make communication more active, such as moving around an exhibition to visit certain booths or seeking out certain individuals at a party or similar situation.It can also be helpful for others to see such a robot moving around, which can provide a greater sense of the operator of the robot actually being there. All in all, this technology makes it possible for others to not only see your face and hear your voice, but to watch your robot react and move&mdash;and therefore more easily determine whether you are really paying attention!<h1 dir="ltr">Remote robots as a means to go anywhere, instantly</h1>The technology of telexistence brings telepresence to the next level. This technology promises to provide an even higher level of communication, in which an individual remotely controls a robot called an avatar that reproduces the operator&#39;s movements and five senses. It is a concept advocated by Susumu Tachi, Professor Emeritus of the University of Tokyo.[1]<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjczMjAwOTQzNDY1LTE2NzMyMDA5NDM0NjUucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="720" height="519" id="isPasted" class="fr-fic fr-dii">Fig. 5: Telepresence takes the capabilities of telexistance devices to the next level by letting us remotely controls a robot&rsquo;s movements and five senses.With telepresence, the operator uses a controller or other device to consciously move a robot. With telexistence, however, sensors are used to detect the operator&#39;s movements and remotely control the robot without any conscious action on the part of the operator. Installing a camera on the robot to feed video to a head-mounted display (HMD) worn by the operator could therefore bring more immersion to remote communication and work. Haptic technology could also be used to provide the operator with a sense of touch mirroring that of the robot for an even greater presence and sense of immersion.In addition to workplaces, telepresence technology could be used for virtual tourism and even in healthcare facilities. Imagine a world where avatar robots are as common as shared bicycle stands. You could take control of such a robot through the Internet and be transported instantly to a tourist spot or other remote location.<h1 dir="ltr">Conclusion</h1>Remote technologies have traditionally served as secondary options for life and work, used only when traveling to a physical location or meeting face-to-face is not possible. This was true even at the start of the COVID-19 pandemic. However, we may soon see a world where remote work becomes predominant, with face-to-face meetings conducted only when absolutely necessary.The remote technologies described here will continue to advance as this comes closer to reality. It seems likely that we will ultimately take the use of such technologies for granted, as we develop means of communicating and working that would not be possible except in remote environments. We will increasingly enter a future in which robots are used to communicate closely with others&mdash;and to experience many things previously not possible.<h2 id="isPasted">About the sponsor: Murata</h2>Murata is a global solution provider and the market leader in the design, manufacture, and supply of advanced, leading-edge electronic components, and multi-functional modules. Murata contributes to the advancement of society and the electronics industry, in close cooperation with its customers and other stakeholders. Their European HQ is based in the Netherlands. &nbsp;&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjc0MTIxOTIwNDU4LW11cmF0YSBsb2dvIGJvdHRvbS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"> <h1 dir="ltr">References</h1>[1] Susumu Tachi, Telexistence, [Online], Available from: <a href="https://tx-inc.com/en/susumu-tachis-bio/" target="_blank">https://tx-inc.com/en/susumu-tachis-bio/</a><hr id="isPasted"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg1MDg2NjcyNTM4LTE2ODUwODY2NzI1MzgucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" class="fr-fic fr-dii">If you have any questions based on this article that you would like to discuss with an expert at Murata, don&#39;t hesitate to reach out. There&#39;s an entire team ready to support you with your question.<a class="button-main-action" href="https://www.murata.com/en-eu/contactform?&Detail=Wevolver%20request" rel="noopener noreferrer" target="_blank">GET IN TOUCH</a><hr>

Immersive technologies and high-speed telecommunication standards are enabling new ways to work remotely.

Expanding the scope of remote work using VR, AR and 5G

In this episode, we talk about how ostriches are inspiring a new generation of bipedal robots and the innovation enabling remote endovascular surgery. As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/e2238814-9212-4010-afb6-ac48d637140f?dark=false" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe><hr>This podcast is sponsored by <a href="https://www2.mouser.com/" id="isPasted" rel="nofollow" target="_blank">Mouser Electronics</a>.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651565842197-Mouser-sponsor-1-a.jpg" style="width: 766px;" class="fr-fic fr-dib"><hr>(0:50) -&nbsp;<a href="https://www.wevolver.com/article/birdbot-a-3d-printed-bipedal-robot-offers-major-efficiency-gains-through-avian-biomimicry" target="_blank">Using Bird Biology To Make Efficient Walking Robots</a>(20:45) -&nbsp;<a href="https://www.wevolver.com/article/joystick-operated-robot-could-help-surgeons-treat-stroke-remotely" target="_blank">Joystick-operated Robot For Treating Strokes</a>Episode 68 was brought to you by Mouser Electronics, Farbod &amp; Daniel&rsquo;s favorite electronics distributor. Click <a href="https://www.mouser.com/applications/future-robotics-singularity/" target="_blank">here</a> to read the article discussing the merger of technology and humanity.<hr>About the podcast:Every day, some of the most innovative universities, companies, and individual technology developers share their knowledge on Wevolver. To ensure we can also provide this knowledge for the growing group of podcast listeners, we started a collaboration with two young engineers, Daniel Scott Mitchell &amp; Farbod Moghaddam who discuss the most interesting content in this podcast series.&nbsp;To learn more about this show, please visit the <a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">shows page</a>. By following the page, you will get automatic updates by email when a new show is published.Be sure to give us a follow and review on <a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" rel="nofollow" target="_blank">Apple</a> podcasts, <a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" rel="nofollow" target="_blank">Spotify</a>, and most of your favorite podcast platforms!Take a few seconds to leave us a review. It really helps! <a href="https://apple.co/2RIsbZ2" rel="nofollow" target="_blank">https://apple.co/2RIsbZ2&nbsp;</a>if you do it and send us proof, we&rsquo;ll give you a shoutout on the show.

The prototype motorized BirdBot, seen here with legs in the locked position, uses a quarter the energy of its immediate predecessor.

In this episode, we talk about how ostriches are inspiring a new generation of bipedal robots and the innovation enabling remote endovascular surgery. 

Podcast: Bird-inspired Bipedal Robots & Robotic Surgeons

The warm feeling of being hugged is known to have a number of positive effects &mdash; and humans are far from the only creatures to enjoy social touch. Robots, meanwhile, are traditionally cold, sterile, and not entirely cuddly &mdash; at least, outside the realm of children&rsquo;s entertainment.But what if a robot could deliver a satisfying hug? That&rsquo;s the question asked by Alexis E. Block and colleagues in the HuggieBot project, with results from a third-generation study having just been accepted for publication. Such a machine could provide comfort to the lonely, have potential medical benefits in care settings, or act as a surrogate when people simply can&rsquo;t be together in person, as has been so common during the ongoing pandemic.Enter HuggieBot 3.0, claimed by its creators to be &ldquo;the first fully autonomous human-sized hugging robot that recognizes and responds to the user&rsquo;s intra-hug gestures.&rdquo;<h1>The birth of HuggieBot</h1>Work on the original HuggieBot began in 2016, when Block was studying for a Master&rsquo;s degree in robotics. The idea: Providing the ability to send a hug anywhere in the world, but in a manner which feels natural and enjoyable &mdash; following what Block and colleagues would come to call the &ldquo;six commandments&rdquo; of robotic hugging, a series of design guidelines which included the need for such a robot to be soft, warm, human-sized, and able to autonomously enter into and exit from an embrace.HuggieBot 2.0, put into testing in 2020, took the concept a stage further by using visual and haptic perception to deliver adaptive hugging in closed-loop system, but it wasn&rsquo;t until HuggieBot 3.0 that the machine would gain a key feature: The ability to autonomously detect, and reciprocate, &ldquo;intra-hugging gestures&rdquo; through a probabilistic behavior algorithm &mdash; the creation of which contributed an additional five new design guidelines, on top of a revision to one of the original six.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxMDA4ODg5NTI0LWh1Z2dpZTQuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="An image of the HuggieBot robot embracing a researcher; its arms, covered in soft fabric, are seen holding the researcher as the researcher leans in to the hug. A screen on the head shows a smiling cartoon face; a skirt is seen to the bottom of the picture, hiding a framework which gives the robot its human-like height.">Built using off-the-shelf robot arms with added soft padding, an inflatable and warmed chest area, and given a welcoming face, HuggieBot is an undeniably huggable robot.In its tested incarnation, HuggieBot 3.0 incorporates a custom sensing system dubbed HuggieChest which uses two inflated chambers of polyvinyl chloride to provide a soft surface. As a hug takes place a barometric pressure sensor and microphone serve to detect human touch, transmitting data via an Arduino Mega microcontroller board to a Robot Operating System (ROS)-based computer located in the robot&rsquo;s 3D-printed head &mdash; behind a display which provides a friendly and welcoming animated visage.The robot&rsquo;s own hugs, meanwhile, are delivered using a pair of Kinova JACO arms, mounted to a custom metal frame and each with six degrees of freedom. &ldquo;These arms were selected for being anthropomorphic, quiet, and safe,&rdquo; the researchers explain. &ldquo;Their movement speed is limited in firmware, and they can be mechanically overpowered by the user if necessary.&rdquo;<h1>Hug autonomy</h1>Initially, the team&rsquo;s testing took place as a &ldquo;Wizard of Oz&rdquo; set-up &mdash; where a remote operator, not seen by participants, controlled the robot&rsquo;s motions. Once data had been gathered from 512 human trials across 32 participants, it was used to train a machine learning system capable of detecting and classifying a range of gestures carried out by humans during a hug and which the team classified into four key intra-hug gestures: Remaining still, moving vertically, tapping or patting on the back, and tightening hold in a squeeze.&ldquo;Rather than maximizing user acceptance for each robot gesture, which would result in the robot only squeezing the user,&rdquo; the researchers note, &ldquo;our behavior algorithm balances exploration and exploitation to create a natural, spontaneous robot that provides comforting hugs.&rdquo;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxMDA4OTA1MjM5LWh1Z2dpZTIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Diagrams of the four &quot;intra-hug gestures&quot; defined by the research team: Remaining still, moving vertically, tapping or patting on the back, and tightening the hold in a squeeze.">The researchers developed a system for detecting, classifying, and responding to four key gestures using a pressure sensor and a microphone in the robot&#39;s torso.Further testing on 16 participants certainly seems to bear that claim out: Twelve of the users were found to hug the robot for longer, subjects declared that they felt &ldquo;significantly more understood&rdquo; by the robot, and stated that the revised robot was &ldquo;significantly nicer to hug&rdquo; in its new interactive form.Perhaps the biggest outcome of the work, however, is in its extension of the earlier &quot;commandments&quot;: The team revised a guideline on hug initiation, creating a new framework for inviting a potential participant and ensuring consent, while adding new guidelines on ensuring a hugging robot is capable of automatically adapting to the user&rsquo;s height in order to avoid some below-the-belt contact which occurred during HuggieBot 3.0&rsquo;s testing, adding a requirement for gesture detection and perception, improving the speed and variety of robot responses to said gestures, and having the robot proactively provide gestures of its own.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxMDA4OTE0MDUyLWh1Z2dpZTMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="A flow chart for the HuggieBot 3.0, showing how the robot starts by waiting for a person then initiates a hug verbally, opens its arms, begins the hug, then responds to or initiates a range of intra-hug gestures with varying probability before releasing the user at the end.">HuggieBot 3.0&#39;s gesture classification system allows it to respond to hug participants in a number of ways, providing variety and offering a more realistic experience.There&rsquo;s more to the team&rsquo;s work than simply building a better hugging robot: The researchers point to the body of work showing the benefits to physical interaction between humans, and how it may be possible to deliver at least some of the those benefits using autonomous or remotely-controlled robotic surrogate &mdash; or, as part of future work on the platform, potentially generalize HuggieBot&rsquo;s capabilities to a robot which could provide companionship and assistance in care homes, helping to address concerns regarding an ageing population and a dearth of care workers.The team&rsquo;s work has been accepted in to the journal <cite><a href="https://dl.acm.org/doi/10.1145/3526110" target="_blank">ACM Transactions on Human-Robot Interaction</a></cite>, and has been made available in preprint form under free access terms.<h1>References</h1>Alexis E. Block, Hasti Seifi, Otmar Hilliges, Roger Gassert, and Katherine J. Kuchenbecker: <cite>In the Arms of a Robot: Designing Autonomous Hugging Robots with Intra-Hug Gestures, ACM Transactions on Human-Robot Interaction, Accepted February 2022</cite>. DOI <a href="https://doi.org/10.1145/3526110" target="_blank">10.1145/3526110</a>.Alexis E. Block, Sammy Christen, Roger Gassert, Otmar Hilliges, and Katherine J. Kuchenbecker: <cite>The Six Hug Commandments: Design and Evaluation of a Human-Sized Hugging Robot with Visual and Haptic Perception, Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction</cite>. DOI <a href="https://doi.org/10.1145/3434073.3444656" target="_blank">10.1145/3434073.3444656</a>.

HuggieBot, with its soft pressure-sensing torso, 3D-printed animated head, and six-degree-of-freedom arms, is a research platform into high-quality autonomous hugging.

While the concept of a hugging robot may sound bizarre, the researchers behind the project — now in its third generation — believe that such a device could have a major impact on everything from social telepresence to elder care.

Intra-hug gestures prove key to making HuggieBot 3.0 the most cuddly robot around

In the not-too-distant future, autonomous, networked vehicles will be able to handle most tasks independently. However, they are not yet capable of dealing with all traffic situations. If they encounter a problem that they do not know how to solve, safety comes first. Most of the time this would mean pulling over to the side of the road and stopping. The German Aerospace Center (Deutsches Zentrum f&uuml;r Luft- und Raumfahrt; DLR) is conducting research into the teleoperation of these vehicles in order to resolve such situations safely and quickly in the future. In unclear situations, the autonomous vehicle can request teleoperation support from a technical supervisor.<h2>Teleoperation &ndash; a technological bridge and a legal requirement</h2>&quot;Teleoperation of autonomous vehicles serves as a bridge to bring existing technologies in the field of automation rapidly into use and onto the roads,&quot; says Michael Oehl of the <a href="https://www.dlr.de/SiteGlobals/Categories/Thesaurus/DLR-Institute/EN/_Research_facility/_DLR/_DLR_site_Berlin/Institute_of_Transportation_Systems.html" target="_blank" title="Institute of Transportation Systems">DLR Institute of Transportation Systems</a> in <a href="https://www.dlr.de/content/en/sites/braunschweig.html" target="_blank" title="Braunschweig">Braunschweig</a>. His team&rsquo;s work focuses on autonomous driving at SAE Level 4, where the automation takes over all of the driving tasks. The occupants do not have to intervene, even though this may remain possible using control elements such as steering wheels.Experts describe the different degrees of automation as SAE levels, based on the classification of systems for autonomous driving devised by SAE International (formerly the Society of Automotive Engineers), the organisation for mobility technology. This classification ranges from Level 0 to Level 5. At Level 0, the vehicle has no automation whatsoever. Level 5 equates to full automation, where the vehicle is completely autonomous and must be able to determine the overall driving behaviour under all circumstances. The occupants are simply passengers. Humans no longer have to intervene, not even by means of teleoperation. SAE Level 4 systems are not yet ready for everyday use, but the legal framework for using them does exist in a number of places, including Germany. The &#39;Autonomous Driving Act&#39; came into force in July 2021 and the rules were specified in greater detail in the &#39;Autonomous Driving Regulations&#39;. The law allows autonomous driving at SAE Level 4 on special routes or within defined areas such as airports, logistics centres and motorways. This opens up possibilities including transporting people or goods with driverless shuttles, for example. For operation and liability, teleoperation or technical supervision &ndash; as defined in the law &ndash; plays a decisive role.<iframe src="https://www.youtube.com/embed/4iF_8waO3Vw?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 767px; height: 438px;"></iframe>Remote Assistance: A Key Enabler for Highly Automated Driving (SAE 4)Highly automated vehicles (SAE 4) are becoming ever more sophisticated. However, due to the sheer number of potential situations the vehicle may encounter, its automation is not yet capable of dealing seamlessly with all of them. Remote Assistance as a type of remote operation is a way to take care...Credit: &copy; DLR. All rights reserved<h2 id="isPasted">Focus on humans and automation working together</h2>&quot;We are at the beginning of development and have many ideas and concepts for making vehicle teleoperation safe and efficient,&quot; says Oehl. &quot;For now, we are concentrating primarily on the interaction between humans and automation, and on the design of the interface that this requires &ndash; the human-machine interface, or HMI for short. For example, we are looking at the form that a teleoperation workstation might take, what the job requires of people and how it has to be designed from the point of view of work and engineering psychology.&quot;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1650451290642-teleoperation-workstation.jpg" style="width: 766px;" class="fr-fic fr-dib">Teleoperation workstation. Credit: &copy; DLR. All rights reserved<h2>The human factor &ndash; requirements and workplace design for teleoperation</h2>In future, several teleoperators equipped with monitors, user interfaces and headsets could sit together in a control centre, guiding autonomous vehicles at SAE Level 4 through difficult situations. Imagine a double-parked delivery vehicle, for instance. In such a scenario, a car will not drive over a solid line of its own accord but remain where it is. Similar situations can arise on construction sites that have not yet been incorporated into the system, so are not known to the automation system.Before they can grapple with these numerous scenarios, however, DLR&#39;s researchers still have to address a number of fundamental questions. What is the best way to map the vehicles for monitoring, and what information is needed? How many vehicles can a single person monitor and oversee on screen at the same time? What kind of live images can cameras in highly automated vehicles provide to help the technical supervisors make decisions? How long does it take to process a request for assistance on average? And how long does it take for operators to put themselves in the position of the vehicle and make a decision?&quot;Vigilance is particularly important in such monitoring tasks. In research, we use this term to describe a state of constant alertness. Even if nothing happens for a long time, you have to be able to react quickly when it does. Teleoperation workstations have to be specially designed in order to prevent a situation where people get distracted, switch off or start experiencing fatigue. Just like train drivers or flight controllers, these operators cannot be constantly overworked, but they must not be underchallenged either,&quot; says Oehl. In order to investigate these questions more closely in the context of the teleoperation of vehicles, the team has constructed such a workstation as a prototype. In this way, the researchers are investigating in a very application-oriented way what information teleoperators will require at their workstation in the future in order to be able to support autonomous vehicles with SAE Level 4 quickly and safely in situations that are unclear for the automation system.

Construction sites pose a challenge for autonomous vehicles. Credit: DLR (CC BY-NC-ND 3.0)

In the not-too-distant future, autonomous, networked vehicles will be able to handle most tasks independently. However, they are not yet capable of dealing with all traffic situations. If they encounter a problem that they do not know how to solve, safety comes first.

Teleoperation steps in when an autonomous vehicle does not know what to do

This article was discussed in our <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1651619086659000&usg=AOvVaw31M1Y-wsRw4c0h8-at-VxZ" href="https://www.wevolver.com/profile/the.next.byte" id="isPasted" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/e2238814-9212-4010-afb6-ac48d637140f?dark=false" class="fr-draggable" data-gtm-yt-inspected-1_19="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe>The full article will continue below.<hr>This article was first published on <a href="https://news.mit.edu/2022/robot-stroke-treatment-remote-0413" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>. &nbsp;<hr>MIT engineers have developed a telerobotic system to help surgeons quickly and remotely treat patients experiencing a stroke or aneurysm. With a modified joystick, surgeons in one hospital may control a robotic arm at another location to safely operate on a patient during a critical window of time that could save the patient&rsquo;s life and preserve their brain function.The robotic system, whose movement is controlled through magnets, is designed to remotely assist in endovascular intervention &mdash; a procedure performed in emergency situations to treat strokes caused by a blood clot. Such interventions normally require a surgeon to manually guide a thin wire to the clot, where it can physically clear the blockage or deliver drugs to break it up.One limitation of such procedures is accessibility: Neurovascular surgeons are often based at major medical institutions that are difficult to reach for patients in remote areas, particularly during the &ldquo;golden hour&rdquo; &mdash; the critical period after a stroke&rsquo;s onset, during which treatment should be administered to minimize any damage to the brain.The MIT team envisions that its robotic system could be installed at smaller hospitals and remotely guided by trained surgeons at larger medical centers. The system includes a medical-grade robotic arm with a magnet attached to its wrist. With a joystick and live imaging, an operator can adjust the magnet&rsquo;s orientation and manipulate the arm to guide a soft and thin magnetic wire through arteries and vessels.The researchers demonstrated the system in a &ldquo;phantom,&rdquo; a transparent model with vessels replicating complex arteries of the brain. With just an hour of training, neurosurgeons were able to remotely control the robot&rsquo;s arm to guide a wire through a maze of vessels to reach target locations in the model.&ldquo;We imagine, instead of transporting a patient from a rural area to a large city, they could go to a local hospital where nurses could set up this system. A neurosurgeon at a major medical center could watch live imaging of the patient and use the robot to operate in that golden hour. That&rsquo;s our future dream,&rdquo; says Xuanhe Zhao, a professor of mechanical engineering and of civil and environmental engineering at MIT.Zhao and his team have <a href="https://www.science.org/doi/10.1126/scirobotics.abg9907" target="_blank">published their findings</a> today in Science Robotics. MIT co-authors include lead author Yoonho Kim, Emily Genevriere, and Jaehun Choe, along with Pablo Harker, Robert Regenhardt, Justin Vranic, Adam Dmytriw, and Aman Patel at Massachusetts General Hospital (MGH), and Marcin Balicki of Philips Research North America.<iframe src="https://www.youtube.com/embed/IYlHl6h8lm4?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 767px; height: 427px;" data-gtm-yt-inspected-1_19="true" id="391874016"></iframe><h2 id="isPasted">Tilt and twist</h2>Endovascular surgery is a specialized, minimally invasive procedure that involves carefully twisting and guiding a thin medical wire through the body&rsquo;s arteries and vessels to a target location, in a way that avoids damaging vessel walls. The procedure typically requires years of training for a surgeon to master.Robotic systems are being explored as assistive technologies in endovascular surgery. These systems mainly involve motor drives that advance and retract a wire while twisting it through the body&rsquo;s vasculature.&ldquo;But having a robot twist with the same level of sophistication [as a surgeon] is challenging,&rdquo; Kim notes. &ldquo;Our system is based on a fundamentally different mechanism.&rdquo;The team&rsquo;s new system builds on <a href="https://news.mit.edu/2019/robot-brain-blood-vessels-0828" target="_blank">work from 2019</a>, in which they demonstrated steering a magnetically controlled thread through a life-sized silicone model of the brain&rsquo;s blood vessels. They did so at the time using a handheld magnet, about the size of a soup can, that they manually manipulated.They have since affixed the magnet to the end of a medical-grade robotic arm, which can be steered using a small joystick knob on a mouse. By tilting the joystick, researchers can tilt the magnet in an orientation that a magnetic wire can follow. Buttons on the mouse control a set of motorized linear drives, which advance and retract the wire to make it move forward and back.The wire is as thin and flexible as a conventional neurovascular guidewire, with a soft, magnetically responsive tip that follows and bends in the direction of a magnetic field.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ5OTIzNDk1NTU1LU1JVC1Sb2JvLVN0cm9rZS0wMi1wcmVzcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">The MIT team envisions that its robotic system could be installed at smaller hospitals and remotely guided by trained surgeons at larger medical centers. Courtesy of the researchers<h2>Finding a path</h2>The team tested the robotic system in MGH&rsquo;s Catheter Lab &mdash; an operating room with standard medical imaging equipment used in endovascular procedures. The researchers installed the robotic arm in the lab, along with a life-sized silicone model of blood vessels. They set the joystick, along with a monitor displaying a live video of the model, in a control room. From there, an operator watched the video while using the joystick to remotely steer the wire through the vessels.The team trained a group of neurosurgeons to use the robotic system. After just one hour of training, each surgeon was able to successfully operate the system to guide the wire through complex vessels that are difficult to navigate with a manual guidewire.The team also used the robotic system to clear simulated clots in difficult-to-reach areas in the model. They steered the guidewire through vessels, and around sharp corners and turns, to reach regions where the researchers simulated clots. Once they guided the wire to the clot, the surgeons proceeded with standard endovascular methods to thread a microcatheter along the wire to the site of the clot. They retracted the wire, leaving the catheter, which they then applied to successfully remove the clot.&ldquo;The primary purpose of the magnetic guidewire is to get to the target location quickly and safely, so that standard devices like microcatheters can be used to deliver therapeutics,&rdquo; Kim says. &ldquo;Our system is like a pathfinder.&rdquo;He hopes that the teleoperated system can help more patients receive time-critical treatment. He also sees benefits for surgeons, who typically perform such vascular procedures in the same room as the patient, while being exposed to radiation from X-ray imaging.&ldquo;The neurosurgeons can operate the robot in another room or even in another city without repeated exposure to X-rays,&rdquo; Zhao says. &ldquo;We are truly excited about the potential impact of this technology on global health, given that stroke is one of the leading causes of death and long-term disability.&rdquo;<hr>&quot;Reprinted with permission of <a href="https://news.mit.edu/2022/robot-stroke-treatment-remote-0413" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>&quot;&nbsp;

MIT engineers developed a telerobotic system to help surgeons remotely treat patients experiencing stroke or aneurysm. With a modified joystick, surgeons may control a robotic arm at another hospital to operate on a patient. Courtesy of the researchers

The system could provide teleoperated endovascular treatment to patients during the critical time window after a stroke begins.

Joystick-operated robot could help surgeons treat stroke remotely

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&amp;amp;-gas, environments.In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video will be showed during the international conference ICRA 2021, on May 31st.With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot&nbsp;combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (&gt; 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (&lt; 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.<hr>Further informationThe research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).&nbsp;Website: <a href="https://advr.iit.it/projects/inail-scc/teleoperazione" target="_blank">https://advr.iit.it/projects/inail-scc/teleoperazione</a>

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation.

Telerobotics

Profiles