Article #8 of Innovations in Electric and Autonomous Vehicles Series: Advancements in voice recognition, touch displays, and other HMI technologies of next-gen vehicles are all set to transform the way we interact with our rides. Car dashboards will be safer, non-distracting, and highly intuitive.

Next-Generation Vehicles Need Safe Human Machine Interfaces

<p id="isPasted">On the 24<sup>th</sup> May 2022 in Uvalde, Texas, a gunman killed 19 children and two adults at an elementary school. This is part of the events <a href="https://www.texastribune.org/2022/05/27/uvalde-texas-school-shooting-timeline/" target="_blank">timeline</a>, focused on the police functioning during this event:&nbsp;</p><ul><li><u>11:28 a.m.</u> - Shooter arrives at school</li><li><u>11:30 a.m.</u> - A teacher calls 911</li><li><u>11:35 a.m.</u> - &ldquo;&hellip;Pete Arredondo, the chief of the school district&rsquo;s police department, arrives at the scene around this time. He does not use his radio. <span dir="RTL">)</span><strong>Arredondo wanted both hands on his gun if he encountered the shooter and believed the radio would have slowed him down</strong>, his attorney said.&rdquo;<span dir="RTL">(</span></li><li><u>12:03 p.m.</u> - A student calls 911 from room 112 for a minute and 23 seconds and identifies herself in a whisper. Meanwhile, as many as 19 officers <strong>are positioned in the school&rsquo;s hallway</strong>.</li><li><u>12:46 p.m.</u> - Breach command is given. Arredondo gives his approval to enter the classroom. &ldquo;If y&rsquo;all are ready to do it, you do it,&rdquo; he says, according to a transcript of police body-camera footage.</li><li><u>12:50 p.m.</u> - Forces kill gunman&nbsp;</li></ul><p><strong>It took one hour and twenty minutes from the first 911 call until the gunman was neutralized!</strong></p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/1NqlAcD-OqE?&ab_channel=WFAA&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 720px; height: 480px;" data-gtm-yt-inspected-11="true" id="559664948" title="Uvalde shooting: Police bodycam footage shows moments when law enforcement entered Robb Elementary"></iframe></span></p><h2><strong>The problem&nbsp;</strong></h2><p>Emergency responders all over the world need an immediate understanding of their surroundings. This critical information makes the difference between lives saved and unfortunately, horrible disasters.<br>All forces involved share the same desire for precise information ASAP in a clear and visual way that will<span dir="RTL">&nbsp;</span>enable them to make smart decisions in real-time.</p><p>From investigating the horrible attack in Texas, we can specify a few points that were handled poorly:</p><h3><strong>Non-unified data management in control rooms led to inefficient management of the forces.&nbsp;</strong></h3><p>Control rooms and dispatches are constructed of multiple sub-systems and&nbsp;multiple&nbsp;screens. These subsystems rely<span dir="RTL">&nbsp;</span>on:</p><ul><li>Extensive operator training</li><li>The misassumption that operators are not suffering from cognitive load&nbsp;</li><li>The ability to communicate with each other simultaneously</li></ul><p>These factors can cause fragmented data, which can easily lead to the loss of important information and in some cases can cost human lives.<span dir="RTL">&nbsp;</span></p><p><a href="https://www.statista.com/statistics/1183457/iot-connected-devices-worldwide/" target="_blank">In a six-year period, the amount of IoT devices in the world is expected to double</a>, which will make the cognitive load even more challenging and almost impossible to process without data fusion systems.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjU4MTc1NjY4NTQ3LXJpb19jb250cm9sX3Jvb21fZ2VvcmdlX3NvYXJlcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 720px; height: 480px;" class="fr-fic fr-dib"></p><h3><strong>Unclear visual picture and lack of critical elements geolocation in the school.</strong></h3><p>Emergency responders that arrived at the scene were missing relevant visual information and understanding of the current status at the elementary school - in particular, regarding the attacker&rsquo;s and civilians&#39; locations.&nbsp;</p><h3><strong>Operators&rsquo; limitation to free his hands for radio &amp; tech devices (tablets, etc.).</strong></h3><p>A major fact is that Chief Arredondo was only focused on his gun, claiming to be unable to free his hands for the radio, even at the price of being cut off from communication with others.</p><h3><strong>The flow of information from control rooms to the field.</strong></h3><p>It is clear from this timeline that the information <strong>was</strong> <strong>available</strong> to the control room streaming constantly to the 911 dispatch via oral information.&nbsp;</p><p>Moreover, the ability to exploit accessible data such as smartphone devices of the students or even the gunman was possible, yet, unfortunately, unavailable to the emergency responders as a result of poor distribution to the field.&nbsp;</p><h2>The Solution</h2><p>The ultimate solution for this type of scenario will involve taking all the data that was streamed in real-time to the 911 dispatch, combined with any other accessible data such as the students&rsquo; smartphone information, static cameras deployed in the school and other friendly forces&rsquo; locations that were on-site in order to bring to any relevant emergency responder a <strong>comprehensive image</strong> of what&#39;s going on in a way that will <strong>improve efficiency</strong> and drastically increase situational awareness.</p><p>Oversight develops <em><strong>Absorber</strong></em><strong>&nbsp;- A visual, 3D, Command-and-Control (C&amp;C) solution based on augmented and virtual reality tools</strong> enabling information sharing and management on top of Geographic Information System (GIS) between many users simultaneously. The system does not require a physical location to run in and can be managed remotely.</p><h3><strong>Complete data fusion - one platform for all information stitched together in real-time</strong></h3><p>This product has a novel unified and distributed data management system.<strong>&nbsp;</strong>All devices that are map-locatable and can produce valuable data to the system, (hereinafter, data collection agents) - <u>Drones and other unmanned aerial vehicles, robots, IoT sensors, smartphones, XR headsets (extended reality), PTZ cameras or other stationery cameras&nbsp;</u>and other sources that produce localization information or other visual information.</p><h3><strong>XR-based display overlaid on reality image</strong></h3><p>Another breakthrough is connecting information to the actual world by displaying data collection agents in real-time on the user reality image. Seeing things without any physical limitations will open a whole new world for managing data.</p><h3><strong>Easy to operate</strong></h3><p>Controlling information on such a system is done using the operator&rsquo;s <strong>hands, head and eye gestures</strong> in a common and natural way. This will reduce the number of operators and the level of training required. Logics can be in-built in order to eliminate the operator needing to deal with other devices. For example, he will be able to talk to a partner only by focusing on them with his eyes.</p><h3><strong>Decentralized multiple-player system</strong></h3><p>This method for controlling agents is especially suitable for a rear control room operation while the user in the field can also participate by managing data. All data can be shared on one system, making the system mobile and not so dependent on hardware-based devices.</p><h3><strong>Bottom line</strong></h3><p><em>Absorber&nbsp;</em>is a breakthrough all-in-one system that can independently operate throughout a developing situation and communicate information in real-time. <strong>It enables an immediate flow of information and smart decision making</strong></p><h3><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/7WTHrbVi7r0?&t=7s&ab_channel=Oversight&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 727px; height: 454px;" data-gtm-yt-inspected-11="true" id="283478871" title="Absorber by Oversight"></iframe></span></h3><h2><em>Absorber&nbsp;</em>and Its Components</h2><p>The different data collection agents <em>Absorber&nbsp;</em>will support have the common ability to continuously convey their current location. This is a basic requirement of the system to allow accurate determining of each agent&rsquo;s location, their spatial relation to others and displaying them on a hologram-based 3D digital table-top view board using XR headsets or smartphones.&nbsp;</p><p><em>Absorber&nbsp;</em>will serve as an add-on or replacement for agent management and can enhance operators&rsquo; situational awareness in real-time.&nbsp;</p><h3>Collectors&nbsp;</h3><p><em>Absorber&nbsp;</em>consists of &ldquo;collectors&rdquo; - software utilities for connecting devices to the system. Collectors convert the protocol of the connected device into <em>Absorber&rsquo;s&nbsp;</em>proprietary protocol - that&rsquo;s what makes the system unified, since each device can have different operating systems, SDK, units of measurement or even global datum reference.</p><p>XR devices serve as the system display and also act as data collection agents since most XR devices can report their relative location, produce a real-time map and report device status. By connecting them to a GPS device they can also be a map-locatable device.</p><p>The very use of immersive technology and holograms to represent information will make it possible to build a multi-user system that can share information even from remote locations.</p><h3>Core &nbsp;</h3><p><em>Absorber&rsquo;s&nbsp;</em>core components are media server for managing videos from multiple devices, map server in order to store relevant geocell information for a comprehensive on-premises solution and an interactive simulation engine in order to create events and scenarios for an offline training solution. Data fusion logics for managing multiple agents will be built on top of that collaborative information.</p><h3>Users and SDK</h3><p>On the other side, we have the end-users who can be control room operators or even emergency responders. The system will also deliver SDK to let other users build solutions on top of this collaborative information.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://player.vimeo.com/video/731725270?autoplay=1&loop=1&autopause=0&background=1&muted=1&quality=1080p" width="560" height="315" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-11="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p>

Using AR/VR technology, Oversight gives emergency responders the ability to literally SEE THROUGH WALLS and strengthens the connectivity between control rooms and field operators.

Enhancing Emergency Responders Situational Awareness Using XR Tools

<h2 id="isPasted">How industrial robots can help address the growing labor shortage in manufacturing</h2><p>Many manufacturers are struggling to find skilled labor for their manufacturing plants? According to the Society of Manufacturing Engineers, 89% of manufacturers are having difficulty finding skilled workers in the United States today. Moreover, a 2018 study by&nbsp;<a href="https://www.themanufacturinginstitute.org/press-releases/manufacturing-industry-faces-unprecedented-employment-shortfall-2-4-million-skilled-jobs-projected-to-go-unfilled-according-to-deloitte-and-the-manufacturing-institute/" target="_blank">Deloitte and the Manufacturing Institute</a> projected that 2.4 million US manufacturing jobs will go unfilled from 2018-2028 due to the lack of skilled talent<sup>1</sup>. The same study notes that manufacturers who are searching for skilled talent like robot programmers, operations managers, and production managers, expect the skilled labor gap to worsen over the next three years, as the aging baby boomer workforce retires and younger workers decline to enter the manufacturing field.</p><p>Not only is the skilled labor shortage impacting businesses&rsquo; ability to hire, but it&rsquo;s also stunting businesses&rsquo; ability to grow and meet production demands. Manufacturers across industries today are having to turn away new business as a result of not having adequate labor resources. According to a 2018&nbsp;<a href="https://www.kornferry.com/press/korn-ferry-study-reveals-global-talent-shortage-could-threaten-business-growth-around-the-world" target="_blank">study by Korn Ferry</a>, the labor skills gap in the global manufacturing industry could result in more than $607 billion in lost revenue by 2030. In this scenario, the U.S. alone would experience a total of $73 billion in unrealized output.</p><h2>What can be done to address the widening skilled labor gap?</h2><p>One effective solution is to implement robotic automation for repetitive tasks. While there were initial fears that industrial robots would replace human jobs, the reality is that industrial robots are actually helping manufacturers to supplement the low-skilled, repetitive jobs that are otherwise going completely unfilled. According to <a href="https://www2.deloitte.com/us/en/insights/focus/human-capital-trends/2018.html" target="_blank">Deloitte&rsquo;s 2018 Global Human Capital Trends report</a>, 47% of manufacturers are already supplementing their current workforce with automation, with 24% using AI and robotics to perform routine tasks and 16% using automation to augment human skills.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1654781735511-shutterstock_681146401.webp" style="width: 766px;" class="fr-fic fr-dib"></p><p id="isPasted">Robotic automation is also helping manufacturers to refocus existing skilled workers on higher-level tasks that require human-only skills. With automation, seasoned human workers are empowered to use their intrinsic knowledge of production best practices to help the manufacturing plant well beyond repetitive or tedious tasks. Refocusing skilled workers on more strategic tasks can create critical competitive advantages for manufacturers, especially as all competitors worldwide struggle to find and retain talent. Additionally, research from the&nbsp;<a href="https://www.weforum.org/agenda/2019/06/industrial-robots-are-bringing-jobs-back-home-but-not-for-low-skilled-workers" target="_blank">Word Economic Forum</a> reveals that increasing robotic automation actually helps to reshore low-skill production jobs from abroad, which in turn helps to increase employment and wages for high-skilled workers.</p><p>Automation doesn&rsquo;t just stop at helping fill open manufacturing jobs. Using industrial robots for tasks that were previously strenuous, or even dangerous, for humans can result in a safer factory environment for all workers. This is especially true for the automotive and electronics industries, where humans were at higher risk for workplace injuries pre-automation; industrial robots in these industries have diverse arrays of computer-vision sensors that help detect when human body parts, like fingers or hands, are in the way of a task being completed. Some industrial robots also have deep machine learning capabilities via artificial intelligence that further help them to learn tasks on the job without pre-programming, all while ensuring that production efforts are being completed in accordance with the highest safety standards.</p><h2>New technologies are enabling automation</h2><p>Advances in industrial automation software are also helping to aid the skilled labor shortage. In the past, manufacturers that purchased industrial robots needed to hire teams of engineer-level programmers to take weeks or months to code the new machines. There are over 30 different manufacturers of industrial robots, each with their own <a href="https://en.wikipedia.org/wiki/Robot_software#Examples_of_programming_languages_for_industrial_robots" target="_blank">unique programming language</a>, meaning that the highly-skilled programmers needed to know how to program the specific industrial robot that was purchased. While these skill requirements made it very difficult in the past for manufacturers to deploy and redeploy robots as quickly as needed, there are now new solutions in automation that make it possible to launch robots without relying on hard-to-find programmers. For example, at READY Robotics, our <a href="https://www.ready-robotics.com/software/forge-os/" target="_blank">Forge/OS</a> software makes it possible for any factory worker to program any robot in minutes via a user-friendly, visual touchscreen &ndash; no coding required. This means that businesses can now rely on existing factory workers to complete the work of rare, high-skilled programmers, which is both more time-efficient and cost-effective.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1654781794102-pasted-image-0-3.webp" style="width: 766px;" class="fr-fic fr-dib"></p><p id="isPasted">Human-machine pairing that is enabled by collaborative robots (cobots) will also be helpful for factories that need to maximize their existing human workforce alongside robots. Similar to industrial robots, cobots can also aid with dangerous and repetitive tasks, improve product quality and consistency, and increase overall productivity. A major benefit of cobots over industrial robots is that they allow robots to work within the same factory space as humans, with minimal safety concerns. In some cases, new product designs can be deployed up to 50% faster with cobots than industrial robots, particularly since they often use human-machine interfaces (HMI). By removing the need to have formal robotics or programming experience, HMIs help to address the skilled labor shortage by making it easier and quicker for all workers to learn how to use cobots.</p><p>As the labor shortage situation in the U.S. manufacturing industry continues to unfold in the coming decade, manufacturers should continue to plan ways that they can meet production demands without the availability of highly skilled labor. Though industrial automation cannot fill all roles left empty by the skilled labor shortage, it can help manufacturers to directly address the areas of production that are most repetitive and easiest to complete by robots. It is uplifting to see how automation worldwide is already helping to supplant, and not replace, human workforces. Robotic automation is transforming the factory of the future, where not only is production more efficient, but the available human workforce also benefits from higher-level tasks and safer work environments. It&rsquo;s evident that by working through the implementation of automation, while also empowering factory workers, manufacturers will be in the best position themselves to be successful in the future.</p>

Many manufacturers are struggling to find skilled labor for their manufacturing plants? According to the Society of Manufacturing Engineers, 89% of manufacturers are having difficulty finding skilled workers in the United States today.

Human-Machine Interaction (HMI)

Latest Posts

Next-Generation Vehicles Need Safe Human Machine Interfaces

Enhancing Emergency Responders Situational Awareness Using XR Tools

Solving the Skilled Labor Shortage with Automation

Profiles