In a broad sense, "machine vision" refers to the utilization of visual information and optical information using digital technologies. However, because it is being introduced to production sites in the manufacturing industry, machine vision generally refers to digital image technologies utilized in automatic inspection and process automation in factories. Machine vision consists of the following:・Imaging section (cameras) ・Interface section (board/general-purpose interface) ・Processing engine section (computer) ・Processing control section (software)etc. The image data obtained by the imaging section is loaded into the processing engine via the interface section. In the processing engine section, the processing control section performs binarization*1, edge detection*2, and other forms of image processing. Next, it outputs the image data that has completed image processing to robots and industrial equipment.*1: Processing that converts the pixel values (threshold values) that are preset for the image to the two values of black and white. *2: Processing that detects the bright and dark areas in an image.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327121894-img-0001-en.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 1: Machine vision configurationMachine vision is used in automatic inspection, automatic recognition, and automatic measurement, etc. For example, automatic inspection and automatic recognition are very fast compared to manual inspection. Moreover, machine vision used in automatic measurement can accurately identify small value differences that humans cannot measure. In addition, linking machine vision to AI functions is expected to produce more intelligent uses going forward.<h2>Differences Between Machine Vision, Computer Vision, and Robot Vision</h2>Computer vision and robot vision are technologies that are similar to machine vision, but they each play different roles. Here, we explain how their roles differ.<h3>Differences Between Machine Vision and Computer Vision</h3>"Computer vision" is a technology that adds equipment control functions to machine vision functions. Computer vision can control various types of equipment based on data output by machine vision. For example, this would make it possible to combine the image data output by machine vision with CAD data generated by the computer to control processors or to adjust the conveyor transport speed based on image data that captured the progress of line work. In addition, computer vision has a broad scope of utilization that naturally includes smart factories as well as referee assistance systems and virtual replays in sports and projection mapping, etc. in entertainment.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327141371-img-0002-en.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 2: Example of computer vision utilization<h3 id="isPasted">Differences Between Machine Vision and Robot Vision</h3>"Robot vision" is a technology that adds machine vision to robot visual functions. Machine vision attaches cameras to various types of equipment to obtain image data, but robot vision attaches cameras to the end of robot arms to obtain image data. This enables the robot to operate based on visual information obtained by itself. For example, a robot that does not use robot vision can only execute certain operations programmed in advance. However, when a robot equipped with robot vision moves its robot arm closer to an object, it then sequentially identifies the target based on image data from the camera to execute operations. Because this type of operational control is possible, a robot equipped with robot vision is capable of detailed, precise, and flexible operations that cannot be specified with a program.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327158790-img-0003-en.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 3: Example of robot vision utilization<h2 id="isPasted">Example of Machine Vision Utilization in a Smart Factory</h2>Machine vision is used in various types of work in a smart factory. Here, we introduce an example of machine vision utilization.<h3>Reading Characters at Ultrahigh-speed – OCR (Optical Character Recognition/Reader)</h3>OCR is a technology that scans handwritten or printed characters with an image scanner or digital camera. Machine vision scans the 2D code, serial number, and date of manufacture, etc. printed on products. The scanned data is utilized in inspections of the print quality or whether the correct characters are printed as well as the screening of the next process, etc. In many cases, the products being scanned by OCR pass through the line at high speeds. Therefore, machine vision uses cameras that can capture images at high speed and image processing systems that are capable of high-speed processing.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327179168-img-0004+%281%29.jpg" style="width: 100%px;" class="fr-fic fr-dib">Figure 4: OCR on a manufacturing line<h3 id="isPasted">Scanning Shapes in 2D and 3D – Visual Inspections</h3>The utilization of machine vision is considered to be the most effective means of visual inspection on a manufacturing line. Because machine vision-based visual inspection is able to inspect the accuracy of products after cutting and molding and the mounting condition of components, etc. with high speed and accuracy, it is essential for improving production efficiency and maintaining quality. The surfaces of target objects can vary widely in terms of their glossiness and light absorption, etc. The lighting used in machine vision enables products with various surface conditions to be inspected by changing the method of illumination and the color of the light rays. In addition, 3D cameras equipped with a stereophonic*3, time-of-flight*4, and other measurement systems as well as structured light*5 and other imaging systems can also measure the depth.*3: Method that uses two cameras to perform triangulation and measure the distance to an object. *4: Method that measures the distance to an object according to the time it takes for light emitted by a light source to reflect off of the object and return to a light detector. *5: Lighting method that emits light with a specific shading pattern.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327197939-img-0005.jpg" style="width: 100%px;" class="fr-fic fr-dib">Figure 5: Visual inspection with a 3D camera<h3 id="isPasted">Grabbing Specific Components – Picking</h3>"Picking" on a manufacturing line refers to the task of collecting (pickup) the necessary components and products. Utilizing machine vision makes it possible to identify and pick components with the specified color and shape even if components and products with various colors and shapes are mixed together in one case.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327216910-img-0006.jpg" style="width: 100%px;" class="fr-fic fr-dib">Figure 6: Picking by a robot<h3 id="isPasted">Providing Visual Functions to Robots – Robot Operational Control</h3>In recent years, industrial robots have also been used in precision work due to their high operating precision. For example, they play an active role in installing, soldering, and welding microcomponents. However, controlling such work with programming requires high-precision positioning devices. With robot vision utilizing machine vision, high-precision positioning devices are not required for the robot to perform precise positioning with the image data from the camera mounted on the end of the robot arm.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327233332-img-0007.jpg" style="width: 100%px;" class="fr-fic fr-dib">Figure 7: Installing microcomponents with a robot<h2 id="isPasted">Future of Machine Vision – AI Vision</h2>As various types of industrial equipment shift to using AI, one form of machine vision evolution that shows promise is "AI vision."<h3>What is AI Vision?</h3>AI vision is a system that equips lightweight AI chips in smartphones, etc. with cameras and camera functions to make inferences about captured image data. AI vision resembles computer vision in that it uses AI (artificial intelligence), but computer vision mainly makes inferences with a cloud AI service (Figure 8). On the other hand, AI vision equips an AI module in an edge device to make inferences, and the inference results are stored on the edge server. The edge server communicates with the cloud AI only when necessary. (Figure 9).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1721327267580-1721327267580.png" class="fr-fic fr-dib">Therefore, AI vision has the following advantages over computer vision: ・Able to decrease the volume of communication with the cloud AI service ・Able to prevent external leaks of data collected by the sensors ・Superior responsiveness Furthermore, the cameras and smartphones used in AI vision are called "edge AI devices" while the method that makes inferences on the edge device is called "edge AI."<h3>What Technologies Are Required to Realize AI Vision?</h3>Although AI vision has the advantages described above, AI modules equipped on edge devices require the following types of functions. ・Superior energy conservation due to restrictions on battery capacity ・Ability to perform high-speed processing due to the responsiveness requirement ・Superior heat dissipation due to installation in a limited space More than anything, it is important that the AI modules be compact.<h2>Summary</h2>This article explained the topic of machine vision and the similar phrases of computer vision and robot vision as well as AI vision. Going forward, it is expected that machine vision and AI will evolve in a close relationship with the evolution in deep learning for image data. In addition, new AI systems and products are expected to be developed through the proper use of cloud AI and edge AI. These trends will likely become essential technologies for smart factory robots, automatic inspection, automatic recognition, and automatic measurement equipment that require visual information that is closer to the human eye and decision-making capabilities that exceed that of humans.

In a broad sense, "machine vision" refers to the utilization of visual information and optical information using digital technologies.

The Eyes of Smart Factories - What is Machine Vision?

Industry 5.0 is a new phase of industrialization that builds upon the fourth industrial revolution (Industry 4.0) by adding a people-centric approach. It is characterized by the extensive use of automation, data exchange, and smart manufacturing technologies in a manner that fosters the well-being of the workers. It bridges the gap of the unfulfilled promise of full automation from Industry 4.0 in processes where a human in the loop is still needed, &nbsp;increasing the competitiveness of the industry and helping attract the best talents.The ultimate goal is to create human-centric, efficient, flexible, and responsive production environments that can quickly adapt to changing market demands.To achieve this goal, this next generation of manufacturing environments leverages the latest available technologies to the entire production process. One of the key operational aspects of manufacturing is intralogistics -understood as the management, control, and optimization of internal material and information flows.&nbsp;Intralogistics 5.0 leverages automated material handling, Internet of Things (IoT), data analytics, and Artificial Intelligence (AI) to create smart, interconnected, data-driven production environments and empower human workers.&nbsp;This intralogistics evolution is not only adding efficiency to industrial processes but is also driving a shift in manufacturing, from large, rigid systems into modular, flexible, and robot-supported self-optimizing ones.<h2>Optimizing Material Handling with Visual AI-Enabled Robotic Vehicles</h2>Autonomous Mobile Robots (AMRs), equipped with Visual AI and orchestrated through a Fleet Management System (FMS) are at the forefront of this transformation. The shape and function of these AMRs range from autonomous forklifts, shuttle systems, tugger trains, and stackers to platform or mouse robots, goods-to-person, and pick-assist robots. Their material transport activities are ideally tracked directly by the Enterprise Resource Planning (ERP) system of the factory.AMRs are designed to navigate with agility and accuracy through complex factory environments using a combination of sensors, cameras, and real-time data processing. The integration of Visual AI enables them to perceive their surroundings, identify objects and obstacles, make informed decisions, and share information in real-time, adding a number of benefits at each step of the process:<ul><li id="isPasted">Enhanced Goods-In: Vehicles equipped with Visual AI contribute to this stage by autonomously transporting goods from receiving areas to designated storage locations reducing manual work and extending operation hours. Additionally, the visual input ensures accurate identification and categorization of incoming goods, feeding the ERP, and laying the foundation for precise inventory management.</li><li>Optimized Storage and Retrieval: Visual AI allows robotic vehicles to analyze available storage space and stack goods efficiently, optimizing storage capacity, ensuring timely retrieval of items, and providing real-time inventory updates.</li><li>Streamlined Production Supply: Visual AI makes autonomous intralogistics vehicles inherently flexible, thus they can optimize routes dynamically, ensuring a continuous and streamlined supply of materials to production lines.&nbsp;</li><li>Accurate Order Picking: Visual AI technology supports workers in order picking, significantly reducing error rates and ensuring accurate order fulfillment.‍</li><li>Efficient Dispatch and Goods-Out: Visual AI enables robotic logistics vehicles to perform order verification and optimize load onto transportation vehicles, adding efficiency to the preparation and organization of finished goods for delivery to customers or distribution centers.</li></ul><h2>Flexible Manufacturing</h2>Besides the optimization across the material handling process, and on account of their inherent reconfigurability, Visual AI AMRs can flexibly support diverse production scenarios, such as batch, mass, or customized production, and different manufacturing layouts, from linear or modular to cellular.&nbsp;Serving as flexible links between individual production stages, these robots are also fundamental in the transition from traditional rigid conveyor-belt-based manufacturing to flexible-cell manufacturing, where production units can be reconfigured and rearranged without downtime.&nbsp;This flexible movement and connection of production cells, create dynamic and modular production environments that can adapt to different product variants and volumes, to respond to changing customer demands and market conditions.<h2>Challenges on the Path Towards the Factories of the Future</h2>Visual AI allows operating this Factory of the Future today. While the technology is already here, its adaption and implementation are only starting to pick up speed. The transition to Intralogistics 5.0 and new levels of automation is mostly gradual and incremental as it requires interoperability, scalability, and human-robot collaboration.<h3>Interoperability</h3>Many manufacturing facilities and warehouses are currently operating with mixed fleets that consist of both manned and unmanned material-handling vehicles, and an initial and crucial challenge in the path toward Intralogistics 5.0 is to integrate both types of shop floor traffic into an all-encompassing FMS and ERP.At this stage, equipping manned vehicles -like forklifts, tuggers, or stackers- with Visual AI-enabled Real Time Location System (RTLS) to track their location and feed the system with the visual data they capture would enable an FMS to orchestrate floor traffic holistically.Adding observability to manned traffic and integrating it into an FMS cancels out typical shortcomings of mixed fleets such as human drivers colliding with AMRs, inefficient manned routing unobservable by the FMS, or inventory compromised by human errors.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyOTAzNTU1NTQzLU1hbm5lZF9Vbm1hbm5lZF9UcmFmZmljX0ludGVncmF0aW9uX1NldmVuc2Vuc2VfUm9ib3RpY3MucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Visual AI-enabled RTLS feeding manned traffic data into an FMSThe Visual AI-enabled RTLS adds value in manufacturing and warehousing unlocking completely new opportunities in material handling applications, where some operations will continue to be performed using manned vehicles for the next 20+ years.To make the benefits of holistically managed mixed fleets broadly accessible to the market, Sevensense Robotics is developing a Visual AI-enabled RTLS, which is already available to early adopters.<h3>Scalability</h3>In many cases, as part of their automation journey, manufacturers opt for initially commissioning AMRs at a small scale in a dedicated sector of their facilities. However, expanding the AMRs’ operating space from a wing of a plant or to the entire factory has traditionally been time-consuming.Until now, it implied a complex and disruptive endeavor of stopping operations and dedicating specialized professionals to the map expansion. including running a vehicle in manual mode to scan the new areas of the facility where robots will operate, and then manually editing and merging the new maps.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1702909540635-Continous+map+updates+V2.gif" style="width: 302px;" class="fr-fic fr-dib">Lifelong Visual SLAM enables seamless map updatesTo solve this scalability blocker, Sevensense has developed a solution that makes the map expansion seamless. Combining continuous map updates -also known as Lifelong Visual SLAM- with swarm intelligence, robots equipped with this technology can automatically map the new sections of the environment and share the extended map with the rest of the fleet. All they need to start their exploration is an anchor point in the existing map and the expansion is computed on-the-fly.By automatically incorporating relevant changes in the environment, Sevensense’s Lifelong Visual SLAM also offers the key benefit of lifelong robust, high precision localization &lt;5mm (&lt;0.2in) without markers, reflectors, or beacons, allowing the robots to operate in dynamic, ever-changing environments.<h3>Human-Robot Collaboration</h3>An increasing number of companies are incorporating AMRs into their operations. However, the lack of seamless human-machine collaboration solutions poses the need for dedicated robotic operations zones or the introduction of robotic shifts, restricting their flexibility and efficiency while also limiting the ROI.Visual AI AMRs have the built-in capability of understanding the context of the environment, distinguishing between unobstructed paths, obstacles -and types of obstacles-, and adapting their behavior accordingly in real-time.When the robot encounters an obstacle, the system identifies the type of object, then estimates its velocity and predicts its trajectory, calculates the most efficient path, and automatically performs an obstacle avoidance maneuver, rerouting the robot.This navigation mode can boost efficiency, in particular in scenarios when robots share the shop floor with other robots, but it may not be the optimal solution when robots share the shop floor with people.&nbsp;On the one hand, as human behavior is less predictable, robots have less visibility of a person’s movement intent than they have on other mobile robots or RTLS-tracked manned vehicles. On the other hand, having a robot overtake people while walking across a facility may be perceived as unpredictable and unsafe, making human workers feel anxious.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyOTAzNzkyNTUyLUh1bWFuX01hY2hpbmVfQ29sbGFib3JhdGlvbl9Sb2JvdF9OYXZpZ3Rpb25fU2V2ZW5zZW5zZV9Sb2JvdGljcy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Visual AI is key to developing collaborative AMRsSolving the human-robot collaboration challenge is key to enabling Industry 5.0 and is motivating Sevensense’s work to solve this issue by combining scene understanding and a people-centric approach, to turn AMRs into Co-AMRs. The solution involves Visual AI perceiving the obstacle as a person and then commanding the robot to perform a people-friendly behavior such as stopping and waiting for the person to continue their movement or complete their task.To learn more about Sevensense Robotics and its Visual AI technology, visit <a href="https://www.sevensense.ai/" target="_blank" rel="noopener noreferrer">sevensense.ai</a>

Visual AI enables Autonomous Mobile Robots to fulfil Industry 5.0 requirements

Visual AI is applied to overcome the obstacles in the path of the Factories of the Future

Autonomous Mobile Robots Powering Industry 5.0

The intralogistics specialist STILL, a subsidiary of the KION Group, leads the German project consortium of the Intelligent Motion Control -IMOCO4.E- research project.The goal of the IMOCO4.E project is to enable intelligent trucks to navigate autonomously in a factory or warehouse.In numerous warehousing and production settings, semi-autonomous trucks are already in use and have already proven to be profitable assets. However, they remain constrained by certain limitations.&nbsp;While these trucks can presently detect obstacles and apply brakes independently, achieving complete self-driving capabilities requires further development. Intelligent trucks should be able to navigate warehouse environments fully autonomously. They should be able to understand their surroundings, avoid obstacles, and find the most efficient routes.The IMOCO4.E project focuses on applying solutions based on Artificial Intelligence (AI) to address challenges in four main scenarios: navigation, collection, and movement of goods, and delivery to their final position.In this context, STILL's iGo neo platform, which already comes very close to the idea of this autonomously driving vehicle due to its 'intelligent' equipment and the resulting capabilities, &nbsp;was equipped with an Alphasense Autonomy unit from Sevensense Robotics.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5NDQ4MDQ2ODU4LUFwbGhhc2Vuc2UgQXV0b25vbXkgaW5zdGFsbGVkIG9uIGlHTyBOZW8ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Alphasense Autonomy unit installed on a STILL iGo neo platform - Image credits STILL">Alphasense Autonomy leverages Visual AI technology to provide precise navigation and obstacle avoidance capabilities in complex and dynamic warehouse environments -with racking, inbound and outbound areas, and even when transitioning from indoors to outdoors.The Visual AI system maps out the environment and provides precise positioning without using any type of infrastructure.In addition, Sevensense is prototyping the implementation of AI to perform Semantic Scene Understanding, which allows the vehicles to extract information about the objects in their environment, to detect, classify, determine the position, -and when applicable, predict the movement of obstacles e.g. people and vehicles, and other objects of interest such as pallets, docking or charging stations.With this information, the vehicles are able to adapt their navigation behavior -stopping, slowing down, or overtaking obstacles-, based on real-time perception, allowing them to interact with the environment, for example, docking, picking, or charging, while safely sharing the floor with moving elements like people and other vehicles.“In the IMOCO4.E4.E project, STILL is partly refining and extending existing approaches with regard to general interactions of his iGo neo platform in an unknown environment. The high precision and reliability of the localization provided by Sevensense's Visual AI technology is of great benefit here",&nbsp;says Ansgar Bergmann, responsible for the IMOCO4.E project at STILL.The collaboration between Sevensense Robotics and STILL within the IMOCO4.E research project marks a significant stride toward the realization of fully autonomous trucks in warehouse and factory environments.The project is scheduled to be completed in the fourth quarter of 2024.<a href="https://www.sevensense.ai/" target="_blank" rel="noopener noreferrer">Learn more about Sevensense Robotics and its Visual AI technology</a>

STILL's iGo neo platform operating in a warehouse setting - image credits STILL

Sevensense’s Visual AI implemented into STILL's iGo neo platform for precise Autonomous Navigation and Pallet Handling

Still Incorporates Visual AI Technology on IMOCO4.E Research Project

<h2 id="isPasted">Its impact on Visual SLAM and beyond</h2>It is clear that Artificial Intelligence (AI) is here to stay and there is a need for an open discussion, to bring some light into the AI black box.At Sevensense Robotics we strongly believe that transparency builds trust, this is why we decided to tackle the elephant in the room and dissect the role that AI plays in our offering.Let’s start by exploring what AI is and the difference between Machine Learning (ML) and Deep Learning (DL).The original definition of AI, coined in 1955 by John McCarthy -one of the founding fathers of the discipline- was really broad: “The science and engineering of making intelligent machines.”Nowadays, we conceive AI as a subset of computer science focused on creating systems that can replicate human intelligence and problem-solving abilities. It involves any technique that enables machines to mimic human intelligence, using logic, if-then rules, decision trees, and machine learning -including deep learning.Machine Learning (ML) is a subset of AI that intersects with statistics. It involves the development of algorithms and statistical models that enable computers to learn and make predictions or decisions without being explicitly programmed; instead, they recognize patterns in the data and make predictions once new data arrives.Deep Learning (DL) is a specialized subset of Machine Learning that analyzes data with a logical structure similar to how a human would draw conclusions.&nbsp;To achieve this, DL applications use a layered structure of algorithms called an Artificial Neural Network (ANN). It is characterized by its deep layers of interconnected nodes or neurons, which allows it to automatically learn hierarchical representations of data, making it well-suited for tasks like image and speech recognition.The three technologies are nested one within the next:<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MDIxMTQzOTQ1LURpZmZlcmVuY2VzIGJldHdlZW4gQUksIE1hY2hpbmUgTGVhcm5pbmcgYW5kIERlZXAgTGVhcm5pbmcgKDIpLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="Differences between AI, ML and DL">Visual AI refers to Artificial Intelligence applications that focus on enabling machines to interpret, understand, and process visual information, such as images and videos. Visual AI technologies employ advanced Machine Learning algorithms, Deep Learning models, and computer vision techniques to analyze and extract meaningful insights from visual data.AI-Powered Robot AutonomySevensense uses Visual AI to allow robots to identify, understand, and act on imagery and visual data -captured by cameras- just like humans so that they can navigate autonomously.By processing camera images with AI, the process known as Visual Simultaneous Localization and Mapping (SLAM) builds a rich 3D map of the environment and precisely localizes within it.Sevensense’s Visual SLAM technology comprises proprietary and lightweight AI sensor fusion algorithms to enable robots to precisely estimate their position. This is achieved by extracting distinctive elements from the images, for example, a corner of a window, and using these to build a representative yet compact 3D model of the environment.&nbsp;To deliver robust localization under all conditions, including lighting and environmental changes, Sevensense employs AI -more concretely Deep Convolutional Neural Networks- for feature extraction. Incorporating AI for this purpose makes it possible to overcome a limiting constraint of legacy SLAM technology, which until now could not cope with significant illumination and other environmental changes, leading to failures during autonomous navigation in difficult settings.Sevensense also uses machine learning to maintain the visual maps of dynamic spaces to ensure robust localization and reusability of the initial map of the environment. The solution known as Lifelong Visual SLAM is basically a multi-session map that uses the initial map as the basis and later on, the changes in the environment are added to it to enrich it. All of this, without the need to run Visual SLAM from scratch on each session, increases operational efficiency.The Lifelong Visual SLAM solution enables building and maintaining always-up-to-date maps. These maps incorporate data from multiple conditions of the environment, such as during cloudy- and sunny weather, bright and dark lighting situations, and changes in the elements present in the area, e.g. changes in factory floor layouts, or simply in the warehouse or shop inventory. All of these changes are detected automatically and incorporated into the lifelong map.This allows the maps to be used for visual positioning in environments that change considerably over time.To enable autonomous navigation even in complex and ambiguous environments, a Convolutional Neural Network is employed to perform Semantic Scene Understanding making it possible to detect, classify, determine the position, -and when applicable, predict the movement of obstacles e.g. people and vehicles, and other objects of interest such as pallets, trolleys, floor markings, docking or charging stations.After the robots have gained insights into their environment and the expected behavior of the elements around them, it is time for them to plan their next move. Reinforcement Learning makes it possible for robots to adapt their navigation behavior -stopping, slowing down, or overtaking obstacles, based on real-time perception, allowing them to interact with the environment, for example, docking, picking, or charging, while safely sharing the floor with moving elements like people and other vehicles.All of these processes happen on the Edge, on the multi-camera Alphasense Edge AI device -a GPU-accelerated compute unit- embedded in the robot. By bringing computational power closer to the data source, Sevensense minimizes latency challenges -enabling nearly simultaneous, real-time perception and decision-making- and addresses security and privacy issues typically present in solutions purely based on cloud computing, where visual data needs to be recorded and transmitted to remote servers for processing.&nbsp;AI-Powered Continuous Performance ImprovementTo exceed the ever-increasing performance requirements of the market, both in terms of accuracy and robustness, Sevensense leverages ongoing cutting-edge developments in the fields of AI, computer vision, and robotics.&nbsp;Sevensense uses AI models to enrich its simulation, training, and testing infrastructure.Combining Generative AI and Digital Twins, Sevensense is able to create the most challenging training and testing environments to continuously validate its software stack.Visual AI AnalyticsThe visual data captured by the cameras, apart from providing inputs for best-in-class localization and autonomy performance, also unlocks the potential for providing ground-breaking additional services.The rich visual data captured by the cameras allows Sevensense to offer further Visual AI analytic services focused on providing organized data and insights -beyond Visual SLAM- such as asset tracking, stocktaking, surveillance, and inspection, as an additional safety or reporting data layer.

Visual AI enables autonomous navigation and an array of object-recognition-based applications

Its impact on Visual SLAM and beyond

AI in Robotics Perception and Autonomous Navigation

<h2>Challenges of self-localization in mixed in- and outdoor environments</h2>Knowing its own location or position within an operating environment (a.k.a. self-localization) is one of the key capabilities any mobile robot must master to be able to get from A to B. Yet, depending on the environment the robot is operating in, self-localization can impose vastly different challenges, rendering the choice of sensor technology and software algorithms far from a trivial task.Indoors, 2D LiDAR technology is widely used, as an example, on AGVs and AMRs for material handling operations. Such a technology allows to measure the geometric structure of the near-by environment (e.g. walls, shelves) to localize in pre-built maps. However, it shows its limitations in environments which do not have unique geometric features (e.g. a long corridor) or are highly dynamic especially if shared with people.&nbsp;On autonomous vehicles for outdoor environments, instead, satellite navigation systems such as GPS or Galileo are often the first choice of sensor technology for self-localization, for good reasons: The GPS receivers are very cheap, and they yield an easily interpretable position in a known global reference coordinate system (e.g., WGS84).&nbsp;However, an accurate GPS position requires unobstructed skies. This is a limitation that is violated in several circumstances like urban surroundings, company campuses, &nbsp;logistic yards and airport tarmac. In fact, these outdoor places often require to operate close to or even inside buildings or tunnels. Due to these restrictions, mobile robots operating outdoors or in mixed indoor-outdoor settings cannot rely solely on GPS for self-localization.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1MTMxNTk3MzM0LU1peGVkIGluZG9vciBhbmQgb3V0ZG9vciBwb3NpdGlvbmluZy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Mixed indoor and outdoor environments are challenging for GPS and LiDAR solutions aloneEven though 2D LiDAR sensors can be employed outdoors too, using them in the open only works in well-structured spaces of limited extent or requires additional infrastructure such as laser reflectors in order to work better., or to a better extent thanks to the installation of additional infrastructure such as laser reflectors. 3D LiDARs, on the other hand, remain a very expensive sensor gear, despite all efforts in cutting down the costs in recent years.<h3>The strength and challenges of visual localization</h3>As an alternative to GPS and LiDAR sensors, cameras are able to reliably sense the environment both indoors and outdoors. Vision thus has the capability to bridge this gap left uncovered by GPS or LIDAR sensors. It offers highly accurate and reliable self-localization in both indoor and outdoor environments with a very cost effective sensor gear.&nbsp;Visual self-localization does not come without its very own challenges. Cameras naturally provide good images in well illuminated surroundings, while during night time outdoors, or in dim indoor areas, visual self-localization becomes more difficult. However, the selection of camera sensor chips with a high dynamic range &nbsp;has shown to allow proper images to be taken even under challenging illumination conditions.<blockquote>Instead, LiDAR sensors can be employed. However, using 2D LiDARs outdoors only works in well-structured spaces of limited extent. As an alternative to GPS and LiDAR sensors, cameras are able to reliably sense the environment both indoors and outdoors.</blockquote>Further, and in contrast to GPS or LiDAR sensor data, images taken by cameras of the same environment can look vastly different under varying seasonal, weather, or lighting conditions. This change in appearance renders accurate localization difficult. Additionally, geometric characteristics are often unfavorable for visual localization outdoors compared to indoors, as there is often less stable, human-made structure visible, plus it is often located at a considerably farther distance. Therefore, smart algorithms to work with visual data are paramount.<h3>Tackling the challenges of visual localization with AI</h3>To master these challenges, Sevensense employs a smart AI algorithm based on lifelong visual mapping that enables building and maintaining always-up-to-date maps of the environment. These maps incorporate data from multiple appearances of the environment, such as before- and after remodeling of a room, during cloudy- and sunny weather, or bright and dark lighting conditions. This allows the maps to be used for visual localization under practically any appearance condition.&nbsp;At the core of this lifelong mapping process is a software algorithm capable of merging visual data from multiple traversals through the same environment, recorded under potentially very different appearance conditions, into a single, geometrically consistent map. This process is fully automated, resulting in having a multi-appearance map ready for reliable localization after traversing through the environment only a handful of times beforehand.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1MTMxNzMxMTk3LUxpZmVsb25nIHZpc3VhbCBtYXBwaW5nIGFsZ29yaXRobXMgdXNlZCBieSBTZXZlbnNlbnNlLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Smart lifelong visual mapping algorithms allow for always-up-to-date mapsFurther, continuously updating this map over time naturally captures structural change in the environment. This way events like remodeling of rooms, construction sites or temporary modifications in street or room layouts don&rsquo;t disrupt everyday operations. These changes are detected automatically and swiftly incorporated into the lifelong map.&nbsp;In particular, no expert knowledge or intervention is necessary at any point in the process. Just as generating the initial map of the environment can be done by anyone without any prior training or expert know-how, updating the maps at any later point is done automatically by Alphasense Position. This considerably lowers the costs of maintaining the system in a long-term horizon.&nbsp;<h3>Vision is the optimal solution for indoor- and outdoor self-localization</h3>By employing this lifelong mapping technique, vision overcomes its limitations and turns out to be the go-to technology for cost-effective, but still reliable and accurate self-localization both indoors and outdoors. In particular, it offers seamless transition from indoor to outdoors and vice-versa, as the same camera sensors and the same map can be used to localize in both types of environments. This is a fundamental advantage over any technology primarily relying on GPS for outdoor localization and navigation that can revolutionize the logistics, delivery, cleaning or supply chain processes.<blockquote>By employing this lifelong mapping technique, vision turns out to be the go-to technology for cost-effective, but still reliable and accurate self-localization both indoors and outdoors.</blockquote>Sevensense has already successfully demonstrated the unique benefits of visual localization in a <a href="https://www.sevensense.ai/blog/kyburz" target="_blank" rel="noopener noreferrer">proof-of-concept study in partnership with KYBURZ Switzerland AG</a>. By equipping a mail delivery robot with Alphasense Position, accurate and reliable self-localization of the vehicle became possible at any time indoors, as well as outdoors, where LiDAR and GPS solutions alone failed.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695132745402-Reliable+positioning+indoors+and+outdors+cameras+overcome+LiDAR+and+GPS+limitations+Sevensense.jpeg" style="width: 300px;" class="fr-fic fr-dib">Delivery robot used to benchmark Alphasense Position, 3D LiDAR and GPS localization performance[the missing image will go here once the CMS issue is fixed]<blockquote>By equipping a mail delivery robot with Alphasense Position, accurate and reliable self-localization of the vehicle became possible at any time indoors, as well as outdoors, where LiDAR and GPS solutions failed.</blockquote>Along a similar vein, the indoor- and outdoor capabilities of Alphasense Position will open up countless other opportunities for bringing cost-effective autonomy to robotic platforms operating in challenging environments. Delivering goods across multiple buildings on a campus, or material handling in warehouses undergoing frequent layout change have never been simpler before.<h3>With vision towards a fully autonomous future</h3>Self-localization is essential for robots to safely navigate in changing environments and achieve full autonomy. By employing cameras for this task together with smart algorithms for building and maintaining maps, autonomy can be provided at a fraction of the costs - be it for the sensor gear itself, as well as operating costs over the robot&rsquo;s lifetime.&nbsp;At Sevensense we are constantly pushing cutting edge technologies forward to deliver affordable autonomy solutions for new application fields.You can find more informative material about Alphasense Position on its <a href="https://www.sevensense.ai/product/alphasense-position" target="_blank" rel="noopener noreferrer">product page</a>.

Test vehicle benchmarking cameras, LiDAR and GPS systems

Breaking Free from LiDAR and GPS Constraints: Harnessing Vision for Reliable Positioning, Indoors and Outdoors

Why reliable positioning indoor and outdoor requires cameras

Sevensense Robotics builds the eyes and brains of mobile robots. &nbsp;We strive to provide the most flexible and robust Visual AI solutions to give robots the ability to identify, understand, and act on imagery and visual data -captured by cameras- just like humans so that they can navigate autonomously.Our technology enables robots to adapt their navigation behavior based on real-time perception, allowing them to complete their missions, for example, docking, picking, or charging, while sharing the floor with moving elements like people and other vehicles.To ensure everything we ship has the high performance our customers expect, we stress-test our technology in different challenging settings.

Sevensense Robotics pushes the boundaries of Visual SLAM technology

Visual AI

Latest Posts

The Eyes of Smart Factories - What is Machine Vision?

Autonomous Mobile Robots Powering Industry 5.0

Still Incorporates Visual AI Technology on IMOCO4.E Research Project

AI in Robotics Perception and Autonomous Navigation

Why reliable positioning indoor and outdoor requires cameras

The Sevensense Labs

Profiles