Optimizing AI for the Future: Model Efficiency and Scalability


Intelligence at Scale Through AI Model Efficiency

This is the first in a series of articles, from Synopsys that look at&nbsp;<a href="https://www.wevolver.com/article/designing-energy-efficient-ai-accelerators-for-data-centers-and-the-intelligent-edge" target="_blank" rel="noopener noreferrer">AI</a>, automotive, and multi-die chip design.&nbsp;<h2>Introduction</h2>According to Allied Market Research, the global artificial intelligence (AI) chip market is projected to reach $263.6 billion by 2031. The AI chip market is vast and can be segmented in a variety of different ways, including chip type, processing type, technology, application, industry vertical, and more. However, the two main areas where AI chips are being used are at the edge (such as the chips that power your phone and smartwatch) and in data centers (for deep learning inference and training).No matter the application, however, all <a href="https://www.synopsys.com/ai/ai-powered-eda.html" target="_blank">AI chips</a> can be defined as <a href="https://www.synopsys.com/glossary/what-is-ic-design.html" target="_blank">integrated circuits (ICs)</a> that have been engineered to run machine learning workloads and may consist of FPGAs, GPUs, or custom-built ASIC AI accelerators. They work very much like how our human brains operate and process decisions and tasks in our complicated and fast-moving world. The true differentiator between a traditional chip and an AI chip is how much and what type of data it can process and how many calculations it can do at the same time. At the same time, new software AI algorithmic breakthroughs are driving new AI chip architectures to enable efficient deep learning computation.Read on to learn more about the unique demands of AI, the many benefits of an AI chip architecture, and finally the applications and future of the AI chip architecture.<section id="isPasted"><h2>The Distinct Requirements of AI Chips</h2>The AI workload is so strenuous and demanding that the industry couldn’t efficiently and cost-effectively design AI chips before the 2010s due to the compute power it required—orders of magnitude more than traditional workloads. AI requires massive parallelism of multiply-accumulate functions such as dot product functions. Traditional GPUs were able to do parallelism in a similar way for graphics, so they were re-used for AI applications.The optimization we’ve seen in the last decade is drastic. AI requires a chip architecture with the right processors, arrays of memories, robust security, and reliable real-time data connectivity between sensors. Ultimately, the best AI chip architecture is the one that condenses the most compute elements and memory into a single chip. Today, we’re moving into multiple chip systems for AI as well since we are reaching the limits of what we can do on one chip.Chip designers need to take into account parameters called weights and activations as they design for the maximum size of the activation value. Looking ahead, being able to take into account both software and hardware design for AI is extremely important in order to optimize AI chip architecture for greater efficiency.</section><section><h2>The Benefits of AI Chip Architecture</h2>There’s no doubt that we are in the renaissance of AI. Now that we are overcoming the obstacles of designing chips that can handle the AI workload, there are many innovative companies that are experts in the field and designing better AI chips to do things that would have seemed very much out of reach a decade ago.As you move down process nodes, AI chip designs can result in 15 to 20% less clocking speed and 15 to 30% more density, which allows designers to fit more compute elements on a chip. They also increase memory components that allow AI technology to be trained in minutes vs. hours, which translates into substantial savings. This is especially true when companies are renting space from an online data center to design AI chips, but even those using in-house resources can benefit by conducting trial and error much more effectively.We are now at the point where AI itself is being used to design new AI chip architectures and calculate new optimization paths to optimize power, performance, and area (PPA) based on big data from many different industries and applications.</section><section><h2>AI Chip Architecture Applications and the Future Ahead</h2>AI is all around us quite literally. AI processors are being put into almost every type of chip, from the smallest IoT chips to the largest servers, data centers, and graphic accelerators. The industries that require higher performance will of course utilize AI chip architecture more, but as AI chips become cheaper to produce, we will begin to see AI chip architecture in places like IoT to optimize power and other types of optimizations that we may not even know are possible yet.It’s an exciting time for AI chip architecture. Synopsys predicts that we’ll continue to see next-generation process nodes adopted aggressively because of the performance needs. Additionally, there’s already much exploration around different types of memory as well as different types of processor technologies and the software components that go along with each of these.In terms of memory, chip designers are beginning to put memory right next to or even within the actual computing elements of the hardware to make processing time much faster. Additionally, software is driving the hardware, meaning that software AI models such as new neural networks are requiring new AI chip architectures. Proven, real-time interfaces deliver the data connectivity required with high speed and low latency, while security protects the overall systems and their data.Finally, we’ll see photonics and multi-die systems come more into play for new AI chip architectures to overcome some of the AI chip bottlenecks. Photonics provides a much more power-efficient way to do computing and multi-die systems (which involve the heterogeneous integration of dies, often with memory stacked directly on top of compute boards) can also improve performance as the possible connection speed between different processing elements and between processing and memory units increases.One thing is for sure: Innovations in AI chip architecture will continue to abound, and Synopsys will have a front-row seat and a hand in them as we help our customers design next-generation AI chips in an array of industries.Read more about how <a href="https://wevlv.co/synopsys-chip-arch" target="_blank" rel="noopener noreferrer">Synopsys is tackling AI chip architecture challenges</a>.</section>

AI chips, designed for machine learning tasks, are spurring new chip architectures for efficient deep learning computation. 

Why AI Requires a New Chip Architecture

This is the first in a series of articles, sponsored by Synopsys that look at AI, automotive, and multi-die chip design.&nbsp;<h2>Introduction</h2>Artificial intelligence (AI) accelerators are deployed in data centers and at the edge to overcome conventional von Neumann bottlenecks by rapidly processing petabytes of information. Even as Moore’s law slows, AI accelerators continue to efficiently enable key applications that many of us increasingly rely on, from ChatGPT and advanced driver assistance systems (ADAS) to smart edge devices such as cameras and sensors.Although AI accelerators are typically 100x to 1,000x more efficient than general-purpose systems, the computational resources needed to generate best-in-class AI models <a href="https://openai.com/blog/ai-and-compute/" target="_blank" id="isPasted">doubles every 3.4 months</a>. Moreover, training a single deep-learning model such as ChatGPT’s GPT3 creates approximately <a href="https://www.theguardian.com/technology/2023/jun/08/artificial-intelligence-industry-boom-environment-toll" target="_blank">500 metric tons of CO2</a>, the equivalent of over a million miles driven by an average gasoline-powered vehicle! To help reduce global carbon emissions, the U.S. Department of Energy (DoE) <a href="https://www.energy.gov/sites/default/files/2022-02/Semiconductor%20Supply%20Chain%20Report%20-%20Final.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">recently recommended</a> a 1,000x improvement in semiconductor energy efficiency.Achieving optimal performance-per-watt—whether for AI training in the data center or inference at the edge—is understandably a top priority for the semiconductor industry. In addition to minimizing environmental impact, reducing energy consumption lowers operating costs, maximizes performance within limited power budgets, and helps mitigate thermal challenges. Read on to learn how chip designers—including edge AI chip developer <a href="https://sima.ai/" target="_blank">SiMa.ai</a>—are leveraging end-to-end power analysis solutions to build a new generation of more energy-efficient AI accelerators.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1ODA4NTA3MzQ4LWNoaXAtc3VzdGFpbmFiaWxpdHkud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Semiconductors must achieve at least 1,000x improvement in energy efficiency.<section id="isPasted"><h2>Optimizing Power for Billion-Plus Gate Designs</h2>An end-to-end approach to energy efficiency for AI accelerators must start at the architectural and micro-architectural levels during the earliest stages of the design flow and conclude at signoff. That’s why AI chip designers rely on architectural exploration platforms to map and evaluate power, performance, and area (PPA) tradeoffs for specific training or inference applications while proactively identifying critical vectors for downstream analysis. As&nbsp;AI hardware typically consists of large arrays with thousands of tiles (processing elements), billion-plus-gate designs require multi-domain hardware and software power verification to minimize energy consumption and leakage. However, analyzing crucial power blocks and time windows requires advanced emulation systems to run billions of cycles and rapidly deliver multiple—and accurate—iterations. Only after completing this step can register transfer level (RTL) power analysis and physical implementation tools effectively optimize dynamic (gate switching) and static (leakage) power dissipation. To consistently deliver accurate results, RTL power analysis tools for AI chip design should include the following capabilities:<ul><li>Timing-driven fast synthesis: Internal power calculation errors are often caused by fanout-based fast synthesis tools that fail to properly size cells following timing constraints. Like their downstream place-and-route counterparts, fast synthesis embedded in RTL power analysis tools must be timing driven.</li><li>Physically aware fast synthesis: RTL power analysis tools should be “physical aware” and capable of obtaining precise net capacitance values by executing first-pass placement of the cells in the design, as well as global routing. Unlike a fanout-based approach, physically aware capacitance estimation results in a unique and accurate value for each net.</li><li>Signoff-quality power computation engine: Traditional RTL power analysis tools using word-level logic inferencing for fast synthesis can only apply heuristic—and therefore inaccurate—methods for <a href="https://www.synopsys.com/ai/what-is-glitch-power.html#:~:text=Glitches%20occur%20if%20signal%20timing%20within%20the%20paths,leading%20to%20up%20to%2040%25%20additional%20dynamic%20power%29." target="_blank">glitch power computation</a>. To accurately calculate glitch power (which can potentially consume up to 40% of a chip’s total power) and reduce highly replicated tiles, RTL power analysis tools must have a signoff-quality power analysis engine, a netlist level design representation, and an integrated timing engine.</li></ul> After completing RTL power analysis and reduction, physical implementation (synthesis and place and route) tools can be used to further optimize PPA. To ensure reliability, scalability, and a frictionless user experience, these implementation tools should include a single, integrated data model architecture, interleaved engines, and a unified shell. Just as importantly, implementation tools should be capable of accurately modeling advanced node effects and glitch power to accelerate engineering change orders (ECOs) and final design closure.</section><section><h2>Exceeding Energy-Efficiency and Performance</h2>Synopsys offers a comprehensive end-to-end power solution to help AI chip designers cost-effectively meet or exceed ambitious performance and energy-efficiency goals while<a target="_blank"></a><a target="_blank">&nbsp;</a>accelerating time to market. Used at the very beginning of the design flow, <a href="https://www.synopsys.com/verification/virtual-prototyping/platform-architect.html" target="_blank">Synopsys Platform Architect™</a> provides AI chip designers with SystemC™ transaction-level modeling (TLM) tools and efficient methods to rapidly model, analyze, and optimize complex silicon architectures.&nbsp;<a href="https://www.synopsys.com/verification/emulation.html#zebuempower" target="_blank">Synopsys ZeBu® Empower</a>, a fast power profiler, is used for the next stage of the AI chip design process: analyzing and debugging energy consumption—based on hundreds of millions of cycles—for real software workloads. Leading semiconductor companies have significantly reduced power draw with Synopsys ZeBu Empower, including&nbsp;<a href="https://sima.ai/" target="_blank">SiMa.ai</a>, a Silicon Valley-based AI chip startup that designs high-performance, low-energy AI chips for the intelligent edge. Specifically, the company realized a 2.5x frames per second (FPS) per watt improvement for its SiMa.ai™ Low Power MLSoC™. During a presentation at the SNUG Silicon Valley 2023 conference this spring, Sounil Biswas, director of silicon engineering at SiMa.ai, noted that subsequent silicon validation demonstrated excellent correlation between Synopsys ZeBu Empower data and board measurements. To complement ZeBu Empower and enable RTL design for low power, we offer&nbsp;<a href="https://www.synopsys.com/implementation-and-signoff/signoff/primepower.html" target="_blank">Synopsys PrimePower RTL</a>, an RTL power analysis and reduction tool that consistently achieves accurate results (within +/- 15% of post-route implementation) by pairing timing-driven, physically aware synthesis capabilities with an integrated computation engine. Synopsys PrimePower RTL also provides step-by-step guidance to help AI chip designers further minimize glitching and reduce overall power consumption. &nbsp; Additional PPA optimization is achieved with <a href="https://www.synopsys.com/implementation-and-signoff/physical-implementation/fusion-compiler.html" target="_blank">Synopsys Fusion Compiler™</a>, a comprehensive and integrated RTL-to-GDSII implementation system. After passing this milestone, the AI chip design is analyzed with <a href="https://www.synopsys.com/implementation-and-signoff/signoff/primepower.html" target="_blank">Synopsys PrimePower</a>, the golden power signoff solution. Certified by leading foundries worldwide down to 3nm processes, Synopsys PrimePower delivers fast runtime performance with distributed processing, achieving high accuracy at signoff within a few percent of SPICE and silicon measurements.</section><section><h2>Designing Differentiated Silicon for Edge AI Inference</h2>AI accelerators enable many popular applications to quickly analyze massive amounts of information and accurately infer results in milliseconds. At the same time, achieving optimal performance-per-watt remains a top priority for chip designers. This is especially true at the edge, where performance is often limited by minimal power envelopes and smaller die sizes. However, these constraints create new opportunities for semiconductor companies to design differentiated silicon by precisely calibrating PPA to match the specific requirements of low-latency, high-bandwidth applications. For example, autonomous navigation demands a computational response latency limit of 20μs, while voice and video assistants must understand spoken keywords in less than 10μs and hand gestures in a few hundred milliseconds. To successfully implement PPA tradeoffs, chip designers should adopt a holistic approach to power optimization by leveraging an end-to-end solution that spans early architectural exploration to golden power signoff. You can learn more about Synopsys <a href="https://wevlv.co/synopsys-energy-efficient" target="_blank" rel="noopener noreferrer">energy-efficient SoCs solutions here</a>.</section>

Leveraging Power Analysis for Energy-Efficient AI Accelerator

Designing Energy-Efficient AI Accelerators for Data Centers and the Intelligent Edge

<h2>Why You&#39;ll Love This Article</h2>Urban planning, the art of designing and shaping the cities we live in, is undergoing a quiet revolution. Researchers have harnessed the power of Artificial Intelligence (AI) to optimize community layouts, making cities more efficient, accessible, and sustainable. In this article, we&#39;ll delve into the world of AI-driven urban planning, exploring the challenges it addresses, the ingenious solution it offers, and the tangible impact it promises.&nbsp;This research was conducted by researchers at Tsinghua University, Beijing and the full article can be found on <a href="https://www.nature.com/articles/s43588-023-00503-5" target="_blank" rel="noopener noreferrer">Nature</a>.<h2>The Woes of Urban Development</h2>Cities are intricate puzzles with myriad pieces: roads, buildings, parks, and services. Urban planners face the daunting task of piecing together these elements to create functional and harmonious communities. Traditionally, this process relied on manual labor and heuristics, often resulting in suboptimal designs and lengthy planning cycles.One of the fundamental challenges in urban planning is optimizing the spatial layout. How can we ensure that essential services like schools and hospitals are easily accessible to residents? How do we strike the right balance between green spaces and built-up areas? And how can we design road networks that are efficient and free from congestion?<h2>What&#39;s The Solution?</h2>The researchers behind this groundbreaking approach have devised a solution that takes urban planning to new heights. At its core is an AI agent trained to make informed decisions about community layouts. But training an AI to plan cities is no small feat. It involves running millions of simulations, a process akin to trial and error, where the AI learns from its mistakes and successes.Central to this solution is a tool called a Graph Neural Network (GNN). Think of it as a virtual brain capable of comprehending and processing geographical data. It constructs a graph to represent the urban environment, where nodes symbolize various urban elements, and edges capture their relationships and contiguity.Why opt for a GNN, you may wonder? Urban planning is complex, with countless interconnections between different elements. Traditional methods often struggle to consider these intricate relationships. Enter the GNN, with its ability to analyze these connections, making it a crucial asset. It examines city layouts, understands the interactions between different components, and uses this knowledge to make informed decisions.<h2>But Does It Actually Work?</h2>Now, let&#39;s talk about the real-world impact. This AI-powered approach isn&#39;t just theoretical; it&#39;s a game-changer for urban planning.<h3>Efficiency Boost</h3>The AI agent can generate spatial plans in milliseconds, a stark contrast to the hours or days traditional methods require. This significant time saving can expedite the planning process.<h3>Service Accessibility</h3>The AI excels at optimizing community services. It ensures that essential services like schools and hospitals are within a short walking or cycling distance for residents. The accessibility metric indicates a remarkable improvement compared to conventional planning methods.<h3>Ecological Enhancements</h3>The AI knows how to make communities greener. It excels at covering residential areas with greenery, improving the quality of life and environmental sustainability.<h3>Traffic Efficiency</h3>The AI is a traffic virtuoso too. It ensures roads are dense, well-connected, and appropriately spaced, leading to smoother traffic flow and reduced congestion.By combining these quantitative results, the impact becomes clear. AI-driven urban planning promises substantial time savings, enhanced accessibility to services, greener environments, and efficient traffic networks. It&#39;s a recipe for creating smarter and more livable cities.<h2>What Are The Limitations?</h2>It&#39;s vital to recognize that the success of this AI system hinges on the quality of input data. Inaccurate or incomplete geographical information can hamper its performance and while AI is a potent tool, it&#39;s not a replacement for human planners; In fact, it should work alongside human experts, balancing technological prowess with human creativity and judgment.<h2>TL;DR (Too Lazy; Didn&#39;t Read)&nbsp;</h2>AI-driven urban planning marks a significant leap forward in the quest to create better cities. It harnesses the power of AI and GNNs to optimize spatial layouts, making cities more efficient, accessible, and sustainable. While challenges and limitations remain, the potential for smarter, more livable cities is within reach, thanks to the fusion of artificial and human intelligence.

The framework which consists of two separate policy networks (b and d) that take actions for the land use planning task (e) and road planning task (f) respectively, and share a GNN state encoder (a) with a value network (c) that estimates the effect of the current planning result.

Researchers have harnessed the power of Artificial Intelligence (AI) to optimize community layouts, making cities more efficient, accessible, and sustainable

From Pixels to Pavement: AI's Impact on Urban Design

<h2 dir="ltr" id="isPasted">Introduction</h2>In the world of engineering, the integration of the Internet of Things (IoT), robotics, and artificial intelligence (AI) is transforming the capabilities of autonomous systems. This article focuses on the importance of connectivity in enabling robots to access and exchange data, leading to more intelligent and adaptable systems.&nbsp;IoT allows robots to connect to networks, while robotics involves the design and operation of autonomous machines. AI enhances IoT-enabled robots by analysing data and making informed decisions. Sensor technology, connectivity modules, and AI processing units are key components in this integration.&nbsp;<h2 dir="ltr">Understanding IoT, Robotics, and AI</h2>When we talk about integrating IoT devices with robotics and adding a layer of AI in the system, it is necessary to understand the basic differences between AI, IoT, and robotics and how they form a base for IoT-enabled robotics. These three terms are the enablers of advanced robotics, impacting how IoT-enabled robots interact with the environment. For that reason, let&#39;s break down these concepts in simple terms.<h3 dir="ltr">IoT</h3>The IoT refers to a network of physical objects that are embedded with sensors, software, and other technologies to connect and exchange data with other devices and systems over the internet. This exchange of data expands the possibilities for automation, monitoring, and control. While IoT is primarily focused on connectivity and data exchange, it does not inherently possess autonomous decision-making capabilities.<h3 dir="ltr">Robotics</h3>Robotics is the science and technology involved in designing, constructing, and operating robots. Robots are autonomous or semi-autonomous machines that can perform complex tasks, interact with their environment, and manipulate objects. Unlike IoT devices, which primarily focus on connectivity and data exchange, robots are physical entities capable of interacting with the physical world. While some robots may be equipped with IoT capabilities, not all robots are inherently part of the IoT ecosystem.In robotics, the IoT presents an opportunity for integrating advanced robots into broader networks, enabling them to act in a more intelligent and autonomous manner. The IoT has transformed robotics with the paradigm shift known as <a href="https://www.frontiersin.org/articles/10.3389/frobt.2020.00104/full" target="_blank">Internet of Robotic Things (IoRT)</a>.<h3 dir="ltr">AI</h3>AI encompasses the development of intelligent systems that can perform tasks requiring human-like intelligence. AI algorithms enable machines to analyse data, identify patterns, make predictions, and even make decisions based on available information. AI augments the capabilities of IoT and robotics by providing advanced data analysis capabilities and enabling autonomous decision-making. While AI can be integrated with IoT devices and robots to enhance their functionality, not all AI systems are directly part of the IoT ecosystem.<h2 dir="ltr">The Role of AI in the Convergence of Robotics and IoT</h2>AI is the core of this convergence between robotics and IoT. While IoT provides the mechanism for connecting robots and gathering data, AI is the engine that processes, learns from this data, and enables autonomous decision-making.&nbsp;For instance, <a href="https://www.conurets.com/use-of-iot-and-robotics-in-daily-life/" target="_blank">an AI-powered IoT robot</a> could leverage machine learning algorithms to recognize patterns in the data, learn from them, and make predictions or decisions based on its learnings. This combination of IoT connectivity and AI-based learning results in a new breed of intelligent, autonomous robots capable of complex operations without requiring constant human oversight.&nbsp;However, not all robots are inherently part of the IoT. A robot becomes an IoT device when it&#39;s equipped with sensors and connectivity capabilities that allow it to collect, transmit, and receive data over the internet. In essence, it&#39;s the ability to participate in the larger, interconnected ecosystem of data and devices that makes a robot an IoT device.&nbsp;<h2 dir="ltr">The Mechanics of IoT in Robotics and AI</h2>Three primary technological areas are integral to this interconnected system: sensor technology, connectivity modules, and AI processing units.<h3 dir="ltr">Sensor Technology</h3>By combining many sensors&mdash;including cameras, radars, lidars, and ultrasonic sensors&mdash;designers can create data that is merged into a single model of sensor fusion to create a robust sense function. As the Internet of Vehicles (IoVs) and edge intelligence are used for training and real-time environment models, the sense functions are evolving towards distributed model-based or AI-inference processing at the sense-node and in the central cognition.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695746433856-shutterstock_2280373681+%281%29+%281%29.jpg" style="width: 300px;" class="fr-fic fr-dib">Figure 2: Robotic vision camera sensors enable seamless navigation, object recognition, and enhanced situational awareness.&nbsp;The advancement of sensor technology in the IoRT system&#39;s capacity to sense the environment using a variety of sensor types, combine the information from the various sensors, and localize itself and other things using GPS/GNSS signals and both local and high-definition maps to create a semantic understanding and local and global models. Because perception information is the input for analytics and AI processing, the data that these sensors acquire must be in a format that can be used by various and unique cognitive processes of robotic things and applications.<h3 dir="ltr">Connectivity Modules</h3>Connectivity is the lifeblood of the IoT, facilitated by communication modules that enable data exchange between devices. These modules employ a variety of technologies, including Wi-Fi&reg;, Bluetooth&reg;, Zigbee, <a href="https://www.wevolver.com/article/an-introduction-to-lorawan" target="_blank">LoRaWAN</a>, and cellular networks.In terms of connectivity, wireless communication methods are dominant for IoRT. When discussing the interactions and pairings of wireless technologies (such as cellular 5G and beyond), IoT/IIoT, and AI approaches and processes, the term &quot;intelligent connectivity&quot; is used.The connectivity of modules for IoT-enabled robotics involves developing robotic objects with reliable wireless and cellular connectivity is essential. IoRT applications may use wired, wireless, cellular, optical, audio, video, voice, or other types of communication channels. Any of a wide range of protocols&mdash;including TCP, UDP, 802.15.1, 802.15.4, 802.11, 4G/LTE/5G&mdash;can be used as the foundation for the communication networks.Edge-distributed processing, including analytics at the edge based on AI concepts and approaches, replaces centralised processing for mission- and safety-critical IoRT applications where latency, dependability, and throughput are crucial requirements. Edge-distributed processing makes use of multi-access edge computing, fog computing, and intelligent connection offered by wireless and cellular communication (4G/5G and beyond). This idea enables IoRT apps to process data locally and use local learning to apply AI algorithms to the data that has been acquired, reducing the need to transport huge volumes of data to the cloud, store data locally, and lighten the stress on connection lines.<h3 dir="ltr">AI Processing Units</h3>AI processing units, also known as AI accelerators, constitute the brainpower behind IoT-enabled robots. The progress of IoRT depends heavily on the AI processing units. The capabilities of particular robotic objects are improved by the AI algorithms. To improve cognition and decision-making, AI technologies are used to optimise the sensor fusion capabilities of IoRT devices (such as cameras for detecting sight, chemical sensors for identifying smell and taste, microphones for hearing, pressure sensors for detecting touch/pressure). They are also used to extract patterns from data. To give analytics and insights and to optimise the functions of the individual robotic things and their cooperative behaviours as a fleet, AI techniques and approaches are incorporated in the various layers of IoRT platforms.Deep neural networks, such as convolutional neural networks (CNNs), are used to assess and extract visual features from images. To improve the performance of the recognition function by attaining high detection rates of quality variations or probable errors, the strategies are created for partitioning and de-noising monitored signals. IoRT applications can now handle extraordinarily huge data sets because of the transition from central compute to edge/fog nodes. To significantly increase the computing efficiency of an inspection model that can identify the type and <a href="http://www.ijmmm.org/vol7/445-ICMEM2019-216.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">severity of defects</a>, this method is also used in the case of effective manufacturing inspection systems. It combines AI techniques with edge processing by adapting a CNN model to the fog computing environment.At the level of robotic things, video capture, video data compression, video picture pre-processing, and segmentation are performed using perception tools like cameras. Better object identification accuracy can be achieved by jointly training a context and scenery-aware adaption model using data from various video capture systems. An ideal offloading method must be established in order to balance the distribution of the deep learning (DL) computation among IoRT devices, edge servers, and the cloud. The trade-offs between important indicators such network health, video compression, data rates/usage, power consumption, processing delay, frame rate, and processing accuracy of analytics must be taken into account. Numerous distributed edge robotic thing devices can work together at the edge to provide better services.&nbsp;The system&#39;s overall performance can be enhanced by edge computing federation and distribution of functions for compressing the DL model at the edge layer. According to <a href="https://ieeexplore.ieee.org/abstract/document/8337810" target="_blank">a research on object detection in edge computing</a>, the cloud can guarantee the integration of distributed learning models with the edge computing layer and update the parameters of these models on edge devices. The computing power and global knowledge in the cloud infrastructure can be used for processing and assisting the edge nodes in updating DL models when the edge infrastructure is unable to provide a reliable service with the required quality (e.g., detecting objects/patterns with low confidence).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695746483237-shutterstock_668202985-scaled.jpg" style="width: 300px;" class="fr-fic fr-dib">Figure 3: The automation of robot arms within IoRT revolutionizing the manufacturing, logistics, logistics, and assembly processes. Image credit: KukaMouser Electronics, an authorised global distributor of electronic components, offers a wealth of products and knowledge that support the development of IoT-enabled robotics and AI systems.&nbsp;These products range from advanced microcontrollers and processors to sensors, connectivity modules, power management devices, and much more. By supplying these essential elements, Mouser is facilitating the development and deployment of IoT-enabled robotics and AI systems across various industries.<h2 dir="ltr">Conclusion</h2>The integration of the IoT with advanced robotics and AI is revolutionizing the capabilities of autonomous systems. Robots connected to IoT networks gain access to a wealth of data and the ability to communicate and collaborate with other devices, leading to more intelligent and adaptable systems.&nbsp;The addition of AI further enhances the capabilities of these IoT-enabled robots, enabling them to learn, reason, and make informed decisions based on real-time data analysis. Together, IoT, robotics, and AI are reshaping industries and everyday life, offering new opportunities for innovation and automation.&nbsp;As engineers and designers embark on the development of IoT-enabled robotics and AI devices, Mouser Electronics serves as a reliable partner, providing the tools to compare and acquire the necessary components for these groundbreaking technologies. Through their comprehensive inventory and user-friendly website, Mouser supports the advancement of IoT-enabled robotics and AI, driving progress and shaping the future of intelligent and autonomous systems.<h2 dir="ltr">References:</h2><a href="https://www.frontiersin.org/articles/10.3389/frobt.2020.00104/full" target="_blank">https://www.frontiersin.org/articles/10.3389/frobt.2020.00104/full</a><a href="https://geekflare.com/digital-transformation-ai-robots-iot/" target="_blank">https://geekflare.com/digital-transformation-ai-robots-iot/</a><a href="https://www.analyticssteps.com/blogs/internet-robotic-things-robotics-iot" target="_blank">https://www.analyticssteps.com/blogs/internet-robotic-things-robotics-iot</a><a href="https://iot.eetimes.com/the-internet-of-robotic-things-how-iot-and-robotics-tech-are-evolving-together/" target="_blank">https://iot.eetimes.com/the-internet-of-robotic-things-how-iot-and-robotics-tech-are-evolving-together/</a><a href="https://ieeexplore.ieee.org/abstract/document/8976180/" target="_blank">https://ieeexplore.ieee.org/abstract/document/8976180/</a>

The integration of AI, IoT, and robotics allows robots to access and exchange data with other devices, enabling them to make informed decisions.


The Role of IoT in Robotics and AI: Understanding Connectivity

In today&rsquo;s evolving technology landscape, AI is not merely &ldquo;artificial&rdquo;; it is artfully intelligent. But unlike art, AI&rsquo;s practical applicability requires high levels of efficiency, a challenge the industry still grapples with.Particularly with deep neural networks, we seem to encounter a challenge. While these networks dazzle us with their ability to recognize faces, translate languages, and make complex decisions, they also come with a voracious appetite for resources. High memory, compute power, and energy consumption have become almost defining characteristics of contemporary AI models, holding AI back from true ubiquity and rendering the need for operational efficiency ever more pressing.In this article, we take a deeper look into Qualcomm AI Research&rsquo;s model efficiency research aimed at enhancing power efficiency and boosting AI performance. From the intricacies of Neural Architecture Search (NAS) to the groundbreaking quantization methods, we will discover how Qualcomm AI Research is leading the charge to unlock the full potential of AI.<h2 dir="ltr">Qualcomm AI Research&#39;s Focus on Model Efficiency</h2><a href="https://www.qualcomm.com/research/artificial-intelligence/ai-research" target="_blank">Qualcomm AI Research</a> is at the cutting edge of AI, committed to tackling the intricate challenge of AI model efficiency through practical strategies that aim to enhance both power efficiency and performance. With its aim to make AI&rsquo;s core capabilities ubiquitous across devices, Qualcomm AI Research has adopted a holistic approach to AI model efficiency, delving into the practical aspects of full-stack AI optimization.&nbsp;Addressing AI&#39;s resource-intensive nature requires an approach that encompasses machine learning hardware, software, and algorithms. As a result, their research efforts aim to ensure that AI systems operate efficiently without compromising functionality, making AI more adaptable and efficient across a diverse range of devices, particularly mobile devices. As Jilei Hou, VP Engineering at Qualcomm AI Research, states, &ldquo;Our holistic systems-approach to full-stack AI is accelerating the pipeline from research to commercialization.&rdquo;But what exactly is this holistic approach to AI model efficiency? How is it implemented, and what angles does Qualcomm AI Research tackle the model efficiency challenge from?<h2 dir="ltr">A Holistic Approach to AI Model Efficiency</h2>Qualcomm AI Research&#39;s mission to unlock the full potential of AI centers around a comprehensive strategy that attacks the model efficiency challenge from multiple angles. This multifaceted approach is driven by the recognition that optimizing AI models requires a combination of techniques.Basically, there is no one-size-fits-all solution to AI model efficiency. &nbsp;Just like art, shrinking AI models down and running them efficiently on hardware can and should be approached in multiple ways, including quantization, compression, Neural Architecture Search (NAS), and compilation.Qualcomm AI Research is a strong believer in <a href="https://www.qualcomm.com/news/onq/2019/03/heres-why-quantization-matters-ai" target="_blank">quantization</a> as evidenced by their leading research and products in the market. Quantization enables the AI model to run efficiently on dedicated hardware, thus enhancing its performance while minimizing its consumption of power and memory bandwidth. With focus on quantization-aware training and post-training quantization, Qualcomm AI Research&rsquo;s results demonstrate how effective integer inference is in improving the tradeoff between accuracy and on-device latency.With compression, Qualcomm AI Research aims at reducing the size of AI models without compromising their functionality by removing parameters. In other words, with no sacrifice to model accuracy, Qualcomm AI Research&rsquo;s compression technique systematically removes activation nodes and connections between nodes, rendering the AI model smaller and more efficient. Quantization and compression of neural networks are supported by Qualcomm Innovation Center&rsquo;s <a href="https://github.com/quic/aimet" target="_blank">AI Model Efficiency Toolkit (AIMET)</a> library.Another approach is Neural Architecture Search (NAS). Their research in NAS focuses on automating the design of efficient neural networks that are much smaller than a large state-of-the-art model. Qualcomm AI Research seeks to develop search algorithms and methodologies that automatically create optimal network architectures tailored to specific tasks. This streamlines the process of designing AI models, enhancing their efficiency and performance.Furthermore, advanced compilation techniques optimize the execution of AI models on various hardware platforms. By tailoring AI models to the underlying hardware architecture, they ensure that the models run efficiently, achieving the desired performance while conserving resources.In the following subsections, we look deeper into quantization and NAS, highlighting how Qualcomm AI Research is enabling these approaches to unlock AI&rsquo;s potential even further.<h3 dir="ltr">A Closer Look at Quantization</h3>Quantization reduces the number of bits necessary for representing information in the weights and activations of AI models, boosting power efficiency and performance while reducing memory bandwidth and storage. It also helps enable simultaneous use of multiple AI models, a use case found in demanding application areas like mobile, automotive, and XR.Qualcomm AI Research has been at the forefront of research in quantization techniques that are revolutionizing power-efficient AI. Their research covers both quantization-aware training and post-training quantization for efficient integer AI inference. Their products support quantization down to 4-bit integers, showcasing their commitment to making AI more power-efficient.&nbsp;In fact, some of their recent<a href="https://arxiv.org/abs/2203.11086" target="_blank">&nbsp;findings</a> revealed issues that affect model accuracy, one of which is oscillating weights during model training. With state-of-the-art (SOTA) methods called oscillation dampening and iterative freezing, accuracy is improved even for under 4-bit quantization. Qualcomm AI Research has also explored the role of <a href="https://www.qualcomm.com/news/onq/2023/04/floating-point-arithmetic-for-ai-inference-hit-or-miss" target="_blank">floating-point arithmetic</a> in AI inference, shedding light on the balance between precision and efficiency. This research considers the trade-offs involved in using floating-point or integer representations in AI models, concluding that integer inference is the best approach. Another recent study on post-training quantization (PTQ) and its impact on softmax activation highlights a technique proposed by Qualcomm AI Research, called <a href="https://arxiv.org/abs/2309.01729" target="_blank">offline bias correction</a>. With this method, one can enhance the quantizability of softmax with no additional compute during deployment, thus significantly improving the accuracy for 8-bit quantized softmax.Quantization is a highly-effective approach to AI model efficiency. Qualcomm AI Research&#39;s leading quantization research combined with Qualcomm&rsquo;s leading processor solutions with AI acceleration supporting 8-bit or 4-bit integers has resulted in industry leading performance per watt, showing the importance of tailoring AI models to the specific constraints of edge devices.<h3 dir="ltr">Neural Architecture Search (NAS)</h3>With Neural Architecture Search (NAS), Qualcomm AI Research focuses on automating the design and training of efficient neural networks, aligning them more closely with dedicated tasks and optimizing their overall performance. NAS consists of four components:<ul><li dir="ltr">A search space (what types of networks can be searched over)</li><li dir="ltr">An accuracy predictor (how accurate a network should be)</li><li dir="ltr">A latency predictor (how fast a network will run)</li><li dir="ltr">A search algorithm (what the best architecture is for a particular task)</li></ul>However, NAS still presents some intricate challenges, particularly in defining an expansive search space, where countless network architectures can be explored. This space represents the potential configurations that an AI model could take. The challenge lies in navigating this vast terrain to discover the most efficient and effective neural network for a given task, as well as addressing the high compute cost, unreliable estimates for hardware performances, and inefficient scaling of existing methods.That&rsquo;s why Qualcomm AI Research has addressed these challenges with their NAS research called <a href="https://arxiv.org/pdf/2012.08859.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">DONNA (short for Distilling Optimal Neural Network Architectures)</a>.<h4 dir="ltr">DONNA: Distilling Optimal Neural Network Architectures</h4>An efficient NAS with hardware-in-the-loop optimization, DONNA is a scalable method capable of finding optimal network architectures with high accuracy and minimal latency for virtually any hardware platform at low cost. This revolutionary method tackles the model deployment challenges in real scenarios effectively thanks to its:<ul><li dir="ltr">Diverse search space</li><li dir="ltr">Low compute cost</li><li dir="ltr">Scalability</li><li dir="ltr">Reliable direct hardware measurement</li></ul>Its diverse search space includes the usual variable kernel-size, expansion-rate, depth, channel number, and cell-type, but can also search over activations and attention, a vital parameter to finding optimal architectures. Its hardware predictor is hardware agnostic with a low startup cost, enabling NAS to scale to multiple hardware devices at low costs. Furthermore, DONNA is able to capture the run-time version and hardware architecture, enabling users to find real and accurate latency values rather than simulated ones.DONNA also has pre-trained blocks that enable fast fine-tuning of neural networks to fast track reaching full accuracy and is a single scalable solution, applying, for instance, directly to downstream tasks and non-CNN neural architectures with no need for conceptual code changes.<h2 dir="ltr">Open-Source AI Model Efficiency Projects</h2>Qualcomm AI Research believes that open-source projects are a great way to share knowledge for scaling model-efficient AI to the masses. That&rsquo;s why Qualcomm Innovation Center has provided two open-source GitHub projects based on Qualcomm AI Research&rsquo;s advanced research:<ul><li dir="ltr"><a href="https://www.qualcomm.com/news/onq/2020/05/open-sourcing-ai-model-efficiency-toolkit" target="_blank">AI Model Efficiency Toolkit (AIMET)</a></li><li dir="ltr"><a href="https://www.qualcomm.com/news/onq/2021/01/aimet-model-zoo-highly-accurate-quantized-ai-models-are-now-available" target="_blank">AIMET Model Zoo</a></li></ul>AIMET is a toolkit that offers advanced quantization and compression techniques, while AIMET Model Zoo offers pre-trained 8-bit quantized models with high accuracy. With these open-source projects, Qualcomm aims to boost the transition to fixed-point quantized models, which would radically enhance AI applications, particularly those requiring high performance, low latency, and low power consumption, such as mobile platforms.<h2 dir="ltr">Advancing the AI Frontier</h2>As Qualcomm&rsquo;s OnQ blog series states, &ldquo;<a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">The future of AI is hybrid</a>&rdquo;, AI is evolving in that direction, with AI processing being distributed between edge devices and the cloud. On-device AI provides benefits in cost, power consumption, performance, personalization, privacy, and security. But for all this to happen, model efficiency is key, and that&rsquo;s where Qualcomm is leading the pack as the on-device AI leader.Qualcomm AI Research envisions a future where AI processing is highly efficient for all devices across all applications all over the world. With generative AI pushing the computing requirements even higher, Qualcomm&rsquo;s focus on efficiency and performance alongside their open ecosystem is enabling generative AI and other upcoming revolutionary technologies.Whether it&#39;s the quantization approach, NAS (and DONNA), compression, or compilation, Qualcomm AI Research will continue to find and develop ways of making AI processing ever more efficient so you and I can enjoy the perks of amazing AI technologies on almost every device you could think of.Join Qualcomm&rsquo;s open-source projects (<a href="https://www.qualcomm.com/news/onq/2020/05/open-sourcing-ai-model-efficiency-toolkit" target="_blank">AIMET</a> and <a href="https://www.qualcomm.com/news/onq/2021/01/aimet-model-zoo-highly-accurate-quantized-ai-models-are-now-available" target="_blank">AIMET Model Zoo</a>) and be part of the transformation towards efficient AI processing. Interested in learning more about Qualcomm AI Research&rsquo;s work, check out the following links:<ul><li dir="ltr"><a href="https://assets.qualcomm.com/MCC-ALL-Q322-0915-WB-ThefutureofmodelefficiencyforedgeAI_01RegistrationPage.html" target="_blank">The Future of Model Efficiency for Edge AI (on-demand webinar)</a></li><li dir="ltr"><a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">The future of edge AI is hybrid (whitepaper)</a></li><li dir="ltr"><a href="https://www.qualcomm.com/news/onq/2023/08/5-benefits-of-on-device-generative-ai" target="_blank">5 benefits of on-device generative AI</a></li></ul><a href="https://assets.qualcomm.com/mobile-computing-newsletter-sign-up.html" target="_blank">Sign up for our What&#39;s Next in AI and Computing newsletter</a>&nbsp;

Live translation devices will benefit from advances in model efficiency. Image credit: TimeKettle

Model efficiency research is enhancing power efficiency and boosting AI performance.

Unlocking the Full Potential of AI: A Deep Dive into Qualcomm's Model Efficiency Research

This is the second article in a short series. Read the<a href="https://www.wevolver.com/article/balancing-the-cloud-and-the-edge-a-close-look-at-enabling-hybrid-ai" target="_blank" rel="noopener noreferrer"> first piec</a>e here.<h2 dir="ltr" id="isPasted">Introduction</h2>Generative AI is a rapidly advancing field of artificial intelligence (AI) focused on creating novel data assets like images, text, or music. Such AI models are trained using extensive datasets, enabling them to create new data resembling what they were trained on. The growth in generative AI&#39;s development owes itself to several key factors: availability of large datasets, improved computing hardware capabilities, and the evolution of novel machine learning (ML) algorithms and architectures.Large language models&ndash;based chatbots like ChatGPT, which can generate coherent and contextually relevant text and even images, have gained the interest and creativity of researchers, businesses, and the public. Generative models have demonstrated remarkable capabilities across a range of applications, from natural language understanding and text generation to creative content creation and even virtual assistants. However, as these models grow in complexity and sophistication, they bring to the forefront an increasingly pressing issue in the form of scalability challenges that apply to both training and inference.&nbsp;These challenges encompass the need for significant computational resources, the growing size of models, and increasing costs. Latency and real-time responses primarily relate to the inference phase, while ethical considerations, interoperability, privacy, energy consumption, and cost management are critical factors throughout the entire lifecycle of AI systems, including training and inference.&nbsp;One promising solution arises in response to these challenges - on-device AI. Deploying AI models directly on user devices (like smartphones and tablets) offers multiple advantages. It reduces reliance on cloud resources, cutting computational and energy costs. It improves privacy by processing data locally, reducing the need to send sensitive information to the cloud. It also lowers latency, supports offline functionality, and aligns with sustainability goals by reducing energy consumption.<h2 dir="ltr">What is Hybrid AI?</h2>To properly define Hybrid AI, we need to introduce the terms on-device AI and hybrid AI. On-Device AI, also known as edge AI or local AI, refers to the deployment of AI algorithms and models directly on user devices (smartphones, tablets, IoT devices, autonomous vehicles, etc.), without heavy reliance on cloud-based infrastructure. It enables real-time processing and decision-making on the device itself, reducing the need for constant data transmission to remote servers. Hybrid AI represents a synergy between local devices and cloud infrastructure, strategically dividing AI computations to optimize both user experiences and resource efficiency.In certain situations, AI processing predominantly occurs on the device, with the option to delegate tasks to the cloud as needed. In cloud-centric setups, devices will opportunistically offload AI workloads from the cloud, where possible and subject to their capabilities. Employing a hybrid AI architecture, or relying solely on on-device AI, yields numerous advantages including cost savings, improved energy efficiency, enhanced performance, heightened privacy and security, and personalized experiences on a global scale.Within a hybrid AI framework, on-device AI processing delivers consistent performance that can rival or surpass cloud-based alternatives, particularly in scenarios characterized by congested cloud servers and network connectivity issues. Instances of high demand for generative AI queries in the cloud can result in lengthy queues and elevated latency, and in extreme cases, lead to service disruptions. These challenges can be mitigated by redistributing computational workloads to edge devices.Furthermore, the presence of on-device processing capabilities in a hybrid AI architecture enables generative AI applications to function seamlessly in locations with limited or no connectivity. In a device-centric hybrid AI setup, the device serves as the central point of operation, with cloud resources primarily utilized for tasks that exceed the device&#39;s processing capacity. Given that many generative AI models can efficiently run on the device, edge devices can handle the bulk of the processing load. As the development of on-device AI continues, it holds significant potential to revolutionize AI service deployment and user experiences while maintaining a balance between performance and ethical considerations.<h2 dir="ltr">Unique Benefits of On-Device AI</h2><a href="https://www.qualcomm.com/news/onq/2023/08/5-benefits-of-on-device-generative-ai" target="_blank">On-device AI offers several unique benefits</a>, including:<h3>Cost-efficiency&nbsp;</h3>By processing data locally, it reduces the need for extensive cloud infrastructure, which can be expensive to maintain and scale. On-device processing eliminates the necessity of sending data to the cloud, a process that can be both expensive and time-consuming. &nbsp;For instance, on-device AI for image generation, can be employed to generate high-quality images in real-time without relying on cloud-based servers. A mobile photography application equipped with generative AI can apply artistic filters, improve image quality, or create artistic effects directly on the user&#39;s device.&nbsp;This ensures that users can quickly and seamlessly edit and generate images without uploading them to a remote server, preserving both privacy and reducing latency. Additionally, it allows for offline image generation, enabling users to create and edit images even when they&#39;re in locations with poor or no internet connectivity. This level of on-device image generation can significantly improve the user experience and provide powerful tools for content creation without compromising data privacy or speed.&nbsp;<h3>Energy Savings&nbsp;</h3>On-device AI mitigates the issue of energy consumption by performing computations locally. This is particularly crucial for devices with constrained battery life, like smartphones and wearables. Comparing the power efficiency of running generative AI models on edge devices versus to the cloud, edge devices offer superior performance per watt and reduced energy consumption, potentially resulting in environmental and cost benefits.&nbsp;<h3>Performance and Low Latency</h3>By processing tasks locally, it minimizes communication and computing delays associated with cloud-based AI. This is particularly important in real-time applications such as autonomous vehicles, medical devices, and interactive content, where even milliseconds of delay can have a significant impact on functionality and user experience.<h3>Enhanced Privacy and Security</h3>This approach reduces the risk of data breaches and unauthorized access during data transfer and cloud storage. Users can have greater confidence that their personal information remains on their devices, reinforcing privacy and security standards, especially for applications handling sensitive data, like financial transactions and medical information.<h3>Personalization and User Experience</h3>These models can adapt to the user&#39;s context and needs, improving user experiences by recommending relevant products or services based on their preferences and current circumstances. For example, a shopping application could use on-device AI to recommend products similar to past purchases or consider the user&#39;s location and weather conditions. Real-time personalization boosts relevance, engagement, and customer satisfaction.<h2 dir="ltr">Case Studies: On-Device AI in Action</h2>The following case studies illustrate the transformative potential of on-device AI across various domains, showcasing its benefits.<h3 dir="ltr" id="isPasted">Automotive</h3>In the automotive industry, AI-driven cockpits are revolutionizing the driving experience. These systems, like those in smartphones and PCs, are starting to employ in-vehicle digital assistants powered by generative AI to create highly personalized and hands-free interactions for drivers and passengers.&nbsp;These digital assistants leverage personal and vehicle sensor data, including cameras and microphones, to improve the driving journey. Third-party services can also integrate seamlessly, improving navigation, offering proactive assistance, and enabling in-car commerce, such as ordering food. Furthermore, the cockpit experience is improved through individual recognition, customization of media content, and the rise of in-vehicle augmented reality. Vehicle maintenance becomes proactive as AI digital assistants analyze data to predict maintenance needs, to seamlessly communicate with users using natural language, and to provide precise repair guidance. This proactive approach not only enhances reliability but also effectively reduces costs.&nbsp;Qualcomm, which leads the field in cockpit and in-vehicle infotainment solutions with numerous global automakers choosing Snapdragon&reg; Cockpit Platforms. These platforms power digital cockpits, with many already in production or design stages. Partners include Honda, Mercedes, Renault, Volvo, and others. The Snapdragon Cockpit Platform&#39;s latest iteration focuses on superior in-vehicle user experiences, safety, comfort, and reliability, setting new standards for connected car digital cockpit solutions. Qualcomm&#39;s Snapdragon Ride&trade; Platform complements this, offering scalable automated driving SoC platforms for ADAS, featuring vision perception, parking, and path planning.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUyNzU0OTg0LXF1YWxjb21tIGNhYmluLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">&nbsp;Cockpit platforms are designed to transform in-vehicle experiences, supporting higher levels of compute and computing intelligence needed for advanced capabilities featured in next generation vehicles. Image credit: Qualcomm.<h3 dir="ltr" id="isPasted">Extended Reality</h3>For Extended Reality (XR), generative AI has the potential to democratize 3D content creation. In the near future, AI rendering tools will enable content creators to generate 3D objects and entire virtual worlds using various inputs like text, speech, images, or video. Furthermore, text-to-text models will facilitate lifelike conversations for avatars, transforming the way we interact with XR content. Despite the exciting possibilities, widespread adoption timelines remain uncertain, though significant progress is expected soon. Immersive XR experiences will benefit from text-to-image models for realistic textures, likely available on smartphones and XR devices within a year.&nbsp;These capabilities will involve distributed processing, with headsets handling perception and rendering, while generative AI models run on paired smartphones or in the cloud. In a few years, text-to-3D and image-to-3D models will likely produce high-quality 3D objects, evolving to generate entire scenes and rooms. Eventually, XR may enable users to immerse themselves in virtual worlds constructed from their imagination. Virtual avatars will undergo a similar evolution, utilizing text-to-text models for natural conversations and text-to-image models for outfits. Over time, avatars may become increasingly photorealistic, fully animated, and mass-producible, driven by image-to-3D and encoder/decoder models. This progression promises to revolutionize how we create and experience immersive XR content, with continual advancements anticipated in the years ahead.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUzMTYyMjIyLUNhbm5lcy1YUi1WaXJ0dWFsLUJyaW5ncy1JbW1lcnNpdmUtQXJ0LXRvLUxpZmUtV2l0aC1WaXJ0dWFsLVJlYWxpdHkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Image credit: Cannes<h2 dir="ltr">Challenges and Solutions</h2>The technical challenges specific to hybrid AI are being actively addressed through ongoing research and innovation:<h3 dir="ltr">Model size</h3>Model size is a notable challenge, particularly on resource-constrained devices. As large models are challenging to store and run efficiently, a potential solution involves employing compression and quantization techniques to reduce model size while maintaining accuracy. <a href="https://stackoverflow.blog/2023/08/23/fitting-ai-models-in-your-pocket-with-quantization/" target="_blank">Qualcomm</a> is actively researching ways to enhance <a href="https://www.qualcomm.com/news/onq/2019/03/heres-why-quantization-matters-ai" target="_blank">quantization</a> techniques, specifically aiming to quantize 32-bit floating point parameters to 8-bit or 4-bit integers in neural networks while maintaining accuracy. The research encompasses various model quantization methods, such as Bayesian learning, adaptive rounding, and transformer quantization, as part of <a href="https://www.qualcomm.com/research/artificial-intelligence/ai-research" target="_blank">Qualcomm AI Research</a>&#39;s ongoing efforts.&nbsp;For instance, the Qualcomm AI Research team employed the <a href="https://www.forbes.com/sites/davealtavilla/2023/05/15/powering-ai-on-mobile-devices-requires-new-math-and-qualcomm-is-pioneering-it/?sh=54cb40c91a16" target="_blank">AIMET</a> tool (AI Model Efficiency Toolkit) to quantize the 32-bit floating point Stable Diffusion AI model into an 8-bit integer format while preserving almost the same accuracy level. By optimizing and scaling the model for INT8 precision operations instead of FP32, Qualcomm successfully enabled Stable Diffusion to run efficiently on handheld devices without internet connectivity, significantly reducing the model&#39;s size while maintaining accuracy and enhancing performance on low-power Snapdragon AI accelerators.<h3>Computational power</h3>Large AI models may place heavy computational demands, particularly on devices with restricted processing capabilities. A potential solution involves leveraging specialized hardware like AI accelerators to enhance AI model performance on such devices. In 2022, Snapdragon 8 Gen 2 provided groundbreaking AI, integrated across the entire system, powered by their fastest, most advanced Qualcomm AI Engine to date. Users can experience faster natural language processing with multi-language translation or have fun with AI cinematic video capture.&nbsp;The latest Hexagon Processor introduced a dedicated power delivery system, which adapts power according to the workload. Special hardware improves group convolution, activation function acceleration, and the performance of the Hexagon Tensor Accelerator. Support for micro-tile inferencing and INT4 hardware acceleration provide even higher performance while reducing power and memory traffic. The transformer acceleration dramatically speeds up inference for multi-head attention that is used throughout generative AI, resulting in AI performance up to 4.35X on certain use cases with MobileBERT.&nbsp;These ongoing research efforts are addressing the technical challenges of hybrid AI, with expectations to overcome them successfully as the technology advances and becomes more widely adopted and utilized.<h2 dir="ltr">Conclusion</h2>On-device AI plays a critical role in addressing the future scalability challenges of generative AI models. It has the potential to revolutionize numerous industries and significantly enhance user experiences. By enabling local processing and real-time personalization, on-device AI reduces the dependence on cloud resources, improves privacy and security as well as the efficiency of AI applications. This approach not only supports the responsible development of AI technologies but also opens new possibilities for scalable and personalized AI solutions across diverse sectors, from healthcare to entertainment, promising a brighter future for both businesses and users alike.To stay up to date on the latest from Qualcomm, <a href="https://assets.qualcomm.com/mobile-computing-newsletter-sign-up.html" target="_blank">join the &ldquo;What&rsquo;s Next in AI and Computing&rdquo; newsletter</a> and read their hybrid AI whitepapers (<a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 1</a>, <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-2-Qualcomm-is-uniquely-positioned-to-scale-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 2</a>)

How on-device and cloud AI will foster the advancement and scalability of next-generation generative AI experiences

The Scalability Challenge: Focusing on Hybrid AI in the Future of Generative AI Models

Many companies in the manufacturing industry around the world are striving to improve business value and raise operational efficiency through digital transformation (DX). On the other hand, there is now a strong demand for decarbonization efforts, which curb greenhouse gases (GHGs) emitted in production activities to achieve carbon neutrality. The response to these two megatrends of DX and decarbonization has become an important issue in encouraging each company in the manufacturing industry sustainable.The purpose of decarbonization for the manufacturing industry since 2021 onward has changed from social contribution to achieving profits by observing conditions in transactions with customers and enhancing cost competitiveness. Some leading electronic equipment manufacturers have already stated that they will not procure parts or materials from suppliers that are not striving to reduce GHG emissions. An increasing number of investors are also making proactive decarbonization efforts a condition for investment. Moreover, at the 26th Conference of the Parties to the United Nations Framework Convention on Climate Change (COP26) held in November 2021, the so-called 1.5&deg;C target*1 was raised to a stricter mandatory target. It is increasingly likely that policies with strong binding force will be implemented in the future. Those policies include strengthening environment-related regulations and introducing carbon pricing*2 such as carbon taxes. As a result, decarbonization efforts have become directly linked to companies&rsquo; earnings in the manufacturing industry. That means they are starting to have the same meaning as productivity improvements.*1: This is the target to keep the rise in the global average temperature within 1.5&deg;C compared to before the Industrial Revolution.&nbsp;*2: This is the general term for the mechanism to pass on a company&#39;s CO2 emissions to costs. It includes a tax called a carbon tax imposed on the use of fossil fuels and emissions trading to buy and sell CO2 emission allowances to clear environmental regulations. According to the World Bank, carbon pricing has already been introduced in 46 countries and 35 regions as of April 10, 2021.The two megatrends surrounding the manufacturing industry of DX and decarbonization appear to be separate trends at first glance. However, in reality, DX leads to more efficient production activities, energy saving, and decarbonization. Therefore, they have an inseparable relationship. We introduce here how to use DX initiatives as a means to promote decarbonization. In particular, we look at the latest trends in the utilization of information processing technologies such as IoT, AI, and digital twins. We then explain decarbonization in smart factories.<h2>Two Approaches to Promoting Decarbonization in Factories</h2>Many machine tools, industrial robots, motor-driven devices and equipment like compressors, electric heaters, control devices, and other devices and equipment operated by electric power are used in factories. The decarbonization of these devices and equipment operated by electric power has become a major issue in the manufacturing industry.The approaches to decarbonization of devices and equipment that use electric power are broadly divided into two (Fig. 1). One is a method of switching to using sources of energy that do not emit greenhouse gases. Specifically, this includes the utilization of solar power, wind power, and other forms of renewable energy. The other is a method to completely cut out unnecessary power consumption. This is so-called power-saving. Promoting DX is an important condition for pushing ahead with these two approaches.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUwNzA5Mzg4LXNtYXJ0ZmFjdG9yeV8wOS0xZW4ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">Fig. 1: Two approaches to reducing greenhouse gas emissions / Source: Agency of Natural Resources and Energy, Ministry of Economy, Trade and IndustryIn general, the amount of electric power obtained from renewable energy is unstable. It depends on the sun and the wind, for example. However, electric power systems must satisfy the principle of electric power supply called the &quot;same amount at the same time&quot; in which the amount of electric power supplied and consumed is constantly kept the same. The reason for this is that if there is&nbsp;a significant difference in the amount of electric power consumed and supplied, an abnormal load may be applied to power generators, power distribution equipment, electrical equipment, and other devices, causing them to fail, or the electric power quality may become unstable. That may cause malfunctions or power outages. In other words, it is necessary to cover only the electric power that corresponds to the amount consumed in the factory with electric power generated at that time, electric power stored in storage batteries, or other sources of electric power.<h2>Aiming for Compliance with RE100 by Utilizing IoT</h2>In recent years, there has been an increasing number of companies in the manufacturing industry joining the RE100 international environmental initiative. This is an initiative aiming to cover the electric power used in business activities with 100% renewable energy. It is important to introduce an electric power system capable of appropriately managing and controlling the electric power supply and demand balance without lowering productivity or quality and a mechanism to cover the shortfall from countermeasures to realize the ideals of RE100 in factories (Fig. 2).<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUwNzI1NTQ1LXNtYXJ0ZmFjdG9yeV8wOS0yZW4ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">Fig. 2: Electric power system and mechanism for the realization of RE100-compliant factoriesTo ensure a factory is compliant with RE100, it is first necessary to build a&nbsp;renewable energy utilization infrastructure by introducing power generation and storage equipment for solar power and other forms of renewable energy. It is also important to have a mechanism to store surplus electric power in storage batteries and to cover the shortfall in electric power by visualizing the amount of electric power generated, which changes moment by moment due to changes in the weather, and the amount of electric power consumed in the factory, which increases or decreases depending on the production activity situation.There are more and more examples of companies introducing an electric power management system called the Factory Energy Management System (FEMS) in advanced factories. This is a system that utilizes IoT to monitor the amount of electric power consumed. It then adjusts peak electric power, controls the operation of air conditioners, lighting equipment, and production lines according to the situation, and performs other similar operations. The system visualizes the electric power consumption situation across the entire factory. It then curbs the nonessential and nonurgent operation of equipment and minimizes unnecessary electric power consumption without a drop in productivity. The range of equipment managed by the FEMS has now expanded to include power generation and storage equipment. It has evolved to being capable of maintaining the supply and demand balance in RE100-compliant factories.<h2>Highly Accurate Prediction and Management of the Electric Power Supply and Demand Balance with AI and Digital Twins</h2>Attempts are also underway to further raise the accuracy of management and control with the FEMS by utilizing the latest information processing technologies such as AI and digital twins*3.*3: A digital twin is a digital model that reproduces the same conditions and properties as in the real world by inputting data that reflects the current situation collected by IoT in the design data of real things.For example, technologies are being developed that estimate the amount of electric power obtained from solar power generation. To achieve this, those technologies analyze weather forecast and past track record data using AI to accurately predict the amount of power generated, which can be obtained from the equipment owned by a company. If it is possible to accurately predict the amount of power generated, it becomes possible to more effectively utilize the electric power generated in a company by shifting the peak of electric power consumption through the adjustment of production plans.Moreover, digital twins, which reproduce the conditions and movements over an entire factory using a computer, are being built, and technologies that increase the effect of power-saving measures are also being developed. It is becoming possible to take measures that allow the maximum effect to be obtained without risk by verifying on a digital twin the power-saving effect and the impact on productivity and product quality from&nbsp;the changes in the control and operation conditions of devices and equipment and alterations to production plans.Decarbonization efforts in the manufacturing industry are inseparably related to DX initiatives in factories. Regarding DX-related efforts, it seems important to keep in mind not only productivity and quality improvements, but also decarbonization.<hr id="isPasted"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695650770461-1695650770461.png" class="fr-fic fr-dii">If you have any questions based on this article that you would like to discuss with an expert at Murata, don&#39;t hesitate to reach out. There&#39;s an entire team ready to support you with your question.<a href="https://www.murata.com/en-eu/contactform?&Detail=Wevolver%20request" target="_blank" rel="nofollow" class="button-main-action">GET IN TOUCH</a>

Many companies in the manufacturing industry around the world are striving to improve business value and raise operational efficiency through digital transformation (DX). On the other hand, there is now a strong demand for decarbonization efforts, which curb greenhouse gases (GHGs) emitted in production activities to achieve carbon neutrality.

The Inseparable Relationship Between Digital Transformation and Decarbonization and Technologies Essential for a Sustainable Manufacturing Industry

Qualcomm's Pioneering Approach to Hybrid AI: Merging Cloud Processing with On-Device Efficiency and Scale


Balancing the Cloud and the Edge: A Close look at Enabling Hybrid AI

BVLOS, swarm, and military-grade encryption will enable wider and more diverse applications of drones, while cutting-edge software platforms enable operators to improve fleet coordination with more data control


How drone management platforms are enhancing the next generation of drone technologies

<h2>Challenges of self-localization in mixed in- and outdoor environments</h2>Knowing its own location or position within an operating environment (a.k.a. self-localization) is one of the key capabilities any mobile robot must master to be able to get from A to B. Yet, depending on the environment the robot is operating in, self-localization can impose vastly different challenges, rendering the choice of sensor technology and software algorithms far from a trivial task.Indoors, 2D LiDAR technology is widely used, as an example, on AGVs and AMRs for material handling operations. Such a technology allows to measure the geometric structure of the near-by environment (e.g. walls, shelves) to localize in pre-built maps. However, it shows its limitations in environments which do not have unique geometric features (e.g. a long corridor) or are highly dynamic especially if shared with people.&nbsp;On autonomous vehicles for outdoor environments, instead, satellite navigation systems such as GPS or Galileo are often the first choice of sensor technology for self-localization, for good reasons: The GPS receivers are very cheap, and they yield an easily interpretable position in a known global reference coordinate system (e.g., WGS84).&nbsp;However, an accurate GPS position requires unobstructed skies. This is a limitation that is violated in several circumstances like urban surroundings, company campuses, &nbsp;logistic yards and airport tarmac. In fact, these outdoor places often require to operate close to or even inside buildings or tunnels. Due to these restrictions, mobile robots operating outdoors or in mixed indoor-outdoor settings cannot rely solely on GPS for self-localization.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1MTMxNTk3MzM0LU1peGVkIGluZG9vciBhbmQgb3V0ZG9vciBwb3NpdGlvbmluZy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Mixed indoor and outdoor environments are challenging for GPS and LiDAR solutions aloneEven though 2D LiDAR sensors can be employed outdoors too, using them in the open only works in well-structured spaces of limited extent or requires additional infrastructure such as laser reflectors in order to work better., or to a better extent thanks to the installation of additional infrastructure such as laser reflectors. 3D LiDARs, on the other hand, remain a very expensive sensor gear, despite all efforts in cutting down the costs in recent years.<h3>The strength and challenges of visual localization</h3>As an alternative to GPS and LiDAR sensors, cameras are able to reliably sense the environment both indoors and outdoors. Vision thus has the capability to bridge this gap left uncovered by GPS or LIDAR sensors. It offers highly accurate and reliable self-localization in both indoor and outdoor environments with a very cost effective sensor gear.&nbsp;Visual self-localization does not come without its very own challenges. Cameras naturally provide good images in well illuminated surroundings, while during night time outdoors, or in dim indoor areas, visual self-localization becomes more difficult. However, the selection of camera sensor chips with a high dynamic range &nbsp;has shown to allow proper images to be taken even under challenging illumination conditions.<blockquote>Instead, LiDAR sensors can be employed. However, using 2D LiDARs outdoors only works in well-structured spaces of limited extent. As an alternative to GPS and LiDAR sensors, cameras are able to reliably sense the environment both indoors and outdoors.</blockquote>Further, and in contrast to GPS or LiDAR sensor data, images taken by cameras of the same environment can look vastly different under varying seasonal, weather, or lighting conditions. This change in appearance renders accurate localization difficult. Additionally, geometric characteristics are often unfavorable for visual localization outdoors compared to indoors, as there is often less stable, human-made structure visible, plus it is often located at a considerably farther distance. Therefore, smart algorithms to work with visual data are paramount.<h3>Tackling the challenges of visual localization with AI</h3>To master these challenges, Sevensense employs a smart AI algorithm based on lifelong visual mapping that enables building and maintaining always-up-to-date maps of the environment. These maps incorporate data from multiple appearances of the environment, such as before- and after remodeling of a room, during cloudy- and sunny weather, or bright and dark lighting conditions. This allows the maps to be used for visual localization under practically any appearance condition.&nbsp;At the core of this lifelong mapping process is a software algorithm capable of merging visual data from multiple traversals through the same environment, recorded under potentially very different appearance conditions, into a single, geometrically consistent map. This process is fully automated, resulting in having a multi-appearance map ready for reliable localization after traversing through the environment only a handful of times beforehand.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1MTMxNzMxMTk3LUxpZmVsb25nIHZpc3VhbCBtYXBwaW5nIGFsZ29yaXRobXMgdXNlZCBieSBTZXZlbnNlbnNlLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Smart lifelong visual mapping algorithms allow for always-up-to-date mapsFurther, continuously updating this map over time naturally captures structural change in the environment. This way events like remodeling of rooms, construction sites or temporary modifications in street or room layouts don&rsquo;t disrupt everyday operations. These changes are detected automatically and swiftly incorporated into the lifelong map.&nbsp;In particular, no expert knowledge or intervention is necessary at any point in the process. Just as generating the initial map of the environment can be done by anyone without any prior training or expert know-how, updating the maps at any later point is done automatically by Alphasense Position. This considerably lowers the costs of maintaining the system in a long-term horizon.&nbsp;<h3>Vision is the optimal solution for indoor- and outdoor self-localization</h3>By employing this lifelong mapping technique, vision overcomes its limitations and turns out to be the go-to technology for cost-effective, but still reliable and accurate self-localization both indoors and outdoors. In particular, it offers seamless transition from indoor to outdoors and vice-versa, as the same camera sensors and the same map can be used to localize in both types of environments. This is a fundamental advantage over any technology primarily relying on GPS for outdoor localization and navigation that can revolutionize the logistics, delivery, cleaning or supply chain processes.<blockquote>By employing this lifelong mapping technique, vision turns out to be the go-to technology for cost-effective, but still reliable and accurate self-localization both indoors and outdoors.</blockquote>Sevensense has already successfully demonstrated the unique benefits of visual localization in a <a href="https://www.sevensense.ai/blog/kyburz" target="_blank" rel="noopener noreferrer">proof-of-concept study in partnership with KYBURZ Switzerland AG</a>. By equipping a mail delivery robot with Alphasense Position, accurate and reliable self-localization of the vehicle became possible at any time indoors, as well as outdoors, where LiDAR and GPS solutions alone failed.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1695132745402-Reliable+positioning+indoors+and+outdors+cameras+overcome+LiDAR+and+GPS+limitations+Sevensense.jpeg" style="width: 300px;" class="fr-fic fr-dib">Delivery robot used to benchmark Alphasense Position, 3D LiDAR and GPS localization performance[the missing image will go here once the CMS issue is fixed]<blockquote>By equipping a mail delivery robot with Alphasense Position, accurate and reliable self-localization of the vehicle became possible at any time indoors, as well as outdoors, where LiDAR and GPS solutions failed.</blockquote>Along a similar vein, the indoor- and outdoor capabilities of Alphasense Position will open up countless other opportunities for bringing cost-effective autonomy to robotic platforms operating in challenging environments. Delivering goods across multiple buildings on a campus, or material handling in warehouses undergoing frequent layout change have never been simpler before.<h3>With vision towards a fully autonomous future</h3>Self-localization is essential for robots to safely navigate in changing environments and achieve full autonomy. By employing cameras for this task together with smart algorithms for building and maintaining maps, autonomy can be provided at a fraction of the costs - be it for the sensor gear itself, as well as operating costs over the robot&rsquo;s lifetime.&nbsp;At Sevensense we are constantly pushing cutting edge technologies forward to deliver affordable autonomy solutions for new application fields.You can find more informative material about Alphasense Position on its <a href="https://www.sevensense.ai/product/alphasense-position" target="_blank" rel="noopener noreferrer">product page</a>.

Test vehicle benchmarking cameras, LiDAR and GPS systems

Breaking Free from LiDAR and GPS Constraints: Harnessing Vision for Reliable Positioning, Indoors and Outdoors

Why reliable positioning indoor and outdoor requires cameras

<h1 dir="ltr" id="isPasted">Introduction</h1>Gesture recognition is not just a buzzword; it&#39;s an indispensable technology that&#39;s changing the way we interact with devices. From smartphones to smart homes, the ability to control devices through simple hand movements offers an unparalleled level of convenience and accessibility. However, implementing robust gesture recognition systems is fraught with challenges, from data acquisition and preprocessing to machine learning model training and deployment.In this article, we will look into the technical aspects of developing a gesture recognition system using Infineon&#39;s PSoC 6 microcontroller. We&#39;ll explore each stage of the development process, from sensor data acquisition to neural network model deployment. We&#39;ll also share a link to a GitHub project detailing a step-by-step guide on implementing gesture classification, empowering you to create your own gesture recognition application. By the end of this article, you&#39;ll have a comprehensive understanding of the complexities involved and how specialized machine learning platforms such as Edge Impulse can simplify the process.<h1 dir="ltr">The Need for Gesture Classification</h1>The demand for intuitive human-machine interfaces is growing exponentially across various sectors. Whether it&#39;s automotive infotainment systems, medical devices, or consumer electronics, the ability to interact through gestures adds a layer of convenience and user-friendliness. However, implementing these systems on embedded platforms like the PSoC 6 involves challenges such as computational limitations, real-time processing requirements, and power constraints.To address these challenges, developers often have to make trade-offs between system performance and complexity. For instance, simpler algorithms may require fewer computational resources but may not provide the desired level of accuracy. On the other hand, complex algorithms may offer high accuracy but may be too resource-intensive for real-time applications. This delicate balance makes the choice of hardware and software platforms crucial in the development process. With these considerations in mind, let&#39;s delve into the capabilities of the Infineon PSoC 6 and see how it stands out as a viable solution for these challenges.<h1 dir="ltr">Infineon PSoC 6: An Overview</h1>Infineon&#39;s PSoC 6 is a versatile microcontroller unit (MCU) designed with low-power IoT applications in mind. It features a dual-CPU architecture, combining an ARM Cortex-M4 and an ARM Cortex-M0+ processor, allowing for efficient task partitioning. The PSoC 6 also offers programmable analog and digital blocks, making it highly customizable for specific application needs. Its rich set of features makes it an ideal candidate for implementing complex tasks like gesture recognition.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1694956600295-image3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="infineon-psoc-6-prototyping-kit">Fig. 1: Infineon Technologies PSoC&trade; 6 Wi-Fi&reg; BT Prototyping Kit. Source:&nbsp;<a href="https://www.mouser.in/new/infineon/cypress-psoc-6-wifi-bt-proto-kit/" target="_blank" rel="noopener noreferrer">Mouser Electronics</a>The PSoC 6 is not just a powerful MCU; it&#39;s a complete ecosystem that provides developers with a range of tools and libraries to accelerate the development process. From ModusToolbox, a comprehensive software development environment, to a rich set of middleware components, the PSoC 6 ecosystem is designed to simplify complex tasks.<h1 dir="ltr">Sensor Data Acquisition</h1>For gesture recognition, sensor data is the raw material. PSoC 6 can interface with a variety of motion sensors, such as accelerometers and gyroscopes, using SPI and UART protocols. These sensors capture the 3D spatial movements that are then classified into specific gestures. However, data acquisition is not just about reading sensor values; it&#39;s about doing so reliably and efficiently. Specialized machine learning platforms like Edge Impulse can offer features like sensor fusion and automated data collection, ensuring that the raw data is as accurate as possible.Suggested Reading:&nbsp;<a href="https://www.wevolver.com/article/what-is-sensor-fusion-everything-you-need-to-know" target="_blank">Sensor Fusion: The Ultimate Guide to Combining Data for Enhanced Perception and Decision-Making</a>Data acquisition in real-world applications is often noisy and inconsistent. Factors like sensor drift, environmental conditions, and hardware limitations can introduce errors into the collected data. Therefore, it&#39;s essential to implement robust data acquisition strategies that can mitigate these issues. Techniques like sensor calibration, data validation, and outlier detection are often employed to improve the quality of the collected data.<h1 dir="ltr">Data Preprocessing and Filtering</h1>Once the raw sensor data is acquired, it needs to be preprocessed before feeding it into a machine learning model. This involves filtering out noise and normalizing the data. Techniques like Infinite Impulse Response (IIR) filtering and min-max normalization are commonly employed for this purpose. While these preprocessing steps are essential for the accuracy of the final model, they can be time-consuming to implement and optimize. Specialized machine learning platforms can automate much of this, allowing developers to focus on other critical aspects of the system.Data preprocessing is not a one-size-fits-all operation; it often requires fine-tuning based on the specific application requirements. For instance, the choice of filter parameters, the window size for normalization, and the sampling rate can all impact the performance of the gesture recognition system. Therefore, it&#39;s crucial to perform iterative testing and validation to fine-tune these parameters, a process that can be significantly streamlined using specialized machine learning platforms. Edge Impulse&nbsp;offers&nbsp;intuitive tools for data preprocessing, allowing developers to easily filter and normalize data, ensuring it&#39;s ready for model training.<h1 dir="ltr">Neural Network Models for Gesture Recognition</h1>Gesture recognition typically employs Convolutional Neural Networks (CNNs) due to their ability to process spatial hierarchies in the data. A standard CNN model for gesture recognition would include multiple layers, such as convolutional layers, Rectified Linear Unit (ReLU) activation functions, max pooling, and batch normalization. These layers work in tandem to extract features from the raw sensor data and classify them into specific gestures. However, training these models requires a deep understanding of machine learning algorithms, data science, and optimization techniques. Specialized machine learning platforms can simplify this process by offering pre-trained models and automated training pipelines.The complexity of the neural network model often depends on the number of gestures to be recognized and the required accuracy. More complex models with additional layers and nodes can offer higher accuracy but may be computationally intensive, making them unsuitable for real-time applications on resource-constrained devices like the PSoC 6. Therefore, model optimization techniques like pruning, quantization, and layer fusion are often employed to reduce the computational complexity without significantly compromising accuracy.<h1 dir="ltr">Implementing Gesture Recognition on PSoC 6</h1>Developing a gesture recognition system involves more than just machine learning models. The development environment needs to be set up, the hardware configured, and the software architecture designed. In the case of PSoC 6, the ModusToolbox software environment provides a range of tools for configuration and development. The code structure for implementing gesture recognition involves various tasks, such as data acquisition, preprocessing, and classification. FreeRTOS is often used for managing these tasks, offering the benefits of real-time operation and future scalability.The software architecture for implementing gesture recognition on PSoC 6 often involves multiple tasks running concurrently. For instance, one task may be responsible for data acquisition, another for data preprocessing, and yet another for gesture classification. Managing these tasks efficiently requires a robust multitasking environment, which is where real-time operating systems like FreeRTOS come into play. FreeRTOS allows for efficient task scheduling, inter-task communication, and resource management, making it easier to develop complex applications like gesture recognition.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2MTc1NTA3LWltYWdlMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 302px;" class="fr-fic fr-dib" alt="edge-AI-gesture-recognition">Fig. 2: Through edge AI techniques, gestures can be translated into data formats comprehensible by computers, enabling efficient real-time processing and decision-makingBeyond the software architecture, the deployment phase is also crucial. Once the model is trained and optimized, it needs to be converted into a format that can be loaded onto the PSoC 6 MCU. This often involves converting high-level model formats like H5 into C or C++ code that can be integrated into the ModusToolbox environment. This step can be cumbersome and error-prone if done manually. However, specialized machine learning platforms offer automated tools for model conversion and deployment, ensuring that the transition from development to production is seamless.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2MjQ4MjM5LXNwYWNlc19HRWdjQ2s0UGtTNVBhNnVCYWJsZF91cGxvYWRzX2dpdC1ibG9iLWQyYjE4YjVmYTMzM2YwYTJjNjMwY2MzYTBlZmM3ZGY3YTNiNDYzYTBfdHV0b3JpYWwtY29udGludW91cy1tb3Rpb24tY3JlYXRlLWltcHVsc2Uud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 304px;" class="fr-fic fr-dib" alt="edge-impulse-studio-gesture-recognition">Fig. 3: Designing an Edge AI application for motion recognition using Edge Impulse studio. Source:&nbsp;<a href="https://docs.edgeimpulse.com/docs/tutorials/end-to-end-tutorials/continuous-motion-recognition" target="_blank" rel="noopener noreferrer">Edge Impulse</a>The final piece of the puzzle is real-time performance monitoring and updates. Once deployed, the system&#39;s performance needs to be monitored to ensure it meets the desired accuracy and latency requirements. Any necessary updates or optimizations can be rolled out as firmware updates, a process that can be automated and streamlined using specialized machine learning platforms.As we&#39;ve seen, the journey from concept to deployment in gesture recognition involves multiple steps, each with its own set of challenges and complexities. While the PSoC 6 provides a robust hardware platform, the software complexities often require specialized tools and platforms for efficient development and deployment.Here is a project by Infineon explaining the step by step process to implementing gesture classification using Infineon PSoC 6: <a href="https://github.com/Infineon/mtb-example-ml-gesture-classification" target="_blank">GitHub - Infineon/mtb-example-ml-gesture-classification</a>. For in-depth exploration of more Edge AI projects, don&#39;t forget to check out: <a href="https://edgeimpulse.com/blog" target="_blank">Edge Impulse Blog</a>.<h1 dir="ltr" id="isPasted">Conclusion</h1>Gesture recognition is poised to become a standard feature in the next generation of human-machine interfaces. While platforms like Infineon&#39;s PSoC 6 provide the hardware capabilities for such advanced applications, the development process involves several complexities. These complexities can be significantly simplified by leveraging the power of specialized machine learning platforms like Edge Impulse, making it easier to develop, deploy, and scale robust gesture recognition systems.Further reading:&nbsp;<a href="https://www.wevolver.com/article/spotlight-on-innovations-in-edge-computing-and-machine-learning-predictive-maintenance" target="_blank">Spotlight on Innovations in Edge Computing and Machine Learning: Predictive Maintenance</a><h1 dir="ltr">About the sponsor: Edge Impulse</h1><a href="https://www.edgeimpulse.com/" target="_blank">Edge Impulse</a> is the leading development platform for embedded machine learning, used by over 1,000 enterprises across 200,000 ML projects worldwide. We are on a mission to enable the ultimate development experience for machine learning on embedded devices for sensors, audio, and computer vision, at scale.&nbsp;From getting started in under five minutes to MLOps in production, we enable highly optimized ML deployable to a wide range of hardware from MCUs to CPUs, to custom AI accelerators. With Edge Impulse, developers, engineers, and domain experts solve real problems using machine learning in embedded solutions, speeding up development time from years to weeks. We specialize in industrial and professional applications including predictive maintenance, anomaly detection, human health, wearables, and more.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2NDM1MDUxLWltYWdlMS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="edge-impulse-logo">

Gesture recognition, leveraging the advanced capabilities of embedded devices and streamlined through specialized platforms is creating new means of human-machine interactions, paving the way for more intuitive and user-friendly device interfaces.

Gesture Recognition and Classification Using Infineon PSoC 6 and Edge AI

On 16 May, the world came to High Tech Campus Eindhoven to celebrate women in tech, their male allies and to get a preview of what the future holds.&nbsp;They were not disappointed.The theme of the fourth annual Fe+male Tech conference was &ldquo;Dare to Grow,&rdquo; and that was the consistent message from speakers: be bold, embrace risk and find your career path in a tech world where women in leadership roles are still a rarity. Or, as ASML&rsquo;s Wendy Gehoel van Ansem put it, an event meant to send you away &ldquo;confident so that you kick ass.&rdquo; If you think women don&rsquo;t belong in tech and STEM jobs, we know 300 women who&rsquo;d passionately disagree.&nbsp;More than 300 women, as well as male allies, came from the Netherlands, other countries in Europe and the United States for the sellout event.Watch the recording of the plenary program here:<iframe width="640" height="360" src="https://www.youtube.com/embed/BmPFVFMz8Mo?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>They came to hear 30 speakers and panel members, including Lori Glover from the Massachusetts Institute of Technology&rsquo;s Computer Science Artificial Intelligence Laboratory, Kimberly Fuqua from Microsoft Netherlands, workshops, panels and off-the-charts networking. Also participating were executives from Eindhoven&rsquo;s signature tech corporations, including Moyra McManus and Marco Pieters from ASML, the Eindhoven-based semiconductor giant.<h2>Trailblazing Women Throughout History</h2>With her keynote &quot;She Who Dares, Wins: Women Leading Fearlessly in Tech,&rdquo; Glover set the tone early, revealing some of the unsung female tech heroes whose trailblazing achievements never got the recognition they should have. They included Hedy Lamarr, the 1930s movie star who invented the frequency-hopping signal that makes wi-fi, Bluetooth and GPS possible, and the women at the Jet Propulsion Lab who did the high-level mathematics and trajectory calculations that made space exploration possible. These are the women who were actually referred to as &ldquo;computers.&rdquo;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwMzc1MTA0LWlLWXBJUURBLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Lori Glover of MIT&nbsp;Glover advised the current generation of women in tech to take full advantage of new opportunities opening up at tech companies: &ldquo;When you go to meetings, take a seat at the table, not along the wall.&rdquo;Then, Glover opened a new chapter in Eindhoven&rsquo;s history by inviting attendees to connect directly with her and her CSAIL program at MIT, the legendary engineering school that&rsquo;s home to dozens of world-class scientists and Nobel Prize laureates, including Sir Tim Berners-Lee, inventor of the Internet. Glover also gave a detailed presentation about MIT and CSAIL at the AI Innovation Center as part of the conference.THAT was just the beginning of the day.In 1988, 35 years ago, women earned 33 percent of computer science degrees, but that fell to less than 20 percent by 2016. Women have only about 30 percent of top jobs in tech. More women are going into tech, but half drop out by age 35. Why are women dropping out? Glover asked. &ldquo;Why aren&rsquo;t women moving up?&rdquo;<h2>Tech to Career: Exploring Breakout Sessions</h2>Conference attendees chose to attend one of six tech and seven career breakout sessions led by inspiring women in their fields and included topics such as data science, personalized medicine, wearable devices and cancer research. A panel of deep-tech female founders talked about the challenges and rewards of their venture journeys. As usual, attendees found it hard to choose only one breakout session!Career breakout topics included imposter syndrome, exhaustion, personal presence, equality for bi-cultural women, confidence boosting and how to be an effective role model. Several presentations and workshops addressed the unique career challenges women in tech face, including the career penalty for taking time out of the workforce. Glover cited statistics and trends indicating so much more has to be done.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwNDQxMTE3LTFHOEE3OTE5LndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Diana M&auml;kel &nbsp;(Thermo Fisher) during her Tech Breakout about data science<h2 id="isPasted">Fix the system, not the girls</h2>Sahar Yadegari and Rian de Heur from VHTO led a workshop on how we can &ldquo;Fix the system, not the girls&rdquo; so more females can be inspired by science and technology from an early age. With women only making up 16 percent of the Dutch STEM workforce and the Netherlands ranking No. 1 in strongest gender stereotypes, their talk was hard-hitting and eye-opening.Many audience members shared deeply personal stories about their experiences fighting gender bias with their children. With advice ranging from women in tech being more visible to encouraging a growth mindset in children, every attendee walked away with valuable insights and actionable tasks.The day was not just about corporates, even though Eindhoven is home to multiple dominant semiconductor companies such as ASML and NXP. It was also about a new generation of female entrepreneurs at startups and scale-ups.<h2>Spectrometer-In-A-Chip</h2>Kaylee Hakkel, COO of MantiSpectra, said her company is taking the functionality of a spectrometer and shrinking it down to the size of a chip. When trained with AI and a robust database, this deep-tech solution uses light to detect various types of materials, and it will be used to solve societal challenges, problems from detecting what type of plastic is pulled from the ocean to quality control in food and beverage to sampling illegal drugs.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwNDkwNjI1LTFHOEE3OTM0LndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Kaylee Hakkel (MantiSpectra)&nbsp;At only 28 years old, Kaylee already has a wealth of experience from being a startup co-founder. Her talk drew an audience of women that filled the room and who asked probing questions about the science and applicability of MantiSpectra&rsquo;s solution.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwNTE5MTI2LTFHOEE3NDQ1LndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib"><h3 id="isPasted">Here are some highlights from the day, curated by conference attendees:</h3><ul><li>Lori Glover noted that leaders&nbsp;aren&rsquo;t necessarily bosses. Leaders are people with intelligent, curious minds and the vision to see what&rsquo;s possible. They have exceptional communication skills and the ability to inspire. They show responsibility and share credit with others. </li><li>In her keynote, Microsoft&rsquo;s Kimberly Fuqua told attendees that women in tech shouldn&rsquo;t underestimate the human touch. &ldquo;Don&rsquo;t call them soft skills anymore. Call them superpower skills,&rdquo; she said. </li><li>Fuqua advised continually reevaluating your relationships. &ldquo;Take note of the energy of the relationship. You are responsible for the energy you bring to the relationship and the energy you take away from it.&quot;</li></ul><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwNTM5MDc4LTFHOEE4MjI1LndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Kimberly Fuqua (Microsoft)&nbsp;<ul id="isPasted"><li>ASML executives and others noted that while men tend not to take &ldquo;no&rdquo; for an answer, reapplying over and over for jobs they were rejected for until they get it, women typically only apply once or twice. </li><li>The male allies panel featured Marco Pieters, ASML; Andy L&uuml;rling, LUMO Labs; Johan Feenstra, SMART Photonics and Ed Heerschap, City of Eindhoven. Eindhoven-based serial entrepreneur Johan Feenstra told the audience during the panel discussion,&nbsp;&ldquo;We need to pay the role and not the person.&rdquo; Feenstra said women and highly skilled internationals often don&rsquo;t negotiate and fall behind when it comes to salary. </li><li>Ed Heerschap, Living In Program Coordinator for the city of Eindhoven, noted that families are crucial to Eindhoven&rsquo;s tech talent shortage because many highly skilled internationals won&rsquo;t come unless they can bring spouses and families.</li></ul><h2>Conference Donates to Gender Diversity Cause</h2>The fourth edition of the Fe+male Tech Heroes conference marked the first time organizers charged for the conference. Attendees paid &euro;30 for a day full of amazing speakers, meaningful content, great food and networking while also contributing to a worthy cause. More than 80 percent of ticket sales went to VHTO, a Dutch foundation and Expertise Center that actively champions and empowers Gender Diversity in STEM, Engineering, and IT.And it wasn&rsquo;t just the attendees with key takeaways. Keynote speakers and breakout session speakers also saw great value in the conference.&nbsp;&ldquo;The best word I can use to describe about how I felt about the conference is GRATEFUL!! The atmosphere was so electric, the people and other presenters were so encouraging with a learning and curious mindset. I will make this event one of my annual &#39;not to miss&#39; going forward,&rdquo; said Kimberly Fuqua.Keynote speaker Dan Jing Wu, CEO and Co-Founder VivArt-X and co-founder of fashion company NOYA NOIR,&nbsp;said &quot;I&#39;m so impressed with the organization and the audience was so kind and responsive. It gave me so much energy!&rdquo;And that&rsquo;s a wrap! Watch this space and mark your calendars for other Fe+male Tech Heroes events. If you&rsquo;re not already a member, join the <a href="https://insights.hightechcampus.com/female-tech-heroes" rel="noopener" target="_blank">Fe+male Tech Heroes community</a> and stay updated about news and events!<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NjkwNTY2MzY1LTFHOEE4MzY4LTIud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Hilde de Vocht (Fe+male Tech Heroes), Rian van Heur (VHTO), Sahar Yadegari (VHTO), Ingelou Stol (Fe+male Tech Heroes)&nbsp;

On 16 May, the world came to High Tech Campus Eindhoven to celebrate women in tech, their male allies and to get a preview of what the future holds. 

Breaking Barriers and Forging Paths: Key Lessons from the Fe+male Tech Heroes Conference 2023

Conversational AI enables machines to respond in a human-like manner. These intelligent systems are designed to understand intent and context, remember user preferences, and engage in meaningful conversations. 


High-Performance MEMS Microphones for Conversational AI: Unlocking New Potential for Voice-Enabled Assistants

In this third part of the series, we will take a deeper look at the development of the Shadow Hand and Glove, the technology behind the glove, and the potential impact of this system on the field of robotics.


Dexterous Robotic Hands Part 3: Shadow Gloves - The Final Piece of the Robotics Puzzle

Edge Impulse-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Edge Impulse-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Edge Impulse-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Edge Impulse-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Edge Impulse-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Matroid

THINC Lab-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

THINC Lab-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

THINC Lab-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

THINC Lab-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

THINC Lab-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Plainsight-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Plainsight-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Plainsight-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Plainsight-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Plainsight-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Artificial intelligence (AI) is a wide-ranging tool that enables people to rethink how we integrate information, analyze data, and use the resulting insights to improve decision making


Artificial Intelligence: A Comprehensive Guide to its Engineering Principles and Applications

A.I.

Clustering - A Common Unsupervised Learning Technique

Understanding What is Unsupervised Learning, the Mechanisms, Types, and Applications of Various Algorithms and Challenges it presents in Machine Learning

What is Unsupervised Learning: A Comprehensive Guide

Exploring the key concepts related to Unsupervised vs Supervised Learning, understanding the fundamental principles, major algorithms and their real-world applications, and practical distinctions between supervised and unsupervised learning.

Unsupervised vs Supervised Learning: A Comprehensive Comparison

Gain a comprehensive understanding of Edge AI by exploring the fundamental concepts, hardware components, software tools, and real-world applications.

Edge AI: A Comprehensive Guide to its Engineering Principles and Applications

When we talk about automation, there is a tendency to picture cutting-edge technologies, like self-driving cars and sophisticated robots. But automation has existed in some form or another for centuries and many automated technologies are now quite commonplace. The primary history of automation dates back to the Industrial Revolution, though we can trace roots back even further. Water mills used to mill grain have existed since the time of Ancient Greece; windmills have existed since the 12th century.&nbsp;In the Industrial Revolution, however, automation really took hold with a slew of new manufacturing technologies that were invented to reduce the reliance on human labor. Devices like conveyor belts, mechanized looms, steam engines, and more led to the proliferation of factories that could produce products on a mass scale like never before. Electrification in the early 1900s and advances in signal processing throughout the early and mid 20th century led to further automation and revolutionized every aspect of society, from communication to manufacturing.&nbsp;Today, it is not quite as simple as talking about &ldquo;automation&rdquo;, because the term can mean a lot of things. Industrial automation can look like a conveyor belt moving products along an assembly line; it can look like a CNC machine milling a part from a metal blank; it can look like a robotic arm manipulating objects; and it can look like an entire lights-out factory working independent of any human intervention. Because of this versatility, it is helpful to categorize the different types of automation.&nbsp;Recommended reading: <a href="https://www.wevolver.com/article/6-industrial-automation-trends-for-2022" target="_blank">6 Industrial Automation Trends For 2022</a><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1671197047280-scada.jpg" width="624" height="393" class="fr-fic fr-dii" alt="Technician in automated factory">Automation is a versatile topic. In industrial manufacturing, there are four main types of automation.<h2 dir="ltr">Four Types of Industrial Automation Systems</h2>Within the context of industrial applications for automated processes, there are four key types of automation: fixed automation, programmable automation, flexible automation, and integrated automation. Let&rsquo;s take a look at what each kind of automation is.&nbsp;<h3 dir="ltr">Fixed Automation</h3>Fixed automation, also known as hard automation, refers to a manufacturing technology that is engineered to automate a specific task. Fixed automation solutions are typically simple and are built for one purpose, like assembling a specific part or material handling. Fixed automation systems are highly efficient and are ideal for achieving fixed, repetitive production steps in mass-scale production lines.&nbsp;An automated conveyor belt is an example of fixed automation: the system operates independently throughout the manufacturing process and its purpose is to carry products along an assembly line. Other common forms of fixed automation are automated assembly machines, which are programmed to assemble specific parts, or automated painting processes designed to coat a specific product with a finishing paint or sealant.Unlike some of the other types of automation we will explore below, fixed automation systems have an internal programming. In other words, it is their hardware that controls them, rather than software. This makes it challenging to adopt fixed automation systems for different types of products or to accommodate product changes. Instead, fixed automation is best fit for mass producing a single type of product or part assembly. Fixed automation solutions are widespread in industrial production: these systems add value by driving production rates and efficiency and decreasing labor costs.[1]<h3 dir="ltr">Programmable Automation</h3>The next type of industrial automation is programmable automation. This subcategory of automation equipment is characterized by its ability to carry out the same task but in different ways. For example, while fixed automation might only be capable of moving from A to B, a programmable automation technology could be programmed to move from A to B, and then subsequently be programmed to move from A to C.&nbsp;Programmable automation solutions are most commonly used for batch production: the equipment is programmed to fulfill a specific task, which it repeats over the course of a single batch. It can then be programmed again to accommodate changes required for a new batch. Examples of programmable automation are CNC machines, which receive instructions from a computer program, industrial robots, whose motions and actions can be programmed, and steel rolling mills, which can be programmed to produce different thicknesses of metal.This type of automation is best suited to batches of up to several thousand parts rather than one-off production because reprogramming the equipment takes time. This means that any changeover between product varieties will necessitate machine down time, which slows down production rates.[2]&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1671197094529-robotic_arm.jpg" width="624" height="496" class="fr-fic fr-dii" alt="Robotic arm">Robotic arms are key to unlocking integrated automation on the factory floor.<h3 dir="ltr">Flexible Automation</h3>Flexible automation, sometimes called soft automation, refers to manufacturing equipment that is controlled by a computer program and is capable of carrying out a diversity of tasks based on the code it has been fed. Unlike programmable automation, which can be programmed for different products but requires significant down-time to do so, flexible automation solutions can be adapted rapidly and often automatically. In most cases, flexible automation only requires software changes (i.e. via a remote computer) rather than hardware changes, minimizing machine downtime and facilitating product changeovers.With flexible automation, it is possible to produce parts in batches or to manipulate different types of parts in succession. For example, a 6-axis robotic arm can be programmed to carry out a range of tasks in a production line, including drilling holes, joining components, and more. It could also be programmed to do something completely different the following day or week if needed. An industrial 3D printer could also be used to fulfill flexible automation: technicians can easily change printer settings and send a different CAD file to start producing a new design.The primary advantage of flexible automation is that it enables high-mix production and flexible manufacturing workflows, easily accommodating changes to product designs as well as a variety of products. One of the current challenges to the broader adoption of flexible automation solutions, however, is its high cost barrier: robotic arms and other flexible automation equipment require a significant upfront investment.<h3 dir="ltr">Integrated Automation</h3>Integrated automation refers to an end-to-end automated manufacturing process that does not require any human intervention. In an integrated automation workflow, a computer software system controls and connects the various pieces of manufacturing equipment on the production floor. Once settings and parameters have been set by an engineer or technician, the integrated automation solution carries out the instructions independently and can respond if any inconsistencies occur thanks to artificial intelligence and machine learning technologies.&nbsp;While the aforementioned types of automation describe specific systems or pieces of equipment, it is perhaps more helpful to think of integrated automation as a sort of framework. It underpins various automated hardware and connects each step in a production chain, resulting in a single, cohesive process rather than multiple distinct steps.&nbsp;Lights-out manufacturing is an example of integrated automation in action. It refers to when a manufacturing facility can operate without any human involvement and even supervision. Robots and automated equipment are responsible for every step in the process, as well as material and part handling. Today, many manufacturers are aiming to achieve lights-out production to increase productivity, achieve high production rates, and minimize the reliance on skilled human operators&mdash;of which there is often a shortage.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1671197140068-cnc.jpg" width="624" height="459" class="fr-fic fr-dii" alt="CNC machining close-up">Lights-out manufacturing is a fully automated production line that requires no human intervention or supervision.<h2 dir="ltr">Advantages and Disadvantages of Automation</h2>Automation is a key part of what is heralded as &ldquo;Industry 4.0&rdquo;, or the Fourth Industrial Revolution. Through a combination of smart machines, control systems, and automation tools, manufacturing operations are able to function more independently, with various steps in the process communicating with each other and process data informing computer system decisions in real-time.[3]Whenever we talk about a new manufacturing or industrial framework, there are two sides to the story. Automation, particularly integrated automation, represents a massive paradigm shift in manufacturing that has many people feeling very optimistic and many feeling quite anxious. It&rsquo;s important to look at both sides to understand where the benefits and disadvantages lie, and how to address them.<h3 dir="ltr">Advantages:</h3><ul><li dir="ltr">Productivity: One of the main reasons that manufacturers are investing increasingly in automation is because it can unlock higher productivity rates. With automation, tasks throughout the production chain&mdash;or the chain as a whole&mdash;can operate independently and with greater efficiency.&nbsp;</li><li dir="ltr">Consistency: With automated technologies in place, manufacturers can benefit from greater process consistency thanks to in-depth process monitoring data. A lower reliance on human labor also minimizes the risk of human error in the production process.</li><li dir="ltr">Labor: This is admittedly a tricky one (you&rsquo;ll also find it in the disadvantages section). There are benefits for both employers and employees when it comes to industrial automation. For one, automation enables manufacturers to mitigate the effects of labor shortages, particularly for skilled technicians. They can keep up production rates and even scale without having to seek out and hire additional workers. For another, automation eliminates the need for humans to undertake many tedious, repetitive tasks. This creates more space for more complex and valuable tasks that require problem solving, such as product development, process supervision, and big-picture strategizing.&nbsp;</li></ul><h3 dir="ltr">Disadvantages:</h3><ul><li dir="ltr">Cost: While there are long-term cost benefits for integrating automation in production workflows, the transition to automation and Industry 4.0 comes with a significant initial cost due to the high price of automated equipment and software tools. This cost can be prohibitive to smaller producers, and may only be viable for large manufacturing companies.&nbsp;</li><li dir="ltr">Labor: On this side of things, automation threatens existing labor forces and can lead to worker displacement. If a whole segment of jobs is suddenly automated, those workers are at risk of losing their employment and their skills may be devalued. Upskilling and trainings are an important strategy for mitigating this effect. They equip workers for new roles and give them new skills that are in demand.</li><li dir="ltr">Cybersecurity: As processes across manufacturing become more connected, they become more efficient but they also increase their vulnerability to cyber attacks. Malware, ransomware, and data breaches can compromise business processes significantly and come with a high cost. Establishing a robust cybersecurity system can help to minimize these risks and keep IP and production chains safe as automation is implemented.[4]</li></ul>Recommended reading: <a href="https://www.wevolver.com/article/is-your-manufacturing-plant-ready-for-automation" target="_blank">Is Your Manufacturing Plant Ready for Automation?</a><h2 dir="ltr">Automation Applications in Manufacturing</h2>In the context of industrial manufacturing, there are countless applications for automation. Many of the steps involved in transforming raw material into a finished product can be automated, including CNC milling, CNC turning, sheet rolling, stamping, pressworking, laser cutting, additive manufacturing, welding, product handling, material handling, part inspection, picking, packaging, and more.Automated material handling, assembly machines, and other automated manufacturing systems are indispensable tools within major manufacturing facilities today, including in the automotive, food packaging, electronics, pharmaceutical, and consumer goods industries, among others.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1671197274809-automated_car_manufacturing.jpg" width="624" height="457" class="fr-fic fr-dii" alt="car factory with robots">The automotive industry is among those driving the adoption of industrial automation.<h2 dir="ltr">Key Takeaways</h2>At its core, automation is about improving productivity and reducing the need for human labor. And it has been evolving for millenia. Thousands of years ago, water and wind mills freed people up from the labor intensive and time-consuming task of hand pounding wheat and grain. Today, the most advanced automated solutions are enabling entire production chains to go ahead without supervision. We covered a lot about automation in this article, if you&rsquo;re looking for a quick summary, here it is:<ul><li dir="ltr">There are four key types of automation: fixed automation, programmable automation, flexible automation, and integrated automation.</li><li dir="ltr">Fixed automation is a type of automation that is designed to fulfill a specific task and that one task only. For example, an automated conveyor belt has only one function, which it performs efficiently.</li><li dir="ltr">Programmable automation is a type of automation that can be reprogrammed to fulfill different variations on the same task. For example, a CNC machine can be reprogrammed to mill various different designs. Programmable automation is ideal for batch automation and requires hardware and software changes.</li><li dir="ltr">Flexible automation is a type of automation that is controlled by a computer program and can carry out a diversity of tasks based on the code it has been fed. Programmable automation often does not require machine down-time.</li><li dir="ltr">Integrated automation is an end-to-end automated manufacturing process that does not require any human intervention and is underpinned by a control system.</li><li dir="ltr">The main advantages of automation are higher productivity and output consistency.</li><li dir="ltr">The main disadvantages of automation are high initial cost and worker displacement.</li><li dir="ltr">Labor is a two-sided issue when it comes to automation: on the one hand, employers can mitigate labor shortages and workers are relieved from tedious, repetitive work. On the other hand, automation can replace jobs and lead to layoffs and short term unemployment issues.</li></ul><h2 dir="ltr">References</h2>[1] &ldquo;Manufacturing applications of automation and robotics.&rdquo; Britannica. [Accessed 2022 December]. <a href="https://www.britannica.com/technology/automation/Manufacturing-applications-of-automation-and-robotics" target="_blank">https://www.britannica.com/technology/automation/Manufacturing-applications-of-automation-and-robotics</a>&nbsp;[2] ibid.[3] Marr, Bernard. &ldquo;What is Industry 4.0? Here&#39;s A Super Easy Explanation For Anyone&rdquo;. Forbes. [Accessed: 2022 December]. <a href="https://www.forbes.com/sites/bernardmarr/2018/09/02/what-is-industry-4-0-heres-a-super-easy-explanation-for-anyone/" target="_blank">https://www.forbes.com/sites/bernardmarr/2018/09/02/what-is-industry-4-0-heres-a-super-easy-explanation-for-anyone/</a>&nbsp;[4] Davies, Vicki. &ldquo;The changing face of automated manufacturing and the risks&rdquo;. Cyber Magazine. December 3, 2021. [Accessed 2022 December]. <a href="https://cybermagazine.com/technology-and-ai/changing-face-automated-manufacturing-and-risks" target="_blank">https://cybermagazine.com/technology-and-ai/changing-face-automated-manufacturing-and-risks</a>&nbsp;

Automation and Industry 4.0 are revolutionizing the manufacturing and industrial sectors.

Everything you need to know about fixed automation, programmable automation, flexible automation, and integrated automation.

The Four Automation Types You Need to Know

This is the 3rd and final article in a 3 part series featuring the applications of tinyML. The series introduces the concept of tinyML and explains some selected applications of the technology in various fields like sustainability, healthcare, and education.Education is the first step towards innovation. It is an integral part of personal development and equips the learners with the tools necessary to go ahead and solve the challenges faced by humanity. While tinyML is a great tool to develop pedagogical interventions for improving the quality of education in different streams, it is in itself a vast subject that every science and technology student must be familiar with.In our <a href="https://www.wevolver.com/article/applications-of-tinyml-in-healthcare-for-a-safer-future" target="_blank">previous article</a> about the applications of tinyML, we explained how it aids medical research and improves healthcare. This article continues the series by showcasing two more applications of tinyML for education.<h2 dir="ltr">Using tinyML for Neuroscience in K12 education: tinyML4STEM</h2>Anything and everything concerned with the central nervous system is a part of neuroscience. It is a multidisciplinary field that involves a scientific study of how the brain coordinates with the body and makes various body functions possible. The study is not just limited to humans but can also be extended to animals and even plants.Roughly 20% of the world’s population has suffered from a neurological disorder at some point in their lives, and none of the disorders have a cure yet. The only way to make new discoveries in the domain is by promoting research and education.It is also interesting to note that the most fundamental unit of Machine Learning (ML) is called a neuron and the whole concept and modelling of Artificial Intelligence (AI) based systems came from neuroscience. This makes it essential to talk about the specialised subject with the students of computer science to make them understand the underlying principles of the technology.But the problem with neuroscience education has always been the affordability and availability of experiment kits. Unlike many other Science Technology Engineering Management (STEM) fields, the apparatus for performing neuroscience experiments can not be acquired easily by schools and colleges. Graduate students opting for a specialisation in the area are the only ones to get their hands on these experiment kits.Greg Gage and Tim Marzullo realised this problem back in 2008 when they were pursuing PhD at the University of Michigan. The duo tried explaining neuroscience concepts to school children but couldn’t do so without showcasing practicals. There was no way to bring expensive graduate college equipment to schools. To solve this issue and make neuro-science kits affordable and readily available, Greg and Tim founded Backyard Brains.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkwNDUyMjI0LWltYWdlNy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="EMG-signals-backyard-brains">Illustrations like these can help kids grasp even complex concepts easily. Source: Backyard BrainsThe organisation today makes low-cost neuroscience trainers, DIY kits, tools, software, conducts workshops and a lot more to democratise neuroscience education.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkwNTI1MDYyLWltYWdlMS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="neuroscience-experiment-kits-backyard-brains">Neuroscience trainer kits by Backyard Brains are great alternatives for schools to introduce the students to the field. Source: Backyard BrainsNeuroscience signals like Electromyogram (EMG), Electrooculogram (EOG), Electrocardiogram (EKG), etc., can be used to get various cues about what is going on inside the body.When neurons fire electrical impulses from the motor cortex, the signals flow through the spinal cord to the parts of the body involved in the action. By putting small electrodes on the skin near the areas where the signals can be picked up, it is possible to observe them via an EMG. The impulses being electrical in nature can be amplified as a single-dimensional signal and visualised easily. These single-dimensional signals are beautiful for ML and are also quite easy to be analysed.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkwNjI0MDcwLWltYWdlNC5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="visualising-neuroscience">A snapshot from tinyML talks where Greg demonstrates how EMG signals can be visualised with a low-cost training kit. Source: tinyML YouTubeThe patterns in these signals might not be as apparent to humans as they are to the tinyML algorithms, which can classify and map them with different human behaviours. An implementation of such a technique can be used to develop a brain-machine interface that can help make neuroprosthetics.&nbsp;EOG is another example of a neuro signal that measures the corneo-retinal standing potential of the eye. In other words, it helps understand eye movements. A trained TinyML system can be used to accurately detect eye blinking and movements by feeding the signal to the input.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkwNjYzNTM5LWltYWdlOC5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="EOG-signals-detect-eye-blinking">The spikes that appear on the EOG are due to eye blinking. Source: tinyML YouTubeThen there’s EKG generated by the electrical impulses from the heart. For different people, the patterns of EKG are observed to be similar when looking at the same object. This can be used to predict if the person is telling the truth. In medical applications, signals can be used to check the functioning of the brain. A sample tone is played to the patient, and electric potential signals are observed. When the tone being played changes, the subconscious mind should ideally change the way it reacts, which should be evident in the electric potential signals.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkxNTU2NjQ2LWltYWdlMy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="hackerhand-copies-gestures-EKG-signals">‘Hackerhand’, a robotic arm mimicking hand movements based on EKG signals. Source: tinyML YouTubeIt is also interesting to note that the signals are generated slightly before someone is about to take action, making it possible for tinyML systems to be able to predict the behaviour in advance. A similar fun project has been developed at Backyard Brains.Readers can participate in many more such projects or share ideas by visiting <a href="https://backyardbrains.com/" target="_blank">Backyard Brains</a>.<h2 dir="ltr">TinyML Open Education Initiative: tinyMLedu</h2>Over the past two decades, ML has been through a journey from a purely academic and theoretical discipline to a widely spread technology in use. It won’t be an understatement to say that technology has touched almost every industry and is now ubiquitous technology. It is expected that the interest in the field will only grow from here, at least in the near foreseeable future. Given its amazing potential in solving some of the greatest challenges faced by humanity, it is important that more people are introduced to technology in a systematic way.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkxNTk4OTQ2LWltYWdlNS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="instructors-tinyML-harvard-google-edx">Instructors of the tinyML specialisationUnlike traditional ML, it is possible to run tinyML algorithms on low-cost and low-power embedded systems, which solves a major problem related to the unavailability of powerful hardware for learners. But the shortage of ML educators is still something that needs to be addressed.A joint initiative by Harvard and Google, developed as an academic and industry collaboration, aims to address the challenges by introducing tinyML to the masses with a Massive Open Online Course (MOOC) on edX.&nbsp;The courses that are a part of the tinyML specialisation go beyond teaching the students about the subject by diving deeper into the practical applications and industrial product development lifecycle. The next generation of students needs to understand the end-to-end full-stack development of tinyML projects to be able to gain a complete understanding of the subject.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkxNjIwMzE4LWltYWdlOS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="tinyML-specialisation-course-structure">A diagram providing an overview of topics explored in tinyML specialisation. Source: [3]The tinyML specialisation, jointly developed by Harvard and Google, is a set of self-paced learning modules that can be taken from anywhere. The specialisation includes four courses:<ol><li dir="ltr">Introduction to tinyML:&nbsp;The first course covers the basics of ML and embedded systems. Everything from the concept to gathering data and training the algorithms is explained.</li><li dir="ltr">Applications of tinyML: The second course builds on the foundation by explaining some parts of the code behind the most widely used tinyML applications. It also introduces the learners to the principles of keyword spotting, visual wake words, anomaly detection, and dataset engineering.</li><li dir="ltr">Deploying tinyML: The third course teaches how code is written, optimised, and deployed for embedded systems. This is where microcontrollers are introduced, and students can purchase a kit for a nominal price (US$49.99) to get hands-on experience.</li><li dir="ltr">Scaling tinyML [Optional]: Once the learners are familiar with the design, development and deployment of tinyML, the final course covers the more advanced aspects related to producing tinyML solutions for the masses.</li></ol>Each course includes several activities such as collabs, hands-on labs, quizzes, readings, assignments, tests, and discussion-forum participation.The key challenges for students taking up the tinyML specialisation can be limited hardware accessibility and the requirement of being familiar with programming. To address the challenges associated with hardware accessibility, open-source emulation platforms like <a href="https://renode.io/" target="_blank">renode.io</a> from Antmicro are being tested to allow learners to try out their code without the hardware. The other challenge can be solved by platforms like <a href="https://www.wevolver.com/profile/edge.impulse" target="_blank">Edge Impulse</a>, which are lowering the barriers to enable people with little to no programming knowledge to implement tinyML solutions.Anyone with interest in tinyML can visit <a href="https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning" target="_blank">tinyML Professional Certificate | edX</a> and enrol for the course for free. The course material is available on <a href="https://github.com/tinyMLx/courseware" target="_blank">GitHub</a>.<h2 dir="ltr" id="isPasted">Conclusion</h2>Though the project kits or MOOCs are not a complete solution to train professional neuroscience researchers or tinyML solution architects, they offer a great place for beginners to start. The kits and courses are designed in such a way that the students can engage in fun activities while they experiment and learn new stuff. This makes the experience memorable and develops an interest in the subject, which could lead more people to choose the specialised fields and make new discoveries.&nbsp;This article concludes our tinyML applications series.About tinyML FoundationWith its workshops, webinars, expert talks, research conferences, competitions, and a plethora of other events, tinyML Foundation enables students, researchers, and industry personnel to come together and share their experiences about the technology. It is the incredibly collaborative nature of the community that allows it to advance rapidly.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQzMjkxNjYyNjY4LWltYWdlMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">The <a href="https://www.wevolver.com/article/tinyml-for-good-where-machine-learning-meets-edge-computing" target="_blank">introductory article</a> familiarised the readers with the concept of tinyML.The <a href="https://www.wevolver.com/article/tinyml-unlocks-new-possibilities-for-sustainable-development-technologies" target="_blank">first article</a> explained some possibilities of tinyML applications in sustainable technologies.The <a href="https://www.wevolver.com/article/applications-of-tinyml-in-healthcare-for-a-safer-future/" target="_blank">second article</a> is about tinyML applications aiding healthcare and medical research.The <a href="https://www.wevolver.com/article/applications-of-tinyml-making-specialised-education-more-accessible" rel="noopener noreferrer" target="_blank">third and final article</a> shall cover the tinyML applications in pedagogy and education.<h2 dir="ltr">References</h2>[1] Marzullo TC, Gage GJ (2012) The SpikerBox: A Low Cost, Open-Source BioAmplifier for Increasing Public Participation in Neuroscience Inquiry. PLOS ONE 7(3): e30837. <a href="https://doi.org/10.1371/journal.pone.0030837" target="_blank">https://doi.org/10.1371/journal.pone.0030837</a>.[2] Gregory J. Gage, The Case for Neuroscience Research in the Classroom, Neuron, Volume 102, Issue 5, 2019, Pages 914-917, ISSN 0896-6273, <a href="https://doi.org/10.1016/j.neuron.2019.04.007" target="_blank">https://doi.org/10.1016/j.neuron.2019.04.007</a>.[3] Vijay J. Reddi et. al., Widening Access to Applied Machine Learning with TinyML, arXiv:2106.04008v2 [cs.LG], Jun 2021, [Online], Available from: <a href="https://arxiv.org/abs/2106.04008" target="_blank">https://arxiv.org/abs/2106.04008</a>.

In this article, we look at two tinyML projects for education. We show how Backyard Brains uses low-cost experiment kits to make neuroscience education more accessible. We also introduce our readers to a specialisation offered by Harvard & Google to help students learn tinyML like never before.

Applications of TinyML making specialised education more accessible

This is the 2nd article in a 3 part series featuring the applications of tinyML. The series introduces the concept of tinyML and explains some selected applications of the technology in various fields like sustainability, healthcare and education.By adopting digital advancements, healthcare providers are updating their services to improve them and make them accessible to more people. 5G networks, Augmented Reality (AR) &amp; Virtual Reality (VR), Wearables, Digital Twins, and many more technologies are enabling this digital transformation.One such technology with great potential in ensuring healthy lives and well-being for all is tinyML. The concept is a crossover between Artificial Intelligence (AI) and embedded systems that involves developing solutions that bring Machine Learning (ML) capabilities to edge devices. Benefits include privacy, affordability, low-power utilisation, the ability to work in remote locations, and a lot more.In our <a href="https://www.wevolver.com/article/tinyml-unlocks-new-possibilities-for-sustainable-development-technologies" target="_blank">previous article</a> about the applications of tinyML, we explained how it unlocks new possibilities for sustainable development technologies. This article continues the series by showcasing two more applications of tinyML for medical research and improved healthcare.<h2 dir="ltr">MaRTiny: a tinyML solution for extreme heat sensing for urban climate science</h2>As a direct effect of global warming, summers are getting warmer. [1] This disruption of the natural balance poses a risk to all living entities on the planet. Higher temperatures are known to cause a degradation in the air quality and induce severe health-related issues like heat strokes. Looking at the data of weather-related fatalities in America, heat is the single most significant cause of weather-related deaths over the last 30 years, surpassing floods and storms. [2]But it’s not just the ambient temperature that causes discomfort. It’s the Mean Radiant Temperature (MRT) which accounts for the influence of all the surfaces in the surrounding that ultimately determines the occupant’s comfort. Often, the effect of roads, buildings and other entities has an adverse effect on the MRT, creating a complex problem intersecting health and urban infrastructure.To aid the collection of information related to the MRT, a tinyML solution called MaRTiny was developed by researchers from Arizona State University (ASU). It is a $200 low-cost mobile platform for bio-metrological sensing.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3MzYxNzgyLTE2NDI1MTczNjE3ODIucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="720" height="297" id="isPasted" class="fr-fic fr-dii" alt="MaRTiny-tech-stack">The tech stack of the MaRTiny platformThe list of sensors embedded on the device include:<ul><li dir="ltr">Ambient temperature sensor</li><li dir="ltr">Globe temperature sensor for sensing the radiant heat</li><li dir="ltr">Humidity sensor</li><li dir="ltr">Ultra Violet (UV) sensor</li><li dir="ltr">Anemometer for measuring wind speed and direction.</li></ul>The platform uses an <a href="https://www.wevolver.com/profile/arduino.pro" target="_blank">Arduino</a> Uno to collect the data and a NodeMCU to provide it with the electronic infrastructure required for wireless data communication. At the core of the system is the Nvidia Jetson Nano development board that provides MaRTiny with the power to run tinyML algorithms.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3NDcxMTAyLU5WSURJQV9KZXRzb25fTmFub19EZXZlbG9wZXJfS2l0Xyg0MDY1MDQyNTUwMykuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="nvidia-jetson-nano-development-board">The Nvidia Jetson Nano development board provides MaRTiny with image processing and ML capabilitiesJetson Nano, with its Camera Serial Interface (CSI), can connect with a camera for shadow detection, object detection and people-in-shade detection. Jetson’s ARM Cortex A57 MPcore, 128 CUDA core Maxwell GPU coupled with 4 GB RAM is just enough to offer advanced ML capabilities on edge.The task of MRT estimation was formulated as a supervised learning problem by the inventors that required a lot of true data for training. This true data was collected with the help of a previously developed, larger and more expensive MRT detecting system going by the name of MaRTy. [3]&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3NTMzMzM5LTE1MDB4NTAwLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="MaRTiny-MaRTy">MaRTiny aims to offer similar functionality as that of MaRTy (in the image) at just 1% of the price. Image source: @ASUMaRTy, TwitterIt was observed that a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel achieved the highest accuracy with the MaRTiny. The method being computationally light, offered an easy way of deployment on Jetson. To improve the estimations, both SVM and Artificial Neural Network (ANN) models were utilised and trained with data collected via sensors. [4]For privacy, the images taken by MaRTiny are not stored in the cloud or even in the device’s memory. It automatically discards personally identifiable information (PII) before reporting anything back to the servers, which is central to the idea of tinyML and a must-have feature for public deployment.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3NTc1MjI0LWltYWdlNy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="MaRTiny-setup">The experimental setup of MaRTinyThe tinyML solution, MaRTiny, can provide crucial insights related to weather conditions and the utilisation of shades and space by the public. Multiple MaRTiny devices can be installed across the city to gather valuable data for architecture and town planning.<h2 dir="ltr">Pneumonia Detection using Embedded Machine Learning</h2>Pneumonia is a respiratory infection caused by viruses, bacterias and fungi. According to the World Health Organisation (WHO) data, it is estimated that as much as 22% of all deaths in children aged from 1 to 5 is due to pneumonia. Bacteria induced pneumonia can be treated with antibiotics and other low-cost and low-tech medical facilities if detected at the right time. [5]Chest X-ray, blood oxygen levels and Complete Blood Count (CBC) are used to detect pneumonia which is time-consuming. Doctors manually review the chest X-ray and decide if the person is infected, which is sometimes inaccurate and requires an automated solution with better accuracy. This is the idea behind the tinyML solution that we are about to discuss in this section.Arijit Das, a 15-year-old tinyML enthusiast, has developed a tinyML model using the <a href="https://www.wevolver.com/profile/edge.impulse" target="_blank">Edge Impulse</a> platform to detect pneumonia from chest X-rays. The model can be deployed on single board development platforms like Raspberry Pi 4, Sony Supersense and a few other devices. With just the main board, a camera shield and the Edge Impulse tinyML model, the tiny setup can detect pneumonia in under a minute which is phenomenal considering the fact that it takes anywhere between 1 to 4 days for detection by manual inspection.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3NzA0Nzk2LVJhc3BiZXJyeV9QaV80X01vZGVsX0JfLV9TaWRlLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="Raspberry-Pi-4">Credit card-sized single-board computers like Raspberry Pi come with enough compute power to run optimised tinyML algorithmsThe tinyML model doesn’t require internet connectivity to work and offers an accuracy of over 80%, which is far better compared to manual inspection. This can prove to be quite helpful in largely affected regions like Sub-Saharan Africa and South Asia, where a huge number of examinations need to be conducted related to the same disease.&nbsp;At the time of writing this article, the project is fully functional and has been clinically tested by over 10 doctors in a local survey conducted by the inventor. The project is available on <a href="https://github.com/Pneumonia-Detection-using-EdgeML" target="_blank">GitHub</a> for everyone to try, test and contribute. [6]<h2 dir="ltr">Conclusion</h2>It’s the third year into the pandemic, and the emphasis on healthcare research is more than ever. Introducing technologies that can work with little power and no network connectivity at an affordable price in healthcare can save several lives. Leveraging interdisciplinary technologies like tinyML is the way forward to a safer and healthier future.About tinyML FoundationWith its workshops, webinars, expert talks, research conferences, competitions, and a plethora of other events, tinyML Foundation enables students, researchers, and industry personnel to come together and share their experiences about the technology. It is the incredibly collaborative nature of the community that allows it to advance rapidly.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQyNTE3NzY5MDkxLWV5SmlkV05yWlhRaU9pSjNaWFp2YkhabGNpMXdjbTlxWldOMExXbHRZV2RsY3lJc0ltdGxlU0k2SW1aeWIyRnNZUzh4TmpNMk9UZ3lNVEUwT1RRNExWUnBibmxOVEdadmNrZHZiMlFnS0RRd01EQWdlQ0EzTlRBZ2NIZ3BMbXB3WnlJc0ltVmthWFJ6SWpwN0luSmxjMmw2WlNJNmV5SjNhV1IwYUNJNk9UVXdMQ0ptYVhRaU9pSmpiM1psY2lKOWZYMD0ud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="tinyML-banner"> The <a href="https://www.wevolver.com/article/tinyml-for-good-where-machine-learning-meets-edge-computing" target="_blank">introductory article</a> familiarised the readers with the concept of tinyML.The&nbsp;<a href="https://www.wevolver.com/article/tinyml-unlocks-new-possibilities-for-sustainable-development-technologies" target="_blank">first article</a> explained some possibilities of tinyML applications in sustainable technologies.The <a href="https://www.wevolver.com/article/applications-of-tinyml-in-healthcare-for-a-safer-future/" rel="noopener noreferrer" target="_blank">second article</a> is about tinyML applications aiding healthcare and medical research.The third and final article shall cover the tinyML applications in&nbsp;pedagogy&nbsp;and education.<hr><h2>References</h2>[1] Extreme Weather Toolkits, Climate Central, [Online], Available from: &nbsp;<a href="https://medialibrary.climatecentral.org/extreme-weather-toolkits/extreme-heat" target="_blank">https://medialibrary.climatecentral.org/extreme-weather-toolkits/extreme-heat</a>[2] Weather Related Fatality and Injury Statistics, National Weather Service, [Online], Available from:&nbsp;<a href="https://www.weather.gov/hazstat/" target="_blank">https://www.weather.gov/hazstat/</a>[3] Ariane Middel, E. Scott Krayenhoff, ‘Micrometeorological determinants of pedestrian thermal exposure during record-breaking heat in Tempe, Arizona: Introducing the MaRTy observational platform’, Science of The Total Environment, Volume 687, 2019, Pages 137-151, ISSN 0048-9697, Available from:&nbsp;<a href="https://doi.org/10.1016/j.scitotenv.2019.06.085" target="_blank">https://doi.org/10.1016/j.scitotenv.2019.06.085</a>[4] Karthik K. Kulkarni, 2021, ‘Machine Learning and Vision Using Edge Devices for Multimodal Chatbots andBio-meteorological Sensing’, MS Thesis, Arizona State University, United States of America, [Online], Available from:&nbsp;<a href="https://keep.lib.asu.edu/_flysystem/fedora/c7/Kulkarni_asu_0010N_21269.pdf" target="_blank">https://keep.lib.asu.edu/_flysystem/fedora/c7/Kulkarni_asu_0010N_21269.pdf</a>[5] Pneumonia Facts Sheet, World Health Organisation, [Online], Available from:&nbsp;<a href="https://www.who.int/news-room/fact-sheets/detail/pneumonia" target="_blank">https://www.who.int/news-room/fact-sheets/detail/pneumonia</a>[6] Arijit Das, Pneumonia-Detection-using-EdgeML, GitHub, [Online], Available from:&nbsp;<a href="https://github.com/Pneumonia-Detection-using-EdgeML" target="_blank">https://github.com/Pneumonia-Detection-using-EdgeML</a>

In this article, we take a look at two tinyML projects related to healthcare. The first project helps gather Mean Radiant Temperature data outdoors to protect people from extreme heat, and the other one is a solution for affordable, accurate, & rapid detection of pneumonia.