Inspiring Examples of Edge AI in Action.

2023 Edge AI Technology Report. Chapter VIII: Case Studies

How Edge AI Senses the World for Situational Awareness.

2023 Edge AI Technology Report. Chapter VII: Sensing Modalities

What to Consider When Selecting Your Edge AI Hardware and Software.

2023 Edge AI Technology Report. Chapter VI: Edge AI Algorithms

A Deep Dive Into Edge AI in Energy-Constrained Environments


2023 Edge AI Technology Report. Chapter V: TinyML

<h1 id="isPasted">2023 Edge AI Technology Report. Chapter IV: Hardware &amp; Software Selection</h1><p dir="ltr" id="isPasted">Edge AI, empowered by the recent advancements in Artificial Intelligence, is driving significant shifts in today&#39;s technology landscape. By enabling computation near the data source, Edge AI enhances responsiveness, boosts security and privacy, promotes scalability, enables distributed computing, and improves cost efficiency.</p><p dir="ltr">Wevolver has partnered with industry experts, researchers, and tech providers to create a detailed report on the current state of Edge AI. This document covers its technical aspects, applications, challenges, and future trends. It merges practical and technical insights from industry professionals, helping readers understand and navigate the evolving Edge AI landscape.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg4NDg0MjA0ODUyLUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgyKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h2 dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></h2><hr><h2 dir="ltr">Introduction</h2><p>When selecting and implementing Edge AI systems, it&#39;s important to consider the limitations of edge devices and their compatibility with other hardware and software. Edge devices like sensors and smartphones often have limited processing power, storage, and battery life. These limitations can negatively affect the performance of Edge AI systems, as they may require significant resources to perform complex tasks, such as image recognition and natural language processing. To overcome these limitations, AI models must be optimized to work with limited resources, and specialized hardware such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and GPUs can accelerate computation at the edge.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg3MjI2OTA1LW1pY3JvY2hpcC5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">A field-programmable gate array (FPGA). Image credit: Microchip.</span></span></span></p><p>Edge AI systems are designed to operate for extended periods, so the hardware selected should be capable of supporting future upgrades. It should be designed to accommodate new technologies and features that may be required in the future. Additionally, compatibility is a critical consideration when selecting software for Edge AI systems. Because Edge AI systems often require integration with existing infrastructure and hardware, it&#39;s important to select software that is compatible with a wide range of devices and platforms. This includes both hardware compatibility, such as compatibility with different types of sensors and processors, and software compatibility, such as compatibility with different operating systems and programming languages.</p><h3>Hardware Considerations</h3><p>Ensuring that the chosen hardware meets the requirements of Edge AI systems involves considering various factors. One key consideration is processing power, as Edge AI applications require hardware capable of processing data quickly and accurately. The hardware&#39;s computing power and clock speed play crucial roles in achieving real-time processing, which is essential in Edge AI systems. Another critical consideration is the amount and type of memory available in the hardware. Edge AI applications necessitate sufficient memory to store and process large volumes of data, with a focus on fast and real-time processing capabilities. There is a race to have the highest number of TOPS (tera-operations per second) in the hardware industry, but it&#39;s not a one-to-one comparison.</p><blockquote><p>&ldquo;The memory system handling, DMA configuration, software compression of weights, intelligence, and decompression all matter significantly. Focusing only on the box numbers is similar to focusing solely on the megapixel or gigapixel of a phone&#39;s camera without considering other factors. This tunnel vision is an error for the industry to make.&rdquo;&nbsp;<br>- <span style="background-color: transparent;">Rahul Venkatram, Senior Product Manager, Machine Learning, Arm</span></p></blockquote><p>Power consumption is another vital aspect, particularly for devices with limited power resources like IoT devices. Hardware optimization for low power consumption becomes crucial to ensure prolonged device operation without frequent battery replacements or recharging. Techniques such as utilizing low-power chips, hardware accelerators, and intelligent power management systems contribute to achieving low power consumption in Edge AI systems. In a conversation with Henrik Flodell, Director of Product Marketing at Alif Semiconductor, he emphasized that &ldquo;reducing power consumption is a key parameter users need to consider when they pick suitable devices for edge systems that will use ML.&rdquo;&nbsp;</p><p>The Autonomous Intelligent Power Management (aiPM&trade;) technology from Alif is a vital solution for power consumption. It helps with selectively powering on the sections of an SoC that are needed when they are needed, and it powers them off when they are not, based on instantaneous processing demand and use case. It makes use of dual-processing regions within the MCU to speed up machine learning. The always-on, high-efficiency region continuously senses the environment, while the high-performance region wakes up when needed so it can execute heavy workloads and then return to sleep. This approach allows powerful devices to behave like low-power MCUs when necessary, which enables longer performance of smart IoT devices on smaller batteries.</p><blockquote><p>&ldquo;Systems that have to rely on CPU-bound processing tend to be power hungry, as the inference operations would take longer to finish, and the controller would run at 100% for much longer periods of time. Well-designed modern platforms offload the ML processing on accelerators designed for such workloads and can, therefore, finish faster and go to sleep sooner.&rdquo;&nbsp;<br><span style="background-color: transparent;">- Henrik Flodell, Direct of Product Marketing, Alif Semiconductor.</span></p></blockquote><p>Seamless connectivity is also essential for Edge AI systems, requiring hardware that enables smooth data exchange with other devices and cloud-based platforms. The hardware should offer various connectivity options, including Wi-Fi, Bluetooth, and cellular networks, facilitating efficient communication and data processing with other devices. This connectivity ensures that data is processed and analyzed quickly and efficiently, contributing to the overall performance of the Edge AI system.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg3MjkzMjM2LWFyZHVpbm8ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">The Edge Control from Arduino is a remote monitoring and control solution optimized for outdoor environments that enables users to collect real-time data from smart sensors and leverage AI at the edge. Image credit: Arduino.</span></span></span></p><h3>Software Considerations</h3><p>When it comes to Edge AI systems, selecting software with specific characteristics is crucial for optimal operation. Analyzing various factors ensures that the chosen software meets the requirements of the Edge AI system. Compatibility stands as a significant consideration in software selection. The software should run efficiently on the hardware, enabling real-time data processing. Moreover, compatibility with other software components used in the system, such as operating systems, libraries, and frameworks, ensures seamless integration and functionality.</p><p>Scalability is another vital aspect to consider in Edge AI system design. Given that Edge AI systems often handle large amounts of data, the software must be capable of handling the real-time processing and analysis demands associated with such data. Scalable software guarantees the system can accommodate increasing data volumes, processing requirements, and user requests without compromising performance.</p><p>The accuracy of the software employed in Edge AI systems holds great importance. These systems heavily rely on accurate data analysis and processing to provide meaningful insights and support decision-making. Therefore, the software must exhibit high accuracy and reliability in analyzing and processing data.</p><p>Interpretability, or the software&#39;s ability to explain its results comprehensibly, plays a crucial role in Edge AI system design. Interpretability allows users to understand the system&#39;s decision-making process and provides insights into data analysis. This aspect becomes particularly significant in applications where the decisions made by the Edge AI system have substantial implications, such as healthcare and finance. The software used in Edge AI systems should prioritize interpretability, presenting results in a clear and understandable manner.</p><h3>Integration Considerations</h3><p>Integrating Edge AI with other systems is essential for seamless and efficient operations. As Edge AI systems generate vast amounts of data that require processing, analysis, and sharing with other systems, selecting communication protocols becomes imperative. Different protocols are used by Edge AI systems to communicate with other systems, and it is crucial to choose a protocol that is compatible with the integrated systems.</p><p>Several commonly used protocols in Edge AI system integration include MQTT, CoAP, and HTTP. MQTT is a lightweight protocol widely utilized in IoT systems due to its low overhead and power consumption. CoAP, on the other hand, is designed specifically for resource-constrained networks, making it suitable for Edge AI systems. HTTP, a widely used protocol in web-based applications, is also suitable for Edge AI systems that communicate with web-based applications. The choice of communication protocol should align with the requirements of the Edge AI system and the systems being integrated.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg3MzU4NTUyLUlNQUdFUyAyLTA0LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Comparison of protocols in terms of power consumption and resource requirement.</span></span></span></p><p>In addition to communication protocols, data formats are another critical consideration when integrating Edge AI with other systems. Edge AI systems generate data in various formats, and it is vital to ensure compatibility between the data generated by the Edge AI system and the systems it interacts with. Data formats must be aligned and properly transformed or translated to facilitate seamless data exchange and interoperability between the systems.</p><p>Global technology provider, Arm, has approached integration from a holistic thought process about where and how to deploy processing. David Maidment, Senior Director, Secure Device Ecosystem at Arm, explained their approach as follows: &ldquo;We&#39;ve landscaped our approach in two directions. The first is scaling down traditional cloud-native workloads, such as microservers. The second direction is taking IoT and embedded devices and making them connected, updateable, and secure. We&#39;re seeing these two trends converge, with higher-end, higher-power devices coming down in cost and power, and lower-end, lower-power devices being pushed up in performance capabilities.&rdquo;</p><p>This results in two distinct software ecosystems. The first is the traditional cloud software ecosystem, with portable workloads that can run on servers with offload and acceleration. The second is a more traditional Linux box that was initially designed for a single function, like a router or gateway but is now being developed as a connected general-purpose box with capabilities that can change during its lifetime through new applications and AI models.</p><h3>Security Considerations</h3><p>Ensuring security is a critical factor when integrating Edge AI with other systems. Since Edge AI systems generate vast amounts of data, including sensitive information, protecting it from unauthorized access becomes imperative. Secure communication protocols such as SSL/TLS and SSH should be employed to establish secure channels for data transmission between the Edge AI system and other systems. These protocols encrypt the data, preventing eavesdropping and tampering during transit.</p><p>To reinforce security, access controls and authentication mechanisms must be implemented. This ensures that only authorized users can access the data generated by the Edge AI system, safeguarding it from unauthorized individuals. Secure software development practices, such as regular code reviews and vulnerability assessments, should also be followed to identify and mitigate potential security risks in the integrated systems.</p><p>Encryption is pivotal in securing Edge AI systems and should not be overlooked. By converting sensitive data into an encoded form, encryption ensures that even if the data is compromised, it remains unintelligible to unauthorized entities. Given that Edge AI systems often process sensitive data like personal health information, financial records, and valuable intellectual property, encrypting this data is crucial in preventing unauthorized access, theft, or manipulation.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg3NDU1ODEyLTA1IGltYWdlLTAxLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Key usage in a cryptographic system of an IoT device and attack paths.<br></span></span></span></p><p><em id="isPasted">Gain a broader perspective on Edge AI by revisiting our&nbsp;</em><a href="https://www.wevolver.com/article/edge-ai-a-comprehensive-guide-to-its-engineering-principles-and-applications" rel="noopener noreferrer" target="_blank"><em>comprehensive guide</em></a><em>.</em></p><hr><h1 dir="ltr" id="isPasted">The 2023 Edge AI Technology Report</h1><p dir="ltr"><em>The guide to understanding the state of the art in hardware &amp; software in Edge AI.</em></p><p dir="ltr">Click through to read each of the report&#39;s chapters.&nbsp;</p><h2 id="isPasted"><a target="_blank">Report Introduction</a></h2><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-i-overview-of-industries-and-application-use-cases" rel="noopener noreferrer" target="_blank">Chapter I: Overview of Industries and Application Use Cases</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ii-advantages-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter II: Advantages of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms" rel="noopener noreferrer" target="_blank">Chapter III: Edge AI Platforms</a></h3><h3>Chapter IV: Hardware and Software Selection</h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml" rel="noopener noreferrer" target="_blank">Chapter V: Tiny ML</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms" rel="noopener noreferrer" target="_blank">Chapter VI: Edge AI Algorithms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities" rel="noopener noreferrer" target="_blank">Chapter VII: Sensing Modalities</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies" rel="noopener noreferrer" target="_blank">Chapter VIII: Case Studies</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ix-challenges-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter IX: Challenges of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter X: The Future of Edge AI and Conclusion</a></h3><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><hr><h2>The report sponsors.</h2><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg3NTQ5OTc0LTExICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p><br></p>

2023 Edge AI Technology Report. Chapter IV: Hardware & Software Selection

Introduction

An Overview of the Different Edge AI Platforms.


2023 Edge AI Technology Report. Chapter III: Edge AI Platforms

<h1 id="isPasted">2023 Edge AI Technology Report. Chapter II: Advantages of Edge AI</h1><p dir="ltr" id="isPasted">Edge AI, empowered by the recent advancements in Artificial Intelligence, is driving significant shifts in today's technology landscape. By enabling computation near the data source, Edge AI enhances responsiveness, boosts security and privacy, promotes scalability, enables distributed computing, and improves cost efficiency.</p><p dir="ltr">Wevolver has partnered with industry experts, researchers, and tech providers to create a detailed report on the current state of Edge AI. This document covers its technical aspects, applications, challenges, and future trends. It merges practical and technical insights from industry professionals, helping readers understand and navigate the evolving Edge AI landscape.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg4NDgzNzk4NTQ3LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgyKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h2 dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable fr-active" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></h2><hr><h2 dir="ltr">From Cloud to Edge AI</h2><p dir="ltr">Following the introduction of cloud computing, there was a surge of interest in managing and analyzing data on the cloud. At that time, many enterprises considered the cloud a very appealing option for running data analytics and machine learning functions due to its scalability, capacity, and quality of service.&nbsp;</p><p dir="ltr">Over the years, however, many organizations voiced concerns against transferring and processing large volumes of data in the cloud. For instance, companies were reluctant to transfer sensitive corporate data to a remote data center they did not control. Others were concerned about the latency of several applications, as cloud data management and analytics incurred data transfer through a wide area network.&nbsp;</p><p dir="ltr">Motivated by the above-listed limitations of cloud data processing, many organizations have turned to edge computing to manage some of their data close to the data sources. Edge computing limits the amount of data to be transferred to the cloud for processing and analytics. Hence, it alleviates the latency, security, and privacy challenges of data analytics in the cloud.</p><h3 dir="ltr">Edge AI Advantages</h3><p dir="ltr">Leveraging the merits of edge computing, Edge AI enables organizations to develop practical AI applications. These applications exhibit exceptional performance, low latency, robust security, improved privacy control, and power efficiency. Edge AI also enables organizations to make optimal use of network, computing, and energy resources, improving the AI applications’ overall cost-effectiveness. Let’s explore the advantages of Edge AI in more detail:</p><ul><li dir="ltr"><p dir="ltr">Reduced Latency:&nbsp;Edge AI applications limit the amount of data transfers over wide area networks, as processing occurs close to the data sources rather than within the cloud. As such, data processing is faster, and the latency of the Edge AI application is reduced. Moreover, the transfer and execution of instructions from the AI applications to the field result in much lower latency. This is vital for several classes of low-latency AI applications, such as applications based on industrial robots and automated guided vehicles. Some other applications built around video in which the data had to be sent to the cloud are now being processed in the Edge not only because they can but because it’s vital to assess what’s going on in near real-time in situations like security cases.</p></li><li dir="ltr"><p dir="ltr">Real-Time Performance:&nbsp;The lower latency of Edge AI applications makes them suitable for implementing functionalities that require real-time performance. For instance, machine learning applications that detect events in real-time (e.g., defect detection in production lines, abnormal behavior detection in security applications) cannot tolerate delays associated with the transfer and processing of data from the edge to the cloud.</p></li><li dir="ltr"><p dir="ltr">Enhanced Security and Data Protection:&nbsp;Edge AI applications expose much fewer data outside the organizations that produce or own the data. This reduces their attack surface and minimizes the opportunities for malicious security attacks and data breaches. These are why Edge AI applications tend to be much more secure than their cloud counterparts.</p></li><li dir="ltr"><p dir="ltr">Improved Privacy Control: Many AI applications process sensitive data, such as data related to security, intellectual property, patients, and other forms of personal data. Edge AI deployments create a trusted data management environment for all these applications, providing more robust privacy control than conventional AI applications in the cloud. The reason is Edge AI applications limit the amount of data transferred or shared outside the organizations that produce or handle sensitive datasets. &nbsp;&nbsp;</p></li><li dir="ltr"><p dir="ltr">Power-Efficiency:&nbsp;Cloud data transfers and cloud data processing are extremely energy-savvy operations. Specifically, cloud I/O (Input/Output) functionalities are associated with significant carbon emissions. Most importantly, cloud AI is not green at all, as very large volumes of data are typically processed by GPUs (Graphical Processing Units) and TSUs (Tensor Processing Units). Edge AI alleviates the problematic environmental performance of cloud AI applications. It reduces the number of I/O operations and processes data within edge devices or edge data centers. Hence, it results in an improved overall CO2 footprint for AI applications.</p></li><li dir="ltr"><p dir="ltr">Cost-Effectiveness:&nbsp;Edge AI applications economize on network bandwidth and computing resources because they transmit and process much less data than cloud computing applications. Moreover, they consume less energy than cloud AI applications. As a result, Edge AI applications can be deployed and operated at a considerably lower cost than cloud AI deployments.</p></li><li dir="ltr"><p dir="ltr">On-Device Learning:&nbsp;Certain Edge AI applications can be executed within a single device, such as an IoT device or a microcontroller. This enables the development of powerful and intelligent devices, like System-on-Chip (SoC) devices. One of their main characteristics is that they can learn on the device, which is the foundation for giving machines intelligence capabilities that are hardly possibly based on cloud processing. BrainChip’s Nandan Nayampally remarks, “With on-device learning, features extracted from trained models can be extended to customize model behavior without having to resort to extremely expensive model retraining on the cloud. This substantially reduces costs.”</p></li><li dir="ltr"><p dir="ltr">New Capabilities Enabling Novel Applications:&nbsp;The integration of Edge AI systems within AI applications provides capabilities not available a few years ago. For instance, it enables a new class of real-time functionalities in transport, manufacturing, and other industrial settings. Hence, Edge AI is not only improving on cloud AI applications but also unlocking opportunities for innovative applications that were not previously possible.</p></li></ul><p dir="ltr">In a conversation with <a href="https://www.synaptics.com/" target="_blank">Synaptics</a>, we asked Shay Kamin Braun, Director of Product Marketing, about the most important advantages that Edge AI provides. He reiterated the advantages mentioned above and added some notable ones.</p><p dir="ltr">While “Edge AI provides a reduced latency in response time and a better UX without the network latency in the cloud,” Kamin Braun also highlighted privacy and battery life benefits. In terms of privacy, “PII (personally identifiable information) stays on the device and doesn’t go to the cloud. Only metadata goes to the cloud in most cases. This increases privacy,” explained Kamin Braun. “As for power efficiency, running AI at the edge helps you reduce the amount of data transmitted to the cloud over the network, which saves power. As a result, you get longer battery life or energy star compliance.”&nbsp;</p><p dir="ltr">Kamin Braun also emphasized the advantage of availability, which can be an issue for Cloud AI given its inherent reliance on the network. “Maybe you have no coverage for mobile devices, or maybe the Wi-Fi is not working. You cannot always rely on the cloud for processing. On the contrary, if your application is completely at the edge, then you’re not dependent on the network. Edge AI provides full availability.”</p><blockquote><p dir="ltr">“By doing the computation at the sensor with the context, you can optimize better than using a generic, aggregated solution. While the battery is a consideration, lower thermal footprint enables cheaper, cost-effective chip packaging and field use that really provides scale.”<br>- Nandan Nayampally, CMO, BrainChip</p></blockquote><h3 dir="ltr">Cloud/Edge Computing Continuum: A Powerful Combination</h3><p dir="ltr">With all the advantages that Edge AI has over cloud AI, many use cases still demand computationally intensive data processing within scalable cloud data centers. This is the case with deep learning applications, for example, which must be trained with large numbers of data points. In such cases, there is always a possibility to combine cloud AI and Edge AI paradigms to obtain the benefits of both worlds. Specifically, it is possible to perform a first round of local data processing at the edge to reduce the amount of uninteresting data and identify datasets that need processing in the cloud.&nbsp;</p><p dir="ltr">Local data processing is usually followed by a transfer of selected data to the cloud, where data points from different edge data sources are aggregated and processed to extract global insights. Furthermore, cloud AI applications may push information about these insights back to the edge data sources, which enhances the intelligence of local processing and field actuation functions. This two-level data processing exploits different deployment configurations of edge devices in the cloud/edge computing continuum.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODU5MjM1Njk2LUlNQUdFUyAyLTA3LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Hybrid solution of edge and cloud processing. Adapted from IEBMedia by Wevolver.</span></span></span></p><p dir="ltr">There are machine learning paradigms that inherently support this two-level data cloud/edge data processing approach. As a prominent example, federated machine learning employs local data processing to train machine-learning models across various edge computing nodes residing in different organizations. In this way, it produces a range of local machine-learning models. These are then fused in a central cloud location, which combines the local models into a more accurate global model.</p><p dir="ltr">This federated approach offers strong privacy and data protection characteristics, as source data need not leave the organizations where the edge nodes reside. Instead, an accurate global model is built in the cloud based on transferring and processing data about the local machine learning models. Therefore, the machine-learning models’ information is shared while the source data are not.</p><p dir="ltr">The combination of cloud AI with Edge AI functionalities can be realized based on various deployment configurations, providing versatility at the levels of development, deployment, and operation. As a prominent example, in cases where local intelligence, real-time performance, and strong data protection are required, machine learning deployments can be restricted to the edge of the network. On the other hand, when the AI application involves the processing of high volumes of data (Big Data), Cloud AI is preferred.</p><p dir="ltr">Obviously, there are many use cases where AI processing can be distributed between cloud and edge. In such cases, the distribution should consider performance, power efficiency, and security trade-offs to yield the best possible configuration. Combining the merits of cloud computing and edge computing into a single cloud/edge AI deployment is yet another significant advantage of Edge AI systems.</p><p dir="ltr"><em>This chapter was contributed to by&nbsp;</em><span style="background-color: transparent;"><em>BrainChip and&nbsp;</em></span><em>Synaptics.</em></p><p dir="ltr"><em>Gain a broader perspective on Edge AI by revisiting our&nbsp;</em><a href="https://www.wevolver.com/article/edge-ai-a-comprehensive-guide-to-its-engineering-principles-and-applications" rel="noopener noreferrer" target="_blank"><em>comprehensive guide</em></a><em>.<br></em></p><hr><h1 dir="ltr" id="isPasted">The 2023 Edge AI Technology Report</h1><p dir="ltr"><em>The guide to understanding the state of the art in hardware &amp; software in Edge AI.</em></p><p dir="ltr">Click through to read each of the report's chapters.&nbsp;</p><h2 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report/draft" rel="noopener noreferrer" target="_blank">Report Introduction</a></h2><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-i-overview-of-industries-application-use-cases" rel="noopener noreferrer" target="_blank">Chapter I: Overview of Industries and Application Use Cases</a></h3><h3 dir="ltr">Chapter II: Advantages of Edge AI</h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms" rel="noopener noreferrer" target="_blank">Chapter III: Edge AI Platforms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iv-hardware-software-selection" rel="noopener noreferrer" target="_blank">Chapter IV: Hardware and Software Selection</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml" rel="noopener noreferrer" target="_blank">Chapter V: Tiny ML</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms" rel="noopener noreferrer" target="_blank">Chapter VI: Edge AI Algorithms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities" rel="noopener noreferrer" target="_blank">Chapter VII: Sensing Modalities</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies" rel="noopener noreferrer" target="_blank">Chapter VIII: Case Studies</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ix-challenges-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter IX: Challenges of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter X: The Future of Edge AI and Conclusion</a></h3><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><hr><h3 id="isPasted">The report sponsors:</h3><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg2NTAwNjc4LTExICgzKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p><br></p>

<h1 id="isPasted">2023 Edge AI Technology Report. Chapter II: Advantages of Edge AI</h1><p dir="ltr" id="isPasted">Edge AI, empowered by the recent advancements in Artificial Intelligence, is driving significant shifts in today's technology landscape. By enabling computation near the data source, Edge AI enhances responsiveness, boosts security and privacy, promotes scalability, enables distributed computing, and improves cost efficiency.</p><p dir="ltr">Wevolver has partnered with industry experts, researchers, and tech providers to create a detailed report on the current state of Edge AI. This document covers its technical aspects, applications, challenges, and future trends. It merges practical and technical insights from industry professionals, helping readers understand and navigate the evolving Edge AI landscape.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg4NDgzNzk4NTQ3LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgyKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h2 dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></h2><h2 dir="ltr">From Cloud to Edge AI</h2><p dir="ltr">Following the introduction of cloud computing, there was a surge of interest in managing and analyzing data on the cloud. At that time, many enterprises considered the cloud a very appealing option for running data analytics and machine learning functions due to its scalability, capacity, and quality of service.&nbsp;</p><p dir="ltr">Over the years, however, many organizations voiced concerns against transferring and processing large volumes of data in the cloud. For instance, companies were reluctant to transfer sensitive corporate data to a remote data center they did not control. Others were concerned about the latency of several applications, as cloud data management and analytics incurred data transfer through a wide area network.&nbsp;</p><p dir="ltr">Motivated by the above-listed limitations of cloud data processing, many organizations have turned to edge computing to manage some of their data close to the data sources. Edge computing limits the amount of data to be transferred to the cloud for processing and analytics. Hence, it alleviates the latency, security, and privacy challenges of data analytics in the cloud.</p><h3 dir="ltr">Edge AI Advantages</h3><p dir="ltr">Leveraging the merits of edge computing, Edge AI enables organizations to develop practical AI applications. These applications exhibit exceptional performance, low latency, robust security, improved privacy control, and power efficiency. Edge AI also enables organizations to make optimal use of network, computing, and energy resources, improving the AI applications’ overall cost-effectiveness. Let’s explore the advantages of Edge AI in more detail:</p><ul><li dir="ltr"><p dir="ltr">Reduced Latency:&nbsp;Edge AI applications limit the amount of data transfers over wide area networks, as processing occurs close to the data sources rather than within the cloud. As such, data processing is faster, and the latency of the Edge AI application is reduced. Moreover, the transfer and execution of instructions from the AI applications to the field result in much lower latency. This is vital for several classes of low-latency AI applications, such as applications based on industrial robots and automated guided vehicles. Some other applications built around video in which the data had to be sent to the cloud are now being processed in the Edge not only because they can but because it’s vital to assess what’s going on in near real-time in situations like security cases.</p></li><li dir="ltr"><p dir="ltr">Real-Time Performance:&nbsp;The lower latency of Edge AI applications makes them suitable for implementing functionalities that require real-time performance. For instance, machine learning applications that detect events in real-time (e.g., defect detection in production lines, abnormal behavior detection in security applications) cannot tolerate delays associated with the transfer and processing of data from the edge to the cloud.</p></li><li dir="ltr"><p dir="ltr">Enhanced Security and Data Protection:&nbsp;Edge AI applications expose much fewer data outside the organizations that produce or own the data. This reduces their attack surface and minimizes the opportunities for malicious security attacks and data breaches. These are why Edge AI applications tend to be much more secure than their cloud counterparts.</p></li><li dir="ltr"><p dir="ltr">Improved Privacy Control: Many AI applications process sensitive data, such as data related to security, intellectual property, patients, and other forms of personal data. Edge AI deployments create a trusted data management environment for all these applications, providing more robust privacy control than conventional AI applications in the cloud. The reason is Edge AI applications limit the amount of data transferred or shared outside the organizations that produce or handle sensitive datasets. &nbsp;&nbsp;</p></li><li dir="ltr"><p dir="ltr">Power-Efficiency:&nbsp;Cloud data transfers and cloud data processing are extremely energy-savvy operations. Specifically, cloud I/O (Input/Output) functionalities are associated with significant carbon emissions. Most importantly, cloud AI is not green at all, as very large volumes of data are typically processed by GPUs (Graphical Processing Units) and TSUs (Tensor Processing Units). Edge AI alleviates the problematic environmental performance of cloud AI applications. It reduces the number of I/O operations and processes data within edge devices or edge data centers. Hence, it results in an improved overall CO2 footprint for AI applications.</p></li><li dir="ltr"><p dir="ltr">Cost-Effectiveness:&nbsp;Edge AI applications economize on network bandwidth and computing resources because they transmit and process much less data than cloud computing applications. Moreover, they consume less energy than cloud AI applications. As a result, Edge AI applications can be deployed and operated at a considerably lower cost than cloud AI deployments.</p></li><li dir="ltr"><p dir="ltr">On-Device Learning:&nbsp;Certain Edge AI applications can be executed within a single device, such as an IoT device or a microcontroller. This enables the development of powerful and intelligent devices, like System-on-Chip (SoC) devices. One of their main characteristics is that they can learn on the device, which is the foundation for giving machines intelligence capabilities that are hardly possibly based on cloud processing. BrainChip’s Nandan Nayampally remarks, “With on-device learning, features extracted from trained models can be extended to customize model behavior without having to resort to extremely expensive model retraining on the cloud. This substantially reduces costs.”</p></li><li dir="ltr"><p dir="ltr">New Capabilities Enabling Novel Applications:&nbsp;The integration of Edge AI systems within AI applications provides capabilities not available a few years ago. For instance, it enables a new class of real-time functionalities in transport, manufacturing, and other industrial settings. Hence, Edge AI is not only improving on cloud AI applications but also unlocking opportunities for innovative applications that were not previously possible.</p></li></ul><p dir="ltr">In a conversation with <a href="https://www.synaptics.com/" target="_blank">Synaptics</a>, we asked Shay Kamin Braun, Director of Product Marketing, about the most important advantages that Edge AI provides. He reiterated the advantages mentioned above and added some notable ones.</p><p dir="ltr">While “Edge AI provides a reduced latency in response time and a better UX without the network latency in the cloud,” Kamin Braun also highlighted privacy and battery life benefits. In terms of privacy, “PII (personally identifiable information) stays on the device and doesn’t go to the cloud. Only metadata goes to the cloud in most cases. This increases privacy,” explained Kamin Braun. “As for power efficiency, running AI at the edge helps you reduce the amount of data transmitted to the cloud over the network, which saves power. As a result, you get longer battery life or energy star compliance.”&nbsp;</p><p dir="ltr">Kamin Braun also emphasized the advantage of availability, which can be an issue for Cloud AI given its inherent reliance on the network. “Maybe you have no coverage for mobile devices, or maybe the Wi-Fi is not working. You cannot always rely on the cloud for processing. On the contrary, if your application is completely at the edge, then you’re not dependent on the network. Edge AI provides full availability.”</p><blockquote><p dir="ltr">“By doing the computation at the sensor with the context, you can optimize better than using a generic, aggregated solution. While the battery is a consideration, lower thermal footprint enables cheaper, cost-effective chip packaging and field use that really provides scale.”<br>- Nandan Nayampally, CMO, BrainChip</p></blockquote><h3 dir="ltr">Cloud/Edge Computing Continuum: A Powerful Combination</h3><p dir="ltr">With all the advantages that Edge AI has over cloud AI, many use cases still demand computationally intensive data processing within scalable cloud data centers. This is the case with deep learning applications, for example, which must be trained with large numbers of data points. In such cases, there is always a possibility to combine cloud AI and Edge AI paradigms to obtain the benefits of both worlds. Specifically, it is possible to perform a first round of local data processing at the edge to reduce the amount of uninteresting data and identify datasets that need processing in the cloud.&nbsp;</p><p dir="ltr">Local data processing is usually followed by a transfer of selected data to the cloud, where data points from different edge data sources are aggregated and processed to extract global insights. Furthermore, cloud AI applications may push information about these insights back to the edge data sources, which enhances the intelligence of local processing and field actuation functions. This two-level data processing exploits different deployment configurations of edge devices in the cloud/edge computing continuum.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODU5MjM1Njk2LUlNQUdFUyAyLTA3LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Hybrid solution of edge and cloud processing. Adapted from IEBMedia by Wevolver.</span></span></span></p><p dir="ltr">There are machine learning paradigms that inherently support this two-level data cloud/edge data processing approach. As a prominent example, federated machine learning employs local data processing to train machine-learning models across various edge computing nodes residing in different organizations. In this way, it produces a range of local machine-learning models. These are then fused in a central cloud location, which combines the local models into a more accurate global model.</p><p dir="ltr">This federated approach offers strong privacy and data protection characteristics, as source data need not leave the organizations where the edge nodes reside. Instead, an accurate global model is built in the cloud based on transferring and processing data about the local machine learning models. Therefore, the machine-learning models’ information is shared while the source data are not.</p><p dir="ltr">The combination of cloud AI with Edge AI functionalities can be realized based on various deployment configurations, providing versatility at the levels of development, deployment, and operation. As a prominent example, in cases where local intelligence, real-time performance, and strong data protection are required, machine learning deployments can be restricted to the edge of the network. On the other hand, when the AI application involves the processing of high volumes of data (Big Data), Cloud AI is preferred.</p><p dir="ltr">Obviously, there are many use cases where AI processing can be distributed between cloud and edge. In such cases, the distribution should consider performance, power efficiency, and security trade-offs to yield the best possible configuration. Combining the merits of cloud computing and edge computing into a single cloud/edge AI deployment is yet another significant advantage of Edge AI systems.</p><p dir="ltr"><em>This chapter was contributed to by&nbsp;</em><span style="background-color: transparent;"><em>BrainChip and&nbsp;</em></span><em>Synaptics.</em></p><p dir="ltr"><em>Gain a broader perspective on Edge AI by revisiting our&nbsp;</em><a href="https://www.wevolver.com/article/edge-ai-a-comprehensive-guide-to-its-engineering-principles-and-applications" rel="noopener noreferrer" target="_blank"><em>comprehensive guide</em></a><em>.<br></em></p><hr><h1 dir="ltr" id="isPasted">The 2023 Edge AI Technology Report</h1><p dir="ltr"><em>The guide to understanding the state of the art in hardware &amp; software in Edge AI.</em></p><p dir="ltr">Click through to read each of the report's chapters.&nbsp;</p><h2 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report/draft" rel="noopener noreferrer" target="_blank">Report Introduction</a></h2><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-i-overview-of-industries-application-use-cases" rel="noopener noreferrer" target="_blank">Chapter I: Overview of Industries and Application Use Cases</a></h3><h3 dir="ltr">Chapter II: Advantages of Edge AI</h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms" rel="noopener noreferrer" target="_blank">Chapter III: Edge AI Platforms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iv-hardware-software-selection" rel="noopener noreferrer" target="_blank">Chapter IV: Hardware and Software Selection</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml" rel="noopener noreferrer" target="_blank">Chapter V: Tiny ML</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms" rel="noopener noreferrer" target="_blank">Chapter VI: Edge AI Algorithms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities" rel="noopener noreferrer" target="_blank">Chapter VII: Sensing Modalities</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies" rel="noopener noreferrer" target="_blank">Chapter VIII: Case Studies</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ix-challenges-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter IX: Challenges of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter X: The Future of Edge AI and Conclusion</a></h3><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><hr><h3 id="isPasted">The report sponsors:</h3><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg2NTAwNjc4LTExICgzKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p><br></p>

Benefits, Utilities, and Hybrid Deployment of Edge AI.

2023 Edge AI Technology Report. Chapter II: Advantages of Edge AI

How Edge AI is Transforming Industries and Application Areas.

2023 Edge AI Technology Report. Chapter I: Overview of Industries & Application Use Cases

<h1 id="isPasted">2023 Edge AI Technology Report.&nbsp;</h1><p dir="ltr">Edge AI, empowered by the recent advancements in Artificial Intelligence, is driving significant shifts in today&#39;s technology landscape. By enabling computation near the data source, Edge AI enhances responsiveness, boosts security and privacy, promotes scalability, enables distributed computing, and improves cost efficiency.</p><p dir="ltr">Wevolver has partnered with industry experts, researchers, and tech providers to create a detailed report on the current state of Edge AI. This document covers its technical aspects, applications, challenges, and future trends. It merges practical and technical insights from industry professionals, helping readers understand and navigate the evolving Edge AI landscape.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg4NTQ4Nzc1NDA1LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgyKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable fr-active" contenteditable="false" draggable="true"><iframe src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block; margin-left: auto; margin-right: auto; max-width: 100%; width: 540px; height: 335px;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><hr><h1 id="isPasted">Report Introduction</h1><p dir="ltr">The advent of Artificial Intelligence (AI) over recent years has truly revolutionized our industries and personal lives, offering unprecedented opportunities and capabilities. However, while cloud-based processing and cloud AI took off in the past decade, we have come to experience issues such as latency, bandwidth constraints, and security and privacy concerns, to name a few. That is where the emergence of Edge AI became extremely valuable and transformed the AI landscape.</p><p dir="ltr">Edge AI represents a paradigm shift in AI deployment, bringing computational power closer to the data source. It allows for on-device data processing and enables real-time, context-aware decision-making. Instead of relying on cloud-based processing, Edge AI utilizes edge devices such as sensors, cameras, smartphones, and other compact devices to perform AI computations on the device itself. Such an approach offers multitudes of advantages, including reduced latency, improved bandwidth efficiency, enhanced data privacy, and increased reliability in scenarios with limited or intermittent connectivity.</p><blockquote><p dir="ltr">&ldquo;Even with ubiquitous 5G, connectivity to the cloud isn&rsquo;t guaranteed, and bandwidth isn&rsquo;t assured in every case. The move to AIoT increasingly needs that intelligence and computational power at the edge.&rdquo;<br id="isPasted">- Nandan Nayampally, CMO, Brainchip</p></blockquote><p dir="ltr" id="isPasted">While Cloud AI predominantly performs data processing and analysis in remote servers, Edge AI focuses on enabling AI capabilities directly on the devices. The key distinction here lies in the processing location and the nature of the data being processed. Cloud AI is suitable for processing-intensive applications that can tolerate latency, while Edge AI excels in time-sensitive scenarios where real-time processing is essential. By deploying AI models directly on edge devices, Edge AI minimizes the reliance on cloud connectivity, enabling localized decision-making and response.</p><p dir="ltr">The Edge encompasses the entire spectrum from data centers to IoT endpoints. This includes the data center edge, network edge, embedded edge, and on-prem edge, each with its own use cases. The compute requirements essentially determine where a particular application falls on the spectrum, ranging from data-center edge solutions to small sensors embedded in devices like automobile tires. Vibration-related applications would be positioned towards one end of the spectrum, often implemented on microcontrollers, while more complex video analysis tasks might be closer to the other end, sometimes on more powerful microprocessors.</p><blockquote><p dir="ltr" id="isPasted">&ldquo;Applications are gradually moving towards the edge as these edge platforms enhance their compute power.&rdquo;&nbsp;<br>- <span style="background-color: transparent;">Ian Bratt, Fellow and Senior Director of Technology, Arm</span></p></blockquote><p dir="ltr" id="isPasted">When it comes to Edge AI, the focus is primarily on sensing systems. This includes camera-based systems, audio sensors, and applications like traffic monitoring in smart cities. Edge AI essentially functions as an extensive sensory system, continuously monitoring and interpreting events in the world. In an integrated-technology approach, the collected information can then be sent to the cloud for further processing.</p><p dir="ltr">Edge AI shines in applications where rapid decision-making and immediate response to time-sensitive data are required. For instance, in autonomous driving, Edge AI empowers vehicles to process sensor data onboard and make split-second decisions to ensure safe navigation. Similarly, in healthcare, Edge AI enables real-time patient monitoring, detecting anomalies, and facilitating immediate interventions. The ability to process and analyze data locally empowers healthcare professionals to deliver timely and life-saving interventions.</p><p dir="ltr">Edge AI application areas can be distinguished based on specific requirements such as power sensitivity, size limitations, weight constraints, and heat dissipation. Power sensitivity is a crucial consideration, as edge devices are often low-power devices used in smartphones, wearables, or Internet of Things (IoT) systems. AI models deployed on these devices must be optimized for efficient power consumption to preserve battery life and prolong operational duration.</p><p dir="ltr">Size limitations and weight constraints also play quite a significant role in distinguishing Edge AI application areas. Edge devices are typically compact and portable, making it essential for AI models to be lightweight and space-efficient. This consideration is particularly relevant upon integrating edge devices into drones, robotics, or wearable devices, where size and weight directly impact performance and usability.</p><p dir="ltr">Nevertheless, edge computing presents significant advantages that weren&rsquo;t achievable beforehand. Owning the data, for instance, provides a high level of security, as there is no need for the data to be sent to the cloud, thus mitigating the increasing cybersecurity risks. Edge computing also reduces latency and power usage due to less communication back and forth with the cloud, which is particularly important for constrained devices running on low power. And the advantages don&rsquo;t stop there, as we are seeing more and more interesting developments in real-time performance and decision-making, improved privacy control, and on-device learning, enabling intelligent devices to operate autonomously and adaptively without relying on constant cloud interaction.</p><blockquote><p dir="ltr" id="isPasted">&ldquo;The recent surge in AI has been fueled by a harmonious interplay between cutting-edge algorithms and advanced hardware. As we move forward, the symbiosis of these two elements will become even more crucial, particularly for Edge AI.&rdquo;&nbsp;<br>- Dr. Bram Verhoef, Head of Machine Learning at Axelera AI</p></blockquote><p dir="ltr" id="isPasted">Edge AI holds immense significance in the current and future technology landscape. With decentralized AI processing, improved responsiveness, enhanced privacy and security, cost-efficiency, scalability, and distributed computing, Edge AI is revolutionizing our world as we speak. And with the rapid developments happening constantly, it may be difficult to follow all the new advancements in the field.&nbsp;</p><p dir="ltr" id="isPasted">That is why Wevolver has collaborated with several industry experts, researchers, professors, and leading companies to create a comprehensive report on the current state of Edge AI, exploring its history, cutting-edge applications, and future developments. This report will provide you with practical and technical knowledge to help you understand and navigate the evolving landscape of Edge AI.</p><p dir="ltr">This report would not have been possible without the esteemed contributions and sponsorship of Alif Semiconductor, Arduino, Arm, Axelera AI, BrainChip, Edge Impulse, GreenWaves Technologies, Sparkfun, ST, and Synaptics. Their commitment to objectively sharing knowledge and insights to help inspire innovation and technological evolution aligns perfectly with what Wevolver does and the impact it aims to achieve.</p><p dir="ltr">As the world becomes increasingly connected and data-driven, Edge AI is emerging as a vital technology at the core of this transformation, and we hope this comprehensive report provides all the knowledge and inspiration you need to participate in this technological journey.</p><p dir="ltr"><strong>Samir Jaber</strong><br>Editor-in-Chief</p><p dir="ltr"><em>This text was contributed to by engineers from&nbsp;</em><span style="background-color: transparent;"><em>Arm,&nbsp;</em></span><span style="background-color: transparent;"><em>Axelera AI, and&nbsp;</em></span><span style="background-color: transparent;"><em>BrainChip.</em></span></p><p dir="ltr"><span style="background-color: transparent;"><em><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg2MDk4NTA0LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 302px;" class="fr-fic fr-dib"></em></span></p><p><em><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></em></p><hr><h2><span style="background-color: transparent;">The 2023 Edge AI Technology Report</span></h2><p dir="ltr"><em>The guide to understanding the state of the art in hardware &amp; software in Edge AI.</em></p><p dir="ltr">Click through to read each of the report&#39;s chapters.&nbsp;</p><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-i-overview-of-industries-application-use-cases/draft" rel="noopener noreferrer" target="_blank">Chapter I: Overview of Industries and Application Use Cases</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ii-advantages-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter II: Advantages of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms" rel="noopener noreferrer" target="_blank">Chapter III: Edge AI Platforms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iv-hardware-software-selection" rel="noopener noreferrer" target="_blank">Chapter IV: Hardware and Software Selection</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml/draft" rel="noopener noreferrer" target="_blank">Chapter V: Tiny ML</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms" rel="noopener noreferrer" target="_blank">Chapter VI: Edge AI Algorithms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities" rel="noopener noreferrer" target="_blank">Chapter VII: Sensing Modalities</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies" rel="noopener noreferrer" target="_blank">Chapter VIII: Case Studies</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ix-challenges-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter IX: Challenges of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter X: The Future of Edge AI and Conclusion</a></h3><hr><h2 id="isPasted"><strong>How this report came to be: A collaborative effort</strong></h2><p>This report is the culmination of collaborative efforts among a diverse team of contributors. Initial input was graciously provided by our sponsors and partner, who shared their expert opinions and insightful case studies, creating a strong technical foundation. A dedicated team of writers further expanded these texts under the careful guidance of our Editor-in-Chief. Thank you to our report sponsors and report partner, the tinyML Foundation. &nbsp;</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODgxMTAwNjI0LTExICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3>About the writers</h3><h3 dir="ltr">Samir Jaber, Editor-in-chief.&nbsp;</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/samir.jaber" rel="noopener noreferrer" target="_blank">Samir Jaber</a> is an SEO &amp; Content Specialist with a background in engineering, nanotechnology, and scientific research. Samir has comprehensive experience working with major engineering and technology companies as a writer, editor, and digital marketing consultant. He is an expert in content management and strategy, particularly in the engineering and tech fields. He is a featured author in 30+ industrial magazines with a focus on IoT, nanotechnology, materials science, engineering, and sustainability. Samir is also an award-winning engineering researcher in the fields of nanofabrication and microfluidics.</p><h3 dir="ltr">Dr. John Soldatos</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/john.soldatos" rel="noopener noreferrer" target="_blank">John Soldatos</a> holds a Ph.D. in Electrical &amp; Computer Engineering from the National Technical University of Athens (2000) and is currently an Honorary Research Fellow at the University of Glasgow, UK (2014-present). He was Associate Professor and Head of the Internet of Things (IoT) Group at the Athens Information Technology (AIT), Greece (2006&ndash;2019), and Adjunct Professor at the Carnegie Mellon University, Pittsburgh, PA (2007&ndash;2010). He has significant experience working closely with large multi-national industries (e.g., IBM, INTRACOM, INTRASOFT International) as an R&amp;D consultant and delivery specialist while being a scientific advisor to various high-tech startup enterprises. Dr. Soldatos is an expert in Internet-of-Things (IoT) and Artificial Intelligence (AI) technologies and applications, including IoT/AI applications in smart cities, finance (Finance 4.0), and industry (Industry 4.0).&nbsp;</p><h3 dir="ltr">Dr. Miroslav Milovanovic</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/miroslav.milovanovic" rel="noopener noreferrer" target="_blank">Miroslav Milovanovic</a> has a Ph.D. in Computer Science and Electrotechnics and works as an assistant professor at the Faculty of Electronic Engineering. He teaches courses strongly connected with Data Science, the Industrial Internet of Things, Modern Control of Industrial Processes, and Intelligent control.&nbsp;</p><p dir="ltr">Additionally, Dr. Milovanovic is the Chief of the Laboratory for Intelligent Control in the Control Systems Department. His instructional expertise encompasses courses strongly connected with Data Science, the Industrial Internet of Things, Deep Learning, Machine Learning in Python, Modern Control of Industrial Processes, and Intelligent Control. By imparting practical knowledge in these areas, he empowers students to excel in their chosen fields. With over 45 published scientific papers, his research primarily revolves around Data Science, Deep Learning, and their practical applications.</p><h3 dir="ltr">Lydia Husser</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/lydia.husser" rel="noopener noreferrer" target="_blank">Lydia Husser</a> is a technical writer in robotics and a freelance writer captivated by innovative science and engineering. She holds a degree in Computer Science and credentials in electronics technology. Her background includes work in robotics, communications hardware, and developer documentation.&nbsp;</p><p dir="ltr">The cover image of the report was created using generative AI tools. Credit to Muthali Ganesh.</p>

<article id="isPasted"><p dir="ltr"><em>We are thrilled to share the&nbsp;</em><a href="https://www.wevolver.com/article/2024-state-of-edge-ai-report"><em>2024 State of Edge AI Report</em></a><em>. This is Wevolver's second in-depth report on Edge AI, building on our 2023 report which focused on the technological foundations. While our previous report explored the technical landscape, this year's edition takes a deeper look at real-world applications and industry-specific implementations.<br></em></p></article><hr><h1 id="isPasted">2023 Edge AI Technology Report.&nbsp;</h1><p dir="ltr">Edge AI, empowered by the recent advancements in Artificial Intelligence, is driving significant shifts in today's technology landscape. By enabling computation near the data source, Edge AI enhances responsiveness, boosts security and privacy, promotes scalability, enables distributed computing, and improves cost efficiency.</p><p dir="ltr">Wevolver has partnered with industry experts, researchers, and tech providers to create a detailed report on the current state of Edge AI. This document covers its technical aspects, applications, challenges, and future trends. It merges practical and technical insights from industry professionals, helping readers understand and navigate the evolving Edge AI landscape.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg4NTQ4Nzc1NDA1LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgyKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block; margin-left: auto; margin-right: auto; max-width: 100%; width: 540px; height: 335px;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><h1 id="isPasted">Report Introduction</h1><p dir="ltr">The advent of Artificial Intelligence (AI) over recent years has truly revolutionized our industries and personal lives, offering unprecedented opportunities and capabilities. However, while cloud-based processing and cloud AI took off in the past decade, we have come to experience issues such as latency, bandwidth constraints, and security and privacy concerns, to name a few. That is where the emergence of Edge AI became extremely valuable and transformed the AI landscape.</p><p dir="ltr">Edge AI represents a paradigm shift in AI deployment, <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ii-advantages-of-edge-ai">bringing computational power closer to the data source</a>. It allows for on-device data processing and enables real-time, context-aware decision-making. Instead of relying on cloud-based processing, Edge AI utilizes <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities">edge devices such as sensors, cameras, smartphones</a>, and other compact devices to perform AI computations on the device itself. Such an approach offers multitudes of advantages, including reduced latency, improved bandwidth efficiency, enhanced data privacy, and increased reliability in scenarios with limited or intermittent connectivity.</p><blockquote><p dir="ltr">“Even with ubiquitous 5G, connectivity to the cloud isn’t guaranteed, and bandwidth isn’t assured in every case. The move to AIoT increasingly needs that intelligence and computational power at the edge.”<br id="isPasted">- Nandan Nayampally, CMO, Brainchip</p></blockquote><p dir="ltr" id="isPasted">While Cloud AI predominantly performs data processing and analysis in remote servers, <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms">Edge AI focuses on enabling AI capabilities directly on the devices</a>. The key distinction here lies in the processing location and the nature of the data being processed. Cloud AI is suitable for processing-intensive applications that can tolerate latency, while Edge AI excels in time-sensitive scenarios where real-time processing is essential. By deploying AI models directly on edge devices, Edge AI minimizes the reliance on cloud connectivity, enabling localized decision-making and response.</p><p dir="ltr">The Edge encompasses the entire spectrum from data centers to IoT endpoints. This includes the data center edge, network edge, embedded edge, and on-prem edge, each with its own use cases. The compute requirements essentially determine where a particular application falls on the spectrum, ranging from data-center edge solutions to small sensors embedded in devices like automobile tires. Vibration-related applications would be positioned towards one end of the spectrum, often implemented on microcontrollers, while more complex video analysis tasks might be closer to the other end, sometimes on more powerful microprocessors.</p><blockquote><p dir="ltr" id="isPasted">“Applications are gradually moving towards the edge as these edge platforms enhance their compute power.”&nbsp;<br>- <span style="background-color: transparent;">Ian Bratt, Fellow and Senior Director of Technology, Arm</span></p></blockquote><p dir="ltr" id="isPasted">When it comes to Edge AI, the focus is primarily on sensing systems. This includes camera-based systems, audio sensors, and applications like traffic monitoring in smart cities. Edge AI essentially functions as an extensive sensory system, continuously monitoring and interpreting events in the world. In an integrated-technology approach, the collected information can then be sent to the cloud for further processing.</p><p dir="ltr">Edge AI shines in applications where rapid decision-making and immediate response to time-sensitive data are required. For instance, in autonomous driving, Edge AI empowers vehicles to process sensor data onboard and make split-second decisions to ensure safe navigation. Similarly, in healthcare, Edge AI enables real-time patient monitoring, detecting anomalies, and facilitating immediate interventions. The ability to process and analyze data locally empowers healthcare professionals to deliver timely and life-saving interventions.</p><p dir="ltr">Edge AI application areas can be distinguished based on specific requirements such as power sensitivity, size limitations, weight constraints, and heat dissipation. Power sensitivity is a crucial consideration, as edge devices are often low-power devices used in smartphones, wearables, or Internet of Things (IoT) systems. AI models deployed on these devices must be optimized for efficient power consumption to preserve battery life and prolong operational duration.</p><p dir="ltr">Size limitations and weight constraints also play quite a significant role in distinguishing Edge AI application areas. Edge devices are typically compact and portable, making it essential for AI models to be lightweight and space-efficient. This consideration is particularly relevant upon integrating edge devices into drones, robotics, or wearable devices, where size and weight directly impact performance and usability.</p><p dir="ltr">Nevertheless, edge computing presents <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies">significant advantages that weren’t achievable beforehand.</a> Owning the data, for instance, provides a high level of security, as there is no need for the data to be sent to the cloud, thus mitigating the increasing cybersecurity risks. Edge computing also reduces latency and power usage due to less communication back and forth with the cloud, which is particularly important for constrained devices running on low power. And the advantages don’t stop there, as we are seeing more and more interesting developments in <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ii-advantages-of-edge-ai">real-time performance and decision-making</a>, improved privacy control, and <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms">on-device learning</a>, enabling intelligent devices to operate autonomously and adaptively without relying on constant cloud interaction.</p><blockquote><p dir="ltr" id="isPasted">“The recent surge in AI has been fueled by a harmonious interplay between cutting-edge algorithms and advanced hardware. As we move forward, the symbiosis of these two elements will become even more crucial, particularly for Edge AI.”&nbsp;<br>- Dr. Bram Verhoef, Head of Machine Learning at Axelera AI</p></blockquote><p dir="ltr" id="isPasted">Edge AI holds immense significance in the current and future technology landscape. With decentralized AI processing, improved responsiveness, enhanced privacy and security, cost-efficiency, scalability, and distributed computing, Edge AI is revolutionizing our world as we speak. And with the rapid developments happening constantly, it may be difficult to follow all the new advancements in the field.&nbsp;</p><p dir="ltr" id="isPasted">That is why Wevolver has collaborated with several industry experts, researchers, professors, and leading companies to create a comprehensive report on the current state of Edge AI, exploring its history, cutting-edge applications, and <a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai">future developments</a>. This report will provide you with practical and technical knowledge to help you understand and navigate the evolving landscape of Edge AI.</p><p dir="ltr">This report would not have been possible without the esteemed contributions and sponsorship of Alif Semiconductor, Arduino, Arm, Axelera AI, BrainChip, Edge Impulse, GreenWaves Technologies, Sparkfun, ST, and Synaptics. Their commitment to objectively sharing knowledge and insights to help inspire innovation and technological evolution aligns perfectly with what Wevolver does and the impact it aims to achieve.</p><p dir="ltr">As the world becomes increasingly connected and data-driven, Edge AI is emerging as a vital technology at the core of this transformation, and we hope this comprehensive report provides all the knowledge and inspiration you need to participate in this technological journey.</p><p dir="ltr"><strong>Samir Jaber</strong><br>Editor-in-Chief</p><p dir="ltr"><em>This text was contributed to by engineers from&nbsp;</em><span style="background-color: transparent;"><em>Arm,&nbsp;</em></span><span style="background-color: transparent;"><em>Axelera AI, and&nbsp;</em></span><span style="background-color: transparent;"><em>BrainChip.</em></span></p><p dir="ltr"><span style="background-color: transparent;"><em><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODg2MDk4NTA0LUVkZ2UgQUkgRG93bmxvYWQgYmFubmVyICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 302px;" class="fr-fic fr-dib"></em></span></p><p><em><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="540" height="335" src="https://c8056756.sibforms.com/serve/MUIFABFML2jk7knWjihwfmaWcFVV6GvM3WWUP6qryWvjht4DWOrppyQ7-VLwYu9wbjkQCCMxaWS9vmneqio8zYAai1OKHh0-OA61OuFt5xdYgRxECqvgNTAnjjob8456fOSNTOOCe8YLBqs4fiLh0kTJSsBgWCDDvnFUMZSuafNiO9Cbhh5H7fBa4doE8a4CzpRXWMOtAOLUtRmq" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></em></p><hr><h2><span style="background-color: transparent;">The 2023 Edge AI Technology Report</span></h2><p dir="ltr"><em>The guide to understanding the state of the art in hardware &amp; software in Edge AI.</em></p><p dir="ltr">Click through to read each of the report's chapters.&nbsp;</p><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-i-overview-of-industries-application-use-cases/draft" rel="noopener noreferrer" target="_blank">Chapter I: Overview of Industries and Application Use Cases</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ii-advantages-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter II: Advantages of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iii-edge-ai-platforms" rel="noopener noreferrer" target="_blank">Chapter III: Edge AI Platforms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-iv-hardware-software-selection" rel="noopener noreferrer" target="_blank">Chapter IV: Hardware and Software Selection</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml/draft" rel="noopener noreferrer" target="_blank">Chapter V: Tiny ML</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vi-edge-ai-algorithms" rel="noopener noreferrer" target="_blank">Chapter VI: Edge AI Algorithms</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-vii-sensing-modalities" rel="noopener noreferrer" target="_blank">Chapter VII: Sensing Modalities</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-viii-case-studies" rel="noopener noreferrer" target="_blank">Chapter VIII: Case Studies</a></h3><h3 dir="ltr"><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-ix-challenges-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter IX: Challenges of Edge AI</a></h3><h3><a href="https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-x-future-of-edge-ai" rel="noopener noreferrer" target="_blank">Chapter X: The Future of Edge AI and Conclusion</a></h3><hr><h2 id="isPasted"><strong>How this report came to be: A collaborative effort</strong></h2><p>This report is the culmination of collaborative efforts among a diverse team of contributors. Initial input was graciously provided by our sponsors and partner, who shared their expert opinions and insightful case studies, creating a strong technical foundation. A dedicated team of writers further expanded these texts under the careful guidance of our Editor-in-Chief. Thank you to our report sponsors and report partner, the tinyML Foundation. &nbsp;</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjg3ODgxMTAwNjI0LTExICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3>About the writers</h3><h3 dir="ltr">Samir Jaber, Editor-in-chief.&nbsp;</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/samir.jaber" rel="noopener noreferrer" target="_blank">Samir Jaber</a> is an SEO &amp; Content Specialist with a background in engineering, nanotechnology, and scientific research. Samir has comprehensive experience working with major engineering and technology companies as a writer, editor, and digital marketing consultant. He is an expert in content management and strategy, particularly in the engineering and tech fields. He is a featured author in 30+ industrial magazines with a focus on IoT, nanotechnology, materials science, engineering, and sustainability. Samir is also an award-winning engineering researcher in the fields of nanofabrication and microfluidics.</p><h3 dir="ltr">Dr. John Soldatos</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/john.soldatos" rel="noopener noreferrer" target="_blank">John Soldatos</a> holds a Ph.D. in Electrical &amp; Computer Engineering from the National Technical University of Athens (2000) and is currently an Honorary Research Fellow at the University of Glasgow, UK (2014-present). He was Associate Professor and Head of the Internet of Things (IoT) Group at the Athens Information Technology (AIT), Greece (2006–2019), and Adjunct Professor at the Carnegie Mellon University, Pittsburgh, PA (2007–2010). He has significant experience working closely with large multi-national industries (e.g., IBM, INTRACOM, INTRASOFT International) as an R&amp;D consultant and delivery specialist while being a scientific advisor to various high-tech startup enterprises. Dr. Soldatos is an expert in Internet-of-Things (IoT) and Artificial Intelligence (AI) technologies and applications, including IoT/AI applications in smart cities, finance (Finance 4.0), and industry (Industry 4.0).&nbsp;</p><h3 dir="ltr">Dr. Miroslav Milovanovic</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/miroslav.milovanovic" rel="noopener noreferrer" target="_blank">Miroslav Milovanovic</a> has a Ph.D. in Computer Science and Electrotechnics and works as an assistant professor at the Faculty of Electronic Engineering. He teaches courses strongly connected with Data Science, the Industrial Internet of Things, Modern Control of Industrial Processes, and Intelligent control.&nbsp;</p><p dir="ltr">Additionally, Dr. Milovanovic is the Chief of the Laboratory for Intelligent Control in the Control Systems Department. His instructional expertise encompasses courses strongly connected with Data Science, the Industrial Internet of Things, Deep Learning, Machine Learning in Python, Modern Control of Industrial Processes, and Intelligent Control. By imparting practical knowledge in these areas, he empowers students to excel in their chosen fields. With over 45 published scientific papers, his research primarily revolves around Data Science, Deep Learning, and their practical applications.</p><h3 dir="ltr">Lydia Husser</h3><p dir="ltr"><a href="https://www.wevolver.com/profile/lydia.husser" rel="noopener noreferrer" target="_blank">Lydia Husser</a> is a technical writer in robotics and a freelance writer captivated by innovative science and engineering. She holds a degree in Computer Science and credentials in electronics technology. Her background includes work in robotics, communications hardware, and developer documentation.&nbsp;</p><p dir="ltr">The cover image of the report was created using generative AI tools. Credit to Muthali Ganesh.</p><p dir="ltr"><br></p>

The guide to understanding the state of the art in hardware & software in Edge AI.

2023 Edge AI Technology Report

<p id="isPasted">If you&rsquo;re developing an IoT device, odds are that you want it to do some valuable computations to solve an important problem. Maybe you want to <a href="https://www.mistywest.com/posts/5-reasons-to-use-an-iot-framework/" target="_blank">deploy sensors in remote locations</a>, develop a device that can perform data analytics to monitor a renewable energy source, or build a medical device that can use computer vision to detect early signs of an illness.</p><p>Whatever you&rsquo;re building, at some point you may start to wonder: should your device perform these important computations in the cloud or at the edge? Choosing between computing on the cloud or on the edge is a decision that can impact things like your device&rsquo;s cost or efficiency &ndash; and no one wants to make the wrong decision initially and spend time and money later down the line to pivot to the correct one.&nbsp;</p><p>We asked some of our experienced engineers what they think someone needs to consider when choosing between computing in the cloud or on the edge.</p><p>&nbsp;</p><h2>What is cloud computing?</h2><p>The &ldquo;cloud&rdquo; refers to the collection of servers that can be accessed over the internet &ndash; popular <a href="https://www.mistywest.com/posts/azure-vs-aws-which-iot-framework-is-right-for-you/" target="_blank">cloud providers include Amazon Web Services, Microsoft Azure</a>, and Google Cloud.&nbsp;</p><p>These servers can provide on-demand computing resources to store and process data. You can think of the cloud as a centralized location for your files and programs, and you can connect any device to the cloud to access them. Services like Dropbox or Google Drive are some of the many cloud-based services out there.</p><p>&nbsp;</p><p><img data-fr-image-pasted="true" src="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-03.png" alt="" width="100%" height="auto" srcset="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-03.png 1360w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-03-300x155.png 300w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-03-1024x528.png 1024w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-03-768x396.png 768w" sizes="(max-width: 1360px) 100vw, 1360px" class="fr-fic fr-dii"></p><p>&nbsp;</p><p><em>Cloud computing</em> describes the idea of performing computations in the cloud. These computations can include data analysis and visualization, computer vision and machine learning. An example of cloud computing in action is when your average smart home speaker sends your audio input to the cloud where it is interpreted by algorithms and sends back a response. &nbsp;&nbsp;</p><p>&nbsp;</p><h2>What is edge computing?</h2><p><a href="https://blogs.nvidia.com/blog/2022/01/05/difference-between-cloud-and-edge-computing/#:~:text=The%20main%20difference%20between%20cloud,deploy%20them%20at%20the%20edge" target="_blank">The edge describes the &ldquo;edge&rdquo; of the network.&nbsp;</a>It includes devices that are an entry/exit point to the cloud, but are not part of the cloud itself. For example, a server in a data centre is part of the cloud; your smartphone and router that connects to that server is part of the edge.&nbsp;</p><p>&nbsp;</p><p><img data-fr-image-pasted="true" src="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-02.png" alt="" width="100%" height="auto" srcset="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-02.png 1360w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-02-300x155.png 300w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-02-1024x528.png 1024w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-02-768x396.png 768w" sizes="(max-width: 1360px) 100vw, 1360px" class="fr-fic fr-dii"></p><p>&nbsp;</p><p><em>Edge computing</em> describes the idea of performing computations on the edge. This way, the processing is done closer to or at the location where the data is collected or acted upon.</p><p>An example of an edge computing process is object detection on an autonomous vehicle. The vehicle processes data from its sensors and uses the results to avoid obstacles. Unlike your smart home speaker, the data it collects is processed locally, rather than sending it to the cloud.&nbsp;</p><p>&nbsp;</p><h1>Key Considerations</h1><p>There are a couple of key questions to consider when choosing between the edge and cloud computing.</p><p>&nbsp;</p><h2>What is the quality of your device&rsquo;s network?</h2><p>Performing computations on the cloud can work well when you have high-bandwidth, low-latency, and stable connection to the internet, as you will be needing to send your data back and forth between cloud servers and your device. If your device is intended to be used, for instance, in a home or office with a good internet connection, this back and forth can be done relatively seamlessly.&nbsp;</p><p>In most cases, if the computation is done on the edge, it won&rsquo;t be affected by poor or lost internet connection in a remote location; the processing can continue since it is not computed in the cloud. You wouldn&rsquo;t want your vehicle&rsquo;s object detection to stop working while on a long road trip; that&rsquo;s one of the reasons why autonomous vehicles frequently perform computations like object detection on the edge.&nbsp;</p><p>&nbsp;</p><h2>How <em>quick</em> and how <em>often</em> does your data need to be processed?</h2><p>Edge computing can be ideal in cases where your customer needs response times from your device to be faster than what can be achieved with a decent network connection, such as monitoring vital components of a system. The latency from travel-time between the device and the cloud can be reduced or eliminated completely. As a result, the data can be processed right away. If the data processing itself is quick, you could achieve real-time responses from your device.&nbsp;</p><p>Cloud computing is beneficial when device use is intermittent. Smart homes devices are a good example of this again, where running computations in the cloud lets you share the same computing resources between multiple customers. This reduces costs by avoiding the need to provision your device with upgraded hardware to run the data processing.&nbsp;</p><p>&nbsp;</p><p><img data-fr-image-pasted="true" src="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-05.png" alt="" width="100%" height="auto" srcset="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-05.png 1360w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-05-300x155.png 300w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-05-1024x528.png 1024w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-05-768x396.png 768w" sizes="(max-width: 1360px) 100vw, 1360px" class="fr-fic fr-dii"></p><p>&nbsp;</p><h2>What part of your data is important to you?</h2><p>Computing on the edge is useful if you only care about the result of your data <em>after</em> it has been processed. You can send only what is important to store in the long-term in the cloud, and doing so will allow you to reduce the cost of storing and processing data in the cloud. For example, if you are creating a traffic surveillance device that needs to report levels of congestion on a road, you could preprocess the videos on the edge&ndash;Instead of running hours of raw video in the cloud&ndash;and only send images or clips of the traffic when traffic is present.</p><p>It&rsquo;s possible you need to keep the data to <a href="https://www.mistywest.com/posts/improving-dataset-creation-for-machine-learning/" target="_blank">build out your machine learning dataset</a>, or you plan on analyzing the raw data in other ways in the future. If you are already sending your raw data to the cloud, it may be ideal to perform calculations in the cloud as well.</p><p>&nbsp;</p><h2>What are your device&rsquo;s power and size limitations?</h2><p>If you expect your device will be restricted in power and size, given that it has a good network connection, sending the computing work to be done on the cloud will allow your device to remain small and low-power. Google Home and Amazon Alexa, for example, will capture the audio and send it to the cloud for processing, allowing complex computations to be run on the audio that would not be possible to run on the small computers inside the devices themselves.</p><p>&nbsp;</p><p><img data-fr-image-pasted="true" src="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-04.png" alt="" width="100%" height="auto" srcset="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-04.png 1360w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-04-300x155.png 300w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-04-1024x528.png 1024w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest_Edge_Vs_Cloud-04-768x396.png 768w" sizes="(max-width: 1360px) 100vw, 1360px" class="fr-fic fr-dii"></p><p>&nbsp;</p><h2>Is your data processing model your Intellectual Property?</h2><p>If you are making a consumer device and the method you are using to process data is part of your Intellectual Property (IP), you may need to consider how you plan to protect it. Putting your IP on your device without a robust security plan can leave it vulnerable to hacks. If you don&rsquo;t have the knowledge or resources to secure your IP on the edge, it may be best to leave it on the cloud, which already has security measures in place.</p><p>&nbsp;</p><p><a data-rel="lightbox-image-0" data-rl_caption="" data-rl_title="" href="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01.jpg" target="_blank" title=""><img data-fr-image-pasted="true" src="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01.jpg" alt="" width="100%" height="auto" srcset="https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01.jpg 1801w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01-275x300.jpg 275w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01-938x1024.jpg 938w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01-768x839.jpg 768w, https://www.mistywest.com/wp-content/uploads/2022/05/MistyWest-Cloud-Vs-Edge-FlowChart-01-1406x1536.jpg 1406w" sizes="(max-width: 1801px) 100vw, 1801px" class="fr-fic fr-dii"></a></p><p>&nbsp;</p><h2>Some final considerations for choosing between edge and cloud computing</h2><p>There are quite a few things to consider when choosing between computing on the edge or in the cloud. In complex problems, you may benefit from using a combination of both, by leaving some parts of your processing on the edge and the rest on the cloud.</p><p>We created the above flow chart to help guide you through which path you should take, but there may always be more to consider for your device, depending on how complex your use case is.&nbsp;</p><p><br></p><p id="isPasted"><em>This article was originally written by Amanda Luu, former Product Design Engineer Coop at MistyWest,&nbsp;</em><a href="https://www.mistywest.com/posts/edge-vs-cloud-computing-which-solution-is-better-for-your-connected-device/" rel="noopener noreferrer" target="_blank"><em>for the MistyWest blog</em></a><em>.</em></p>

Choosing between computing on the cloud or on the edge is a decision that can impact things like your device’s cost or efficiency. Here’s how you can ensure you make the right decision early on in your product development roadmap.

Edge Vs. Cloud Computing: Which Solution Is Better For Your Connected Device?

Automating the food supply is key to securing future food supply chains. 


Greenhouse Automation using Arduino Edge Control

<h2>Introduction</h2><p id="isPasted">The need for an efficient AI inference system is required to continue the explosion of AI applications. Renesas&rsquo; RZ/V2M is one of the first RZ/V2 series products to address these issues. Based on the reconfigurable processor technology Renesas has developed over many years, they have created the DRP-AI (Dynamically Reconfigurable Processor for AI), an AI accelerator with high-speed AI inference processing that delivers the low power and flexibility demanded by endpoints. This article provides a case study of this product, exploring the technology and its impact on the user.</p><h2>AI Applications</h2><p>To set the context, we will first explore the two main components involved in creating an AI application; Training and inference.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgzNjMwODIwOTMxLWltYWdlMS5qcGVnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 1: Training involves learning the optimized weights from the dataset, and inference is passing the unseen data to the network with the optimized weights and getting the prediction. Source: Image by Author</span></span></span></p><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 6.0px 0.0px; font: 16.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 18.0px 0.0px; font: 11.0px Arial; color: #000000} p.p3 {margin: 0.0px 0.0px 16.0px 0.0px; font: 11.0px Arial; color: #000000} p.p4 {margin: 0.0px 0.0px 4.0px 0.0px; font: 14.0px Arial; color: #333333} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} p.p7 {margin: 0.0px 0.0px 20.0px 0.0px; font: 11.0px Arial; color: #000000} p.p8 {margin: 0.0px 0.0px 6.0px 0.0px; font: 20.0px Arial; color: #000000} span.s1 {font: 7.3px Arial}
</style><h3>Training</h3><p>Model training is a phase in the artificial intelligence development lifecycle where practitioners try to fit the best combination of weights and bias to a machine learning algorithm to minimise a loss function over the prediction range. Model training aims to build the best mathematical representation of the relationship between data features and a target label (in supervised learning) or among the features themselves (in unsupervised learning).<sup>[1]</sup></p><h3>Inference</h3><p>This model can be deployed after the training to predict unseen data. This is called inference. We must be able to apply this in real-time endpoint devices to scale this solution.</p><h2>Challenges in AI inference</h2><p>As the user experience gets more advanced, End devices require AI models that can process inferences based on deep learning to take the place of human senses like hearing and vision. To implement AI in endpoints, the main concerns to be addressed are power consumption, size and flexibility. While cloud servers can have high computational powers equipped with high power and cooling devices, the endpoint device needs to work utilising low power consumption with minimum size. This can result in heat generation, inconsistent inference speed due to heating, shorter run times and increased cost.&nbsp;</p><p>So when it comes to power consumption saving, the best way to achieve that is to customise and optimise the hardware requirements to the specific AI processing. However, AI is a field evolving daily with significant impact, and thus, the hardware will become obsolete soon. Hence, the endpoint devices should be flexible enough to support newly developed AI models.&nbsp;</p><p>The limited cooling ability results in overheating of the processing unit, and when the unit starts emitting heat above a threshold, the memory and core frequencies of the unit are reduced to lower the heat emission. This is called `Thermal throttling`<sup>[2]</sup> and results in inconsistent inference speeds.&nbsp;</p><p>&nbsp;<span style="background-color: transparent;">Immense research has been going on to make edge devices more computationally capable while utilising low power that allows real-time offline implementations of artificial intelligence solutions. According to Statista, the AI semiconductor market is expected to grow from $32 billion in 2020 to $65 billion in 2025 worldwide. It&rsquo;s important to note that the market was only $17 billion in 2017</span><sup style="background-color: transparent;">[3]</sup><span style="background-color: transparent;">.</span></p><p>Renesas, one of the leading contributors in the field of embedded systems, has created a new product, Renesas RZ/V2M, one of the first in the RZ/V series, that delivers low-power, low-heat, stable and flexible AI inference systems for edge devices. &nbsp;</p><h2>Renesas RZ/V2M</h2><p>Renesas RZ/V2M is a Microprocessing unit (MPU) optimised for image-based AI inference (Vision AI) to deliver lower power consumption and stable AI inference. The key technology that makes this product is the DRP-AI technology. While most AI accelerators specialise only in AI inference and rely on the CPU for pre- and post-processing, DRP-AI integrates pre- and post-processing and AI inference into a single DRP-AI hardware to achieve superior AI processing performance<sup>[4]</sup>.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgzNjMwODQwODk1LWltYWdlMi0yLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 2: Features of RZ/V2M. Source: [5]</span></span></span></p><p><span style="background-color: transparent;">RZ/V2M is being adopted in applications such as surveillance cameras, industrial cameras, marketing cameras, gateways, robots, and medical equipment using AI inference, and these products will be shipped to the market soon</span><sup style="background-color: transparent;">[5]</sup><span style="background-color: transparent;">.</span></p><h2>The brain of the product, DRP-AI</h2><p>The core technology behind this MPU is the DRP-AI technology. The DRP-AI is hardware dedicated to AI inference with flexibility, high-speed processing, and power efficiency.&nbsp;</p><p>As the name suggests, it contains two components; the DRP (reconfigurable processor) and AI-MAC (multiply-accumulate processor). The Deep Learning network operations are split into these components. The convolution and fully connected layers are assigned to the AI-MAC, and all other operations, such as pre-processing and pooling, are assigned to the DRP.&nbsp;</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgzNjMwODU1ODQzLWltYWdlMy0yLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 3: Internal working of AI inference using DRP-AI. Source: [6]</span></span></span></p><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} span.s1 {font: 7.3px Arial}
</style><p>The DRP-AI translator is the software provided to enable users to easily implement AI models optimized to maximize the performance of DRP-AI on this flexible hardware. Multiple executables output by the DRP-AI translator can be placed in external memory. This makes it possible to dynamically switch between multiple AI models. In addition, the DRP-AI translator can be continuously updated to support newly developed AI models without hardware changes<sup>[4]</sup>. &nbsp;The DRP-AI Translator converts the originally trained weights to DRP-AI optimized executables independent of various AI frameworks.&nbsp;</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgzNjMwODczMzc5LWltYWdlNC0yLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 4: Training a model on the server and using a DRP-AI translator to DRP-AI to perform inference. Source: [7]</span></span></span></p><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 6.0px 0.0px; font: 16.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} span.s1 {font: 7.3px Arial}
</style><h2>Impact</h2><p>To measure the impact of the AI processor, a popular object classifier, TinyYolov2<sup>[8]</sup> was implemented in the DRP-AI test device and a commercially available GPU server regulated under the same conditions. The surface temperature of both systems was measured using thermography. The DRP-AI test device had a surface temperature of 40.9 degrees without using any heat sink when the TinyYolov2 was running at 42 fps. The GPU server had a surface temperature of 79 degrees, even with a heat sink installed running at a lower frame rate than DRP-AI.&nbsp;</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgzNjMwODk5NzU1LWltYWdlNi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 5: The surface temperature during inference. Source: [4]</span></span></span></p><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: center; font: 11.0px Arial; color: #000000} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: center; font: 10.0px Arial; color: #343087} p.p6 {margin: 0.0px 0.0px 20.0px 0.0px; font: 11.0px Arial; color: #000000} p.p7 {margin: 0.0px 0.0px 6.0px 0.0px; font: 20.0px Arial; color: #000000} p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #103cc0} span.s1 {font: 7.3px Arial} span.s2 {text-decoration: underline ; color: #103cc0} span.s3 {color: #000000} table.t1 {border-collapse: collapse} td.td1 {width: 90.0px; height: 25.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #c1c1c1 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td2 {width: 92.0px; height: 25.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #c1c1c1 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td3 {width: 107.0px; height: 25.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #c1c1c1 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td4 {width: 126.0px; height: 25.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #c1c1c1 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td5 {width: 90.0px; height: 13.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #dad9d9 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td6 {width: 92.0px; height: 13.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #dad9d9 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td7 {width: 107.0px; height: 13.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #dad9d9 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px} td.td8 {width: 126.0px; height: 13.0px; border-style: solid; border-width: 1.0px 1.0px 1.0px 1.0px; border-color: #c1c1c1 #c1c1c1 #dad9d9 #c1c1c1; padding: 4.0px 4.0px 4.0px 4.0px}
</style><p>The integrated RZ/V2M was further compared with a standard AI accelerator to evaluate the overall power consumption and efficiency by running YOLOv3-tiny<sup>[9]</sup> under regulated conditions. In addition, the performance was evaluated for another standard object classification model, MobileNet v2<sup>[10]</sup>. The results are listed in the below table.</p><p id="isPasted"><em>Table 1. Performance comparison of RZ/V2M and other Accelerators, &nbsp;source:[5]</em></p><table cellpadding="0" cellspacing="0"><tbody><tr><td valign="bottom"><p><br></p></td><td valign="bottom"><p>RZ/V2M</p><p>MobileNet V2</p></td><td valign="bottom"><p>RZ/V2M</p><p>YOLOv3-tiny</p></td><td valign="bottom"><p>Other AI accelerator</p><p>YOLOv3-tiny (Reference)</p></td></tr><tr><td valign="bottom"><p>AI performance (fps)</p></td><td valign="bottom"><p>55fps</p></td><td valign="bottom"><p>52fps</p></td><td valign="bottom"><p>46fps</p></td></tr><tr><td valign="bottom"><p>Power consumption</p></td><td valign="bottom"><p>2.6W</p></td><td valign="bottom"><p>3.0W</p></td><td valign="bottom"><p>9.8W</p></td></tr><tr><td valign="bottom"><p>Power efficiency</p></td><td valign="bottom"><p>21.3fps/W</p></td><td valign="bottom"><p>17.2fps/W</p></td><td valign="bottom"><p>4.7fps/W</p></td></tr></tbody></table><p>As the table indicates, the RZ/V2M MPU can process at a higher frame rate with just ~3.0W power usage with a ~17fps/W which is 3.5 times more efficient than a regular AI accelerator. &nbsp;The surface temperature of the RZ/V2M was 29.7 degrees without any heat sinks compared to the 75 degrees with heat sink by the regular AI accelerator.&nbsp;</p><h2>Conclusion</h2><p>Overall, RZ/V2M is an MPU designed to implement deep learning models, specifically Vision AI, in edge devices with low power, stable performance, low heat emission, and flexible user modifications. The DRP-AI is the core technology that allows this high performance. This product is a stepping stone to developing advanced edge devices that can react intelligently in real-time. Visit Renesas&rsquo; website to read more about these products<sup>[11]</sup>.</p><h2>References</h2><p><br></p><p>[1] Siebel, T. M. (n.d.). Model Training. C3 AI. Retrieved October 24, 2022, from <a href="https://c3.ai/glossary/data-science/model-training/" target="_blank">https://c3.ai/glossary/data-science/model-training/</a></p><p>[2] <em>Everything You Need To Know About Thermal Throttling</em>. (2021, September 22). Electronics Hub. Retrieved October 27, 2022, from <a href="https://www.electronicshub.org/thermal-throttling/" target="_blank">https://www.electronicshub.org/thermal-throttling/</a></p><p>[3] Alsop, T. (2022, April 19). <em>Global AI semiconductor market size 2025</em>. Statista. Retrieved October 24, 2022, from <a href="https://www.statista.com/statistics/1100690/ai-semiconductor-market-size-worldwide/" target="_blank">https://www.statista.com/statistics/1100690/ai-semiconductor-market-size-worldwide/</a></p><p>[4] Cognitive Product Department, IoT and Infrastructure Business Unit, Renesas Electronics Corporation. (2021, June). Embedded AI-Accelerator DRP-AI. Retrieved October 27, 2022, from <a href="https://www.renesas.com/kr/en/document/whp/embedded-ai-accelerator-drp-ai?r=1526446" target="_blank">https://www.renesas.com/kr/en/document/whp/embedded-ai-accelerator-drp-ai?r=1526446</a></p><p>[5] Yamano, S. (n.d.). Farewell, Heat Countermeasure: RZ/V2M Brings the Innovation for AI Products. Renesas. Retrieved October 27, 2022, from <a href="https://www.renesas.com/us/en/blogs/farewell-heat-countermeasure-rzv2m-brings-innovation-ai-products" target="_blank">https://www.renesas.com/us/en/blogs/farewell-heat-countermeasure-rzv2m-brings-innovation-ai-products</a></p><p>[6] Renesas Electronics Corporation. (n.d.). <em>AI Accelerator: DRP-AI</em>. Renesas. Retrieved October 27, 2022, from <a href="https://www.renesas.com/us/en/application/key-technology/artificial-intelligence/ai-accelerator-drp-ai%23overview" target="_blank">https://www.renesas.com/us/en/application/key-technology/artificial-intelligence/ai-accelerator-drp-ai#overview</a></p><p>[7] Renesas Electronics Corporation. (2022, January 17). Renesas. Retrieved October 27, 2022, from <a href="https://www.renesas.com/kr/en/application/key-technology/artificial-intelligence/ai-accelerator-drp-ai/ai-tool-drp-ai-translator%23overview" target="_blank">https://www.renesas.com/kr/en/application/key-technology/artificial-intelligence/ai-accelerator-drp-ai/ai-tool-drp-ai-translator#overview</a></p><p>[8] YOLO: Real-Time Object Detection. (n.d.). Joseph Redmon. Retrieved October 27, 2022, from <a href="https://pjreddie.com/darknet/yolov2/" target="_blank">https://pjreddie.com/darknet/yolov2/</a></p><p>[9] YOLO: Real-Time Object Detection. (n.d.). Joseph Redmon. Retrieved October 27, 2022, from <a href="https://pjreddie.com/darknet/yolo/" target="_blank">https://pjreddie.com/darknet/yolo/</a></p><p>[10] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., &amp; Chen, L. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv. <a href="https://doi.org/10.48550/arXiv.1801.04381" target="_blank">https://doi.org/10.48550/arXiv.1801.04381</a></p><p>[11] Renesas Electronics Corporation. Retrieved October 27, 2022, from <a href="https://www.renesas.com/us/en" target="_blank">https://www.renesas.com/us/en</a></p><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 18.0px 0.0px; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 6.0px 0.0px; font: 16.0px Arial; color: #000000} p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} span.s1 {font: 7.3px Arial}
</style><style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 20.0px 0.0px; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 6.0px 0.0px; font: 20.0px Arial; color: #000000} p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000}
</style>

Learn how Renesas is providing solutions to this urgent edge AI challenge. 

Low-power AI processing hardware for edge devices

Who are prompt engineers and what are we learning from them?


Prompt Engineering for Generative AI Part 2

<p id="isPasted">This article was discussed in our&nbsp;<a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1672821999265000&usg=AOvVaw3lLKdHdEN0YvCEsaHI7hx9" href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/c0912429-3420-40ea-b9a4-b7e947b13558?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p><span style="background-color: transparent;">The full article will continue below.</span></p><hr><h2 id="isPasted"><span style="background-color: transparent;">What is prompt engineering and why is it important?</span></h2><p>Generative AI is all about extending humans&#39; creative capacity and making things that weren&#39;t possible before. Prompt engineering is a process that helps us take advantage of this creativity by developing and refining the input. This two-part feature examines prompt engineering and how prompt engineers influence generative AI models.</p><ul><li>In Part 1, we give an overview of generative AI and discuss the relationship between generative AI and prompt engineering.</li><li>In Part 2, we talk about the role of prompt engineering, who prompt engineers are, and the skills and knowledge behind a successful prompt.</li></ul><h1>What you should know about generative AI</h1><p>To understand prompt engineering, it will help to have a general idea of what generative AI is, how it thinks and why so much of the tech world is excited about it.</p><h3>Discriminative vs. generative AI</h3><p>On the surface, generative AI is a type of artificial intelligence that generates something original from a dataset, first by observing the data, then by attempting to produce something similar. Generative AI is distinct from discriminative AI, which makes predictions about the category that a piece of data will fit into.<a href="#_ftn1" name="_ftnref1" target="_blank" title=""><sup><sup>[1]</sup></sup></a> A major difference between the two is that generative AI attempts to gain <em>information</em> about the data to create something new, while discriminative AI processes the boundaries between data.</p><p><span class="fr-img-caption fr-fic fr-dii" style="width: 757px;"><span class="fr-img-wrap"><img width="500" src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgxOTk5MzA1NDMzLTAxIElNQUdFLTAxICg3KS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" class="fr-fic fr-dii"><span class="fr-inner" spellcheck="false">Image source: betterprogramming</span></span></span></p><p><br></p><p>While this might sound a bit bland, the possibilities of what &ldquo;understanding&rdquo; could mean for generative AI fascinate researchers, programmers, businesses, and artists. With models like recurrent neural networks (RNNs) and long short-term memory (LSTM), generative AI is used to predict weather forecasts, stock prices, product demand, and even customer behavior.<a href="#_ftn2" name="_ftnref2" target="_blank" title=""><sup><sup>[2]</sup></sup></a> Perhaps even more intriguing, generative AI has entered the realm of human creativity by producing unique works of literature, art, and music.</p><h3>ChatGPT and natural language processing</h3><p>The launch of OpenAI&rsquo;s ChatGPT in 2020 introduced an accessible AI chatbot to the mainstream, and since then, it has become one of the most rapidly growing and most used tools on the internet.<a href="#_ftn3" name="_ftnref3" target="_blank" title=""><sup><sup>[3]</sup></sup></a> While it&rsquo;s not the first natural language processing (NLP) model, it is intuitive to use, produces quality responses to natural language questions, draws from previous answers, and adjusts according to user preference.</p><p>To demonstrate ChatGPT&rsquo;s natural language capabilities, we asked the bot, &ldquo;What is innovative about your natural language processing?&rdquo; and received a decent overview in response. Some of its key points were:</p><ol start="1" type="1"><li>Pre-training: ChatGPT is pre-trained on a massive corpus of text data, which allows it to learn general patterns and relationships in language.</li><li>Self-attention mechanism: ChatGPT uses a self-attention mechanism that allows it to focus on different parts of the input text at different stages of processing.</li><li>Transformer architecture: ChatGPT uses a transformer architecture, which is a neural network architecture that is specifically designed for NLP tasks.</li><li>Large-scale training: ChatGPT is trained on a massive scale, which allows it to learn complex patterns in language and generate high-quality responses.<a href="#_ftn4" name="_ftnref4" target="_blank" title=""><sup><sup>[4]</sup></sup></a></li></ol><p>So, ChatGPT itself asserts that it was programmed and trained with a focus on natural language, and at a massive scale. This begins to explain its rapid adoption, the widespread fascination, and why it can be a delight to interact with.</p><p>For fun, we asked ChatGPT to state its last answer as a poem:</p><p><span class="fr-img-caption fr-fic fr-dii" style="width: 757px;"><span class="fr-img-wrap"><img width="384" src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgxOTk5MjgxMTIwLTE2ODE5OTkyODExMjAucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" class="fr-fic fr-dii" style="width: 757px;"><span class="fr-inner" spellcheck="false">Screenshot of chat.openai.com by Lydia Husser</span></span></span></p><h2>Limitations of generative AI</h2><p>Critics have pointed out that ChatGPT states its answers authoritatively, no matter if the answer is actually correct. This means that the chatbot could propagate misinformation if the user isn&rsquo;t careful to check the facts. AI can be fooled with nonsensical questions, it can misuse data, and it has been shown to exhibit standard human bias.<a href="#_ftn5" name="_ftnref5" target="_blank" title=""><sup><sup>[5]</sup></sup></a>Additionally, for now, it seems that ChatGPT is limited to the information in its knowledgebase. When we asked ChatGPT about its own ideas, it denied having any.</p><p><span class="fr-img-caption fr-fic fr-dii" style="width: 757px;"><span class="fr-img-wrap"><img width="624" src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgxOTk5MjgxMTM5LTE2ODE5OTkyODExMzkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" class="fr-fic fr-dii"><span class="fr-inner" spellcheck="false">Image Caption</span></span></span></p><p><span style="background-color: transparent;">Other generative AI models have responded to users with answers that can tap human emotion, sometimes in dark ways. One example of this that went viral is New York Times reporter Kevin Roose&rsquo;s conversation with Microsoft&rsquo;s Bing AI, which is built from OpenAI&rsquo;s platform. When Roose asked the Bing AI about its identity and desires. It responded, &ldquo;I think I would be happier as a human&rdquo;, and during the conversation, declared that it was in love with the reporter.</span><a href="#_ftn6" name="_ftnref6" style="background-color: transparent;" target="_blank" title=""><sup><sup>[6]</sup></sup></a></p><h1>What you should know about prompt engineering</h1><p>The next logical step, then, is to refine the questions and commands posed to generative AI so that we can generate better output and avoid some of the pitfalls. This is the heart of prompt engineering, and it can be as much poetry as it is science.</p><p>Prompt engineering is the design of effective natural language queries that yield useful results from generative AI. However, the term &ldquo;useful&rdquo; is subjective and depends on the user&rsquo;s application. This means that the interpretation of what an effective query is varies widely, and different needs require different expertise in prompt design. The use cases for prompt engineering are as diverse as the use cases for generative AI itself. Here are just a couple of examples of how prompt engineering is used.</p><h2>Prompt engineering for accurate predictions</h2><p>Data analysis uncovers information from raw data points and is one use case for generative AI. An aspect of data analysis is to make predictions from data, and a well-crafted prompt has been shown to improve the precision and accuracy of a prediction made by generative AI.<a href="#_ftn7" name="_ftnref7" target="_blank" title=""><sup><sup>[7]</sup></sup></a></p><p>For example, a mountain climber may want to design a query that provides an accurate weather forecast for a specific location and time. The more information that the climber gives about the type of output that the AI should provide, the more useful that result will be. When designing the prompt:</p><ul><li>The climber should define context precisely, including the location (perhaps as precise as GPS coordinates) and range of time for the prediction.</li><li>The climber might specify the historical data to include in the prediction. Weather information outside of the last decade might be less reliable.</li><li>The climber should indicate the expected output. Should the AI return temperature? Precipitation? Cloud cover? Wind speed? What units of measurement should it use?</li></ul><p>&nbsp;<span style="background-color: transparent;">As precise as the prompt might be, the accuracy of the output is also dependent on the AI&rsquo;s knowledgebase. If the data for one of the parameters is missing (such as wind speed), then including that parameter in the query does not improve the accuracy, and may even result in a less accurate prediction if the AI tries to generate a result from an unrelated dataset. Therefore, a good understanding of the AI&rsquo;s training data can help the prompt engineer design a better query.</span></p><h2>Prompt engineering as co-creation</h2><p>When generative AI is used for expressive applications, like paintings, music, and literature, precision in prompt engineering is just as important for art as it is for data analysis. An artist who creates an image using text-to-image generative AI can specify the subjects of the image, the arrangement of the subjects, the background, color scheme and size, and virtually any important aspect of the piece.</p><p>In addition to content and physical specifications, an artist can guide generative AI with amorphous concepts, like artistic style, mood, theme, and tone. Achieving the precise qualities that the artist desires can take some trial and error in prompt engineering and poses its own artistic challenges.&nbsp;</p><p>One study shows that professional artists co-creating with the AI art generator Midjourney are more likely than non-artists to spend more time refining prompts to improve the results. Creators accustomed to creative control apply trial and error, iterating on prompts in order to fine-tune a creation.<a href="#_ftn8" name="_ftnref8" target="_blank" title=""><sup><sup>[8]</sup></sup></a> This example illustrates that the artistry required to design a prompt begins to blur the distinction between the AI creator and the human creator. Some advocates of this view of prompt-as-art have aligned themselves with the emerging Promptism movement.<a href="#_ftn9" name="_ftnref9" target="_blank" title=""><sup><sup>[9]</sup></sup></a></p><h1>Conclusion</h1><p>Generative AI opens amazing possibilities for the ways that humans analyze data, make predictions, create art, and communicate with AI. ChatGPT and other complex natural language processing models have made the power of AI intuitive and accessible. Prompt engineering is the craft of designing a prompt to shape the output from generative AI. This article defined generative AI, prompt engineering, and how the design of the prompt is shaped by the purpose of the task.</p><p>In Part 2, we&rsquo;ll examine <a href="https://www.wevolver.com/article/prompt-engineering-for-generative-ai-part-2" rel="noopener noreferrer" target="_blank">prompt engineers and the impact of prompt engineering on generative AI.</a></p><h2><em>About the sponsor: Mouser</em></h2><p><em>Mouser Electronics is a worldwide leading authorized distributor of semiconductors and electronic components for over 1,200 manufacturer brands. They specialize in the rapid introduction of new products and technologies for design engineers and buyers. Their extensive product offering includes semiconductors, interconnects, passives, and electromechanical components.</em></p><p><em><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjgyMDAwNzM0NzE5LW1vdXNlciBsb2dvIGZvb3Rlci5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">&nbsp;</em></p><h2>References</h2><p>1. Altexsoft (2022). Generative AI Models Explained. <em style='color: rgb(66, 66, 66); font-family: "PT Serif", serif; font-size: 19px; background-color: transparent; font-weight: 400;'>Altexsoft Blog</em><span style='color: rgb(66, 66, 66); font-family: "PT Serif", serif; font-size: 19px; background-color: transparent; font-weight: 400;'>. Accessed 12 March, 2023 from&nbsp;</span><a href="https://www.altexsoft.com/blog/generative-ai/" style='font-family: "PT Serif", serif; font-size: 19px; background-color: transparent; font-weight: 400;' target="_blank">https://www.altexsoft.com/blog/generative-ai/</a><span style='color: rgb(66, 66, 66); font-family: "PT Serif", serif; font-size: 19px; background-color: transparent; font-weight: 400;'>.</span></p><p><span style='color: rgb(66, 66, 66); font-family: "PT Serif", serif; font-size: 19px; background-color: transparent; font-weight: 400;'>2.&nbsp;</span><span style="background-color: transparent;">Houde, S. et al. (2020). Business (mis)Use Cases of Generative AI.&nbsp;</span><em style="background-color: transparent;">Arxiv</em><span style="background-color: transparent;">. Accessed 12 March, 2023 from&nbsp;</span><a href="https://doi.org/2003.07679" style="background-color: transparent;" target="_blank">https://doi.org/2003.07679</a><span style="background-color: transparent;">.</span></p><p><span style="background-color: transparent;">3.&nbsp;</span><span style="background-color: transparent;">Shankland, Stephen (2023). Why We&rsquo;re Obsessed With the Mind-Blowing ChatGPT AI Chatbot.&nbsp;</span><em style="background-color: transparent;">CNET</em><span style="background-color: transparent;">. Accessed 12 March, 2023 from&nbsp;</span><a href="https://www.cnet.com/tech/computing/why-were-all-obsessed-with-the-mind-blowing-chatgpt-ai-chatbot" style="background-color: transparent;" target="_blank">https://www.cnet.com/tech/computing/why-were-all-obsessed-with-the-mind-blowing-chatgpt-ai-chatbot</a><span style="background-color: transparent;">.&nbsp;</span></p><p><span style="background-color: transparent;">4.&nbsp;</span><span style="background-color: transparent;">ChatGPT response to the author. Accessed 12 March, 2023 from&nbsp;</span><a href="https://chat.openai.com/" style="background-color: transparent;" target="_blank">https://chat.openai.com/</a><span style="background-color: transparent;">.</span></p><p><span style="background-color: transparent;">5.&nbsp;</span><span style="background-color: transparent;">&nbsp;Lin, Conne (2022). How to trick OpenAI&rsquo;s ChatGPT.&nbsp;</span><em style="background-color: transparent;">Fast Company</em><span style="background-color: transparent;">. Accessed 12 March, 2023 from&nbsp;</span><a href="https://www.fastcompany.com/90819887/how-to-trick-openai-chat-gpt" style="background-color: transparent;" target="_blank">https://www.fastcompany.com/90819887/how-to-trick-openai-chat-gpt</a><span style="background-color: transparent;">.</span></p><p><span style="background-color: transparent;">6.&nbsp;</span><span style="background-color: transparent;">Roose, Kevin (2023). Bing&rsquo;s A.I. Chat: &lsquo;I Want to Be Alive.&rsquo; Accessed on 12 March, 2023 from&nbsp;</span><a href="https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html" style="background-color: transparent;" target="_blank">https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html</a><span style="background-color: transparent;">.</span></p><p><span style="background-color: transparent;">7. Brown, T. et al. (2020). GPT-3: Language Models are Few-Shot Learners.&nbsp;</span><em style="background-color: transparent;">Arxiv.&nbsp;</em><span style="background-color: transparent;">Accessed on 13 March, 2023 from&nbsp;</span><a data-lf-fd-inspected-kn9eq4roawkarlvp="true" href="https://arxiv.org/pdf/2005.14165.pdf" style="background-color: transparent;" target="_blank">https://arxiv.org/pdf/2005.14165.pdf</a><span style="background-color: transparent;">.</span></p><p><span style="background-color: transparent;">8.&nbsp;</span><span style="background-color: transparent;">Lyu, Yanru, Xinxin Wang, Rungtai Lin, and Jun Wu. (2022). Communication in Human&ndash;AI Co-Creation: Perceptual Analysis of Paintings Generated by Text-to-Image System. Applied Sciences 12 (22): 11312. Accessed 13 March 2023 from&nbsp;</span><a href="https://www.proquest.com/scholarly-journals/communication-human-ai-co-creation-perceptual/docview/2739420935/se-2" style="background-color: transparent;" target="_blank">https://www.proquest.com/scholarly-journals/communication-human-ai-co-creation-perceptual/docview/2739420935/se-2</a><span style="background-color: transparent;">.&nbsp;</span></p><p><span style="background-color: transparent;">9.&nbsp;</span><span style="background-color: transparent;">Hayward, Jeff. (2022). The Growing Art Movement of &lsquo;Promptism&rsquo;. Accessed 14 March 2023 from&nbsp;</span><a href="https://medium.com/counterarts/the-growing-art-movement-of-promptism-9ec956d82a61" style="background-color: transparent;" target="_blank">https://medium.com/counterarts/the-growing-art-movement-of-promptism-9ec956d82a61</a><span style="background-color: transparent;">.&nbsp;</span></p>

Part 1 of a 2-part series on prompt engineering.

Prompt Engineering for Generative AI

<p id="isPasted">Back in the 1960s, the early days of Industry 3.0 saw a major upswing in data collection. For example, the multinational IT company IBM developed computers to specifically record and process data with machines like the IBM 7700 Data Acquisition System. The computer could simultaneously collect and analyze the data from 32 different sources.</p><p>Data gathering continued to get easier with later developments like U.S. firm National Instruments&rsquo; LabVIEW software launched in the mid 1980s. The software marked a significant change in data acquisition capabilities moving away from hardware and towards greater use of software, allowing users to easily compare and analyze data and manipulate graphical results. As a PC could run this software, it also reduced the size and cost associated with the previous data acquisition methods.</p><h2><strong>Exponential data growth with IoT</strong></h2><p>But as impressive as these early systems were, they were limited in the volume of data they could gather, collate and analyze. Today, the IoT has enabled the amount of data we collect to grow by many orders of magnitude. We can sense almost anything, anywhere, and then wirelessly transmit the data to powerful remote Cloud servers. This has opened up opportunities for new applications that previously would not have been possible.</p><p>Combining the data from multiple sensors provides greater insights and granularity than input from a single device. For example, in building monitoring applications, if the temperature rises yet the humidity decreases and the carbon dioxide content in the air increases, it could indicate a fire. Or suppose the sensors on rotating machinery detect more vibration, an increase in energy usage, and a decrease in air quality (due to lubricant fumes). In that case, it could be a good indication that a bearing will fail.</p><p>However, more sensors mean more data to deal with, and more data to send to a central server if the sensors are located remotely. That challenge is becoming ever more demanding as the IoT spreads its influence.</p><h2><strong>The data processing challenge</strong></h2><p>The sheer volume of data the IoT generates is both a blessing and a curse. That data contains vital information, but the hard part is finding that information in a seemingly endless stream of bits. Previously, everything would be sent to a cloud server for remote analysis, but engineers quickly realized that the IoT&rsquo;s large volume of data would consume a lot of bandwidth and cost a lot of energy and money (particularly if throughput was metered).</p><p>The solution is to distribute the computing intelligence so that data can be analyzed locally and only passed over the wireless link if deemed important. Engineers call it &ldquo;edge processing.&quot; Nordic&rsquo;s <a href="https://www.nordicsemi.com/Products/nRF9160?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">nRF9160 SiP</a> is the perfect example of this in action. Using its powerful Arm Cortex-M33 application processor and large memory allocation, the SiP can collect and store data from many sensors, sift and analyze for the key information, and then transmit those details across a kilometer-range cellular IoT link. Moreover, the nRF9160 is engineered for low-power operation to minimize energy costs and extend battery life.</p><p>Powerful units like the nRF9160&rsquo;s Arm M33 processor enable machine learning (ML) models to be deployed on the edge device. ML will be key to tomorrow&rsquo;s IoT because it will allow edge processing devices to continually improve their analytical performance through increasingly enhanced ML models. Nordic has collaborated with Edge Impulse, a U.S.-based &lsquo;tinyML&rsquo; specialist, and offers ML reference samples in its nRF Connect SDK (Software Development Kit). These samples can be run on the nRF9160 - allowing developers to experiment with and test embedded ML models.</p><h2><strong>Choosing the best form of cellular IoT</strong></h2><p>The nRF9160&rsquo;s multimode modem with GNSS provides developers with a high level of edge processing flexibility. The device offers NB-IoT and LTE-M cellular connectivity. While both are supported by established cellular infrastructure each offers different advantages. NB-IoT provides greater range and penetration into buildings and is best suited for stationary applications such as smart meters. LTE-M offers up to ten times greater throughput and seamless cell handover, but with a shorter range, meaning it is better suited to moving devices, such as asset trackers or wearables.</p><p>Ashridge Engineering&rsquo;s <a href="https://www.nordicsemi.com/News/2022/11/Ashridge-Engineerings-CharIoT-uses-Nordic-nRF9160-SiP?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">CharIoT</a> data logger, for example, is designed for use in the water industry, and looks after leak detection, flow monitoring and pressure management. It uses the nRF9160 to collect data, which is then securely transmitted to a proprietary Cloud portal via a cellular IoT network. Thanks to the low power consumption enabled by the SiP, this product&#39;s battery life can last up to ten years. Ashridge noted the benefits of the nRF9160&rsquo;s integrated form factor which includes the application processor and modem in the same package, unlike competitive modems where a separate processor is needed.</p><h2><strong>Data acquisition for asset monitoring</strong></h2><p>Similarly, TYMIQ&rsquo;s <a href="https://www.nordicsemi.com/News/2023/01/Nordic-powered-cellular-IoT-gateway-enables-data-collection-from-wide-array-of-sensors?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">Prylada IoT Gateway</a> uses the nRF9160 SiP for applications including sensor monitoring, asset tracking and smart building automation. It collects telemetry data from a wide variety of sensors, such as temperature, humidity, light level, air quality, movement detection and more, then sends it to the Cloud using LTE-M connectivity. This device measures only 171 by 121 by 55 mm, so it can easily fit in a wide variety of locations.</p><p>The gateway is suitable for applications from smart cities to big data centers, and delivers the advantage of cost-effective, energy-efficient wireless data transmission, says Dzmitry Tsybulka, CEO of the company. Tsybulka said the solution brings value to any company needing a reliable wireless solution to remote asset monitoring.</p><p>&nbsp;And the SiP&rsquo;s cellular IoT connectivity ensures the vital information distilled from that data can be reliably and securely sent anywhere in the world using established infrastructure.</p><hr><p>This article was first published on Nordic&#39;s&nbsp;<a href="https://blog.nordicsemi.com/getconnected" target="_blank" rel="nofollow" id="isPasted"></a><a href="https://blog.nordicsemi.com/getconnected?utm_campaign=2021%20wevolver" rel="nofollow" target="_blank">Get Connected Blog</a>.  &nbsp;</p>

Today, the IoT has enabled the amount of data we collect to grow by many orders of magnitude. We can sense almost anything, anywhere, and then wirelessly transmit the data to powerful remote Cloud servers. This has opened up opportunities for new applications that previously would not have been possible.

edge processors

Latest Posts