<p dir="ltr" id="isPasted">Edge Impulse recently announced official support for the <a href="https://www.syntiant.com/ndp101" target="_blank">Syntiant NDP101</a> processor. The NDP101 (Neural Decision Processor™) is an always-on sensor and speech recognition processor, with ultra-low power consumption of less than 140 uW while recognizing words. It features a Dense Neural Network Architecture of four layers with 256 inner nodes and runs real-time audio inferencing up to 100 times per second. The NDP101 is a solution for wake word detection, speech classification or any audio-related application.</p><p dir="ltr"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p dir="ltr">Equipped with the ultra-low-power Syntiant <a href="https://www.globenewswire.com/Tracker?data=hVCGZ6kMB-cDkZNKwE61sOEIRHpn97IpKQBQdH_1nVFus9Ct-Q9YnTOq2yFHiWA3lgDtI1M213rZ8aS7v3sIaQ==" target="_blank">NDP101</a> Neural Decision Processor, the new TinyML Board can enable speech and sensor applications to run at under 140 and 100 microwatts, respectively, delivering 20x more throughput and 200x efficiency improvement compared to traditional MCU-based systems.&nbsp;</p><p dir="ltr">Sized at 24 mm x 28 mm, the Syntiant TinyML Board is a small, self-contained system that allows trained models to be easily downloaded via Edge Impulse through a micro-USB connection without the need for any specialized hardware. The board also is fully compatible with Arduino’s open source platform.</p><p dir="ltr">The TinyML Board’s onboard digital microphone and motion sensor enable easy configuration for any speech, event detection or six-axis motion- and vibration-related application. Shipping with a built-in “Alexa play music” model, key product features include:</p><ul><li dir="ltr"><p dir="ltr">Syntiant NDP101 Neural Decision Processor™</p></li><li dir="ltr"><p dir="ltr">Syntiant Core 1™ neural network</p></li><li dir="ltr"><p dir="ltr">Supports up to 560k parameters and 64 output classifications</p></li><li dir="ltr"><p dir="ltr">Host processor: Arm Cortex-M0+ 32-bit microcontroller, including:</p><ul><li dir="ltr"><p dir="ltr">256KB flash memory</p></li><li dir="ltr"><p dir="ltr">32KB host processor SRAM</p></li></ul></li><li dir="ltr"><p dir="ltr">5 digital I/Os compatible with Arduino MKR series boards</p></li><li dir="ltr"><p dir="ltr">2MB on-board serial flash</p></li><li dir="ltr"><p dir="ltr">48MHz system clock</p></li><li dir="ltr"><p dir="ltr">6-axis motion sensor</p></li><li dir="ltr"><p dir="ltr">Digital microphone</p></li></ul><p dir="ltr">To make it easy to build and deploy audio classification models on ultra-low power silicon, Edge Impulse has partnered with <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/syntiant&source=gmail&ust=1638971436649000&usg=AOvVaw1Wadj23Bavue5Sm1Qe1Ea9" href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant</a> to bring support for the NDP101 processor to Edge Impulse Studio - giving developers a quick and efficient way to collect data, build models, and then deploy to the <a data-saferedirecturl="https://www.google.com/url?q=https://www.syntiant.com/tinyml&source=gmail&ust=1638971436649000&usg=AOvVaw2CJOwWnOHkiLeqNFjAEFQe" href="https://www.syntiant.com/tinyml" rel="nofollow" target="_blank">Syntiant TinyML Board</a>.</p><p dir="ltr">Best of all, you can collect audio data directly through the Edge Impulse Studio. The TinyML Board supports the Arduino environment to make it easier for developers to get started.</p><p dir="ltr"><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="640" height="360" src="https://www.youtube.com/embed/uK3v8yXynhY?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p>

Edge Impulse recently announced official support for the Syntiant NDP101 processor, an always-on sensor and speech recognition processor, with ultra-low power consumption of less than 140 uW while recognizing words. 

Build Speech, Audio and Sensor Machine Learning Applications with Syntiant's TinyML Platform and Edge Impulse

<h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">Artificial Intelligence (AI) refers to the ability of a system to learn new things. There are several approaches to do so, but Machine Learning (ML) is the one that is the centre of attraction these days. It involves training a neural network with a lot of data to make it understand a situation and make some decisions.</p><p dir="ltr">Traditionally, an AI-powered system acquires analog signals with the help of a transducer. This can be acoustic, visual, chemical, thermal or any signal available in nature the transducer is developed for. In the next step, the data is conditioned by the microprocessor and sent to the cloud servers with the help of a wireless transmission link. Cloud servers powered by multiple interconnected CPUs and GPUs process this data and share the results back.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 300px;"><span class="fr-img-wrap"><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjM2OTg1MzE4NjUxLXNlcnZlcl9yb29tLTIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="servers-data-centres-tinyml"><span class="fr-inner" contenteditable="true">Conventional ML requires servers and data centres with high compute power</span></p></span></span></div><p dir="ltr" id="isPasted">While most ML and training used to happen on powerful cloud servers, over the last couple of years, the experiments have led us to a point where some part of the ML can be performed on the edge, right where the data source is.</p><p dir="ltr">In this article, we are going to discuss the fundamentals, key advantages and applications of TinyML. Through this article, we have tried to give readers a basic idea of the TinyML project development flow and how the technology holds the potential to revolutionise the way data is processed by electronic systems.</p><p dir="ltr">But what exactly led us to a point where we started trying to fit machine learning algorithms running on clouds into edge computing devices, bringing together such isolated fields? Let’s find out.</p><h2 dir="ltr">What is TinyML good for?</h2><p dir="ltr">By taking machine learning to the edge, TinyML aims to improve the products on four fronts. They are:</p><ul><li dir="ltr"><strong>Privacy:&nbsp;</strong>Consider, for example, smart home devices that always listen to the conversation you have at home while they wait patiently to listen to the hot word that activates them. One wouldn’t prefer to share their private conversations that take place at home/office with third-party entities. By running machine learning programs on the edge, it is possible to eliminate the risk of such a privacy incident.</li><li dir="ltr"><strong>Power:&nbsp;</strong>Transmission of data in Internet of Things (IoT) devices is probably the operation that consumes most of the energy. &nbsp;Processing data on the device cuts off a large portion of the energy consumed in the data communication layer.</li><li dir="ltr"><strong>Cost:</strong> By not transmitting data, the cost of setting up servers and the on-device radio is cut, which greatly brings down the overall price of the device.</li><li dir="ltr"><strong>Reliability:&nbsp;</strong>Not sending data anywhere means that the response time is significantly reduced. Processing on the edge also means that the least amount of data is lost during transmission. The problem of devices not being able to work during server maintenance or outrage is also solved.</li></ul><p dir="ltr">The case with embedded systems (especially the ones interacting with us) is that we expect these devices to respond to our actions instantly. Any delay in the response has a direct effect on user experience. Embedded systems are almost always powered by batteries. This means that the device should be able to function in an extremely energy-constrained environment. Embedded Systems also have very limited memory and a lot of sensors.</p><p dir="ltr">So how can the issue of designing a system that can process so much data in such a constrained environment be solved? The answer is TinyML.</p><h2 dir="ltr">What is TinyML?</h2><p dir="ltr">TinyML stands for Tiny Machine Learning, a field of study lying at an intersection of Embedded Systems and AI that involves developing systems that run ML models on ultra-low-power microcontrollers. The idea is to push the implementation to where the information source is.</p><p dir="ltr">There was a time when microcontrollers used proprietary programming languages and tools, which made it too intimidating for beginners and hobbyists to even start with the technology. Things have changed a lot since then. With the development of the Arduino platform, microcontrollers have become more accessible than ever. The introduction of ARM’s Cortex M series chips is also considered a prominent event since it made 32-bit microcontrollers mainstream.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 300px;"><span class="fr-img-wrap"><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjM2OTg1Mzk0NTgzLWFyZHVpbm9fbWljcm9jb250cm9sbGVyLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="arduino-microcontroller-tinyml"><span class="fr-inner" contenteditable="true">Rapid microcontroller prototyping platforms like Arduino reduce the steep learning curve for getting started with embedded systems, IoT and TinyML</span></p></span></span></div><p dir="ltr" id="isPasted">Collaborations between companies like Google, Arduino, ARM, Qualcomm, <a href="https://www.wevolver.com/profile/edge.impulse" target="_blank">Edge Impulse</a>, etc, is making it increasingly simpler with each passing day to get started with TinyML devices.</p><h2 dir="ltr">Overview of the development flow of TinML projects</h2><p dir="ltr">The initial stage of development involves designing and training a machine learning model in frameworks like TensorFlow. Google has introduced a special version of the tool called TensorFlow Lite, which comes with a binary footprint of less than 16 KB and doesn’t have any library dependencies.</p><p dir="ltr">The generated model from TensorFlow Lite is combined with the application code with the help of Integrated Development Environments (IDEs) like Arduino IDE, Keil MDK and Mbed studio. Next up, the software compiles the code into a binary file before deploying it on the appropriate hardware. Many development boards like Arduino Nano 33 BLE, Sparkfun Edge, STM32F746NG MCU discovery kit, ESP32-DevKitC, Sony Spresense etc., can be used to develop TinyML projects.</p><p dir="ltr">Don’t forget to check out the video links in the references section to find out more about getting started with TinyML projects.</p><h2 dir="ltr">Applications of TinyML</h2><p dir="ltr">TinyML can be used wherever conventional ML is required. But there are some specific areas where there’s no alternative to TinyML. Real-time applications, applications involving extremely personal data or applications where a continuous power source is not available are especially suited for TinyML. Here is a list of the most common applications of TinyML in domestic and industrial environments:</p><p dir="ltr"><strong>Computer Vision:</strong> TinyML can be used for face recognition, text recognition and various applications where cameras are involved. It is possible to make TinyML devices that can respond to you when you look at them or wave at them. Appliances that detect the presence of someone, smartphones that use face unlock and all other devices which take a decision based on visual cues can be built with TinyML.</p><p><strong>Audio recognition &amp; Natural Language Processing:</strong> Small neural networks of size in the order of a few KBs are trained to detect wake words. These algorithms are deployed on a dedicated DSP chip and are used to process audio continuously, looking for the wake words.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 300px;"><span class="fr-img-wrap"><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjM2OTg1NDczNDI3LWVkZ2VfZGV2aWNlcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="google-home-amazon-echo-tinyml"><span class="fr-inner" contenteditable="true">Smart home appliances and smartphones come with a dedicated chip for identifying wake words locally</span></p></span></span></div><p dir="ltr" id="isPasted">The chip requires just a few milliWatts (mWs) of power which is far less compared to what a typical smartphone processor uses. And since the data doesn’t leave the device, privacy concerns are largely addressed. Google Assistant’s implementation of “OK Google” wake word detection remains one of the most iconic TinyML applications ever.</p><p dir="ltr"><strong>Time-Frequency Signal analysis:&nbsp;</strong>ML, in general, performs well at finding patterns in messy data. By training models with proper data sets, it is possible to monitor the condition of electrical machines.&nbsp;</p><p dir="ltr">Motors and Drives can be fitted with TinyML devices that monitor the vibrations and other electromechanical parameters to detect an anomaly in operation for intelligent and predictive maintenance.</p><p dir="ltr">TinyML applications can potentially be applied to all the sectors in some way or the other. Energy, retail, financial services, travel, tourism, hospitality, manufacturing, healthcare, life sciences are just some fields where the technology can bring massive improvements.</p><h2 dir="ltr">TinyML for good</h2><p dir="ltr">TinyML for good is an online conference showcasing some inspirational thinkers that used TinyML to develop innovative solutions and products in the field of education, healthcare and the environment sectors.&nbsp;</p><p dir="ltr"><strong>Visit the event page of <a href="https://www.tinyml.org/event/tinyml-for-good" target="_blank">TinyML for good</a> to know more about it.</strong>&nbsp;</p><p dir="ltr">Alternatively, you can directly head over to the <a href="https://us02web.zoom.us/webinar/register/6716342408482/WN_cquU_7lRRzOOKhzvVX2X-Q" target="_blank">registration page</a> to get yourself enrolled for the event. Dive into the world of TinyML and learn about the possibilities of the technology on 17th November 2021. No prior technical experience is required.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjM2OTgyMTE0OTQ4LVRpbnlNTGZvckdvb2QgKDQwMDAgeCA3NTAgcHgpLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="TinyML-for-good-event-logo"></p><p dir="ltr" id="isPasted"><strong>About TinyML Foundation</strong></p><p dir="ltr">With its workshops, webinars, expert talks, research conferences, competitions, and a plethora of other events, TinyML Foundation enables students, researchers, and industry personnel to come together and share their experiences about the technology. It is the incredibly collaborative nature of the community that allows it to advance rapidly.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">We can arrive at a consensus that the research on TinyML devices and algorithms is fueled by the growing applications of the technology. It wouldn’t be an understatement to say that in the next couple of years, we can see this technology going to every portable electronic device there is. It is really exciting to witness how things change in unique and unexpected ways.</p><p dir="ltr" id="isPasted">The <a href="https://www.wevolver.com/article/tinyml-for-good-where-machine-learning-meets-edge-computing" target="_blank">introductory article</a> familiarised the readers with the concept of tinyML.</p><p dir="ltr">The <a href="https://www.wevolver.com/article/tinyml-unlocks-new-possibilities-for-sustainable-development-technologies" target="_blank">first article</a>&nbsp;explains some possibilities of tinyML applications in sustainable technologies.</p><p dir="ltr">The <a href="https://www.wevolver.com/article/applications-of-tinyml-in-healthcare-for-a-safer-future/" rel="noopener noreferrer" target="_blank">second article</a> is about tinyML applications aiding healthcare and medical research.</p><p dir="ltr">The third and final article shall cover the tinyML applications in pedagogy and education.</p><hr><h2 dir="ltr">References</h2><p dir="ltr">[1] M. Shafique, T. Theocharides, V. J. Reddy and B. Murmann, "TinyML: Current Progress, Research Challenges, and Future Roadmap," 2021 58th ACM/IEEE Design Automation Conference (DAC), 2021, pp. 1303-1306, DOI: 10.1109/DAC18074.2021.9586232.</p><p dir="ltr">[2] Banbury et al. (2020). Benchmarking TinyML Systems: Challenges and Direction.</p><p dir="ltr">[3] TensorFlow Lite team. (2021). On-device training in TensorFlow Lite. Available from: <a href="https://blog.tensorflow.org/2021/11/on-device-training-in-tensorflow-lite.html" target="_blank">https://blog.tensorflow.org/2021/11/on-device-training-in-tensorflow-lite.html</a></p><p dir="ltr">[4] "Get Started with TinyML Webinar". &nbsp;2021. YouTube. Uploaded by: Hackster.io. <a href="https://www.youtube.com/watch?v=7OCIm_gh55s" target="_blank">https://www.youtube.com/watch?v=7OCIm_gh55s</a></p>

The article covers the basics of TinyML, a technology that improves the privacy, energy efficiency, affordability and reliability of devices utilising artificial intelligence. Be a part of TinyML for good, an online event and showcase of ideas that aims at exploring the topic in more detail.

TinyML for Good: where machine learning meets edge computing

A closer look at the recent market trends and audio technologies that enhance the functionalities of TWS devices.

TWS Evolution: What is next in True Wireless Audio?

<p dir="ltr"><em>Wevolver and&nbsp;<a href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant</a> are creating a series that explore the work of innovators and the future of pervasive AI. Syntiant is developing ultra-low-power AI processors. Because they believe in the importance of innovation, Syntiant is engaging in these fireside chats with engineers and designers who are on the cutting edge of their field.</em></p><p dir="ltr">The fourth conversation in our Voice Of Innovation series was held between Syntiant CEO Kurt Busch and Andreas Urschitz, Division President Power &amp; Sensor Systems at Infineon. Infineon is one of the largest semiconductor companies in the world, they have partnered with Syntiant for many years with the shared goal of innovating artificial intelligence at the edge. In this wide-ranging conversation, Kurt and Andreas discuss the potential of deep learning, strategies to improve efficiency, and the importance of system innovation.&nbsp;</p><h1 dir="ltr">Building the five senses into semiconductors</h1><blockquote><p dir="ltr">“Infineon is pursuing its vision of emulating the five human senses, eventually even the sixth human sense, into each and every kind of electronics.” - Andreas Urschitz.</p></blockquote><p dir="ltr">To make electronics understand in a meaningful way and without latency, such as understanding voice through a microphone and interpreting a specific command or making electronics understand a particular gas combination in the environment or make equipment smell, requires artificial intelligence at the edge through machine learning.&nbsp;</p><p dir="ltr">We have always tried to move the real world into the visual into the virtual world in some way, says Kurt, and with the advent of deep learning, we were able to do things very very quickly. For example, in a very short period of time speech recognition was developed, and within a couple of years, devices such as Amazon Echoes have appeared. This is the sort of thing that deep learning can bring to the table - both rapid innovation and entirely new types of interfaces.&nbsp;</p><h1 dir="ltr">Prioritizing sustainability&nbsp;</h1><p dir="ltr">With processing at the edge, no matter how good a device is there is going to be a demand for more, continues Kurt. What we are creating is a never-ending cycle of increased intelligence inside these devices and it's actually truly exciting because a solution that you have today that works well, you know that next year you are going to have 10 times the performance and that will generate seamless user interfaces.&nbsp;</p><p dir="ltr">Of course, this increase in compute has power requirements. According to Kurt and Andreas, both Syntiant and Infineon are dedicated to prioritizing ways to increase the efficiency of their solutions in order to strive for better environmental credentials and clear sustainability. Syntiant has ramped up its research and investment into power flow solutions, but just energy saving is not enough. What’s crucial is the data that is collected through the millions of sensors we are now surrounded by is analyzed and pre-filtered so that the actual amount of data that needs to be managed in the data center is reduced to the bare minimum. The solution to this is edge processing.</p><h1 dir="ltr">The importance of system innovation</h1><p dir="ltr">Digitalization is changing so many things in our environment, says Andreas. One of the things we have observed over the last couple of years is that formerly we are confronted with a world with so much information and technology available but often quite spread out over the planet.&nbsp;</p><p dir="ltr">Andreas: “We must ask ourselves how could you combine the different technologies - &nbsp;software, &nbsp;hardware - to create meaningful use cases that are helping humanity? We see systemic proceedings such as bringing partners together who are owners of different bits and pieces of technologies to come up with things that either solve customers’ problems or create opportunities as an essential way to make meaning of these technological advances. I believe there are a lot of chances for different companies to team up jointly to strive for system innovation to truly make an impact that goes beyond what is possible alone.”&nbsp;</p><hr><p>This video is part of a series of fireside chats in which Wevolver and Syntiant partner to engage with global innovators. In <a href="https://www.wevolver.com/tag/fireside-chat" rel="noopener noreferrer" target="_blank">previous discussions </a>we spoke with Arduino CEO Fabio Violante, Star Wars Animatronic Designer Gustav Hoegen and Fashiontech Designer Anouk Wipprecht. &nbsp;I encourage you to stay tuned and to follow <a href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant's profile</a> for more of these conversations.</p>

#4 of our 'Voice of Innovation' fireside chat series, in which Syntiant CEO Kurt Busch talks with Andreas Urschitz Division President Power & Sensor Systems at Infineon.

Fireside Chat With Andreas Urschitz from Infineon

An overview of the architecture of IIoT and how it is developing to drive the scalability of Industry 4.0.

The engineer's guide to Industrial IoT and Industry 4.0

From Single Models to Full MLOps, Edge Impulse Is The New Instruction Engine For A Smarter Industrial Sector. 


Embedded Machine Learning Is Giving Industrial Machines The Brains They Always Wanted.

<p dir="ltr"><em>This article was co-authored by Raj Senguttuvan and Vikram Shrivastava.&nbsp;</em>&nbsp;</p><h1 dir="ltr">Introduction</h1><p dir="ltr">The last decade, but particularly 2020, saw a massive increase in video calling and engagement with voice-enabled devices primarily through the work-from-home trend created by the pandemic. However, our use of video calls isn&rsquo;t restricted to just work Zoom meetings. We are now using video calling for teaching, exercise classes, to experience live music, as a tool for interacting at conferences, and <a href="https://www.eatwith.com/" target="_blank">even to go to restaurants.</a> This virtual engagement occurs via laptops, smartphones, tablets, and other IoT devices such as Amazon Echo Show, Facebook Portal, Peloton, <a href="https://imp.i277339.net/c/1141873/823006/11505?u=https%3A%2F%2Ftempo.fit%2Fpdp%2Ftempo-studio&subId2=[]st[p]cjv2u8o050017o5yeyn1o93c5[i]H2ac18[t]w[r]google.com[d]D[z]m" target="_blank">Tempo Studio</a>, etc. We have also embraced the use of voice-controlled devices such as home assistants and wireless earphones, and other smart home applications.&nbsp;</p><p dir="ltr">The core point that limits enjoyable and engaging interactive audio, video call, or home assistant experiences is consistent sound quality in the presence of noise and other distractors. The ability of your device to intelligently manage sound is what makes or breaks your ability to communicate.&nbsp;</p><h2 dir="ltr">Increasing audio expectations</h2><p dir="ltr">Dubbed &lsquo;smart sound&rsquo; by product makers, audio intelligence is the ability of a device to process sound to deliver the best user experience. As the increase in voice-led device use for communication, entertainment, and health management increases, so does the demand for a seamless, low-barrier experience with increased functionality. Users now expect devices to understand more than simple wake or keywords (such as Alexa) and seek the ability to move between devices and applications with superior sound quality to enable an immersive and seamless experience, whether for professional conferencing or personal entertainment.&nbsp;</p><p dir="ltr">Devices should be able to combine your voice and/or voice command with your personal preferences and environmental data to adapt sound processing to your specific context. This is known as contextual awareness.&nbsp;</p><h1 dir="ltr">What is Contextual Awareness?</h1><p dir="ltr">Contextual-aware devices combine user-specific information, such as location, preferences, and ambient sensor data, to better understand what the user is asking for and more accurately execute functions in response to a certain command or trigger.&nbsp;</p><p dir="ltr">Always-listening devices use signal processing technology combined with <a data-lf-fd-inspected-kn9eq4roawkarlvp="true" href="https://mediatum.ub.tum.de/doc/1463108/document.pdf" rel="noopener noreferrer" target="_blank">machine learning</a> to differentiate between sound types such as natural sounds, voices, background interference, etc. These sounds are often divided into &ldquo;scenes&#39;&#39; and &ldquo;events&rdquo;. A scene is the users setting such as a noisy airport terminal or a quiet working space. The events are someone speaking, a glass breaking, or a dog barking. A context-aware device can process these groups of sounds to ensure the intent of the action, whether it is a video call or a voice command, and is processed for optimal experience.&nbsp;</p><p dir="ltr"><br></p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 788px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE0NzkxMjg1ODE2LUlNQUdFIDQ2LTAxLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Figure 1: Context-aware system schematic.</span></span></span><p class="fr-img-space-wrap2">&nbsp;Dedicated audio edge processors with a focus on audio fidelity and with machine learning optimized cores are the key to contextually aware and high-quality audio communication devices. These processors can deliver enough compute power to process complex audio using just a small percentage of the energy of a generic Digital Signal Processing (DSP) implementation. These audio edge processors use ML algorithms for superior sound processing in addition to providing a &nbsp;more contextual audio experience.&nbsp;</p></div><p><br></p><h2 dir="ltr">&ldquo;I think you are on mute&rdquo;</h2><p dir="ltr">Video conference calls have been a focal point in 2020 and continue to dominate work and social lives in 2021. The gaps in technology are revealed quickly when devices, people, and environments fail to keep up with the demands of the calls - to the point where the phrase &lsquo;I think you&#39;re on mute&rsquo; was dubbed the<a href="https://www.humanresourcesonline.net/the-most-used-business-phrases-of-2020-i-think-you-re-on-mute" rel="noopener noreferrer" target="_blank">&nbsp;most used business term</a> in 2020. Dedicated audio processors with ML is the necessary approach to solve some of the common video conferencing pain points.&nbsp;</p><p dir="ltr">ML on dedicated audio processors are already addressing some common video conference pain points:</p><ul><li dir="ltr"><strong>Selective noise canceling</strong>. By understanding a range of scenes and action noises, certain background elements such as a fan, heater, or street noise can be removed, thereby, improving overall call quality and the ability to work in a wider variety of environments.&nbsp;</li><li dir="ltr"><strong>Hi-fi audio reconstruction.</strong> Analogous to the way budget smartphones use smart software to increase the image quality from low-quality cameras, ML in real-time audio processing can improve the audio quality from low-quality microphones.&nbsp;</li><li dir="ltr"><strong>Spatial audio for improved user experience.</strong> Already available on advanced gaming headphones and some TWS devices, advanced digital sound optimization techniques provide users a spatial audio experience. Spatial audio can create a natural listening experience for the user, which can help mitigate &ldquo;<a href="https://news.stanford.edu/2021/02/23/four-causes-zoom-fatigue-solutions/" target="_blank">Zoom Fatigue</a>&rdquo;.</li></ul><p dir="ltr">In the future, ML on audio processors will have even more advanced capabilities such as real-time speech processing that would enable automatic language translations. Additionally, ML can solve the &lsquo;on mute&rsquo; problem by becoming more aware of the user&rsquo;s context and offer prompts to unmute.</p><h1 dir="ltr">A look at specialized audio edge processors</h1><p dir="ltr">There are several areas audio edge processors get right to deliver exceptional audio and voice command experiences.&nbsp;</p><h2 dir="ltr">Noise and Distance</h2><p dir="ltr">Beamforming is the process where the device can engage its ability to focus the listening or output sound in a particular direction. Noise suppression makes conversations in everyday scenarios possible. A listening device determines the direction(s) of the voice source(s) and that of noise sources. ML classification techniques are then used to determine which beams have voice or noise in them. &nbsp;The DSP then focuses on beams with voice content only for further voice UI processing. For example, in a conferencing system, the device must identify the direction of arrival of sound, and multiple speakers must be tracked at all times in a 360-degree fashion. Noise sources can also be classified further for audio event detection like glass break, fire alarms, and gunshots, etc for further expanding the role into audio aware smart home systems.&nbsp;</p><p dir="ltr">Proximity detection is also essential for dynamic listening and speaking experience. &nbsp;The device detects user proximity to the microphone and adjusts the microphone&rsquo;s gain. This feature enables active video conferencing for presentations, workouts, and learning environments. These features are central to the design of advanced video conferencing devices such as the new Amazon Show which features a screen that rotates to where it senses the user.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/fxUSSgrHL-o?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 789px; height: 504px;" data-gtm-yt-inspected-8="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><h2 dir="ltr">Latency</h2><p dir="ltr">Humans can <a href="https://towardsdatascience.com/real-time-noise-suppression-using-deep-learning-38719819e051" rel="noopener noreferrer" target="_blank">generally tolerate</a> up to 200ms of end-to-end latency on a call before we begin to talk over each other. Low latency processing in edge processors is, therefore, a critical requirement for ensuring high-quality voice communication.</p><h2 dir="ltr">Power consumption</h2><p dir="ltr">Audio edge processors designed with proprietary architectures, hardware accelerators, and special instruction sets can optimally run audio and ML algorithms. These optimizations help with reducing power consumption in audio-heavy use cases such as video conferencing.</p><h2 dir="ltr">Integration</h2><p dir="ltr">Audio edge processors that open up their architectures and development environments accelerate innovation by providing audio application developers the tools and support to create new devices and applications. Future audio devices will be a collaborative effort.</p><h2 dir="ltr">Security</h2><p dir="ltr">Edge processing can minimize the need for cloud processing providing many benefits, including increased data security. For example, most consumers are not comfortable with data from personal smart home devices being transferred continuously to the cloud for processing. These fears have proven true with several <a href="https://www.theguardian.com/technology/2020/dec/23/amazon-ring-camera-hack-lawsuit-threats" target="_blank">major breaches</a> by leading device manufacturers in the last years. Peace of mind can be achieved when personal data is processed for analysis or inference at the device. An excellent example of this is a smart home security device that has been trained to hear certain event sounds, such as glass breaking, that act as a trigger to alert the homeowner. As the processing of the sound and the alert occurs on the device, it does not need to be in continuous connection to the cloud, which enhances the security of the system.</p><h1 dir="ltr">Delivering Audio Intelligence at the Edge</h1><p dir="ltr"><a href="https://www.knowles.com/products/audio-processors" target="_blank">Knowles AISonic&trade; IA8201: Dual Core</a> is an audio edge processor specifically designed for advanced audio and machine learning applications, enabling power-efficient intelligence and privacy at the edge. It offers robust voice activation and multi-microphone audio processing with sufficient compute power to perform advanced voice processing and audio output, context awareness, and gesture control for today&rsquo;s most advanced consumer electronics. Processing voice and audio voice with an audio edge processor like IA8201 can keep the power-hungry host processor off as long as possible to extend battery life. &nbsp;</p><p dir="ltr">In addition, it also enables new use cases for enhanced user experience by having the capability to run ML algorithms on multiple sensor inputs to deliver a more natural and contextual user experience.</p><p dir="ltr">Audio edge processors like IA8201 have a place in a variety of audio and voice-enabled applications including smart speakers, video conferencing devices, tablets, Bluetooth headphones, and smart home devices.</p><p dir="ltr"><strong id="isPasted">Recommended reading:&nbsp;</strong><span style="background-color: transparent;"><a href="https://www.wevolver.com/article/enhancing-the-utility-of-audio-ai-in-everyday-devices-using-audio-source-classification" rel="noopener noreferrer" target="_blank">Enhancing the utility of Audio AI in everyday devices using Audio Source Classification</a></span></p><h1 dir="ltr">Conclusion</h1><p dir="ltr">Dedicated audio edge processors will define the next generation of audio and voice-enabled devices creating more context-aware, immersive, and seamless audio communication experiences. Their ability to enable efficient processing for low power and low latency voice communications, noise reduction, context awareness, and accelerated ML inferencing of sensor inputs opens the possibilities to an explosion of new user experiences for the human-machine interfaces.&nbsp;</p><hr><h1><em>References and further reading:</em><a href="https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt" target="_blank"></a></h1><p><a href="https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt" target="_blank">https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt</a></p><p dir="ltr">Antonini, Mattia &amp; Vecchio, Massimo &amp; Antonelli, Fabio &amp; Ducange, Pietro &amp; Perera, Charith. (2018). Smart Audio Sensors in the Internet of Things Edge for Anomaly Detection. IEEE Access. PP. 1-1. 10.1109/ACCESS.2018.2877523.&nbsp;</p><p dir="ltr">Do, Ha &amp; Sheng, Weihua &amp; Liu, Meiqin &amp; Zhang, Senlin. (2016). Context-aware sound event recognition for home service robots. 739-744. 10.1109/COASE.2016.7743476.&nbsp;</p><p dir="ltr"><a data-lf-fd-inspected-kn9eq4roawkarlvp="true" href="https://www.knowles.com/docs/default-source/default-document-library/knowles-ia8201-product-brief-final1b6a11c731dff6ddbb37cff0000940c19.pdf" target="_blank">https://www.knowles.com/docs/default-source/default-document-library/knowles-ia8201-product-brief-final1b6a11c731dff6ddbb37cff0000940c19.pdf</a></p><p dir="ltr"><a href="https://voicebot.ai/2019/06/01/dedicated-audio-processors-at-the-edge-are-the-future-here-are-the-reasons-why/" target="_blank">https://voicebot.ai/2019/06/01/dedicated-audio-processors-at-the-edge-are-the-future-here-are-the-reasons-why/</a></p><p dir="ltr"><a href="https://www.networkworld.com/article/3262105/how-edge-computing-makes-voice-assistants-faster-and-more-powerful.html" target="_blank">https://www.networkworld.com/article/3262105/how-edge-computing-makes-voice-assistants-faster-and-more-powerful.html</a></p>

Dedicated audio edge processors with machine learning will deliver increasingly natural communication experiences.

Audio edge processors deliver sublime context-aware communication experiences

Embedded Machine Learning is not just another cool technology; Rather it is an enabler for better business results and improved bottom lines for enterprises in many different sectors

Embedded Machine Learning Applications with Proven ROI

How Embedded Machine Learning unblocks the potential of the currently unused enterprise data and enables a new wave of real-time, sustainable, and ROI generating applications.

The Embedded Machine Learning Revolution: The Basics You Need to Know

Synopsys Inc

Voltera

embedded machine learning

embedded machine learning

ORGANIZATIONS.

SHAPING THE INDUSTRY.

Edge Impulse

Knowles Corporation

Arduino Pro

Latest Posts