It is not only about AI in MEMS and sensors

Sensors are devices designed to detect and measure physical properties or environmental changes and convert this information into a format that machines can understand in the digital world. They serve as the eyes, ears, skin, tongue, and nose, enabling to gather data and make informed decisions. The integration of Artificial Intelligence (AI) with Micro-Electro-Mechanical Systems (MEMS) sensors has revolutionized the way we perceive and interact with the environment and ourselves. This article explores the significant advancements in AI-driven MEMS sensors, focusing on STMicroelectronics' (ST) cutting-edge technologies, their applications, and the ecosystem supporting these innovations.

ST MEMS sensor AI technology insights

In ST we have been talking about incorporating AI into their product designs since 2017 when we launched in the market the first AI technology inside our MEMS sensors. This dialogue reflects the growing importance and interest in leveraging AI in sensors to enhance customers product functionality, improve user experiences with longer battery life, and gain a competitive edge in the market. We created more than 20 products deployed in the market with embedded data processing capabilities as finite state machine (FSM), and two AI technologies, machine learning core (MLC) and intelligent sensor processing unit (ISPU). 

Finite State Machine (FSM)

The Finite State Machine (FSM) is a behavioral model embedded in ST MEMS sensors. It consists of a finite number of states with specific transitions between them, similar to a flowchart. FSMs can process both internal and external data, making them ideal for recognizing user-defined gesture patterns. Very easy to use and flexible to enable product customization and to trigger events to be identified. FSM is available on accelerometers (i.e. LIS2DUX12), and IMUs (i.e. LSM6DSV16X, LSM6DSV80X, ISM330BX, ASM330LHHX) among other ST MEMS product categories.

Machine Learning Core (MLC)

MLC is a hard-coded engine configurable for decision trees logic that can be trained to trigger an action when a specific event is detected. With this AI technology inside the sensor, it’s possible to recognize precise movements and communicate the event to a processor with the best system energy efficiency. The MLC has 512 nodes, an operative data rate up to 960 Hz, and a computational block based on a 2nd order IIR filter and 12 different features extraction. Capable to run up to 8 decision trees, it enhances performance and speeds-up development by avoiding developing complex algorithms. Offloading processing and reducing data transferred to MCU, the MLC decreases latency and enhances data privacy as processing is done inside the ST MEMS sensor. For example, an activity recognition algorithm will see its power consumption divided by 10 when using MLC. MLC is available on motion sensors as accelerometers (i.e. LIS2DUX12), IMU (i.e. LSM6DSV16X, LSM6DSV80X, ISM330BX, ASM330LHHX) and many others.

Intelligent Sensor Processing Unit (ISPU)

 The ISPU is an integrated C-programmable digital signal processor (DSP), allowing for complex data processing and advanced AI algorithms. It supports both internal and external sensors data processing, making it highly versatile. Based on a 32-bit RISC Harvard architecture core, the ISPU embodies 40KiB SRAM, where 8 KiB and 32 KiB are respectively the data and program memory. Non-volatile memory is not integrated, and it depends on the system architecture of the application. The ISPU features multiple bits precision integer multiply and accumulate, as well as floating point unit 32 bits.  The ISPU can run several AI algorithms (machine learning and neural network) but not only: developers can use it for their own algorithms to process data from internal sensor and externals connected as sensor hub. ISPU is available on IMUs (LSM6DSO16IS, ISM330IS) and key benefits are:

• embedded in ST MEMS sensor ASIC eliminates the physical distance between the sensing and computing elements with huge latency minimization 

• pin to pin compatibility with standard products in its category 

• efficiency in energy, data privacy, performance

• flexibility as customer can easily program ISPU with tools provided free of charge by ST and in C language. 

Any best application with smart MEMS sensors?

ST smart MEMS sensors with embedded data processing, AI included, are for applications where the energy and computational efficiency have a key role. As examples:

Activity recognition

In wearables devices, smart motion sensors are particularly well adapted to track activities like walking, running, driving, and cycling, enhancing user daily insights with low power and high level of accuracy.

Wellness monitoring

For wellness, healthcare applications, it enables to monitor patient activities, detect falls, and track rehabilitation progress, improving patient safety and care.

Laptop and tablet carry-on detection

AI-enabled sensors can detect when a user is carrying their laptop, optimizing power consumption and performance. This capability enhances user experience and extends battery life.

Anomaly detection in motors & pump devices

In industrial settings, smart sensors are instrumental in monitoring the performance of both motors and pump devices, detecting anomalies in real-time. Early identification of issues is crucial, as it prevents costly downtime and maintenance, thereby enhancing efficiency and reliability.

Vehicle status monitoring

In the automotive industry, AI-enabled sensors monitor various vehicle parameters, such as alarm events and vehicle behavior when the driver is not there. ST sensors provide valuable insights that enhance the vehicle monitoring status in all the situations.

How to break the barrier of AI adoption in smart sensors?

ST offers a complete MEMS ecosystem to support development and fast prototyping. From hardware tools to application examples, every developer can find the right solution and reduce time-to-market and related costs.

For the smart MEMS sensors, the offer is MEMS Studio, a desktop tool that support the data collection and creation of decision trees for MLC, the optimization of tiny NN for ISPU and the deployment of the models in the sensor. ST AIoT Craft is another intuitive tool cloud-based offering developers to create a project from scratch, bring their data, generate their decision trees for MLC, and tweak it to optimize inference times and accuracy in the smart MEMS sensor for a low-power IoT solution. Finally, Nano Edge AI Studio offers a comprehensive solution by integrating NEAI software library directly into the sensors with ISPU. This powerful combination ensures seamless and effective anomaly detection, enabling prompt identification and resolution of potential problems. These tools can be used with available use cases in ST's GitHub repository. They contain application and configuration examples for AI features mentioned above.

Thanks to Mouser you may also get samples for the hardware tools to run quickly your AI algorithm in ST smart sensors.

Conclusion

The integration of AI with MEMS sensors represents a significant leap forward, enabling sensors to do much more than just collect data. With AI, sensors can process information locally, make real-time decisions, and interact intelligently with their environment. STMicroelectronics is at the forefront of this innovation, with technologies like FSM, MLC, and ISPU leading the way. These AI-enabled sensors have vast applications, from personal electronics to industrial automation and automotive monitoring. Supported by a robust and wide ecosystem, including MEMS Studio and ST AIoT Craft, developers have the tools they need to create the next generation of smart, efficient, and sustainable devices.

Written by Gildas Henriet, EMEA MEMS Product Marketing Manager and Mohamed Saadna, EMEAProduct Marketing Engineer, STMicroelectronics.

Links:

Ecosystem: https://www.st.com/content/st_com/en/ecosystems/MEMS-Sensors-Ecosystem-for-Machine-Learning.html 

Blog MEMS Studio: https://blog.st.com/mems-studio/ 

Blog AIoT Craft: https://blog.st.com/st-aiot-craft/ 

AIoT Craft : https://staiotcraft.st.com/index.html#/wiki?id=1_introduction.md

Example for MLC: https://blog.st.com/machine-learning-core-repository/