Before diving into the details of Embedded Machine Learning it is essential to address - the obvious. Often new terms and technologies are approached with a must-have attitude, however, the technical application or product where Embedded Machine Learning makes sense needs to be established in the first place. In some cases, a simple threshold approach is just fine. Once this is cleared then the insightful journey can begin.
Embedded machine learning applications consist of machine learning pipelines that are deployed on embedded devices. The development of such pipelines in computationally constrained environments presents most of the challenges of developing any machine learning application. However, the challenges are intensified due to the need for capturing data from embedded devices and deploying machine learning models on them. Specifically, the challenges of embedded machine learning span the following areas:
Fortunately, IoT and data science experts of the embedded machine learning community are intensively working on solutions that alleviate these challenges and bring the merits of this new evolutionary paradigm closer to industry practitioners. In principle, the solution lies in the provision of tools, techniques, and blueprints that simplify the development and deployment of embedded machine learning applications. In several cases, the automation of development and deployment steps can also accelerate the implementation process, while hiding the complexity from industrial end-users. Moreover, the creation of a vibrant community of embedded machine learning developers and experts is a key to the proliferation and spreading of knowledge within relevant industrial sectors.
To address the complexity of embedded programming and the diversity of the various embedded systems platforms, the embedded machine learning community is gradually providing ready-to-use examples and blueprints on how to develop applications for specific platforms. Building on past knowledge and sample applications for specific platforms, it is possible to significantly reduce development times and costs. Beyond providing blueprints, the embedded machine learning community is continually contributing knowledge on how to select the best possible embedded device for a task at hand. This selection is critical for the success of machine learning applications and must be carefully planned.
Embedded machine learning development can benefit from datasets collected for certain devices and application contexts. Such datasets could lower the data collection effort needed in the scope of embedded machine learning projects. They can support different stages of a project’s implementation, including for example the initial experimentation and prototyping phase. In several cases, data collected for specific embedded devices and applications can be repurposed and reused in other contexts. This is possible thanks to machine learning techniques like transfer learning , but also thanks to community efforts to repurpose these datasets through their appropriate processing. In several cases, industry practitioners and domain experts can also benefit from pre-trained machine learning models, which can greatly lower data collection and preparation efforts. The embedded machine learning community is already offering pre-trained models for several devices.
To lower the application development and deployment effort for machine learning applications, industry practitioners must use visual tools for IoT and Machine Learning applications. Such tools provide access to popular features for programmers, such as flow programming, reusable libraries, access to one-click-samples, and various “drag-and-drop” functionalities. In this direction, embedded machine learning developers assemble end-to-end tooling solutions based on the combination of mainstream IoT (e.g., Arduino IDE) and Machine Learning (e.g., Jupyter Notebooks) tools. The ultimate vision of the growing embedded machine learning community is to provide AutoML functionalities to domain experts, which will help them develop end-to-end solutions in a single integrated environment, without a need for deep knowledge of embedded systems and data science processes.
Most importantly, domain experts and industrial end-users must be provided with easy access to large and well-structured knowledge about embedded machine learning applications, including knowledge about the business benefits and the practical use cases of embedded machine learning, as well as access to resources that facilitate application development.
The above-listed solutions are not in the inception stage. Rather they are already available to thousands of developers and many hundreds of enterprises that are riding the wave of embedded machine learning solutions. As a prominent example, Edge Impulse, one of the leading platforms for embedded machine learning, has already a vibrant ecosystem and an active community of many thousands of developers and experts. Edge Impulse is a one-stop shop for embedded machine learning resources. It provides a single access point to fully integrated and best-in-class resources for developing and deploying real-life embedded machine learning applications. Edge Impulse alleviates the lack of visual tools for embedded machine learning applications, which are a key to facilitating development and deployment of industrial applications.
By joining the Edge Impulse ecosystem, business enterprises can instantly gain access to a pool of cutting-edge machine learning resources and solutions, in these areas:
a) Access to data sets for model training: Within the Edge Impulse ecosystem developers find datasets that can be used to train machine learning models for embedded devices. These datasets are used for training and experimentation. However, they can be also useful as a starting point for production-ready, enterprise-scale deployments, following some fair effort for their enhancement and repurposing. Furthermore, within the Edge Impulse ecosystem, users can discover publicly available projects and related datasets that accelerate their embedded machine learning projects. Edge Impulse provides insights on how such public datasets can be enhanced, repurposed, and used in conjunction with embedded devices.
b) MLOps for Embedded Devices: Edge Impulse supports modern MLOps practices for collaboration and communication between data scientists, machine learning engineers, and ML operations experts. Edge Impulse is currently offering a complete platform for organizations to cross collaborate and manage their MLOps pipelines. It is the sole platform that offers MLOps support in the embedded devices space. Documenting data and processing steps along the way for compliance and tech consistency (data lineage).
a) Access to Cloud-based API: The functionalities of the Edge Impulse Studio are accessible through the Edge Impulse API. The latter can be used to build AutoML pipelines, which find the best parameters for machine learning models (e.g., feature extraction using various digital signalling processing blocks), and enable domain experts to optimize their embedded machine learning applications without in-depth IoT and data science knowledge. Edge Impulse provides access to documentation and other knowledge resources about using the Edge Impulse Studio as a cloud service.
b) Delivery of Best-in-Class Embedded Models for Industrial Use Cases: Edge Impulse enables the delivery of best-in-class optimized embedded models. The Edge Impulse tools go beyond enabling embedded machine learning developments to the delivery of optimal models that deliver the best possible performance and accuracy. This is one of the main benefits offered by Edge Impulse to industrial enterprises, which have very demanding performance requirements.
a) Edge Impulse Studio for Embedded Machine Learning as a Service: The Edge Impulse platform offer access to the Edge Impulse Studio, an integrated visual environment that enables the development of end-to-end embedded machine learning pipelines. Using the Studio, it is possible to control embedded devices and to experiment with data ingestion functions. Moreover, the Studio provides AutoML functionalities which facilitates and speed up machine learning development. Furthermore, the Edge Impulse Studio is accessible as a cloud service, which enables an Embedded Machine Learning as a Service paradigm.
b) Examples and Tutorials for Embedded Machine Learning on various devices: Edge Impulse provides readily available examples and blueprints on how to build solutions for a rich set of embedded devices, such as ST B-L475E-IOT01A, Arduino Nano 33 BLE Sense, Eta Compute ECM3532 AI Sensor, Himax WE-I Plus, Nordic Semiconductor nRF52840 DK, Nordic Semiconductor nRF5340 DK, and Silicon Labs Thunderboard Sense 2. Moreover, there are resources for users that would like to build embedded machine learning in any smartphone. These resources facilitate enterprises that use the above-listed boards to kick-start their developments and to lower relevant learning curves. They are extremely useful even for those that plan to use other devices, as they provide tangible examples of how embedded machine learning works in practice.
a) Tutorials and Resources on Machine Learning Modules (including deep learning): In the scope of the Edge Impulse platform, it is possible to access tutorials about how to leverage different machine learning models. There are also resources about building modular applications, where different processing blocks are plugged in applications. For instance, there are resources about plugging in signal processing blocks and building neural networks based on popular deep learning frameworks like Keras.
b) Embedded Machine Learning Knowledge Base: Edge Impulse provides many knowledge assets about embedded machine learning, including tutorials, examples, blueprints, as well as documentation. Moreover, the company has recently sponsored the development of the course, “Introduction to Embedded Machine Learning” on the popular Coursera platform. The course fosters the development of embedded machine learning skills for many students, researchers, and practitioners worldwide.
Many of the above-listed functionalities are offered freely to thousands of developers that active members of the Edge Impulse ecosystem. However, Edge Impulse offers a richer set of resources, including datasets and cloud-based tools, to enterprises that aim at using embedded machine learning to improving their business results. While the free services of Edge Impulse facilitate developers to get started with embedded machine learning and to improve their skills, the enterprise version is a vehicle for improving business competitiveness and productivity. Specifically, as earlier outlined, Edge Impulse delivers optimized, best-in-class embedded machine learning models, which make the Edge Impulse ecosystem the number one choice for industrial applications with stringent performance requirements. Moreover, Edge Impulse offers industrial enterprises access to AutoML and MLOps tools, which improve collaboration and reduce development costs.
In conclusion, embedded machine learning and TinyML are currently revolutionizing the IoT and AI landscape. AI is no longer reserved for high-end computing infrastructures that comprise GPUs and TPUs. Rather, it can also be deployed on IoT devices and microcontrollers, in order to deliver tangible performance, latency, privacy, cost, and sustainability benefits. Industrial enterprises need not wait to capitalize on these benefits. They must already explore how they can leverage embedded machine learning to harness unused data and to improve their business processes. By joining the Edge Impulse ecosystem today, they can gain access to a wealth of resources that can help them adopt and fully leverage embedded machine learning, in ways that could set them apart from their competitors. This is what makes it easy for an embedded engineer to deploy Machine Learning - at all - in the field and "hyper-fast" compared to the time before Edge Impulse.
Edge Impulse is the leading development platform for embedded machine learning, used by over 1,000 enterprises across 10,000 ML projects worldwide. We are on a mission to enable the ultimate development experience for machine learning on embedded devices for sensors, audio, and computer vision, at scale.
From getting started in under five minutes to MLOps in production, we enable highly optimized ML deployable to a wide range of hardware from MCUs to CPUs. With Edge Impulse, developers, engineers, and domain experts solve real problems using machine learning in embedded solutions, speeding up development time from years to weeks. We specialize in industrial and professional applications including predictive maintenance, asset tracking, and monitoring, and human and animal sensing.
1. G. Berry, "Challenges and potential solutions for complex embedded systems," 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT), Taipei, 2011, pp. 1-1, doi: 10.1145/2038642.2038644.
2. Wirth, R. & Hipp, Jochen. (2000). CRISP-DM: Towards a standard process model for data mining. Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining.
3. Y. Roh, G. Heo and S. E. Whang, "A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective," in IEEE Transactions on Knowledge and Data Engineering, doi: 10.1109/TKDE.2019.2946162.
4. Xin He, Kaiyong Zhao, Xiaowen Chu, “AutoML: A survey of the state-of-the-art”, Knowledge-Based Systems, Volume 212, 2021, ISSN 0950-7051, https://doi.org/10.1016/j.knosys.2020.106622.
5. H. Jin, Q. Song and X. Hu, "Auto-Keras: An efficient neural architecture search system", Proc. Int. Conf. Knowl. Discovery Data Mining, pp. 1946-1956, 2019.
6. X. Zhang, Z. Huang and N. Wang, "You only search once: Single shot neural architecture search via direct sparse optimization", Proc. ICLR, pp. 1-11, 2019.
7. F. Zhuang et al., "A Comprehensive Survey on Transfer Learning," in Proceedings of the IEEE, vol. 109, no. 1, pp. 43-76, Jan. 2021, doi: 10.1109/JPROC.2020.3004555.