<h1 dir="ltr" id="isPasted">Introduction</h1>The modern workplace is rapidly changing and AI is emerging as a transformative force. With the growing use of AI-powered tools, companies are rethinking how to approach hiring, employee onboarding, and enhancing the efficiency, development, and well-being of employees.In this article, we delve into the role of AI in the workplace, highlighting its impact on hiring processes, improving employee retention, and workplace conditions. We present case studies of companies who have found success with AI tools. We also explore future trends and ethical considerations, providing an overview of how AI is positioned to help companies hire more effectively and create better places to work.<h1 dir="ltr">AI in the Hiring Process</h1>Hiring is time-consuming and costly for companies, and often frustratingly involves processes that overlook the most qualified candidates. Some problems with candidate screening, matching, and interviewing are:<ul><li dir="ltr">Over-reliance on keywords and algorithms, which may screen out very qualified candidates whose resumes do not precisely match job descriptions.</li><li dir="ltr">Bias in recruitment, which can hinder diversity and inclusion efforts and lead to a homogenized workforce.</li><li dir="ltr">High-volume applications and initial screenings, a task that relies heavily on the judgment of recruiters and is subject to many human factors.[1]</li></ul>Companies can address these challenges through the implementation of AI technologies in the recruitment process. The following are a few examples of the potential for AI to help companies make hiring more efficient and inclusive.<h2 dir="ltr">Streamlining candidate screening</h2>AI-powered applicant tracking systems can streamline sourcing and screening automating resume analysis and identifying qualified candidates efficiently and accurately. These are some case studies where AI is already improving efficiency in candidate screening.<ul><li dir="ltr">IBM Watson recruitment at Hilton&nbsp;– By integrating IBM Watson for candidate screening and matching, the Hilton hospitality chain saw a 50% reduction in time-to-fill, improved candidate quality, and enhanced hiring manager satisfaction.</li><li dir="ltr">Pymetrics gamified assessments at Deloitte&nbsp;– Deloitte, the US auditing and tax services company incorporated Pymetrics’ AI platform for gamified assessments into their candidate screening process. As a result, the company saw a 60% reduction in time spent on initial screenings, a 30% increase in diverse hires, and enhanced candidate experience.</li><li dir="ltr">HireVue video interviews at McDonald’s&nbsp;– The fast food chain McDonald’s implemented HireVue’s AI-enhanced video screening to streamline their high-volume hiring process. Use of the platform resulted in a 40% reduction in time-time hire, improved candidate engagement, and enhanced scalability.[2]</li></ul><h2 dir="ltr">Enhancing candidate matching</h2>AI algorithms can be used to match job skills and requirements without the over-reliance on specific job description keywords that is often the case with traditional algorithms. AI algorithms can use natural language processing to analyze candidate data and match actual qualifications with job requirements, improving the accuracy of candidate matching.[3]Additionally, by focusing on skills, qualifications, and experience rather than personal characteristics, AI can help reduce unconscious bias in recruitment, leading to a more diverse and inclusive workforce.[4]<h2 dir="ltr">Interview and assessment automation</h2>Companies can harness the power of AI using deep learning and natural language processing to improve the interview process for both candidates and recruiters. Image credit: mikemacmarketing | Source: wikimedia.orgAI-driven recruitment strategies can help make the interview process a much better experience for candidates and recruiters alike. AI tools can automate interview scheduling, conduct initial screenings, ask follow-up questions, and use natural language processing to analyze responses, making the interview process more effective and efficient.[5] In the case studies discussed above, the introduction of AI tools improved the recruitment experience for hiring managers and candidates, and improved candidate engagement.By incorporating AI tools into hiring, organizations can streamline candidate screening, improve matching accuracy, reduce bias, enhance efficiency, and ultimately create a more inclusive and effective hiring environment.<h1 dir="ltr">AI's Role in Employee Onboarding and Retention</h1>While it is important to find and hire the best candidates for the job, it is equally important to make sure that new employees are quickly brought up to speed and productive. Perhaps even more critical is to keep turnover rates low and retain employees by making sure that they are satisfied with the job. Companies can use AI-powered tools to build a more effective and engaged workforce.<h2 dir="ltr">Automated onboarding processes</h2>AI-powered recruitment software can analyze the experience, knowledge, and learning style of a new hire and generate tailored onboarding programs. This type of onboarding takes the employee’s personality and background into account, to help a new employee quickly become engaged and part of company culture.This type of streamlined onboarding means that a new employee becomes productive faster and may be more satisfied with the job overall. Research shows that a successful onboarding experience leads to greater employee retention — employees are twice as likely to be satisfied at work and contribute to a positive company culture.[6]<h2 dir="ltr">Predictive analytics for employee turnover</h2>AI is a powerful tool because it can collect and process large volumes of data. This ability to sift through data and identify patterns and correlations can be leveraged to pinpoint the things that contribute to employee turnover. Companies can use historical turnover data to develop models that predict potential turnover risks.With the power of AI, companies can better understand the specific reasons for turnover, make improvements, and test the outcome. For example, if historical turnover data points to dissatisfaction with limited opportunities for growth, the company can offer career development programs, use AI to monitor the result, and make improvements to the programs.[7]AI tools can have a significant impact on employee onboarding and retention, leading to more engaged, productive, and loyal employees.<h1 dir="ltr">Improving Workplace Conditions with AI</h1>Working conditions are another factor in employee satisfaction and retention. Companies can leverage AI-powered tools to optimize the workspace, support career development, and maintain employee health.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1710149861252-1NAGBQZbUUYzSHV_ckw0Z_sG2he6dCo8.jpg" style="width: 100%px;" class="fr-fic fr-dib">Employers can leverage AI to optimize the working environment while improving employee well-being and retention. Image credit: Alice Mai Ah | Source: wikimedia.org<h2 dir="ltr" id="isPasted">AI for workspace optimization</h2>Using AI, companies can create smart work environments that adapt to employee patterns, preferences, and usage. Automatic environmental adjustments like lighting, temperature, and desk configurations can lead to increased comfort, productivity, and energy cost savings for the company.AI systems can also be used to optimize resources like meeting rooms, workstations, and common areas, making sure facilities are used effectively and that employee needs are supported.<h2 dir="ltr">AI-enhanced training and development</h2>As discussed above, AI can be used during onboarding to get new employees working quickly. In addition to new hire training, AI tools can be used for ongoing growth, tailoring development programs to each employee’s learning styles, career goals, and performance. Using AI analytics, organizations can track the effectiveness of development programs and continuously improve them, leading to better career growth opportunities and satisfied employees.<h2 dir="ltr">Ensuring health and well-being</h2>Predictive AI analytics can play a crucial role in improving employee mental health by analyzing patterns in employee data and feedback to catch signs of stress or burnout. Companies can use this data to take proactive measures that support the well-being of employees.AI-powered ergonomic assessments and activity reminders can also help improve the workplace. Wearable devices and AI applications can provide data on workplace ergonomics and suggest improvements. Wearables can also remind employees to take breaks and stretch to maintain both mental and physical health.<h1 dir="ltr">Case Studies: AI-Driven Success in Hiring and Workplace Enhancement</h1>Companies today are already leveraging powerful AI tools to enhance hiring and the workplace. These are just a few case studies that show how companies are seeing success with these tools.<ul><li dir="ltr">ClearCompany&nbsp;– ClearCompany, a recruiting software platform, has automated employment verification, document submission, and analysis of learning pase using AI-driven onboarding software. This has resulted in a more efficient onboarding process and increased focus on strategic work that drives company growth.[6]</li><li dir="ltr">Genpact&nbsp;– This company has used AI to automate their entire onboarding experience, providing personalized experiences for new hires, which has helped speed up the onboarding process and improve retention.[8]</li><li dir="ltr">Tier 4 Group&nbsp;– This talent acquisition firm worked with TechScreen to streamline their tech hiring processes, resulting in a more effective, ethical, and considerate candidate screening process.[9]</li><li dir="ltr">Iceland Foods – During the pandemic, Iceland Foods partnered with Sapia.ai to adapt their recruitment processes to a changing world. AI automation helped the company find the right candidates in the midst of global upheaval, demonstrating the resilience of AI-driven recruitment.[10]</li></ul>These examples illustrate the diverse applications of AI in hiring and workplace enhancement, from improving candidate experience and streamlining the recruitment process, to fostering internal development and supporting ethical hiring practices.<h1 dir="ltr">Looking ahead: the next generation of AI in the workplace</h1>The integration of AI into the workplace is poised to have a huge impact on how we work. Here are some ways that we predict AI will transform the workplace in specific industries.<ul><li dir="ltr">Healthcare&nbsp;– By integrating data like medical imaging, lab results, and checkups, AI advancements will likely lead to more efficient healthcare systems, as well as more effective care, with better outcomes for patients.</li><li dir="ltr">Finance&nbsp;– The role of AI in fraud detection, personalized financial advising, and record keeping is likely to grow, but a strong emphasis on human oversight and ethical decision-making will remain a key part of the finance industry.</li><li dir="ltr">Education&nbsp;– In education, AI will be able to analyze and provide insights into student learning, and will help institutions create more engaging and personalized experiences for both students and teachers.</li><li dir="ltr">Retail&nbsp;– Predictive analytics will enable retailers to understand customer behavior, manage inventory, and improve customer service. AI has the potential to boost customer satisfaction and operational efficiency.</li><li dir="ltr">Manufacturing&nbsp;– AI will significantly impact how work is done in the manufacturing industry as well. Collaborative robots (cobots), predictive maintenance, and smart factories are AI-powered tools that will likely improve the safety and efficiency of manufacturing.</li></ul><h1 dir="ltr">Ethical considerations: balancing technology and humanity</h1>AI is a promising tool for improving workplaces and the efficiency, health, and well-being of workers. And because AI is for the benefit of humanity, it is important to examine ethical considerations and maintain a balance between technology and human values.When using AI tools in hiring and HR functions, It is critical for companies to remain vigilant against bias and discrimination. Laws, standards, and guidelines that require AI tools to be audited for bias are already in place. It is also important for companies to monitor AI tools for bias during recruiting, hiring, and promotion.[11]In an increasingly automated workplace, it is also essential for companies to balance AI and human interaction. While AI magnifies our human capabilities and streamlines the work that we do, it is important for companies to foster human relationships and empathy, and to take an ethical stance to put humans first when it comes to AI strategies.<h2 dir="ltr">Conclusion</h2>The integration of AI into the hiring process, employee onboarding, and workplace improvement is a significant shift towards a more data-driven, efficient, and employee-centric approach to company management. As underscored by the case studies above, there are already tangible benefits to AI-driven strategies, from streamlined hiring processes to personalized onboarding experiences, to optimized workplace conditions that prioritize employee health and well-being.As AI continues to evolve and is increasingly integrated into our workplaces, it is important to address ethical considerations and maintain a balance between technological advancements and human values. The future of AI in the workplace holds great promise for improving how we hire and work, and has the power to transform industries, improve the lives of workers, and shape how we think about work.<h2>References</h2>[1] “Navigating the Maze: The Fracture’s in Today’s IT Recruiting Process &amp; the AI Solution”. TechScreen. November 27, 2023. Accessed from <a href="https://techscreen.com/navigating-the-maze-the-fractures-in-todays-it-recruiting-process-the-ai-solution/" target="_blank">https://techscreen.com/navigating-the-maze-the-fractures-in-todays-it-recruiting-process-the-ai-solution/</a>&nbsp;[2] Jessica Dennis. “AI Recruiting: Uses, Advantages, &amp; Disadvantages”. TechnologyAdvice. 22 May 2023. Accessed from <a href="https://technologyadvice.com/blog/human-resources/ai-recruiting/" target="_blank">https://technologyadvice.com/blog/human-resources/ai-recruiting/</a>&nbsp;[3] Ranganath Ponnaboyina, et al. “Smart Recruitment System Using Deep Learning with Natural Language Processing”. Intelligent Systems and Sustainable Computing, Vol. 289. 29 May 2022. Accessed from <a href="https://link.springer.com/chapter/10.1007/978-981-19-0011-2_58" target="_blank">https://link.springer.com/chapter/10.1007/978-981-19-0011-2_58</a>&nbsp;[4] “Pros and Cons of Using AI in Recruiting”. Recruiter.com. Accessed from <a href="https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/" target="_blank">https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/</a>&nbsp;[5] Paul Arolye. “Navigating Recruitment Challenges: 5 Ways AI Can Revolutionize HR”. eLearning Industry. 27 January 2024. Accessed from <a href="https://elearningindustry.com/navigating-recruitment-challenges-ways-ai-can-revolutionize-hr" target="_blank">https://elearningindustry.com/navigating-recruitment-challenges-ways-ai-can-revolutionize-hr</a>&nbsp;[6] ClearCompany. “3 Benefits of Automating Employee Onboarding with AI-Driven Software”. 5 September 2023. Accessed from <a href="https://blog.clearcompany.com/benefits-of-automating-employee-onboarding-with-ai" target="_blank">https://blog.clearcompany.com/benefits-of-automating-employee-onboarding-with-ai</a>&nbsp;[7] Cabot Jaffee. “Leveraging AI to predict and reduce employee turnover.” Talent Management. 14 August 2023. Accessed from <a href="https://www.talentmgt.com/articles/2023/08/14/leveraging-ai-to-predict-and-reduce-employee-turnover/" target="_blank">https://www.talentmgt.com/articles/2023/08/14/leveraging-ai-to-predict-and-reduce-employee-turnover/</a>&nbsp;[8] Maria Korolov. “AI takes aim at employee turnover”. CIO. 7 April 2022. Accessed from <a href="https://www.cio.com/article/307980/ai-takes-aim-at-employee-turnover.html" target="_blank">https://www.cio.com/article/307980/ai-takes-aim-at-employee-turnover.html</a>&nbsp;[9] TechScreen. “Case Studies in Tech Hiring: AI Success Stories and Lessons Learned from Top Companies”. 28 December, 2023. Accessed from <a href="https://techscreen.com/case-studies-in-tech-hiring-ai-success-stories-and-lessons-learned-from-top-companies/" target="_blank">https://techscreen.com/case-studies-in-tech-hiring-ai-success-stories-and-lessons-learned-from-top-companies/</a>&nbsp;[10] Sapia.ai. “How Iceland Foods smashed its volume hiring goals”. Accessed from <a href="https://sapia.ai/resources/case-study/iceland/" target="_blank">https://sapia.ai/resources/case-study/iceland/</a>&nbsp;[11] Roy Mauer. “HR Must Be Vigilant About the Ethical Use of AI Technology”. SHRM. 26 September 2022. Accessed from <a href="https://www.shrm.org/topics-tools/news/technology/hr-must-vigilant-ethical-use-ai-technology" target="_blank">https://www.shrm.org/topics-tools/news/technology/hr-must-vigilant-ethical-use-ai-technology</a>&nbsp;

 AI is having an impact on hiring processes, employee retention, and workplace conditions. 

How AI will help companies hire more people and make the workplace better

We’ve all heard of Large Language Models, or LLMs – massive scale deep learning models trained on huge amounts of text that form the basis for chatbots like OpenAI’s ChatGPT. Next-generation Multimodal Models (MMs) can learn from inputs beyond text, including video, images, and sound.Creating MM models at a smaller scale poses significant challenges, including the problem of being robust to non-random missing information. This is information that a model doesn’t have, often due to some biased availability in resources. It is thus critical to ensure the model does not learn the patterns of biased missingness in making its predictions.<h2>MultiModN turns this around</h2>In response to this problem, researchers from the&nbsp;<a href="https://www.epfl.ch/labs/ml4ed/" target="_blank">Machine Learning for Education</a> (ML4ED) and&nbsp;<a href="https://www.epfl.ch/labs/mlo/" target="_blank">Machine Learning and Optimization</a> (MLO) Laboratories in<a href="https://www.epfl.ch/schools/ic/" target="_blank">&nbsp;EPFL’s School of Computer and Communication Sciences</a> have developed and tested the exact opposite to a Large Language Model.Spearheaded by Professor Mary-Anne Hartley, head of the Laboratory for<a href="https://www.epfl.ch/labs/mlo/igh-intelligent-global-health/" target="_blank">&nbsp;intelligent Global Health Technologies</a> hosted jointly in the MLO and the Yale School of Medicine and Professor Tanja Käser, head of ML4ED,&nbsp;MultiModN&nbsp;is a unique&nbsp;<a href="https://arxiv.org/abs/2309.14118" target="_blank" rel="noopener">Modular Multimodal Model</a>, presented recently at the&nbsp;<a href="https://neurips.cc/Conferences/2023" target="_blank" rel="noopener">NeurIPS2023</a> conference.Like existing Multimodal Models&nbsp;MultiModN can learn from text, images, video, and sound. Unlike existing MMs, it is made up of any number of smaller, self-contained, and input-specific modules that can be selected depending on the information available, and then strung together in a sequence of any number, combination, or type of input. Itcan then output any number, or combination, of predictions.“We evaluated&nbsp;MultiModN across ten real-world tasks including medical diagnosis support, academic performance prediction, and weather forecasting. Through these experiments, we believe that&nbsp;MultiModN is the first inherently interpretable, MNAR-resistant approach to multimodal modelling,” explained Vinitra Swamy, a PhD student with ML4ED and MLO and joint first author on the project.<h3>A first use case: medical decision-making</h3>The first use case for&nbsp;MultiModN will be as a clinical decision support system for medical personnel in low-resource settings. In healthcare, clinical data is often missing, perhaps due to resource constraints (a patient can’t afford the test) or resource abundance (the test is redundant due to a superior one that was performed).&nbsp;MultiModN is able to learn from this real-world data without adopting its biases, as well as adapting predictions to any combination or number of inputs.“Missingness is a hallmark of data in low-resource settings and when models learn these patterns of missingness, they may encode bias into their predictions. The need for flexibility in the face of unpredictably available resources is what inspired&nbsp;MultiModN,” explained Hartley, who is also a medical doctor.<h3>From the lab to real life</h3>Publication, however, is just the first step towards implementation. Hartley has been working with colleagues at Lausanne University Hospital<a href="https://www.chuv.ch/fr/chuv-home/" target="_blank" rel="noopener">&nbsp;(CHUV</a>) and Inselspital, University Hospital Bern&nbsp;<a href="https://www.insel.ch/fr/" target="_blank" rel="noopener">uBern</a> to conduct clinical studies focused on pneumonia and tuberculosis diagnosis in low resource settings and they are recruiting thousands of patients in South Africa, Tanzania, Namibia and Benin.The research teams undertook a large training initiative, teaching more than 100 doctors to systematically collect multimodal data including images and ultrasound video, so that&nbsp;MultiModN can be trained to be sensitive to real data coming from low resource regions.“We are collecting exactly the kind of complex multimodal data that&nbsp;MultiModN is designed to handle,” said Dr Noémie Boillat-Blanco, an infectious diseases doctor at CHUV. “We are excited to see a model that appreciates the complexity of missing resources in our settings and of systematic missingness of routine clinical assessments,” added Dr Kristina Keitel at Inselspital, University Hospital Bern.<h3>Machine learning for the public good</h3>The development and training of&nbsp;MultiModN is a continuation of EPFL efforts to adapt machine learning tools to reality and for the public good. It comes not long after the launch of<a href="https://actu.epfl.ch/news/epfl-s-new-large-language-model-for-medical-knowle/" target="_blank">&nbsp;Meditron</a>, the world’s best performing open source LLM also designed to help guide clinical decision-making.Both tools align with the mission of the new&nbsp;<a href="https://actu.epfl.ch/news/opening-of-the-new-epfl-ai-center-a-hub-for-ai-i-2/" target="_blank">EPFL AI Center</a> that focuses on how responsible and effective AI can advance technological innovation for the benefit of all sectors of society.Dr. Mary-Anne Hartley is a keynote speaker at the upcoming Applied Machine Learning Days (AMLD) at the SwissTech Convention Center March 23-26, 2024. Vinitra Swamy will be presenting MultiModN at AMLD’s Applied eXplainable AI track. Discover other keynotes<a href="https://2024.appliedmldays.org/media-9-speakers" target="_blank" rel="noopener">here</a>&nbsp;and register<a href="https://2024.appliedmldays.org/media-12-registration" target="_blank" rel="noopener">here</a>.

Machine Learning Neural Network © iStock

Researchers at EPFL have developed a new, uniquely modular machine learning model for flexible decision-making. It is able to input any mode of text, video, image, sound, and time-series and then output any number, or combination, of predictions. 

Anything-in-anything-out: a new modular AI model

Behrooz Tahmasebi — an MIT PhD student in the Department of Electrical Engineering and Computer Science (EECS) and an affiliate of the Computer Science and Artificial Intelligence Laboratory (CSAIL) — was taking a mathematics course on differential equations in late 2021 when a glimmer of inspiration struck. In that class, he learned for the first time about Weyl’s law, which had been formulated 110 years earlier by the German mathematician Hermann Weyl. Tahmasebi realized it might have some relevance to the computer science problem he was then wrestling with, even though the connection appeared — on the surface — to be thin, at best. Weyl’s law, he says, provides a formula that measures the complexity of the spectral information, or data, contained within the fundamental frequencies of a drum head or guitar string.Tahmasebi was, at the same time, thinking about measuring the complexity of the input data to a neural network, wondering whether that complexity could be reduced by taking into account some of the symmetries inherent to the dataset. Such a reduction, in turn, could facilitate — as well as speed up — machine learning processes.Weyl’s law, conceived about a century before the boom in machine learning, had traditionally been applied to very different physical situations — such as those concerning the vibrations of a string or the spectrum of electromagnetic (black-body) radiation given off by a heated object. Nevertheless, Tahmasebi believed that a customized version of that law might help with the machine learning problem he was pursuing. And if the approach panned out, the payoff could be considerable.He spoke with his advisor, Stefanie Jegelka — an associate professor in EECS and affiliate of CSAIL and the MIT Institute for Data, Systems, and Society — who believed the idea was definitely worth looking into. As Tahmasebi saw it, Weyl’s law had to do with gauging the complexity of data, and so did this project. But Weyl’s law, in its original form, said nothing about symmetry.He and Jegelka have now succeeded in modifying Weyl’s law so that symmetry can be factored into the assessment of a dataset’s complexity. “To the best of my knowledge,” Tahmasebi says, “this is the first time Weyl’s law has been used to determine how machine learning can be enhanced by symmetry.”The&nbsp;<a href="https://arxiv.org/abs/2303.14269" target="_blank">paper</a> he and Jegelka wrote earned a “Spotlight” designation when it was presented at the December 2023 conference on Neural Information Processing Systems — widely regarded as the world’s top conference on machine learning.This work, comments Soledad Villar, an applied mathematician at Johns Hopkins University, “shows that models that satisfy the symmetries of the problem are not only correct but also can produce predictions with smaller errors, using a small amount of training points. [This] is especially important in scientific domains, like computational chemistry, where training data can be scarce.”In their paper, Tahmasebi and Jegelka explored the ways in which symmetries, or so-called “invariances,” could benefit machine learning. Suppose, for example, the goal of a particular computer run is to pick out every image that contains the numeral 3. That task can be a lot easier, and go a lot quicker, if the algorithm can identify the 3 regardless of where it is placed in the box — whether it’s exactly in the center or off to the side — and whether it is pointed right-side up, upside down, or oriented at a random angle. An algorithm equipped with the latter capability can take advantage of the symmetries of translation and rotations, meaning that a 3, or any other object, is not changed in itself by altering its position or by rotating it around an arbitrary axis. It is said to be invariant to those shifts. The same logic can be applied to algorithms charged with identifying dogs or cats. A dog is a dog is a dog, one might say, irrespective of how it is embedded within an image.&nbsp;The point of the entire exercise, the authors explain, is to exploit a dataset’s intrinsic symmetries in order to reduce the complexity of machine learning tasks. That, in turn, can lead to a reduction in the amount of data needed for learning. Concretely, the new work answers the question: How many fewer data are needed to train a machine learning model if the data contain symmetries?There are two ways of achieving a gain, or benefit, by capitalizing on the symmetries present. The first has to do with the size of the sample to be looked at. Let’s imagine that you are charged, for instance, with analyzing an image that has mirror symmetry — the right side being an exact replica, or mirror image, of the left. In that case, you don’t have to look at every pixel; you can get all the information you need from half of the image — a factor of two improvement. If, on the other hand, the image can be partitioned into 10 identical parts, you can get a factor of 10 improvement. This kind of boosting effect is linear.To take another example, imagine you are sifting through a dataset, trying to find sequences of blocks that have seven different colors — black, blue, green, purple, red, white, and yellow. Your job becomes much easier if you don’t care about the order in which the blocks are arranged. If the order mattered, there would be 5,040 different combinations to look for. But if all you care about are sequences of blocks in which all seven colors appear, then you have reduced the number of things — or sequences — you are searching for from 5,040 to just one.Tahmasebi and Jegelka discovered that it is possible to achieve a different kind of gain — one that is exponential — that can be reaped for symmetries that operate over many dimensions. This advantage is related to the notion that the complexity of a learning task grows exponentially with the dimensionality of the data space. Making use of a multidimensional symmetry can therefore yield a disproportionately large return. “This is a new contribution that is basically telling us that symmetries of higher dimension are more important because they can give us an exponential gain,” Tahmasebi says.&nbsp;The NeurIPS 2023 paper that he wrote with Jegelka contains two theorems that were proved mathematically. “The first theorem shows that an improvement in sample complexity is achievable with the general algorithm we provide,” Tahmasebi says. The second theorem complements the first, he added, “showing that this is the best possible gain you can get; nothing else is achievable.”He and Jegelka have provided a formula that predicts the gain one can obtain from a particular symmetry in a given application. A virtue of this formula is its generality, Tahmasebi notes. “It works for any symmetry and any input space.” It works not only for symmetries that are known today, but it could also be applied in the future to symmetries that are yet to be discovered. The latter prospect is not too farfetched to consider, given that the search for new symmetries has long been a major thrust in physics. That suggests that, as more symmetries are found, the methodology introduced by Tahmasebi and Jegelka should only get better over time.According to Haggai Maron, a computer scientist at Technion (the Israel Institute of Technology) and NVIDIA who was not involved in the work, the approach presented in the paper “diverges substantially from related previous works, adopting a geometric perspective and employing tools from differential geometry. This theoretical contribution lends mathematical support to the emerging subfield of ‘Geometric Deep Learning,’ which has applications in graph learning, 3D data, and more. The paper helps establish a theoretical basis to guide further developments in this rapidly expanding research area.”

New MIT research provides a theoretical proof for a phenomenon observed in practice: that encoding symmetries in the machine learning model helps the model learn with fewer data. Image: Alex Shipps/MIT CSAIL

Exploiting the symmetry within datasets, MIT researchers show, can decrease the amount of data needed for training neural networks.

How symmetry can come to the aid of machine learning

In this episode, we talk about MIT researchers making a smart tool (like a robot scientist) that uses AI to understand and explain how other AI brains (neural networks) work.

Podcast: AI Model Tests AI Models To Tell Us How They Work

<section id="isPasted">Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers at Texas A&amp;M University achieved when combining machine learning, neural networks and novel compression tactics in a new project advancing reservoir production forecasts.Why is the research important? Because advancing forecasts improves the availability and accuracy of information oil and gas companies use to make sound financial and operating decisions."Production forecasting can be done several ways, but everything depends on it," said researcher Mohammad Elkady. "Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce."&nbsp;Elkady and fellow researcher Veena Kumar created a flexible method they say could apply to understanding and evaluating any complex reservoir. Both are graduate students in the Harold Vance Department of Petroleum Engineering and work under the guidance of their advisor, Dr. Siddharth Misra, the Ted H. Smith, Jr. '75 and Max R. Vordenbaum '73 DVG Associate Professor.</section><blockquote>Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce.<cite>Mohammad Elkady</cite></blockquote><section>Misra and Elkady introduced the work in the paper, "<a href="https://doi.org/10.2118/216935-MS" rel="noopener" target="_blank">Ultrafast Multiphase Production Forecasting for Large Gas Condensate Shale Reservoirs</a>," during the Abu Dhabi International Progressive Energy Congress in Oct 2023.&nbsp;Misra also presented the work in the paper,&nbsp;"Rapid Production Forecasting for Hydraulically Fractured Wells in Large, Heterogeneous Shale Reservoir," at the Society of Petroleum Engineers’ Workshop in New Orleans, Louisiana in Nov. 2023.<h2>Production forecasting history</h2>To make a profit, oil and gas companies weigh equipment, human resources, and other costs against the profitable hydrocarbons they can produce from subsurface reservoirs. Such decisions would be easy if petroleum engineers could peek inside the earth and see what's there. Since the subsurface is opaque, it's tricky to accurately predict oil recovery from reservoirs as conditions change over time.In the past, engineers used two main methods for forecasting. The first and oldest plots a well's oil recovery rates and declines over time, and that decline curve analysis can help estimate the recoveries in nearby or similar wells. The second is newer and uses computational resources to create a visual map of a reservoir by compressing the massive amounts of data gathered from surface and subsurface sensors and simulating the information into simplified renderings.<h2>Refining data to enhance forecasting</h2>This research is not the first to use machine learning or even neural networks to get a more accurate forecast, but it is the first to combine those processes with geomodel data compression. Elkady said data compression isn't new, as it's been used successfully in areas like image and video analysis. High-resolution videos must be compressed when shared around the world on restrictive bandwidths, yet it must be done selectively to keep the subjects true to the original imaging. Though the application in petroleum is novel, data compression makes sense for similar reasons."Geological data usually has a lot of data, and some may not be as important as others," said Elkady. "That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data."</section><blockquote>That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data.<cite>Mohammad Elkady</cite></blockquote><section>Elkady and Kumar began by creating 4,000 virtual homogenous or uniformly structured reservoirs containing 88,000 grid cells. Each cell had three randomly sampled geological properties: permeability, porosity, water saturation, fracture space, etc. The total data generated for each reservoir equaled over a quarter of a million points. The researchers ran these original reservoirs through a simulator for the most accurate production forecasts possible.The researchers then compressed the original data using two different methods simultaneously to crunch the numbers down with a 50,000 to 1 ratio. After training in Python, they taught the neural network to do production forecast workflows of the compressed reservoir model. Results showed the workflow was much faster than the simulator – under a second for every 700 seconds the simulator took – and had an error rate of 3% when compared to simulated forecasts.Next, the researchers used simulated data guided by Aramco that mimicked actual reservoir information to create about 3,000 virtual reservoirs that were heterogeneous or complex in structure and content yet still had the same number of grid cells.“We used geostatistical tools to generate the geology to be as realistic as we could," said Elkady. "The Aramco team supervised the generation of these datasets to make sure that, from their experience, this looks like a real reservoir."When that set of comparisons between workflow and simulator forecasts was completed, the researchers added more parameters, starting with six or seven. Eventually, they included human-related parameters, such as when operators choose how much gas they wish to produce despite what the reservoir is capable of, a complex concept for neural networks to factor in.</section><blockquote>Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations.<cite>Veena Kumar</cite></blockquote><section>"Initially, the amount of accuracy to error was very bad when we added human variations to the data set," Kumar said. "I am proud of getting the error percentage down, but we're looking into the reason for the initial higher percentages."The model-building stage – including reservoir parameter definitions, geomodel compressions, and prediction parameter factoring – required about one hour, depending on the data complexity. Once the models were trained and tuned, the time ratio for generating forecasts from the models remained less than one second for the researcher’s workflow to every 700 seconds, or about 11 minutes, when using a commercial software simulator.<h2>Future direction</h2>Currently, the work is expanding the number of parameters, refining the techniques and input factors, and testing methods to reduce the training times needed to learn the software. After that, the researchers want to test it out on actual reservoir data, ideally a reservoir where some production has already been completed.They are optimistic that the work can be applied effectively in many different formations."The goal of refining the multivariate forecast model for long-term projections, potentially over a decade, is one of the approaches we're excited to try out," said Kumar. "Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations."</section>

Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers achieved when combining ML, neural networks and novel compression in a new project advancing reservoir production forecasts.

Ultrafast Reservoir Forecasting Boasts Speed and Accuracy

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/8a9e30e0-92a2-49c6-afc7-1ca5a7d907f4?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Explaining the behavior of trained neural networks remains a compelling puzzle, especially as these models grow in size and sophistication. Like other scientific challenges throughout history, reverse-engineering how artificial intelligence systems work requires a substantial amount of experimentation: making hypotheses, intervening on behavior, and even dissecting large networks to examine individual neurons. To date, most successful experiments have involved large amounts of human oversight. Explaining every computation inside models the size of GPT-4 and larger will almost certainly require more automation — perhaps even using AI models themselves.&nbsp;Facilitating this timely endeavor, researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a novel approach that uses AI models to conduct experiments on other systems and explain their behavior. Their method uses agents built from pretrained language models to produce intuitive explanations of computations inside trained networks.Central to this strategy is the “automated interpretability agent” (AIA), designed to mimic a scientist’s experimental processes. Interpretability agents plan and perform tests on other computational systems, which can range in scale from individual neurons to entire models, in order to produce explanations of these systems in a variety of forms: language descriptions of what a system does and where it fails, and code that reproduces the system’s behavior. Unlike existing interpretability procedures that passively classify or summarize examples, the AIA actively participates in hypothesis formation, experimental testing, and iterative learning, thereby refining its understanding of other systems in real time.&nbsp;Complementing the AIA method is the new “function interpretation and description” (<a href="https://multimodal-interpretability.csail.mit.edu/FIND-benchmark/" target="_blank">FIND</a>) benchmark, a test bed of functions resembling computations inside trained networks, and accompanying descriptions of their behavior. One key challenge in evaluating the quality of descriptions of real-world network components is that descriptions are only as good as their explanatory power: Researchers don’t have access to ground-truth&nbsp;labels of units or descriptions of learned computations. FIND addresses this long-standing issue in the field by providing a reliable standard for evaluating interpretability procedures: explanations of functions (e.g., produced by an AIA) can be evaluated against function descriptions in the benchmark. &nbsp;For example, FIND contains synthetic neurons designed to mimic the behavior of real neurons inside language models, some of which are selective for individual concepts such as “ground transportation.” AIAs are given black-box access to synthetic neurons and design inputs (such as “tree,” “happiness,” and “car”) to test a neuron’s response. After noticing that a synthetic neuron produces higher response values for “car” than other inputs, an AIA might design more fine-grained tests to distinguish the neuron’s selectivity for cars from other forms of transportation, such as planes and boats. When the AIA produces a description such as “this neuron is selective for road transportation, and not air or sea travel,” this description is evaluated against the ground-truth description of the synthetic neuron (“selective for ground transportation”) in FIND. The benchmark can then be used to compare the capabilities of AIAs to other methods in the literature.&nbsp;Sarah Schwettmann PhD '21, co-lead author of a <a href="https://arxiv.org/pdf/2309.03886.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper on the new work</a> and a research scientist at CSAIL, emphasizes the advantages of this approach. “The AIAs’ capacity for autonomous hypothesis generation and testing may be able to surface behaviors that would otherwise be difficult for scientists to detect. It’s remarkable that language models, when equipped with tools for probing other systems, are capable of this type of experimental design,” says Schwettmann. “Clean, simple benchmarks with ground-truth answers have been a major driver of more general capabilities in language models, and we hope that FIND can play a similar role in interpretability research.”<h2>Automating interpretability&nbsp;</h2>Large language models are still holding their status as the in-demand celebrities of the tech world. The recent advancements in LLMs have highlighted their ability to perform complex reasoning tasks across diverse domains. The team at CSAIL recognized that given these capabilities, language models may be able to serve as backbones of generalized agents for automated interpretability. “Interpretability has historically been a very multifaceted field,” says Schwettmann. “There is no one-size-fits-all approach; most procedures are very specific to individual questions we might have about a system, and to individual modalities like vision or language. Existing approaches to labeling individual neurons inside vision models have required training specialized models on human data, where these models perform only this single task. Interpretability agents built from language models could provide a general interface for explaining other systems — synthesizing results across experiments, integrating over different modalities, even discovering new experimental techniques at a very fundamental level.”&nbsp;As we enter a regime where the models doing the explaining are black boxes themselves, external evaluations of interpretability methods are becoming increasingly vital. The team’s new benchmark addresses this need with a suite of functions with known structure, that are modeled after behaviors observed in the wild. The functions inside FIND span a diversity of domains, from mathematical reasoning to symbolic operations on strings to synthetic neurons built from word-level tasks. The dataset of interactive functions is procedurally constructed; real-world complexity is introduced to simple functions by adding noise, composing functions, and simulating biases. This allows for comparison of interpretability methods in a setting that translates to real-world performance. &nbsp; &nbsp; &nbsp;In addition to the dataset of functions, the researchers introduced an innovative evaluation protocol to assess the effectiveness of AIAs and existing automated interpretability methods. This protocol involves two approaches. For tasks that require replicating the function in code, the evaluation directly compares the AI-generated estimations and the original, ground-truth functions. The evaluation becomes more intricate for tasks involving natural language descriptions of functions. In these cases, accurately gauging the quality of these descriptions requires an automated understanding of their semantic content. To tackle this challenge, the researchers developed a specialized “third-party” language model. This model is specifically trained to evaluate the accuracy and coherence of the natural language descriptions provided by the AI systems, and compares it to the ground-truth function behavior.&nbsp;FIND enables evaluation revealing that we are still far from fully automating interpretability; although AIAs outperform existing interpretability approaches, they still fail to accurately describe almost half of the functions in the benchmark. Tamar Rott Shaham, co-lead author of the study and a postdoc in CSAIL, notes that “while this generation of AIAs is effective in describing high-level functionality, they still often overlook finer-grained details, particularly in function subdomains with noise or irregular behavior. This likely stems from insufficient sampling in these areas. One issue is that the AIAs’ effectiveness may be hampered by their initial exploratory data. To counter this, we tried guiding the AIAs’ exploration by initializing their search with specific, relevant inputs, which significantly enhanced interpretation accuracy.” This approach combines new AIA methods with previous techniques using pre-computed examples for initiating the interpretation process.The researchers are also developing a toolkit to augment the AIAs’ ability to conduct more precise experiments on neural networks, both in black-box and white-box settings. This toolkit aims to equip AIAs with better tools for selecting inputs and refining hypothesis-testing capabilities for more nuanced and accurate neural network analysis. The team is also tackling practical challenges in AI interpretability, focusing on determining the right questions to ask when analyzing models in real-world scenarios. Their goal is to develop automated interpretability procedures that could eventually help people audit systems — e.g., for autonomous driving or face recognition — to diagnose potential failure modes, hidden biases, or surprising behaviors before deployment.&nbsp;<h2>Watching the watchers</h2>The team envisions one day developing nearly autonomous AIAs that can audit other systems, with human scientists providing oversight and guidance. Advanced AIAs could develop new kinds of experiments and questions, potentially beyond human scientists’ initial considerations. The focus is on expanding AI interpretability to include more complex behaviors, such as entire neural circuits or subnetworks, and predicting inputs that might lead to undesired behaviors. This development represents a significant step forward in AI research, aiming to make AI systems more understandable and reliable.“A good benchmark is a power tool for tackling difficult challenges,” says Martin Wattenberg, computer science professor at Harvard University who was not involved in the study. “It's wonderful to see this sophisticated benchmark for interpretability, one of the most important challenges in machine learning today. I'm particularly impressed with the automated interpretability agent the authors created. It's a kind of interpretability jiu-jitsu, turning AI back on itself in order to help human understanding.”Schwettmann, Rott Shaham, and their colleagues presented their work at NeurIPS 2023 in December. &nbsp;Additional MIT coauthors, all affiliates of the CSAIL and the Department of Electrical Engineering and Computer Science (EECS), include graduate student Joanna Materzynska, undergraduate student Neil Chowdhury, Shuang Li PhD ’23, Assistant Professor Jacob Andreas, and Professor Antonio Torralba. Northeastern University Assistant Professor David Bau is an additional coauthor.The work was supported, in part, by the MIT-IBM Watson AI Lab, Open Philanthropy, an Amazon Research Award, Hyundai NGV, the U.S. Army Research Laboratory, the U.S. National Science Foundation, the Zuckerman STEM Leadership Program, and a Viterbi Fellowship.

FIND is a new benchmark suite for evaluating automated interpretability methods in neural networks, featuring functions that mimic real-world network components and their complexities. It also presents a novel interactive method using automated interpretability agents, which employ pretrained language models to generate descriptions of function behavior, demonstrating the agent's ability to infer function structure while highlighting the need for further refinement in capturing local details. Image: Alex Shipps/MIT CSAIL

MIT researchers introduce a method that uses artificial intelligence to automate the explanation of complex neural networks.

AI agents help explain other AI systems

<h2>Introduction to Temporal Event-Based Neural Networks (TENNs)</h2>In the realm of Artificial Intelligence (AI), the evolution of neural network technologies has reached a pivotal juncture with the advent of Temporal Event-Based Neural Networks (TENNs). Developed by BrainChip, TENNs represent a groundbreaking approach to processing sequential spatiotemporal data, addressing the limitations of traditional neural network models like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).CNNs, while proficient in processing spatial information in images, falter in effectively utilizing temporal data. RNNs, on the other hand, capture sequential dependencies but face challenges like vanishing gradients and difficulty in parallelization. TENNs emerge as a solution, efficiently combining spatial and temporal convolutions, thus excelling in processing sequential data like video and time series data.TENNs leverage the strengths of both CNNs and RNNs, integrating spatial and temporal characteristics of data. This integration enables TENNs to learn spatial and temporal correlations effectively, distinguishing them from traditional state-space models. Their flexibility allows for operation in either temporal convolution mode or recurrent mode, adapting to specific application requirements.BrainChip's 2nd Generation Akida platform supports TENNs, offering state-of-the-art performance across various domains of sequential data. This includes applications in speech recognition, patient monitoring in medical equipment, and streaming video object detection and tracking. With their superior performance, low computational requirements, and versatility, TENNs are poised to revolutionize intelligent Edge solutions, providing highly accurate models for sequential data analysis with a fraction of the computational resources required by other network architectures.For a deeper understanding of TENNs and their applications, BrainChip's white paper, "Temporal Event-based Neural Networks: A New Approach to Temporal Processing," offers comprehensive insights into this innovative technology.<a href="https://wevlv.co/brainchip-tenns-whitepaper" target="_blank" rel="noopener noreferrer">Read the whitepaper here. </a><h3 dir="ltr">More about BrainChip: Making AI ubiquitous</h3>The integration of sophisticated IoT infrastructure with Artificial Intelligence technologies, particularly those modeled after human neural processes, results in a significant enhancement of operational efficiency. This amalgamation facilitates proactive problem-solving, augments interactions between humans and machines, and fortifies security measures.A crucial aspect of proliferating AI ubiquitously lies in simplifying its deployment and scalability. Achieving this necessitates a reduction in costs, autonomy from cloud reliance, and the localization of intelligence in proximity to sensory inputs.Brainchip's comprehensive and decentralized strategy alleviates the burden on cloud resources, thereby catalyzing a surge in capable and intelligent sensing devices at the Edge. This approach is instrumental in accelerating the global advancement of artificial intelligence, marking a significant milestone in the AIoT domain.Brainchip’s holistic, distributed approach frees up the cloud and creates an explosion of capable, intelligent sensing devices on the Edge that accelerate global artificial intelligence.

Learn more about a groundbreaking solution called Temporal Event-based Neural Networks (TENNs) developed by BrainChip, which efficiently combines spatial and temporal convolutions to process sequential data like never before.

TENNs: Unlocking the Power of Efficient Processing for Sequential Data

<h1 dir="ltr" id="isPasted">Introduction</h1>The brain controls the body through a complex network of neurons that send, receive, and process sensory signals. These signals are responsible for controlling every limb, muscle, appendage, and bodily function. To better understand these signals, neurotechnology combines the fields of neuroscience and engineering to investigate the structure and working principles of the nervous system and to develop tools to interact with it.The inception of neurotechnology can be traced back to the late 19th century when neurological structures were mapped by scientists like Santiago Ramon Y. Cajal using various staining methods.[1] In the 1920s, German neurologist Hans Berger developed electroencephalography (EEG), enabling the measurement of the brain’s electrical activity. Then, in 1971, physician Raymond Damadian developed the first magnetic resonance image (MRI) of a human finger. With the onset of the 21st century, innovations such as deep brain stimulation (DBS), brain-computer interface (BCI), and neural prosthetic devices have advanced neurotechnology. Today, neurotechnology plays an important role in the diagnosis and treatment of neurological and psychological illnesses while also contributing to the improvement of cognitive and sensory capabilities.[2]&nbsp;Neurotechnology also has applications in education; for instance, EEG can be used to measure the brain waves of a learner and provide feedback regarding the level of attentiveness, engagement, and memory.<h1 dir="ltr">Understanding the Core of Neurotechnology</h1>Neurotechnologies fall into four categories: neural interfaces, neuromodulation technologies, neuroprostheses, and BCIs. These techniques have myriad applications in vastly diverse fields, such as medicine, neuroscience, psychology, education, and entertainment. From pharmaceuticals that improve quality of life to brain imaging that revolutionizes our conception of human consciousness, neurotechnologies stand to change our understanding of ourselves and harness the power of the brain and nervous system’s myriad functions to promote human thriving.<h2 dir="ltr">Neural Interfaces</h2>One of the main applications of neurotechnology is the development of neural interfaces. These devices can record and stimulate neural activity. Depending on how they are placed on or in the body, neural interfaces can be classified into three types:&nbsp;<ul><li dir="ltr">Invasive, examples of which include implanted stimulators for the treatment of Parkinson's disease and cochlear implants for hearing restoration.&nbsp;</li><li dir="ltr">Non-invasive, comprising EEG electrodes outside the body that can control digital objects or interface with the computer.&nbsp;</li><li dir="ltr">Partially invasive, which are placed under the skin but do not enter the brain (for example, electrical foot stimulators).[3]</li></ul>Neural interfaces are effective for restoring motor functions, improving cognitive abilities, and monitoring health conditions, and enabling novel forms of human-machine interaction.[4]<h2 dir="ltr">Brain-Computer Interfaces</h2>BCIs are systems where a computer communicates with an individual's brain to control outside objects or applications via invasive or non-invasive electrodes implanted in the brain or on the attachment to the scalp. Subsequently, algorithms such as machine learning and deep learning are capable of analyzing brain waves to generate commands, linking the outside world directly to the brain.[5] Potential applications of BCIs include restoring function for patients with neuromuscular disorders[6], enhancing rehabilitation from strokes or traumatic injuries, and creating virtual reality experiences.<h2 dir="ltr">Neuroprosthetics</h2>Neuroprosthetics is an amalgamation of neuroscience and bioengineering aimed at developing medical apparatuses that act as substitutes for or upgrade normal functions of nerve systems. Neuroprosthetics are mainly used to restore functions such as sensory, motor, and cognitive abilities impaired by trauma and diseases. Illustratively, cochlear implants assist people who are deaf or hard of hearing in understanding speech. In contrast, paralyzed individuals manipulate artificial limbs[7] or a computer using thoughts through BCIs.Neuroprosthetics is based on two key ideas. Firstly, electrical signals generated within the brain can be picked up and deciphered by external equipment. Secondly, these devices may also give feedback to the brain using electrical stimulation.[8] Neuroprosthetics is growing rapidly using new technology like artificial intelligence (AI), brain imaging, or nanomaterials that provide novel opportunities in terms of health and well-being.[9]<h2><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwODM3NjExNTkwLW5ldXJvIGFybS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">Prosthetic hand using advanced BMI (brain/machine interfaces) technology to deliver capability and usability. Image credit:&nbsp;<a href="https://brainrobotics.com/" target="_blank" id="isPasted">Brain Robotics</a> Neural Modulation by Stimulation</h2>Neural modulation by stimulation involves manipulating nervous system activity through external stimuli such as electric current, magnetic field, or chemical substances. These stimuli may change the excitability, firing patterns, connectivity, and plasticity of neurons, which in turn influences the functions and behavior of neurons.[10]DBS is one of the many methods used in neural modulation by stimulation. It is a process whereby electrodes are placed on selected areas of the brain, sending electrical signals into them to alter their output. It is used for treating movement disorders like Parkinson's disease, essential tremors, dystonia, and psychiatric conditions such as obsessive-compulsive disorder (OCD) and depression.[11]Another method, transcranial magnetic stimulation (TMS), has the same principle as EEG but includes placing a coil on the scalp and generating a magnetic field, causing an electric current below brain tissue.[12] The main uses of TMS have been for treating different neurological and psychiatric illnesses like stroke, migraine, and schizophrenia.<h1 dir="ltr">The Technological Innovations Driving Neurotechnology</h1>Neurotechnology has experienced tremendous growth in areas like sensing, stimulus deliveries, neural signal captures, and manipulation. For instance, many technical advancements in microelectrode arrays, machine learning and artificial intelligence, advanced imaging techniques, biomaterials, and nanotechnology drive these achievements.<h2 dir="ltr">Microelectrode Arrays</h2>Microelectrode arrays (MEAs) are versatile devices with multiple microelectrodes that facilitate the extraction and delivery of neural signals. MEAs act as vital connections between neurons and electronic circuitry, functioning as neural interfaces. Their primary purpose is to record and stimulate neural activity in various settings, in vitro or in vivo, based on the specific type and design of the MEA.[13]To capture neural activity, the electrodes within an MEA convert the ionic currents resulting from neuronal depolarization into measurable electronic currents. Conversely, to induce neural activity, these same electrodes transform electronic currents into ionic currents that activate voltage-gated ion channels present on neuronal membranes. They can help restore both sensory and motor functions, as well as provide treatment for neurological disorders.[14]<h2 dir="ltr">Machine Learning and AI</h2>The intersection of machine learning, AI, and neurotechnology is a fascinating field of research that plays a crucial role in advancing neurotechnological research by providing powerful tools for analyzing complex datasets like brain imaging, neural recordings, and behavioral data.[15] These tools aid in uncovering patterns and principles that govern brain function while identifying the neural correlates of cognitive processes, emotions, and mental disorders.Additionally, machine learning and AI facilitate the development of innovative neurotechnologies that can interact with the brain in real-time, including BCIs, neurofeedback systems, and neural prosthetics. This synergy between disciplines holds immense potential for unlocking new insights into our most complex organ.<h2 dir="ltr">Advanced Imaging Techniques</h2>Neurotechnology researchers can access advanced imaging techniques such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET). Using these powerful tools, they can measure brain activity in living subjects when exposed to different stimuli or performing various tasks. By doing so, they can uncover the neural correlates of cognition, emotion, perception, and behavior to help identify specific brain regions and networks involved in mental processes like memory, attention, language, and decision-making.[16]<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwODM3Njc3OTY1LW5ldXJvIHR1bm5lbC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">PET scan machine uses radiotracers to evaluate tissue functions. Image credit:&nbsp;<a href="https://healthjade.net/pet-scan/" target="_blank" id="isPasted">Health Jade</a><h2 dir="ltr" id="isPasted">Biomaterials and Nanotechnology</h2>The design of neurotech devices must take into consideration the biocompatibility with the brain and other neural tissues in order to avoid any harmful effects on biological systems. One approach to achieving biocompatibility is by utilizing biomaterial substances that can be applied to various components of neurotech devices, including coatings, electrodes, scaffolds, or implants.[17] However, the challenges faced by neurotech devices can be fully addressed with the complimentary use of nanotechnology.Nanotechnology focuses on exploiting matter at the nanoscale, between 1 and 100 nanometers in scale, where matter displays unique and improved properties that can be utilized for applications in neurotechnology.[18] Combining biomaterials with nanotechnology enhances the biocompatibility and biodegradability of nanotech devices while improving their electrical, optical, and magnetic characteristics.[19]&nbsp;<h1 dir="ltr">Ethical Implications and Future Prospects</h1>Despite the revolutionary aspects of neurotechnology, it brings forth essential ethical considerations that must be addressed before its widespread adoption. How can we guarantee the privacy and autonomy of neurotechnology users when it involves accessing or manipulating their brain data or functions? This crucial question demands thoughtful exploration and robust solutions. The misuse of neurotechnology by malicious individuals or authoritarian governments poses a significant threat to the rights and security of individuals. Therefore, it is crucial to establish effective regulations and monitoring mechanisms for developing and implementing such technologies.[20]Despite the ethical concerns, the future of neurotechnology is filled with excitement and challenges alike. It necessitates collaboration among scientists, engineers, ethicists, policymakers, and stakeholders from various disciplines. Each application must be thoroughly assessed to weigh its benefits and potential risks for individuals and society as a whole. To ensure the responsible and beneficial use of neurotechnology for humanity, ethical principles, and guidelines need to be developed.[21]<h1 dir="ltr">Conclusion</h1>The field of neurotechnology is advancing rapidly, offering exciting possibilities for enhancing human capabilities and improving health outcomes. However, along with these promises come essential ethical, social, and legal challenges that demand attention from researchers, policymakers, and stakeholders. This article explores current and emerging applications of neurotechnology, such as brain-computer interfaces, neurostimulation, neuroimaging, and neurofeedback. We also overview the potential benefits and risks associated with these technologies while examining the ethical principles and frameworks that can guide their development and use. In conclusion, we recognize the transformative potential of neurotechnology for society but emphasize the need for thoughtful governance to ensure safety, efficacy, and respect for human dignity.<h1 dir="ltr">References</h1>1. Ghosh SK, Narayan RK. Anatomy of nervous system and emergence of neuroscience: a chronological journey across centuries. Morphologie. 2020 Dec 1;104(347):267-79.2. White SW, Richey JA, Gracanin D, Bell MA, LaConte S, Coffman M, et al. The promise of neurotechnology in clinical translational science. Clinical Psychological Science. 2015;3(5):797-815.3. Zhang M, Tang Z, Liu X, Van der Spiegel J. Electronic neural interfaces. Nature Electronics. 2020;3(4):191-200.4. Jackson A, Zimmermann JB. Neural interfaces for the brain and spinal cord—restoring motor function. Nature Reviews Neurology. 2012;8(12):690-9.5. Ramadan RA, Vasilakos AV. Brain-computer interface: control signals review. Neurocomputing. 2017;223:26-44.6. Nicolas-Alonso LF, Gomez-Gil J. Brain computer interfaces, a review. sensors. 2012;12(2):1211-79.7. Normann RA. Technology insight: future neuroprosthetic therapies for disorders of the nervous system. Nature Clinical Practice Neurology. 2007;3(8):444-52.8. Borton D, Micera S, Millán JdR, Courtine G. Personalized neuroprosthetics. Science translational medicine. 2013;5(210):210rv2-rv2.9. Warwick K. Neuroengineering and neuroprosthetics. Brain and neuroscience advances. 2018;2:2398212818817499.10. De Andrés J, Van Buyten JP. Neural modulation by stimulation. Pain Practice. 2006;6(1):39-45.11. Breit S, Schulz JB, Benabid A-L. Deep brain stimulation. Cell and tissue research. 2004;318:275-88.12. Lefaucheur J-P. Transcranial magnetic stimulation. Handbook of clinical neurology. 2019;160:559-80.13. Obien MEJ, Deligkaris K, Bullmann T, Bakkum DJ, Frey U. Revealing neuronal function through microelectrode array recordings. Frontiers in neuroscience. 2015;8:423.14. Ordonez J, Schuettler M, Boehler C, Boretius T, Stieglitz T. Thin films and microelectrode arrays for neuroprosthetics. MRS bulletin. 2012;37(6):590-8.15. Giattino CM, Kwong L, Rafetto C, Farahany NA, editors. The seductive allure of artificial Intelligence-powered Neurotechnology. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society; 2019.16. Aine CJ. A conceptual overview and critique of functional neuroimaging techniques in humans: I. MRI/FMRI and PET. Critical reviews in neurobiology. 1995;9(2-3):229-309.17. Wellman SM, Eles JR, Ludwig KA, Seymour JP, Michelson NJ, McFadden WE, et al. A materials roadmap to functional neural interface design. Advanced functional materials. 2018;28(12):1701269.18. Acarón Ledesma H, Li X, Carvalho-de-Souza JL, Wei W, Bezanilla F, Tian B. An atlas of nano-enabled neural interfaces. Nature nanotechnology. 2019;14(7):645-57.19. Fattahi P, Yang G, Kim G, Abidian MR. A review of organic and inorganic biomaterials for neural interfaces. Advanced materials. 2014;26(12):1846-85.20. Ienca M, Andorno R. Towards new human rights in the age of neuroscience and neurotechnology. Life sciences, society, and policy. 2017;13(1):1-27.21. Roelfsema PR, Denys D, Klink PC. Mind reading and writing: The future of neurotechnology. Trends in cognitive sciences. 2018;22(7):598-610.

An interdisciplinary venture to shape the future of healthcare, communication, and human capability.


Understanding Neurotechnology: A multidisciplinary venture to elevate potential

This article is based on an article titled, <a href="https://cambrian-ai.com/wp-content/uploads/2023/08/BrainChip-Sees-Gold-Final_2.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">BrainChip Sees AI Gold in Sequential Data Analysis at the Edge</a>, from the Cambrian AI website.Amidst the whirlwind of excitement surrounding Large Language Model (LLM) generative AI, it's easy to lose sight of other AI domains that have seemingly dissolved into the shadows cast by ChatGPT's prominence. One underappreciated domain is the analysis of sequential data streams, which includes monitoring fluctuating stock prices and interpreting video feeds.&nbsp;BrainChip has identified this niche — the adept handling of sequential data — as a pivotal niche for deploying its Akida technology. Akida stands out for its prowess in Event-Based Neuromorphic computing, adeptly handling various neural network architectures like Vision Transformers (ViT), Convolutional Neural Networks (CNN), Temporal Evolution Network (TENN), and Recurrent Neural Networks (RNN).&nbsp;This article explores the need for efficient edge AI for time-series data.<h2 dir="ltr">Sequential data analysis</h2>Sequential analysis refers to the process of analyzing and extracting insights from data that is collected and organized in chronological order. This data type typically involves measurements or observations taken at regular intervals over time. Time-series analysis techniques aim to understand data patterns, trends, and dependencies and make predictions or forecasts based on historical patterns.A few use cases of sequential data analysis include:1. Financial Analysis: Time-series analysis is extensively used in finance to study stock market trends, analyze economic indicators, and forecast future market conditions. It helps model asset prices, trends, risk assessment, and portfolio optimization.2. Demand Forecasting: Sequential analysis is crucial in demand forecasting for retail, supply chain management, and manufacturing industries. By analyzing historical sales or demand data, businesses can predict future demand patterns and optimize their production, inventory, and supply chain accordingly.3. Predictive Maintenance: Predictive maintenance can monitor equipment and machinery in real-time. Analyzing sensor data and historical patterns can help detect anomalies and predict potential failures, enabling proactive maintenance and minimizing downtime.As well as applications in Energy Consumption Analysis and IoT Sensor Data Analysis.The market size for applying AI in time-series data analysis is continually growing as organizations recognize the value of extracting insights and making accurate predictions from temporal data. While specific market size figures for this realm are not readily available, the broader AI market, including applications in time-series analysis, is expected to grow substantially. According to a report by Grand View Research, the global AI market size was valued at USD 62.35 billion in 2020 and is projected to expand at a compound annual growth rate (CAGR) of 40.2% from 2021 to 2028. This growth encompasses various AI applications, including time-series analysis, across multiple industries.<h1 dir="ltr">Introducing the Dimension of Time to Neural Networks</h1>Traditional CNNs have been around for 30+ years and combine multiple hidden layers trained in a supervised manner. These are sequential and hence referred to as feed-forward neural networks. Bi-directional networks, also called RNNs, invented at the turn of the century, added capability for more complex learning, such as language modeling. But for applications to time series, a machine learning engineer needed a combination of CNNs and a temporal network for spatial-temporal analysis. While academia developed networks that did temporal convolution, they have yet to be power efficient or easy to train to make it to the far Edge.<h1 dir="ltr">Temporal Event-based Neural Networks</h1>Temporal Artificial Neural Networks (TENNs). This approach simplifies training and reduces model size while maintaining accuracy, resulting in improved performance and efficiency for Edge AI devices.Using TENNs for data analysis offers several advantages, such as the ability to learn the temporal structure of the data, which is crucial for tasks such as forecasting and anomaly detection. TENNs can make predictions for future time steps, and they are capable of learning from large datasets of time series data. Overall, these features make TENNs a valuable tool for data analysis in various domains.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529042393-image3.png" style="width: 300px;" class="fr-fic fr-dib">Traditional time series analysis applies to all kinds of signals. Image credit: Cambrian AIBeyond traditional Edge time series applications, BrainChip suggests that TENNs could minimize or eliminate the need for DSP filtering of raw audio signals and vital signs in health monitoring. Thus, offering compact solutions for wearable, hearing, and implantable devices with minimal power consumption is a significant advance for preventative healthcare.Additionally, TENNs also excel at treating streaming inputs as a time series of frames, performing 3D convolutions composed of a temporal convolution on the time axis and a spatial convolution on the XY axis. This is efficiently achieved to improve the detection of higher-resolution video objects in low-power scenarios.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529079641-image2.png" style="width: 300px;" class="fr-fic fr-dib">Treating streaming inputs as a 3D time series of frames with a highly efficient 3D convolution. Image credit: Cambrian AI.TENNs offer developers the flexibility to configure them in either buffered temporal convolution or recurrent modes, enabling them to adapt the network to their specific application requirements. Furthermore, TENNs can be efficiently trained on parallel hardware, such as GPUs and TPUs in the cloud, similar to convolutional networks, while retaining the compactness of RNNs for efficient inference at the edge. This approach helps to minimize the exponentially growing cost of training, which is a constant concern.<h1 dir="ltr">Akida 2nd&nbsp;Generation Processor and TENNs</h1>The BrainChip Akida processor is a digital portable processor IP inspired by the energy-efficient functionality of the human brain. Unlike traditional neuromorphic approaches, which are analog, it utilizes fully digital technology and offers a range of capabilities, including image classification, semantic segmentation, odor recognition, and time-series analysis. It supports various neural network architectures, including TENNs.The Akida processor utilizes highly parallel event-based neural processing cores, merging neuromorphic processing with native support for traditional convolutional capabilities and functions, along with hardware support for TENN networks. Its neuromorphic processing cores communicate using sparse, asynchronous events, making it ideal for efficient time-series data analysis and managing high-speed, asynchronous, and continuous data streams.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529133867-image4.png" style="width: 300px;" class="fr-fic fr-dib">The TENN operation may combine transform from convolutions in training to recurrence operation for inference. Image credit: Cambrian AI.BrainChip’s Akida allows the analysis of vision, video, and three-dimensional data as a time series, improving video object detection. Akida's support for spatial-temporal convolutions enhances speed and reduces energy consumption. This ability is possible because the processor features a high-speed, low-power digital design optimized for edge computing applications, offering real-time processing and low-latency analysis. It can handle structured and unstructured data, learning and recognizing patterns from streaming data, making it suitable for time-series data analysis.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529192601-image4.png" style="width: 300px;" class="fr-fic fr-dib">The foundations of the Akida architecture. Image credit: Cambrian AI.Additionally, Akida includes a traditional CNN accelerator, as well as TENN and ViT logic, providing a comprehensive solution for sequential processing. The Akida processor is particularly effective for real-time data classification, anomaly detection, and predictive analytics. Consequently, the second-generation Akida processor, with TENN support, extends its efficiency and accelerated hardware solutions to multi-dimensional applications like video object detection and vision using event-based processing paradigms.<h1 dir="ltr">Conclusion</h1>The rapid expansion of the AI field has primarily focused on image processing and LLMs, leaving a significant gap in the development of sequential data analysis. BrainChip's Akida technology, featuring TENNs, simplifies training, reduce model size, and enhance performance and efficiency in time series data analysis.&nbsp;This innovative approach can be applied in different industries, including finance, demand forecasting, predictive maintenance, energy consumption analysis, and IoT data analysis, meeting the growing demand for effective AI solutions in these domains. The Akida processor extends its efficiency to multi-dimensional applications improving real-time data analysis, predictive analytics, and video object detection for event-based processing.

How BrainChip's Akida technology is revolutionizing time series data analysis.


The huge potential of sequential data analysis at the edge

Training robust AI models can be challenging when using crowdsourced or publicly available datasets. The most useful neural networks are ones that replace expert knowledge to solve real problems. One example would be a visual inspection model for the factory floor.<h2>Human level performance</h2>Human performance comes into tiers: general or expert. An expert is someone who has spent a minimum amount of time or has received practical training in the field.The accuracy of your AI model is directly correlated to the quality of data used to train it. When using general datasets for training, the model usually surpasses the knowledge level embedded in the training samples, but it remains well below expert-level performance. A computer is more attentive than an operator performing the same inspection task, so the model trained will probably reach an accuracy that is marginally better than the human labelled dataset.If you think it's a good idea to use services like Amazon Mechanical Turk to label your dataset, let me ask you a question. Would you take a person and after reading a quick instruction manual get them to work inside the factory?That does not sound like a great idea. But it turns out that is what every one of us is doing when leveraging services like these. The bulk of your dataset can be generated in this manner, but is it helping your model?<h2>Practical considerations</h2>Tricky samples may not make it to the dataset because someone was not able to understand your labeling instructions, and that sample may get discarded. While human-level performance is achievable with large crowd labeled datasets, the truth is that the easiest samples will be labeled first.I had seen numerous occasions in which turks were simply targeting the lowest hanging fruit by submitting no labels for the empty samples. In that case, it is a first come first rewarded economy. The difficult samples may end up unlabeled.<h2>Model hardening</h2>Synthetic data allows the possibility to make the task at hand harder than it will be in real life.Variance or noise is good for your model, while generating samples programmatically can make the model more resilient to unseen inputs. Human labeled data is often curated and less noisy than the real inputs.This may work out, in the beginning, to quickly get started with your model. But believe me, you don't want a crowd trained model to be put in production across a factory floor without verifying that its performance surpasses expert level or that it gets as closely as possible to those expectations.<h2>Methods for generating data</h2>There are several methods you can leverage when generating synthetic data.<h3>Simulator data</h3>Use a simulator to generate new data. Custom shaders can provide RGB+D and segmentation labels. Microsoft is now focusing on aerospace-grade data generation because it is open source, AirSim 3D simulator project obtained promising results hence a lot of companies were using it for internal purposes.<iframe width="640" height="360" src="https://www.youtube.com/embed/b59sxe-M-bg?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe><h3 id="isPasted">Synthetic augmentation</h3>Simple computer vision enhancements can work as well. Using the OpenCVs inpaint function the price was rewritten as per the examples shown bellow. A mixed approach was implemented here, having the human labelers define the polygon in which the price was contained.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697620602389-synthetic_inpaint_example-768x376.jpg" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">Development focus</h2>Validating crowd samples is going to take a similar amount of effort as developing a dataset generator.This is the most essential part of AI and a lot of people get it wrong. If the dataset is poorly curated and not enough time is spent at this stage, your project may end up failing. The dataset should be one of your main priorities, and if you have enough quality data, you can train almost every model to perfection.<h2>Conclusion</h2>The most successful models deployed in production use a mix of the synthetic and real data to achieve great results.<a href="https://www.linkedin.com/company/adapta-robotics" target="_blank">Let us know</a> what other techniques you are using to give your model that extra edge in performance.

Training robust AI models can be challenging when using crowdsourced or publicly available datasets. The most useful neural networks are ones that replace expert knowledge to solve real problems. One example would be a visual inspection model for the factory floor.

Synthetic datasets-Or why you can't train robust models with generic data

In a recent episode of the <a href="https://www.youtube.com/@BrainChipInc" target="_blank">BrainChip podcas</a>t, Rob Telson, the Vice President of Sales and Marketing at BrainChip, spoke to Michael Azoff, an industry analyst with a focus on AI at the edge. He has 18 years of experience as an analyst and has previously worked hands-on with neural networks. We highlight some of the key discussion points below but encourage you to listen to the <a href="https://www.youtube.com/watch?v=xsIWZXlin7I&ab_channel=BrainChipInc" target="_blank">whole episode here.</a><h2 dir="ltr">A Dive into the Past and Present</h2>During the podcast, Azoff recounted his extensive experience with AI, which began in the 1990s. He reminisced about the early days of neuromorphic computing and how it was just taking root. As the conversation progressed, he highlighted the significant milestones, emphasizing the advancements in training neuromorphic architecture-based neural networks. He also acknowledged BrainChip's notable ascent in recent years, pointing to it as evidence of the growing momentum in the neuromorphic field.<h3>Neuromorphic's Niche in Edge AI</h3>Azoff's recent contributions to Gigaom's reports have centered on Edge AI, a field that emphasizes inferencing at the edge. In his analysis, the Edge AI domain bifurcates into three distinct categories: ultra-low power, low power, and automotive. Intriguingly, neuromorphic architectures have carved a niche for themselves in the ultra-low power segment. Their inherent efficiency and minimal power consumption make them a natural fit for this category.<h3>Applications: From Surveillance to Automotive</h3>Neuromorphic architectures excel in scenarios characterized by sparse data. For instance, in visual surveillance, where a static scene is occasionally interrupted by noteworthy events, these architectures prove invaluable. Their design allows them to operate in close proximity to sensors, processing data in real-time. The applications are diverse: smart doorbells, earbuds, industrial fault detection, and even monitoring crowd density at train stations. While vision and audio remain the primary focus areas, the potential applications are vast.In the automotive sector, a distinction emerges between in-cabin applications and driving functionalities. While the latter, especially autonomous driving, demands robust processing, neuromorphic computing finds its place in the former. In-cabin applications, ranging from infotainment to human-machine communication, can benefit immensely from neuromorphic's capabilities.<h2>Consumer Electronics: The Next Frontier</h2>As neuromorphic technology matures, its integration into consumer products is anticipated to grow. The allure of low power consumption cannot be overstated. For application developers, this presents a golden opportunity to innovate, pushing the boundaries of what's possible. In a decade, we might witness a paradigm shift where handheld devices and consumer electronics come equipped with neuromorphic processors, either as accelerators or integral components of the system's ecosystem.<h3>AI's Societal Impact and the Dream of Understanding the Human Brain</h3>Beyond applications, the podcast touched upon a profound topic: the beneficial impact of AI on society. AI's prowess lies in its ability to automate tasks that surpass human capabilities, especially when scale and complexity come into play. The horizon is vast, with the ultimate goal being the enhancement of AI architectures' intelligence, inching closer to the elusive general artificial intelligence.Azoff's vision for an AI superpower is both ambitious and profound: to unravel the mysteries of the human brain. If AI can aid in understanding our brain's intricacies, the implications for neuroscience, psychology, and AI itself could be groundbreaking.<h3>In Conclusion</h3>This episode of BrainChip's podcast offers a panoramic view of neuromorphic computing's past, present, and future. As we stand on the cusp of AI's next revolution, neuromorphic computing emerges as a beacon, guiding the way forward. The journey is just beginning, and the possibilities are boundless.<a href="https://www.wevolver.com/profile/brainchip" target="_blank">&nbsp;Stay up to date with BrainChip's new content here.</a>

In this episode, Rob Telson speaks with Industry Expert, Michael Azoff, about neuromorphic computing in the AI space including edge AI, neural network architectures, sensory-based applications and more.

BrainChip Podcast: Exploring Neuromorphic Computing with Industry Expert Michael Azoff

<h1 dir="ltr" id="isPasted">Introduction</h1>Gesture recognition is not just a buzzword; it's an indispensable technology that's changing the way we interact with devices. From smartphones to smart homes, the ability to control devices through simple hand movements offers an unparalleled level of convenience and accessibility. However, implementing robust gesture recognition systems is fraught with challenges, from data acquisition and preprocessing to machine learning model training and deployment.In this article, we will look into the technical aspects of developing a gesture recognition system using Infineon's PSoC 6 microcontroller. We'll explore each stage of the development process, from sensor data acquisition to neural network model deployment. We'll also share a link to a GitHub project detailing a step-by-step guide on implementing gesture classification, empowering you to create your own gesture recognition application. By the end of this article, you'll have a comprehensive understanding of the complexities involved and how specialized machine learning platforms such as Edge Impulse can simplify the process.<iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> <h1 dir="ltr">The Need for Gesture Classification</h1>The demand for intuitive human-machine interfaces is growing exponentially across various sectors. Whether it's automotive infotainment systems, medical devices, or consumer electronics, the ability to interact through gestures adds a layer of convenience and user-friendliness. However, implementing these systems on embedded platforms like the PSoC 6 involves challenges such as computational limitations, real-time processing requirements, and power constraints.To address these challenges, developers often have to make trade-offs between system performance and complexity. For instance, simpler algorithms may require fewer computational resources but may not provide the desired level of accuracy. On the other hand, complex algorithms may offer high accuracy but may be too resource-intensive for real-time applications. This delicate balance makes the choice of hardware and software platforms crucial in the development process. With these considerations in mind, let's delve into the capabilities of the Infineon PSoC 6 and see how it stands out as a viable solution for these challenges.<h1 dir="ltr">Infineon PSoC 6: An Overview</h1>Infineon's PSoC 6 is a versatile microcontroller unit (MCU) designed with low-power IoT applications in mind. It features a dual-CPU architecture, combining an ARM Cortex-M4 and an ARM Cortex-M0+ processor, allowing for efficient task partitioning. The PSoC 6 also offers programmable analog and digital blocks, making it highly customizable for specific application needs. Its rich set of features makes it an ideal candidate for implementing complex tasks like gesture recognition.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2NjAwMjk1LWltYWdlMy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="infineon-psoc-6-prototyping-kit">Fig. 1: Infineon Technologies PSoC™ 6 Wi-Fi® BT Prototyping Kit. Source:&nbsp;<a href="https://www.mouser.in/new/infineon/cypress-psoc-6-wifi-bt-proto-kit/" target="_blank" rel="noopener noreferrer">Mouser Electronics</a>The PSoC 6 is not just a powerful MCU; it's a complete ecosystem that provides developers with a range of tools and libraries to accelerate the development process. From ModusToolbox, a comprehensive software development environment, to a rich set of middleware components, the PSoC 6 ecosystem is designed to simplify complex tasks.<h1 dir="ltr">Sensor Data Acquisition</h1>For gesture recognition, sensor data is the raw material. PSoC 6 can interface with a variety of motion sensors, such as accelerometers and gyroscopes, using SPI and UART protocols. These sensors capture the 3D spatial movements that are then classified into specific gestures. However, data acquisition is not just about reading sensor values; it's about doing so reliably and efficiently. Specialized machine learning platforms like Edge Impulse can offer features like sensor fusion and automated data collection, ensuring that the raw data is as accurate as possible.Suggested Reading:&nbsp;<a href="https://www.wevolver.com/article/what-is-sensor-fusion-everything-you-need-to-know" target="_blank">Sensor Fusion: The Ultimate Guide to Combining Data for Enhanced Perception and Decision-Making</a>Data acquisition in real-world applications is often noisy and inconsistent. Factors like sensor drift, environmental conditions, and hardware limitations can introduce errors into the collected data. Therefore, it's essential to implement robust data acquisition strategies that can mitigate these issues. Techniques like sensor calibration, data validation, and outlier detection are often employed to improve the quality of the collected data.<h1 dir="ltr">Data Preprocessing and Filtering</h1>Once the raw sensor data is acquired, it needs to be preprocessed before feeding it into a machine learning model. This involves filtering out noise and normalizing the data. Techniques like Infinite Impulse Response (IIR) filtering and min-max normalization are commonly employed for this purpose. While these preprocessing steps are essential for the accuracy of the final model, they can be time-consuming to implement and optimize. Specialized machine learning platforms can automate much of this, allowing developers to focus on other critical aspects of the system.Data preprocessing is not a one-size-fits-all operation; it often requires fine-tuning based on the specific application requirements. For instance, the choice of filter parameters, the window size for normalization, and the sampling rate can all impact the performance of the gesture recognition system. Therefore, it's crucial to perform iterative testing and validation to fine-tune these parameters, a process that can be significantly streamlined using specialized machine learning platforms. Edge Impulse&nbsp;offers&nbsp;intuitive tools for data preprocessing, allowing developers to easily filter and normalize data, ensuring it's ready for model training.<h1 dir="ltr">Neural Network Models for Gesture Recognition</h1>Gesture recognition typically employs Convolutional Neural Networks (CNNs) due to their ability to process spatial hierarchies in the data. A standard CNN model for gesture recognition would include multiple layers, such as convolutional layers, Rectified Linear Unit (ReLU) activation functions, max pooling, and batch normalization. These layers work in tandem to extract features from the raw sensor data and classify them into specific gestures. However, training these models requires a deep understanding of machine learning algorithms, data science, and optimization techniques. Specialized machine learning platforms can simplify this process by offering pre-trained models and automated training pipelines.The complexity of the neural network model often depends on the number of gestures to be recognized and the required accuracy. More complex models with additional layers and nodes can offer higher accuracy but may be computationally intensive, making them unsuitable for real-time applications on resource-constrained devices like the PSoC 6. Therefore, model optimization techniques like pruning, quantization, and layer fusion are often employed to reduce the computational complexity without significantly compromising accuracy.<h1 dir="ltr">Implementing Gesture Recognition on PSoC 6</h1>Developing a gesture recognition system involves more than just machine learning models. The development environment needs to be set up, the hardware configured, and the software architecture designed. In the case of PSoC 6, the ModusToolbox software environment provides a range of tools for configuration and development. The code structure for implementing gesture recognition involves various tasks, such as data acquisition, preprocessing, and classification. FreeRTOS is often used for managing these tasks, offering the benefits of real-time operation and future scalability.The software architecture for implementing gesture recognition on PSoC 6 often involves multiple tasks running concurrently. For instance, one task may be responsible for data acquisition, another for data preprocessing, and yet another for gesture classification. Managing these tasks efficiently requires a robust multitasking environment, which is where real-time operating systems like FreeRTOS come into play. FreeRTOS allows for efficient task scheduling, inter-task communication, and resource management, making it easier to develop complex applications like gesture recognition.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2MTc1NTA3LWltYWdlMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 302px;" class="fr-fic fr-dib" alt="edge-AI-gesture-recognition">Fig. 2: Through edge AI techniques, gestures can be translated into data formats comprehensible by computers, enabling efficient real-time processing and decision-makingBeyond the software architecture, the deployment phase is also crucial. Once the model is trained and optimized, it needs to be converted into a format that can be loaded onto the PSoC 6 MCU. This often involves converting high-level model formats like H5 into C or C++ code that can be integrated into the ModusToolbox environment. This step can be cumbersome and error-prone if done manually. However, specialized machine learning platforms offer automated tools for model conversion and deployment, ensuring that the transition from development to production is seamless.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2MjQ4MjM5LXNwYWNlc19HRWdjQ2s0UGtTNVBhNnVCYWJsZF91cGxvYWRzX2dpdC1ibG9iLWQyYjE4YjVmYTMzM2YwYTJjNjMwY2MzYTBlZmM3ZGY3YTNiNDYzYTBfdHV0b3JpYWwtY29udGludW91cy1tb3Rpb24tY3JlYXRlLWltcHVsc2Uud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 304px;" class="fr-fic fr-dib" alt="edge-impulse-studio-gesture-recognition">Fig. 3: Designing an Edge AI application for motion recognition using Edge Impulse studio. Source:&nbsp;<a href="https://docs.edgeimpulse.com/docs/tutorials/end-to-end-tutorials/continuous-motion-recognition" target="_blank" rel="noopener noreferrer">Edge Impulse</a>The final piece of the puzzle is real-time performance monitoring and updates. Once deployed, the system's performance needs to be monitored to ensure it meets the desired accuracy and latency requirements. Any necessary updates or optimizations can be rolled out as firmware updates, a process that can be automated and streamlined using specialized machine learning platforms.As we've seen, the journey from concept to deployment in gesture recognition involves multiple steps, each with its own set of challenges and complexities. While the PSoC 6 provides a robust hardware platform, the software complexities often require specialized tools and platforms for efficient development and deployment.Here is a project by Infineon explaining the step by step process to implementing gesture classification using Infineon PSoC 6: <a href="https://github.com/Infineon/mtb-example-ml-gesture-classification" target="_blank">GitHub - Infineon/mtb-example-ml-gesture-classification</a>. For in-depth exploration of more Edge AI projects, don't forget to check out: <a href="https://edgeimpulse.com/blog" target="_blank">Edge Impulse Blog</a>.<h1 dir="ltr" id="isPasted">Conclusion</h1>Gesture recognition is poised to become a standard feature in the next generation of human-machine interfaces. While platforms like Infineon's PSoC 6 provide the hardware capabilities for such advanced applications, the development process involves several complexities. These complexities can be significantly simplified by leveraging the power of specialized machine learning platforms like Edge Impulse, making it easier to develop, deploy, and scale robust gesture recognition systems.Further reading:&nbsp;<a href="https://www.wevolver.com/article/spotlight-on-innovations-in-edge-computing-and-machine-learning-predictive-maintenance" target="_blank">Spotlight on Innovations in Edge Computing and Machine Learning: Predictive Maintenance</a><h1 dir="ltr">About the sponsor: Edge Impulse</h1><a href="https://www.edgeimpulse.com/" target="_blank">Edge Impulse</a> is the leading development platform for embedded machine learning, used by over 1,000 enterprises across 200,000 ML projects worldwide. We are on a mission to enable the ultimate development experience for machine learning on embedded devices for sensors, audio, and computer vision, at scale.&nbsp;From getting started in under five minutes to MLOps in production, we enable highly optimized ML deployable to a wide range of hardware from MCUs to CPUs, to custom AI accelerators. With Edge Impulse, developers, engineers, and domain experts solve real problems using machine learning in embedded solutions, speeding up development time from years to weeks. We specialize in industrial and professional applications including predictive maintenance, anomaly detection, human health, wearables, and more.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0OTU2NDM1MDUxLWltYWdlMS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="edge-impulse-logo"><iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>

Gesture recognition, leveraging the advanced capabilities of embedded devices and streamlined through specialized platforms is creating new means of human-machine interactions, paving the way for more intuitive and user-friendly device interfaces.

Gesture Recognition and Classification Using Infineon PSoC 6 and Edge AI

Article #5 of Spotlight on Innovations in Edge Computing and Machine Learning: A computer vision project that monitors vehicle traffic in real-time using video inferencing performed on the Brainchip Akida Development Kit.

Real-Time Traffic Monitoring with Neuromorphic Computing

This is the third of a&nbsp;<a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">series of articles</a>&nbsp;exploring the benefits of Edge AI for a variety of applications. &nbsp;<style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p3 {margin: 0.0px 0.0px 2.0px 0.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Carlito; color: #244084; background-color: #ffffff} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; background-color: #ffffff} p.p7 {margin: 0.0px 0.0px 0.0px 36.0px; line-height: 18.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p8 {margin: 0.0px 0.0px 0.0px 36.0px; font: 12.0px Helvetica; color: #000000; min-height: 14.0px} p.p9 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 18.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p10 {margin: 0.0px 0.0px 2.0px 0.0px; line-height: 18.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p11 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Carlito; color: #244084} p.p12 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 18.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff} p.p13 {margin: 0.0px 0.0px 12.0px 0.0px; font: 12.0px Helvetica; color: #000000; background-color: #ffffff; min-height: 14.0px} p.p14 {margin: 0.0px 0.0px 0.0px 0.0px; font: 16.0px Carlito; color: #244084; background-color: #ffffff} p.p15 {margin: 0.0px 0.0px 0.0px 0.0px; font: 16.0px Carlito; color: #244084} li.li6 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 18.0px; font: 12.0px Helvetica; color: #000000} span.s1 {font: 12.0px Symbol} span.s2 {background-color: #ffffff} span.s3 {font: 8.0px Helvetica} ul.ul1 {list-style-type: disc}
</style>Climate change is putting pressure on water reserves like never before, while gas supplies and electricity production &nbsp;have been impacted by geopolitical instability and surging post-COVID-19 demand.&nbsp;The key focus for businesses and governments alike is a strategic shift towards sustainably, where waste is minimized and efficiency is prioritized. Crucial to this new direction are advanced technologies such as edge AI.&nbsp;By enabling the energy and utilities infrastructure to monitor efficiency and make real-time smart decisions, Internet of Things (IoT) devices equipped with edge AI capabilities can be one of the cornerstones of this new tech-enabled sustainability landscape. &nbsp;<h2>The scale of the challenge&nbsp;</h2>We live in a new world. Relentless global demand for resources means meeting basic needs such as stable access to water and electricity are becoming increasingly difficult even in developed nations. In August, the UK Met Office declared a drought emergency in the Southwest of England following the driest conditions in 90 years. It meant urging businesses and the public to conserve water as much as possible.&nbsp;The European Commission also recently announced it was taking &lsquo;extraordinary&rsquo; measures to intervene in European energy markets. This was due to the rocketing costs of gas and electricity brought about by surging post-pandemic demand and Russia&rsquo;s invasion of Ukraine. The measures aimed to reduce demand for electricity while also imposing energy caps, and any additional demand-driven profits by energy companies redistributed to curb consumer costs.The outlook is quite clear. Electricity, gas, and water will continue to be in high demand across both the developed and developing worlds, while supply is expected to remain constrained, so maximizing efficiency, both in consumption and production, will be of vital importance going forward. &nbsp;<h2>Addressing the problem&nbsp;</h2>So beyond being able to drastically reduce usage or secure more supplies from somewhere, what are the solutions to this global dilemma? Put simply &ndash; increased efficiency.&nbsp;Electricity, gas, and water production and distribution requires hugely sophisticated operations with a many opportunities for waste and lack of efficiency at any stage of each process. To reduce waste and maximize efficiency, the whole process has to become smarter. A cost effictive way to accomplish this efficiency, safely, close to each production/redistribution step, and in a timely fashion is by using Internet of Things (IoT) devices.IoT devices can be used to collect a whole range of information from sensing the flow pressure in a pipeline to detecting changes in temperature which could indicate an imminent disaster. But that information is most useful once it is turned into actionable data. This has traditionally been done in the Cloud, with the information collected by IoT devices being transmitted wirelessly to a data center, where artificial intelligence algorithms analyze the information and make decisions.&nbsp;But relying on all information to be analyzed in the cloud can present a host of challenges. Firstly, the system is reliant on a stable wireless connection. Oftentimes in industrial settings, this is not guaranteed, especially as physical barriers may impact signal quality. Another challenge is the latency caused by the transmission of the information to the cloud and the respective transmission of the action back from the cloud to the point where information was collected. In safety critical cases, minutes and even seconds can mean the difference between averting a disaster or not. What is more, the transmission of large amounts of data consumes a lot of energy, which in turn can contribute to the energy scarcity problems that the industry aims to solve. &nbsp;An answer to these challenges is edge technology &ndash; where the decision making, or inference capability, is contained in the IoT device iteself. Edge-AI on IoT is being implemented across a range of settings, from oil rigs to wind farms, and is helping to to streamline production, avoid waste, and spot safety concerns.&nbsp;Examples of how this technology is being used today include:<ul><li>Edge enabled smart grids: IoT is used to monitor energy production and use across the grid, with AI making decisions on how to best distribute that energy, in what quantities and when. It means, for example, energy production doesn&rsquo;t always have to be on if there&rsquo;s already enough power available, preventing waste.</li><li>Enabling autonomous operations. IoT and edge AI can be used across industrial settings to enable autonomous operations. This cuts down on the need for human staff while improving efficiency. &nbsp;</li><li>Maintenance and condition-based monitoring. IoT sensors with edge AI can monitor industrial processes and devices in real-time, detecting problems and acting immediately to take corrective action. They can also record data and by using deep learning, predict when problems might arise in the future. This helps spot problems before they occur and helps avoid costly downtime. &nbsp;</li><li>Pipeline optimization using AI: Edge AI can also help find the best-fit routes to transfer gas and oil to refineries. This helps protect the long-term safety and health of the pipeline.&nbsp;</li><li>Water infrastructure enabled with AI: Water infrastructure, from reservoir to domestic and business end use, can be a source of waste. Being able to spot leaks early is essential and that&rsquo;s where AI can help. Edge-enabled IoT sensors can monitor control valves, reservoirs, distribution pipelines, supply tanks and other aspects of critical water infrastructure and supply networks. AI can then redirect supplies and notify engineers if repairs are needed. &nbsp;</li></ul><h2>Making edge AI smarter with neural networks&nbsp;</h2>The rationale for having edge-AI enabled IoT in the energy and utilities sectors is clear. However, &nbsp;the adoption of AI on the edge has been hampered by the size of the devices in the past: IoT devices are often small, and it has been difficult to fit the required amount of compute power to implement decision making on the edge effectively.&nbsp;After all, when crucial operations are entrusted to AI, how smart that AI is could mean the difference between success and potentially catastrophic failure. Decisions must be made in real-time about how to act on sensor readings to enable efficiency or predict a problem. It stands to reason then that the more intelligence the AI, the better its decision-making capabilities.One of the challenges of edge computing is that, unlike cloud computing where strategic decisions can be made with potentially unlimited data processing power at its disposal, decisions edge is constrained by size and sometimes an energy source (in the case of battery operated IoT devices).&nbsp;The AI has to be able to be essentially fit in to the IoT technology on site. The advantages of this, when implemented effectively, is little or no latency because there is no dependency on wireless connectivity. To get the best of both worlds, with real-time efficiency, low latency and high-quality decision making, IoT at the edge has to have high calibre AI. That&rsquo;s where neural networks come in.Neural networks have been nothing short of a game-changer for edge technology, remaining sufficiently small and compact to fit into IoT devices, while still delivering artificial intelligence decision-making capabilities required.&nbsp;<h2>Need for a strategic vision</h2>Energy and water crises are likely to last years if not decades so it&rsquo;s imperative that companies and nations alike have a bold, long-term strategic vision where waste reduction and improved efficiency are right at the top of the agenda.&nbsp;Creating smarter energy and utilities infrastructure must be one of the key components to this strategic vision, with IoT underpinned by neural network edge-AI technology that enables us to do more with what we have through improving efficiency and reducing waste at every stage.&nbsp;<h2>Moving towards sustainability with Syntiant</h2>Syntiant&rsquo;s technology is just one example of this new generation of edge-AI solutions. Its small yet powerfull Neural Decision ProcessorsTM&nbsp; utilize breakthrough silicon architecture, optimized for inferences on the edge, and are capable of delivering 100 times the performance when compared to traditional processing technologies.&nbsp;It means larger neural workloads can be moved from a high-powered CPU and GPUs in the data center to a low-power, always-on domain directly on the IoT device, allowing the deployment of intelligent devices across sites without compromising on AI processing power.&nbsp;The first article discussed <a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">about r</a><a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">educing AI&#39;s Vulnerable Attack Surface with Edge Computing.</a>The second article talked about <a href="https://www.wevolver.com/article/why-edge-ai-for-wearables" rel="noopener noreferrer" target="_blank">Edge AI in wearables.</a>The third article explored <a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">h</a><a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">ow edge AI is enabling cutting-edge advances in sustainability.</a>The forth article explained <a href="https://www.wevolver.com/article/why-edge-ai-is-a-win-for-automotive" rel="noopener noreferrer" target="_blank">why Edge AI is a win for automotive.</a>The fifth article analyzed <a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">c</a><a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">omputer Vision on Compute-Constrained Embedded Devices.</a>The sixth article explained <a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">why e</a><a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">dge AI is essential for EV Battery Management.</a><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY2MjAzMTYwMzA3LVVudGl0bGVkIGRlc2lnbi02ICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">

Article #3 of the "Why Edge?" Series. From soaring costs to dwindling resources, the energy and utility sectors face unprecedented challenges in the coming years. 


How edge AI is enabling cutting-edge advances in sustainability.

Researchers from the McKelvey School of Engineering at Washington University in St. Louis have developed a machine learning algorithm that can create a continuous 3D model of cells from a partial set of 2D images that were taken using the same standard microscopy tools found in many labs today.Their findings were published Sept. 16 in the journal <a href="https://www.nature.com/articles/s42256-022-00530-3" target="_blank">Nature Machine Intelligence</a>.&ldquo;We train the model on the set of digital images to obtain a continuous representation,&rdquo; said <a href="https://engineering.wustl.edu/faculty/Ulugbek-Kamilov.html" target="_blank">Ulugbek Kamilov</a>, assistant professor of electrical and systems engineering and of computer science and engineering. &ldquo;Now, I can show it any way I want. I can zoom in smoothly and there is no pixelation.&rdquo;The key to this work was the use of a neural field network, a particular kind of machine learning system that learns a mapping from spatial coordinates to the corresponding physical quantities. When the training is complete, researchers can point to any coordinate and the model can provide the image value at that location.A particular strength of neural field networks is that they do not need to be trained on copious amounts of similar data. Instead, as long as there is a sufficient number of 2D images of the sample, the network can represent it in its entirety, inside and out.The image used to train the network is just like any other microscopy image. In essence, a cell is lit from below; the light travels through it and is captured on the other side, creating an image.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1663764715370-2Dto3DKamilovcrop.jpg" style="width: 766px;" class="fr-fic fr-dib">The imaging system can zoom in on a pixelated image and fill in the missing pieces, creating a continuous 3D representation.&nbsp;&ldquo;Because I have some views of the cell, I can use those images to train the model,&rdquo; Kamilov said. This is done by feeding the model information about a point in the sample where the image captured some of the internal structure of the cell.Then the network takes its best shot at recreating that structure. If the output is wrong, the network is tweaked. If it&rsquo;s correct, that pathway is reinforced. Once the predictions match real-world measurements, the network is ready to fill in parts of the cell that were not captured by the original 2D images.The model now contains information of a full, continuous representation of the cell &mdash; there&rsquo;s no need to save a data-heavy image file because it can always be recreated by the neural field network.And, Kamilov said, not only is the model an easy-to-store, true representation of the cell, but also, in many ways, it&rsquo;s more useful than the real thing.&ldquo;I can put any coordinate in and generate that view,&rdquo; he said. &ldquo;Or I can generate entirely new views from different angles.&rdquo; He can use the model to spin a cell like a top or zoom in for a closer look; use the model to do other numerical tasks; or even feed it into another algorithm.

The imaging system can zoom in on a pixelated image and fill in the missing pieces, creating a continuous 3D representation.

Neural field network doesn’t need to be trained on other samples

Machine learning generates 3D model from 2D pictures

Mikrocentrum-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Mikrocentrum-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Mikrocentrum-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Mikrocentrum-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Mikrocentrum-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Syntiant-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Syntiant-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Syntiant-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Syntiant-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Syntiant-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

The Things Network-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

The Things Network-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

The Things Network-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

The Things Network-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

The Things Network-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Festo Corporation-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Festo Corporation-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Festo Corporation-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Festo Corporation-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Festo Corporation-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

SPE - Industrial Partner Network-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

SPE - Industrial Partner Network-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

SPE - Industrial Partner Network-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

SPE - Industrial Partner Network-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

SPE - Industrial Partner Network-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Monogoto-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Monogoto-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Monogoto-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Monogoto-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Monogoto-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Thought3D Ltd-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Thought3D Ltd-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Thought3D Ltd-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Thought3D Ltd-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Thought3D Ltd-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Engineering Clicks-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Engineering Clicks-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Engineering Clicks-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Engineering Clicks-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Engineering Clicks-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Xometry Europe-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Xometry Europe-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Xometry Europe-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Xometry Europe-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Xometry Europe-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Monogoto

AVSystem

IoT Stars

neural network

neural network

Profiles