Edge AI Technology Report: Generative AI Edition Chapter 2. Generative AI and edge computing are transforming industries by enabling low-latency, real-time AI on edge devices, allowing efficient, private, and personalized applications without reliance on data centers.

Innovations and Advancements in Generative AI at the Edge

Edge AI Technology Report: Generative AI Edition
Chapter 1. Generative AI's demand for real-time insights is driving a shift from cloud to edge computing, enabling faster, local processing on devices and reducing cloud latency and bandwidth constraints.

Leveraging Edge Computing for Generative AI

Large Language Model (LLM) training involves teaching AI models to understand and generate human-like text by processing vast amounts of data, significantly enhancing their language comprehension and production capabilities.

LLM Training: Mastering the Art of Language Model Development

<p>We once believed the cloud was the final frontier for artificial intelligence (AI), but it turns out the real magic happens much closer to home—at the edge, where devices can now think, generate, and respond in real-time. The rapid evolution of AI, particularly generative AI, is fundamentally reshaping industries and challenging the existing computing infrastructure.&nbsp;</p><p>Traditional models, especially resource-intensive ones like Large Language Models (LLMs), have long relied on centralized cloud systems for the necessary computational power. However, as the need for AI-driven interactions grows across sectors—from autonomous vehicles to personalized content generation—there is a clear shift toward edge computing.&nbsp;</p><p>Our new report, drawing from discussions with industry leaders and technologists, explores how generative AI is being harnessed and integrated into edge environments and what this means for the future of technology.</p><p>Download your copy today to stay ahead of generative AI at the edge.&nbsp;</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1730905294088-Mitsubishi+Chemical+Group++Growth+Garage++Open+Innovation+Program+%2810%29.png" style="width: 100%px;" class="fr-fic fr-dib"></p><p>&lt;Form&gt;</p><h2>Why this report matters?</h2><p id="isPasted">Learning about GenAI at the edge is crucial for engineers to meet growing industry demands and stay competitive in a rapidly evolving field. As industries like autonomous vehicles, healthcare, IoT, and industrial automation increasingly rely on real-time AI solutions, engineers must adapt their skills to handle resource-constrained environments and optimize AI models for efficient edge computing. Understanding how to deploy GenAI at the edge allows engineers to develop faster, more secure systems, addressing key industry challenges such as low-latency performance, privacy, and data security.</p><p>Staying ahead in this complex field requires mastering techniques like model optimization, federated learning, and energy-efficient processing. By learning about GenAI at the edge, engineers enhance their expertise in interdisciplinary areas such as machine learning, embedded systems, and networking, positioning themselves at the forefront of technological advancements. This knowledge not only boosts career prospects but also ensures engineers can innovate solutions for future technologies like 5G, IoT, and intelligent systems.</p><p id="isPasted"><strong>Report Highlights:</strong></p><ul><li><strong>Cutting-Edge Applications:</strong> Real-world examples show how generative AI at the edge is revolutionizing operations in fields like autonomous driving, medical diagnostics, industrial automation, and retail.</li><li><strong>Optimization Techniques for Edge Deployment:</strong> Insights on deploying complex AI models on resource-limited devices using techniques such as pruning, quantization, and federated learning.</li><li><strong>Industry Perspectives:</strong> Contributions from EmbedUR, Ambiq, and BrainChip offer expert guidance on how generative AI is reshaping sectors requiring real-time data processing and intelligent responses.</li><li><strong>Challenges and Solutions:</strong> Analysis of key challenges faced when deploying AI at the edge and practical solutions, including orchestration frameworks, MLOps, and secure data protocols.</li></ul><p>Download the full report with custom illustrations and case studies below:</p><p>&lt;Form&gt;</p><h2>Report introduction</h2><p dir="ltr" id="isPasted">Unlike cloud-centric approaches, edge computing brings data processing closer to where it is generated—on devices such as sensors, microcontrollers (MCUs), gateways, and more powerful edge servers. This shift is essential for applications requiring low latency, high bandwidth efficiency, and enhanced data privacy. The first chapter of this report dives into the foundational role that edge computing plays in enhancing generative AI, providing a detailed look at the various devices involved and the practical considerations for integrating these technologies. The chapter also explores how&nbsp;orchestration and machine learning operations (MLOps)&nbsp;are critical in managing AI workloads across the edge-to-cloud continuum, with insights from industry perspectives on the benefits and challenges of edge deployments.</p><p dir="ltr">As we move through the report, we delve into the innovations and advancements driving this convergence. The second chapter highlights recent breakthroughs in generative AI and edge research, examining how AI workloads are coordinated from the far edge to the cloud. This chapter not only covers the technical advancements but also provides a market analysis of the trends and drivers behind industry adoption. The focus is on understanding the collaborative efforts and partnerships pushing the boundaries of what is possible in this space.</p><p dir="ltr">Real-world applications bring these concepts to life, so we dedicated the third chapter to showcasing use cases for deploying generative AI at the edge across various industries. The report illustrates how these technologies transform operations in robotics, healthcare, manufacturing, automotive, retail, and smart cities. These examples highlight the tangible benefits of deploying AI at the edge, including improved efficiency, real-time decision-making, and enhanced user experiences.</p><p dir="ltr">Of course, with innovation comes challenges. The fourth chapter addresses the key hurdles organizations face when implementing generative AI at the edge, from managing the complexity of distributed networks to ensuring the reliability of AI models in resource-constrained environments. This section offers strategies and best practices for overcoming these challenges, informed by insights from leading experts in the field.</p><p dir="ltr">Finally, the report considers the broader impact of these technologies on business and society. The fifth chapter discusses the economic, social, and environmental implications of deploying generative AI at the edge. It explores how these technologies can drive cost savings, enhance accessibility, and contribute to sustainability goals,&nbsp;with perspectives from industry leaders on the long-term effects.</p><p dir="ltr">This report reflects the authors’, contributors’, and Wevolver’s deep commitment to providing industry leaders, technologists, and decision-makers with the insights needed to navigate the complex landscape of generative AI and edge computing. The contributions from the Wevolver team, along with extensive research and expert input, ensure that the content not only informs but also inspires further innovation. As you engage with this report, we hope it serves as a comprehensive guide to understanding and harnessing the power of generative AI at the edge, offering a roadmap for future advancements and industry growth.</p><h2 dir="ltr">Download the full report now.</h2><p>&lt;form&gt;</p><h2>Acknowledgments</h2><p>This report is made possible by the support of our sponsors, EmbedUR, Ambiq, and BrainChip.&nbsp;</p><p>Samir Jaber was the editor-in-chief and provided incredible leadership and expertise. We encourage you to share this report with your colleagues and network.&nbsp;</p><p><br></p>

<p>We once believed the cloud was the final frontier for artificial intelligence (AI), but it turns out the real magic happens much closer to home—at the edge, where devices can now think, generate, and respond in real-time. The rapid evolution of AI, particularly generative AI, is fundamentally reshaping industries and challenging the existing computing infrastructure.&nbsp;</p><p>Traditional models, especially resource-intensive ones like Large Language Models (LLMs), have long relied on centralized cloud systems for the necessary computational power. However, as the need for AI-driven interactions grows across sectors—from autonomous vehicles to personalized content generation—there is a clear shift toward edge computing.&nbsp;</p><p>Our new report, drawing from discussions with industry leaders and technologists, explores how generative AI is being harnessed and integrated into edge environments and what this means for the future of technology.</p><p>Download your copy today to stay ahead of generative AI at the edge.&nbsp;</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1733837684916-download+below.png" style="width: 100%px;" class="fr-fic fr-dib"></p><p><br></p><h2><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe id="JotFormIFrame-243114070410337" title="Edge AI Technology Report: Generative AI Edition - Report Form" allowtransparency="true" src="https://form.jotform.com/243114070410337" frameborder="0" style="min-width: 100%; max-width: 100%; height: 460px; border: none;" scrolling="yes" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span> </iframe></span>Why this report matters?</h2><p id="isPasted">Learning about GenAI at the edge is crucial for engineers to meet growing industry demands and stay competitive in a rapidly evolving field. As industries like autonomous vehicles, healthcare, IoT, and industrial automation increasingly rely on real-time AI solutions, engineers must adapt their skills to handle resource-constrained environments and optimize AI models for efficient edge computing. Understanding how to deploy GenAI at the edge allows engineers to develop faster, more secure systems, addressing key industry challenges such as low-latency performance, privacy, and data security.</p><p>Staying ahead in this complex field requires mastering techniques like model optimization, federated learning, and energy-efficient processing. By learning about GenAI at the edge, engineers enhance their expertise in interdisciplinary areas such as machine learning, embedded systems, and networking, positioning themselves at the forefront of technological advancements. This knowledge not only boosts career prospects but also ensures engineers can innovate solutions for future technologies like 5G, IoT, and intelligent systems.</p><h3 id="isPasted"><strong>Report Highlights:</strong></h3><ul><li><strong>Cutting-Edge Applications:</strong> Real-world examples show how generative AI at the edge is revolutionizing operations in fields like autonomous driving, medical diagnostics, industrial automation, and retail.</li><li><strong>Optimization Techniques for Edge Deployment:</strong> Insights on deploying complex AI models on resource-limited devices using techniques such as pruning, quantization, and federated learning.</li><li><strong>Industry Perspectives:</strong> Contributions from Particle, Syntiant and Edge Impulse offer expert guidance on how generative AI is reshaping sectors requiring real-time data processing and intelligent responses.</li><li><strong>Challenges and Solutions:</strong> Analysis of key challenges faced when deploying AI at the edge and practical solutions, including orchestration frameworks, MLOps, and secure data protocols.</li></ul><p>Download the full report with industry insights and case studies below:</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe id="JotFormIFrame-243114070410337" title="Edge AI Technology Report: Generative AI Edition - Report Form" allowtransparency="true" src="https://form.jotform.com/243114070410337" frameborder="0" style="min-width:100%;max-width:100%;height:460px;border:none;" scrolling="yes" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span> </iframe></span></p><h2>Report introduction</h2><p dir="ltr" id="isPasted">Unlike cloud-centric approaches, edge computing brings data processing closer to where it is generated—on devices such as sensors, microcontrollers (MCUs), gateways, and more powerful edge servers. This shift is essential for applications requiring low latency, high bandwidth efficiency, and enhanced data privacy. The first chapter of this report dives into the foundational role that edge computing plays in enhancing generative AI, providing a detailed look at the various devices involved and the practical considerations for integrating these technologies. The chapter also explores how&nbsp;orchestration and machine learning operations (MLOps)&nbsp;are critical in managing AI workloads across the edge-to-cloud continuum, with insights from industry perspectives on the benefits and challenges of edge deployments.</p><p dir="ltr">As we move through the report, we delve into the innovations and advancements driving this convergence. The second chapter highlights recent breakthroughs in generative AI and edge research, examining how AI workloads are coordinated from the far edge to the cloud. This chapter not only covers the technical advancements but also provides a market analysis of the trends and drivers behind industry adoption. The focus is on understanding the collaborative efforts and partnerships pushing the boundaries of what is possible in this space.</p><p dir="ltr">Real-world applications bring these concepts to life, so we dedicated the third chapter to showcasing use cases for deploying generative AI at the edge across various industries. The report illustrates how these technologies transform operations in robotics, healthcare, manufacturing, automotive, retail, and smart cities. These examples highlight the tangible benefits of deploying AI at the edge, including improved efficiency, real-time decision-making, and enhanced user experiences.</p><p dir="ltr">Of course, with innovation comes challenges. The fourth chapter addresses the key hurdles organizations face when implementing generative AI at the edge, from managing the complexity of distributed networks to ensuring the reliability of AI models in resource-constrained environments. This section offers strategies and best practices for overcoming these challenges, informed by insights from leading experts in the field.</p><p dir="ltr">Finally, the report considers the broader impact of these technologies on business and society. The fifth chapter discusses the economic, social, and environmental implications of deploying generative AI at the edge. It explores how these technologies can drive cost savings, enhance accessibility, and contribute to sustainability goals,&nbsp;with perspectives from industry leaders on the long-term effects.</p><p dir="ltr">This report reflects the authors’, contributors’, and Wevolver’s deep commitment to providing industry leaders, technologists, and decision-makers with the insights needed to navigate the complex landscape of generative AI and edge computing. The contributions from the Wevolver team, along with extensive research and expert input, ensure that the content not only informs but also inspires further innovation. As you engage with this report, we hope it serves as a comprehensive guide to understanding and harnessing the power of generative AI at the edge, offering a roadmap for future advancements and industry growth.</p><h3 dir="ltr">Read an excerpt from the report's chapters here:</h3><p>Introduction Chapter<br><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1736520814848-1.png" style="width: 100%px;" class="fr-fic fr-dib">Our new report, drawing from discussions with industry leaders and technologists, explores how generative AI is being harnessed and integrated into edge environments and what this means for the future of technology.</p><p><a href="https://www.wevolver.com/article/chapter-i-leveraging-edge-computing-for-generative-ai">Chapter 1<br id="isPasted"></a><a href="https://www.wevolver.com/article/chapter-i-leveraging-edge-computing-for-generative-ai"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1736520908314-2.png" class="fr-fic fr-dii"></a>Generative AI's demand for real-time insights is driving a shift from cloud to edge computing, enabling faster, local processing on devices and reducing cloud latency and bandwidth constraints.</p><p id="isPasted"><a href="https://www.wevolver.com/article/chapter-i-leveraging-edge-computing-for-generative-ai" class="button-main-action">READ MORE HERE</a></p><p><a href="https://www.wevolver.com/article/chapter-ii-innovations-and-advancements-in-generative-ai-at-the-edge">Chapter 2</a></p><p><a href="https://www.wevolver.com/article/chapter-ii-innovations-and-advancements-in-generative-ai-at-the-edge"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1736521012489-3.png" style="width: 100%px;" class="fr-fic fr-dib"></a></p><p>Generative AI and edge computing are transforming industries by enabling low-latency, real-time AI on edge devices, allowing efficient, private, and personalized applications without reliance on data centers.</p><p id="isPasted"><a href="https://www.wevolver.com/article/chapter-ii-innovations-and-advancements-in-generative-ai-at-the-edge" class="button-main-action">READ MORE HERE</a></p><p><a href="https://www.wevolver.com/article/real-world-applications-of-generative-ai-at-the-edge">Chapter 3<br id="isPasted"></a><a href="https://www.wevolver.com/article/real-world-applications-of-generative-ai-at-the-edge"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1736521035272-4.png" class="fr-fic fr-dii"></a>Advances in model optimization techniques like pruning, quantization, and knowledge distillation pave the way for more efficient deployment of generative AI at the edge.</p><p id="isPasted"><a href="https://www.wevolver.com/article/real-world-applications-of-generative-ai-at-the-edge" class="button-main-action">READ MORE HERE</a></p><p><a href="https://www.wevolver.com/article/challenges-and-opportunities-in-edge-based-generative-ai">Chapter 4<br id="isPasted"></a><a href="https://www.wevolver.com/article/challenges-and-opportunities-in-edge-based-generative-ai"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1736521050459-5.png" class="fr-fic fr-dii"></a>Generative AI Edition explores the technical hurdles organizations face as they attempt to leverage edge-based generative AI. It also examines strategic opportunities for innovation in hardware, deployment configurations, and security measures.</p><p id="isPasted"><a href="https://www.wevolver.com/article/challenges-and-opportunities-in-edge-based-generative-ai" class="button-main-action">READ MORE HERE</a></p><h2 dir="ltr"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1733837598489-download+below.png" style="width: 100%px;" class="fr-fic fr-dib"></h2><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe id="JotFormIFrame-243114070410337" title="Edge AI Technology Report: Generative AI Edition - Report Form" allowtransparency="true" src="https://form.jotform.com/243114070410337" frameborder="0" style="min-width:100%;max-width:100%;height:460px;border:none;" scrolling="yes" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span> </iframe></span></p><h2>Acknowledgements</h2><p>This report is made possible by the support of our sponsors, <a href="https://www.wevolver.com/profile/particle">Particle</a>, <a href="https://www.wevolver.com/profile/syntiant">Syntiant&nbsp;</a>and <a href="https://www.wevolver.com/profile/edge.impulse">Edge Impulse</a> and the support of the EDGE AI FOUNDATION, the global place of innovation, collaboration, advocacy, and education for energy-efficient, affordable, and scalable edge AI technologies, from tinyML to the edge of AI.</p><p>Samir Jaber was the editor-in-chief and provided incredible leadership and expertise.&nbsp;</p><p>We encourage you to share this report with your colleagues and network.&nbsp;</p>

Introduction

A deep dive into the convergence of generative AI and edge computing.

Edge AI Technology Report: Generative AI Edition

Curious about how AI predicts trends or creates new content? In this piece, we dive into Predictive AI vs. Generative AI, exploring their core differences and real-world impact. Discover how these technologies shape everything from recommendations to creativity.

Generative AI vs Predictive AI: Unveiling the Titans of Artificial Intelligence

<p data-block-key="of49a" id="isPasted">The computing industry's energy appetite is notoriously ravenous. Currently, this results in the industry producing between 2.1 and 3.9 percent of all global carbon emissions. And that is before the true impacts of new energy-consuming applications such as generative AI (artificial intelligence) and upgraded virtual communal spaces have kicked in.</p><p data-block-key="6e2r3">The National Science Foundation (NSF) recently announced a&nbsp;<a href="https://seas.harvard.edu/news/2024/05/sustainable-computing-project-awarded-12-million-nsf" target="_blank">new $12 million multi-institutional research initiative</a> over the next five years to delve deeply into this topic, calling for a rethinking of the infrastructure for information and communication technology (ICT), and paving the way for sustainable computing. Co-led by David Brooks of Harvard and Benjamin Lee of the University of Pennsylvania, the initiative is called NSF Expeditions in Computing. Its overarching goal is admirably high: to reduce computing's carbon footprint by 45 percent by 2030.</p><p data-block-key="9ab3r">Caltech's&nbsp;<a href="https://www.eas.caltech.edu/people/adamw" target="_blank">Adam Wierman</a>, the Carl F Braun Professor of Computing and Mathematical Sciences and director of Information Science and Technology, is part of the new NSF Expeditions team. For the past decade he has spearheaded the design of algorithms to make data centers more sustainable, and his group has worked with industry to improve standards of measurement and reporting of the carbon costs of computation.</p><p data-block-key="c3h0g">We recently talked with him about the AI boom, how data centers could actually help the electrical grid, the new NSF Expeditions initiative, and his role in a recent startup company called&nbsp;<a href="https://www.verrusdata.com/" target="_blank">Verrus</a> that aims to build sustainable data centers.</p><h2 data-block-key="85oer"><strong>What is the challenge that this new NSF Expeditions initiative will be taking on?</strong></h2><p data-block-key="26ct4">The challenge is that AI is taking off and there is this massive excitement around it. And the things underlying that are data centers, GPUs (graphics processing units), and computing. These are using enormous amounts of energy already, and all indications are that this is going to be growing at absurd rates in the coming years. A company that adopts generative AI for something where they used to use more traditional optimization methods is probably using 10 to 20 times more carbon to do that, so if improvements aren't made, you're talking doubling or more in a five-year period in terms of the carbon footprint of the information technology that we do.</p><p data-block-key="d8tfc">Already in many areas where data centers are located, like Northern Virginia or Ireland or Amsterdam, you're talking about 20 or 30 percent of the electricity usage in those areas being devoted to data centers. And that's&nbsp;<em>before</em> the boom of AI training is really hitting.</p><p data-block-key="a8ebc">Utilities are reaching the point where they can no longer handle the demand. So, for example, you can't build a data center in some regions right now because they just can't handle the energy demand. And the big data centers that places like OpenAI are planning to build are gigawatt-scale data centers. That is as much energy as medium-sized cities use going into a single campus for training.</p><h2 data-block-key="fj8sp"><strong>Can you talk about the impact of these data centers?</strong></h2><p data-block-key="8sk67">For a data center built in the traditional way, if you think about carbon usage, you have the carbon that goes into the manufacturing of the hardware and the building. This is called embodied carbon. And then you've got the operational carbon from running and doing the training. And both of those are enormous. Then additionally, beyond that, you've got the water usage that is running into these data centers to aid with cooling. And the water demands are also enormous.</p><p data-block-key="aupcr">But another aspect of all of this is the local impact. When you look especially at new campuses that are being built in areas where there haven't been many data centers before, you're plopping down major energy- and water-using resources in moderate-size cities. This has an impact on energy and water rates for people who live there. There's the pollution associated with the backup generators at the data centers, which have to be run in order to do regular maintenance. And so there's major local environmental and economic impacts from them in addition to just the global usage of carbon. These centers are also major construction projects. But beyond construction, they don't create many jobs for the neighborhoods where they go because there's not a lot of human needs in terms of running them once they're built. So, there are huge challenges around their construction.</p><p data-block-key="2solk">And yet they're essential for advancing AI and all of the improvements that come with that.</p><p data-block-key="9b2bm"><em>That relates to my next question about the Sidewalk Infrastructure Partners (SIP) startup company Verrus. How are you involved with that and what is the goal there?</em></p><p data-block-key="15l79">I'm involved on the advisory board, and I was consulting with SIP as they thought about launching Verrus. Our goal is to build the most sustainable data centers operating at these huge scales.</p><p data-block-key="80fsq">Data centers today often are powered by large-scale backup generators and are just users of the grid and utilities, but they don't need to be. In fact, a lot of technology already exists from our research and from many other groups like ours that have been working on this for the last decade. But it has been very hard for this technology to make its way into the data-center world until now.</p><p data-block-key="2djdr">All the challenges that we just talked about actually provide a huge market opportunity in the sense that utilities in many places where companies want to build data centers just won't accept them, or at least there will be a very large interconnect queue unless a data center can help the utility meet its needs in some way. But if you can be a partner with the utility instead of a drain, then you can get access to capacity. You might be able to build in places where it would otherwise be hard.</p><p data-block-key="8hi7">To do that, there's a lot of optimization work to be done—optimization of the way power works in the data center, of the way scheduling works, of the type of storage and backup power systems that you build in the center. And Verrus is going to be optimizing all of that really at the cutting edge so that it can be a partner for utilities in a way that data centers typically are not.</p><h2 data-block-key="c6smb"><strong>Going back to the NSF Expeditions, how does the initiative plan to make ICT greener?</strong></h2><p data-block-key="9f7ne">I'd say that a decade ago it was a hard sell to have companies pay attention to the impact of the operational carbon in what they did. Gradually, the environmental impact of data centers started to become clearer, and operationally they started to measure and understand their carbon usage. A bunch of our students were some of the first ones to build the measurement tools at a number of these companies.</p><p data-block-key="9qfbp">Then some companies started making purchasing agreements for solar and wind power whenever they built data centers. Some had these net-zero yearly basis goals of trying to counteract the impact of their energy in the operational sense. But that doesn't really solve the problem because you're not feeding green energy in when you're using it, so you're still creating massive strain on the grid.</p><p data-block-key="agsqr">It's only recently that companies have been moving toward actual 24/7 net-zero promises in terms of their operating impact. But achieving that is a huge challenge. Part of the Expedition is trying to move toward a data-center design, from the operational perspective, that can achieve that goal. That means that you need to be very responsive to the grid, to energy availability, and to moving workloads—to both shift them in time to be able to do the work when you have more energy available and do less when there's less green energy available, but also to move them across locations so that you can do work where there's green energy available. That's a big algorithmic problem.</p><p data-block-key="1qvtq">And then, as we discussed with relation to Verrus, data centers can actually help the grid. Much of the operating constraints of an electrical grid come in just a few hours a year where there's really the peak need, like during the hottest time of the hottest summer day. That means that a data center being flexible and doing what we call "peak shaving" and reducing its loads in those moments of peak demand can help the grid itself operate much more efficiently and reduce the need to install new conventional generators. The transition from a data center as a huge load for the grid to one where it's a resource that helps the grid operate more efficiently is a crucial one for the coming years.</p><p data-block-key="ce932">On the embodied carbon side, there is a need to minimize the carbon that's used in manufacturing the computing resources themselves. Here we are talking about your typical three R's: reduce, reuse, recycle. In this area, there is not even an agreed-upon measure for the impact—the amount of embodied carbon that's going into data centers, how you measure it, how you pay attention to where and when things were manufactured. Once you can measure and understand that effectively, you can start to evaluate how companies are doing and give them tools to reduce their embodied carbon for these data centers.</p><p data-block-key="9jmc8">The ambitious goal is to actually reduce the carbon impact of ICT by 45 percent despite a massive growth in the use of computing infrastructure. And honestly, I think that's doable if you can have a broad industry shift in reuse and recycling of hardware and operational integration with the grid.</p><h2 data-block-key="1p4eu"><strong>I know that when you first came to Caltech as a computer scientist this work on sustainable computing wasn't really on your radar. How do you feel about the work that you are doing today?</strong></h2><p data-block-key="6mbv1">I think for me, there is a societal moment here that is just crucial. This is maybe one of the biggest challenges around energy at the moment. Because of the power of AI to help sustainability problems broadly—to design better materials, to design better storage, all of that—we&nbsp;<em>need</em> to enable it. So we need these data centers, and we need this computing infrastructure in order to help make all industries more sustainable. And, at the same time, we're going to be constrained in the way we can use AI unless we can solve AI's sustainability problem directly. So, this is a hugely important moment.</p><p data-block-key="ftgnh">There's a sense that what we've been talking about in my group and in this research area for 10 years is finally here and visible to the rest of the industry and the research community. We've been talking about sustainability, and we've been making progress along the way, but really the moment has arrived where this stuff is front and center. And it's exciting to be able to make this impact on what is a really important problem at a really important time.</p>

The computing industry's energy appetite is notoriously ravenous. Currently, this results in the industry producing between 2.1 and 3.9 percent of all global carbon emissions.

Sustainable Computing in the New World of AI

<p dir="ltr" id="isPasted">In recent years, the rise of Generative Artificial Intelligence (AI) has been nothing short of revolutionary. This technology, capable of producing content that closely mimics human output, has paved the way for innovations across various fields. From writing convincingly real emails to creating impressive images, generative AI's capabilities are vast and transformative.</p><p dir="ltr">However, as with any technological advancement, there are two sides to the story. The same capabilities that offer numerous benefits also pose significant risks, particularly concerning cybersecurity. Specifically, one of the most concerning developments has been the enhancement of phishing schemes, making them more sophisticated and challenging to detect than ever before. This article explores the rise of generative AI, how it's changing the phishing landscape, and best practices to avoid becoming a victim.&nbsp;</p><h1 dir="ltr">The Evolution of Generative AI</h1><p dir="ltr">Generative AI is marked by its ability to learn from a vast dataset of existing digital content and generate new, unique outputs that can pass for human-created. This process involves complex algorithms and neural network models that analyze patterns, styles, and structures within the data they're fed. As a result, generative AI can produce a wide range of outputs, including textual content, images, music, and even synthetic voices. Its applications are far-reaching, offering potential benefits in areas such as content creation, entertainment, education, and beyond. <sup>[1]</sup></p><p dir="ltr">However, the power of generative AI extends beyond positive use cases. Its very capabilities, particularly in mimicking human communication styles and generating personalized content, have opened new avenues for cybercriminals. These advancements have led to a new generation of phishing attacks that are significantly more difficult to identify and combat.</p><h1 dir="ltr">The Transformation of Phishing Tactics</h1><p>Phishing schemes have long been a staple in the arsenal of cybercriminals. Traditionally, these attacks involved sending generic, often poorly written emails, attempting to lure recipients into divulging sensitive information or downloading malicious software. While effective to a degree, these attempts were easier to spot and avoid due to their lack of personalization and sophistication. <sup>[2]</sup></p><p>Enter generative AI, and the landscape of phishing has changed dramatically. With the ability to analyze and replicate communication styles, generative AI enables cybercriminals to craft highly convincing and personalized messages. These messages can mimic the tone, style, and even specific phrases used by individuals or organizations, making them far more likely to deceive recipients.</p><h1 dir="ltr">Understanding AI-Enhanced Phishing</h1><p>The core threat of AI-enhanced phishing is its personalization and realism. Generative AI can create emails, text messages, or social media messages that look and sound like they're from a trusted source, such as a colleague, friend, or a reputable organization. This level of personalization is achieved by analyzing publicly available data or previously breached information to tailor attacks to individual targets. <sup>[3]</sup></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 790px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE0Mzg4NjY3Mjg2LWN5YmVyMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"></span></span></span></p><p dir="ltr" id="isPasted">Phishing attacks begin with tricking users out of their important credentials. Image credit:&nbsp;<a href="https://www.valimail.com/guide-to-phishing/" target="_blank">Vailmail</a></p><p><br></p><p dir="ltr" id="isPasted">For instance, a generative AI-powered phishing email might reference recent transactions, use language that matches the supposed sender's usual tone, or mimic the layout and design of legitimate correspondence. With such attention to detail, these fraudulent messages are incredibly persuasive, increasing the likelihood that recipients will comply with their requests, whether that involves providing passwords, financial information, or other sensitive data.</p><h1 dir="ltr">The Signs of AI-Generated Phishing Attacks</h1><p>Despite their sophistication, AI-generated phishing attacks are not foolproof. There are still telltale signs that can help discerning individuals identify them. These include slight inconsistencies in language or format, unusual requests, or links to websites that, upon closer inspection, have subtle differences from legitimate addresses. Moreover, these messages might lack the personal touch or in-depth knowledge that genuine communications from friends or colleagues would have, relying instead on information that could be publicly accessed or inferred.</p><h1 dir="ltr">Strategies for Personal and Organizational Protection</h1><p dir="ltr">Naturally, the advancement of AI-enhanced phishing schemes necessitates stronger defensive measures, both at personal and organizational levels. Awareness is the first step; understanding that no one is immune to these attacks is crucial. From there, adopting a multi-layered approach to security can significantly mitigate risks. <sup>[4]</sup></p><h2 dir="ltr">For individuals:</h2><ul><li dir="ltr"><p dir="ltr">Be Skeptical of Unsolicited Communications: Always approach unexpected requests for information or action with caution, even if they appear to come from known contacts or organizations.</p></li><li dir="ltr"><p dir="ltr">Verify Independently:&nbsp;If a message asks for sensitive information or to perform a specific action, verify its authenticity through an independent channel. For example, if you receive an email from your bank asking for personal details, contact the bank directly using a phone number from their official website.</p></li><li dir="ltr"><p dir="ltr">Use Advanced Security Features:&nbsp;Enable two-factor authentication (2FA) wherever possible. This adds an extra verification step that can protect your accounts even if your password is compromised.</p></li><li dir="ltr"><p dir="ltr">Regularly Update Software:&nbsp;Ensure that your operating system, applications, and anti-virus software are up to date to protect against known vulnerabilities.</p></li></ul><h3>For organizations:</h3><ul><li dir="ltr"><p dir="ltr">Conduct Regular Training: Employees should be regularly trained on the latest cybersecurity threats and how to recognize phishing attempts. Simulated phishing exercises can be particularly effective.</p></li><li dir="ltr"><p dir="ltr">Implement Strict Information Handling Policies: Limit the amount of personal and sensitive information that can be shared or accessed based on job roles.</p></li><li dir="ltr"><p dir="ltr">Adopt Advanced Security Technologies: Utilize email filtering, intrusion detection systems, and AI-based security solutions that can identify and block phishing attempts.</p></li><li dir="ltr"><p dir="ltr">Establish Clear Reporting Procedures:&nbsp;Employees should know how to report suspected phishing attempts promptly, allowing IT security teams to respond quickly.</p></li></ul><h1 dir="ltr">The Role of Continuous Education</h1><p dir="ltr">As generative AI continues to evolve, so too will the tactics used by cybercriminals. For this reason, staying informed about the latest developments in AI and cybersecurity is essential. For individuals, this means staying abreast of the latest security advice and best practices. For organizations, it involves regular training sessions and updates on cybersecurity trends and threats. <sup>[5]</sup></p><p dir="ltr">Awareness campaigns, workshops, and online courses can play significant roles in educating the public and employees about the dangers of AI-enhanced phishing schemes and how to protect against them. Knowledge is power, and in the context of cybersecurity, it's also the best defense.</p><h1 dir="ltr">Looking Ahead: The Future of AI in Cybersecurity</h1><p dir="ltr">The dual-use nature of generative AI in cybersecurity presents a continuous game of cat and mouse between cybercriminals and defenders. As AI technologies become more sophisticated, so too do the tools and strategies to combat malicious use. Future developments in AI could offer more robust detection mechanisms, identifying and neutralizing phishing attempts before they reach their intended targets. Similarly, advancements in natural language processing could enable more effective filtering of phishing content, while machine learning models might predict and block new phishing tactics.<sup>[6]</sup></p><p dir="ltr">However, as technology advances, the complexity and novelty of phishing attacks will likely increase as well. A proactive and adaptive approach to cybersecurity, emphasizing technological solutions and human vigilance and education, will be necessary to combat these advances.</p><h1 dir="ltr">Conclusion</h1><p dir="ltr">The rise of generative AI has brought with it an era of enhanced phishing schemes, presenting new challenges in the realm of cybersecurity. While these sophisticated attacks pose significant risks, the combination of advanced security measures, continuous education, and heightened awareness provides a formidable defense. As we navigate this evolving landscape, staying informed, cautious, and prepared is essential for protecting personal and organizational integrity in the digital age. Through collective effort and the strategic application of technology, we can mitigate the threats posed by AI-enhanced phishing and secure our digital future.</p><h3 dir="ltr">References</h3><ol><li dir="ltr"><p dir="ltr"><a href="https://www.techtarget.com/searchenterpriseai/definition/generative-AI" target="_blank">https://www.techtarget.com/searchenterpriseai/definition/generative-AI</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://www.phishing.org/what-is-phishing" target="_blank">https://www.phishing.org/what-is-phishing</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://withpersona.com/blog/ai-phishing-attacks" target="_blank">https://withpersona.com/blog/ai-phishing-attacks</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://consumer.ftc.gov/articles/how-recognize-and-avoid-phishing-scams" target="_blank">https://consumer.ftc.gov/articles/how-recognize-and-avoid-phishing-scams</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://www.cisa.gov/secure-our-world/teach-employees-avoid-phishing" target="_blank">https://www.cisa.gov/secure-our-world/teach-employees-avoid-phishing</a></p></li><li dir="ltr"><p dir="ltr">https://sennovate.com/the-role-of-artificial-intelligence-in-detecting-phishing-attacks/</p></li></ol>

Generative AI has opened up an entirely new world of cybersecurity threats. 


The Rise of Generative AI and Enhanced Phishing Schemes

<h2 id="isPasted">Introduction&nbsp;</h2><p id="isPasted">The field of organoid intelligence is recognized as groundbreaking. In this field, scientists utilize human brain cells to enhance computer functionality. They cultivate tissues in laboratories that mimic real organs, particularly the brain. <a id="_anchor_2" href="#_msocom_2" language="JavaScript" name="_msoanchor_2" target="_blank"></a>These brain organoids can perform brain-like functions and are being developed by <a href="https://www.frontiersin.org/journals/science/articles/10.3389/fsci.2023.1017235/full" target="_blank">Dr. Thomas Hartung and his team at the Johns Hopkins Bloomberg School of Public Health</a>.</p><p>For nearly two decades, scientists have used organoids to conduct experiments without harming humans or animals. Hartung, who has been cultivating brain organoids from human skin samples since 2012, aims to integrate these organoids into computing. This approach promises more energy-efficient computing than current supercomputers and could revolutionize drug testing, improve our understanding of the human brain, and push the boundaries of computing technology.</p><p>The conducted research highlights the potential of biocomputing to surpass the limitations of traditional computing and AI. Despite AI's advancements, it still falls short of replicating the human brain's capabilities, such as energy efficiency, learning, and complex decision-making. The human brain's capacity for information storage and energy efficiency remains unparalleled by modern computers. Hartung's work with brain organoids, inspired by Nobel Prize-winning stem cell research, aims to replicate cognitive functions in the lab. This research could open new avenues for understanding the human brain by allowing ethical experimentation. The team envisions scaling up the size of brain organoids and developing communication tools for input and output, enabling more complex tasks.</p><p>A significant innovation by Hartung's team is a brain-computer interface device, resembling an EEG cap for organoids, facilitating signal transmission to and from the brain organoids. This development lays the groundwork for a synergistic relationship between AI and organoid intelligence, exploring each other's capabilities and potentially leading to technological advancements previously unimagined.</p><h2>Understanding Organoids</h2><p>Brain organoids are tiny, lab-created clusters of brain cells that simulate the brain's structure and function. Starting from skin cells, scientists reprogram these cells into stem cells and eventually into brain cells, creating organoids around 500 micrometers in size with fewer than 100,000 cells. Although these organoids are much smaller than a human brain, they offer a model for studying brain functions and developing computing systems.</p><p id="isPasted">Organoid-based bio computers have several advantages over traditional silicon computers. While computers excel at number crunching, the human brain is far superior in handling complex computations efficiently, such as learning to differentiate objects with minimal data. For example, the AI system AlphaGo required training on <a href="https://www.frontiersin.org/journals/science/article-hubs/organoid-intelligence-a-new-biocomputing-frontier/explainer" target="_blank">160,000 game scenarios to master Go</a>, something far beyond human capability in terms of time and energy consumption. Moreover, the brain's energy efficiency and data storage capabilities are unmatched by current technology, with an estimated storage capacity of 2,500 terabytes.</p><p>Despite not being miniature brains, brain organoids share important functional and structural characteristics with the human brain, suggesting their potential to achieve similar computational abilities. A study showing neurons in a dish learning to play Pong illustrates this potential. However, for organoid intelligence to be fully realized, brain organoids need to be larger and more complex, with millions of cells and a diversity of cell types that are important for learning and memory.</p><p>Despite not being miniature brains, brain organoids share important functional and structural characteristics with the human brain, suggesting their potential to achieve similar computational abilities. A study showing neurons in a dish learning to play Pong illustrates this potential. However, for organoid intelligence to be fully realized, brain organoids need to be larger and more complex, with millions of cells and a diversity of cell types that are important for learning and memory.</p><p>Improvements in brain organoid technology are essential for advancing organoid intelligence that includes increasing the size and cell diversity of organoids and improving their ability to express genes that improve learning and memory. Additionally, developing a way to nourish larger organoids, potentially through artificial blood vessels created by microfluidic technology, is considered to be challenging task. This technology would not only support the organoids' health but also enable communication through chemical signals, enabling more sophisticated biological computing systems.</p><h2>Mechanisms of Organoid Intelligence</h2><p id="isPasted">The integration of stem cell biology, bioengineering, and AI is forming novel approaches in the field of organoid research. This synergy is maximizing development of the stem cell, precision engineering techniques, and the analytical advantage of AI to transform the comprehension of how organisms develop, how diseases arise, and how we can devise new treatment methods. Machine learning algorithms have confirmed how closely organoids resemble actual human organs by analysing complex data sets, such as <a href="https://www.sciencedirect.com/science/article/pii/S2590093523000711" target="_blank">comparing organoid RNA sequencing data to that of real organs</a>, which helps in accurately identifying cell types and understanding gene regulatory networks within organoids.</p><p>One study on brain organoids utilized <a href="https://www.sciencedirect.com/science/article/pii/S1934590919303376" target="_blank">UMAP (Uniform manifold approximation and projection) to analyse RNA sequencing data</a> at different developmental stages, uncovering the composition of cell subtypes over time. This analysis, combined with data from neonatal electroencephalogram (EEGs) of preterm infants, highlighted the developmental parallels between cortical organoids and fetal human brains, especially in terms of electrophysiological network changes. Furthermore, the creation of the brain and organoid manifold alignment (BOMA) algorithm has facilitated the comparison of <a href="https://www.sciencedirect.com/science/article/pii/S2667237523000206?via%3Dihub" target="_blank">gene expression between real brains and organoids</a>, uncovering patterns of gene expression over time and across species, thereby validating the use of brain organoids for neural system research.</p><p>Moreover, machine learning extends beyond analysing genetic data. For example, <a href="https://cmbl.biomedcentral.com/articles/10.1186/s11658-022-00347-3" target="_blank">robust support-vector machines (RSVM) algorithms</a> have been used to categorize salivary gland organoids based on changes detected in Raman spectral data during organoid formation. Similarly, a study on cardiac organoids demonstrated that a random forest approach could effectively sort heart cells from non-heart cells in single-cell RNA sequencing data from heart organoids derived from <a href="https://www.nature.com/articles/s42003-022-03346-4" target="_blank">human induced pluripotent stem cells (hiPSCs)</a>. This model could also distinguish between heart tissues created from 3D organoids, traditional 2D cultures, and organoids from genetically mutated hiPSC lines, showcasing the nuanced differences in cell types between ventricular and atrial cardiomyocytes.</p><p>It should be emphasized that deep learning has significantly contributed development in the field of organoid research by improving the analysis of images and videos captured through <a href="https://www.sciencedirect.com/science/article/pii/S2590093523000711" target="_blank">microscopy</a>. These visual data sources contain information about organoid development, including changes in morphology, size, and function, to understand their response to environmental changes. The amount of data generated, however, has created a bottleneck in interpreting these experimental results, necessitating the adoption of deep learning techniques for <a href="https://www.sciencedirect.com/science/article/abs/pii/S0010482521002845?via%3Dihub" target="_blank">efficient organoid image analysis and tracking</a>. Significant efforts have been made to develop tools that are using deep learning to automate the quantification of changes in organoids, addressing the challenges of manual screening which is both time-consuming and error-prone.</p><p>A specific dataset designed for high-throughput organoid image analysis employs Convolutional Neural Networks (CNN) architecture, Residual Network (ResNet), a type of CNN architecture designed to handle very deep networks, both enabling the tracking of organoid growth and morphological changes with high speed and accuracy. <a href="https://ieeexplore.ieee.org/document/7780459" target="_blank">This breakthrough</a> demonstrates deep learning's potential to improve organoid research. Open-source tools like <a href="https://journals.biologists.com/dev/article/148/18/dev199611/272032/MOrgAna-accessible-quantitative-analysis-of" target="_blank">MOrgAna</a> combine deep learning with traditional machine learning to analyse brain organoid images, using deep architectures to differentiate organoid pixels from noise, thereby improving upon existing quantification methods.</p><h2>Applications of Organoid Intelligence</h2><p id="isPasted">Integrating organoids with AI aims to refine models of human organ functionality and disease, offering a new approach for developing and advancing the drug discovery, diagnostics, and therapeutic innovations. The role of AI is in improving organoid technology across <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10511344/" target="_blank">five aspects</a>: swift identification of construction methods, cost-effective analysis of complex image data, efficient multi-omics data interpretation, accurate preclinical assessments, and practical applications. These areas provide a comprehensive view of AI's potential to accelerate and refine the development of AI-improved and supported organoids.</p><p>Rapid identification of construction methods is important for selecting the best procedures for organoid creation, while cost-effective analysis of complex image data allows for deeper insights into organoid structure and function from various perspectives. Streamlined interpretation of multi-omics data enables a thorough understanding of organoid biology, encompassing gene expression, proteomics, and metabolomics. Accurate preclinical assessments and applications are essential for predicting AI's real-world clinical utility.</p><p>Practical strategies for applying these concepts will maximize AI's impact on developing organoid technology. More specifically, a machine learning framework has been developed to forecast the efficacy of <a href="https://www.nature.com/articles/s41467-020-19313-8" target="_blank">anti-cancer drugs using organoid models</a>, highlighting the predictive biomarkers in cancer treatment optimization. However, current machine learning techniques often face challenges in extracting reliable biomarkers from preclinical models. To address this, a novel network-based strategy utilizing <a href="https://www.nature.com/articles/s41565-023-01323-4" target="_blank">pharmacogenomic data from 3D organoid cultures</a> has been introduced, demonstrating precise drug response predictions in colorectal and bladder cancers. This approach underscores the potential of AI in refining cancer therapies through better biomarker identification.</p><p>A notable <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10511344/#bib176" target="_blank">study introduces IRENE</a>, a transformer-based model that improves multimodal inputs for clinical diagnostics, showing superior performance in diagnosing and predicting clinical outcomes compared to traditional models. This study indicates the growing importance of machine learning in disease modelling and treatment development, poised to significantly impact clinical diagnostics and personalized medicine as our technological capabilities and understanding of organoids and disease processes continue to evolve.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 769px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1710925021949-organs.jpg" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner">AI interfacing enables efficient organoid construction, multiscale image analysis, and precise preclinical evaluation. IMage credit: Long Bai et al.</span></span></span></p><h2>Challenges and Future Directions</h2><p>The integration of AI with organoid technology presents several ethical considerations and societal implications. One of the primary ethical concerns revolves around the use of human stem cells for creating organoids. This includes questions about consent, especially when cells are derived from embryos or patients who may not be fully aware of the potential future uses of their cells. Societal implications include issues of accessibility and equity. As organoid technologies advance, there's a risk that treatments developed using these cutting-edge methods could be expensive and only available to a privileged few, exacerbating existing healthcare disparities. Moreover, the potential for creating patient-specific organoids for precision medicine raises privacy concerns regarding genetic data and the need for robust data protection measures.</p><p>From a technical perspective, several challenges and limitations need to be addressed. One of the main challenges is the complexity of human organs, which are difficult to replicate in full of current organoid technology. While organoids can mimic certain aspects of organ structure and function, they often lack the complete cellular diversity and organ-level functions, such as blood supply and immune system interactions.</p><p>Addressing these challenges requires a multidisciplinary approach that combines recent findings in stem cell biology, tissue engineering, computational sciences, and ethics. Future research should focus on improving the fidelity and complexity of organoids to more closely mimic human organs, including the development of vascularized and innervated organoids that can better replicate organ functions.</p><h2>Conclusion&nbsp;</h2><p>AI-enabled organoids have the potential to unlock deeper understandings of the biological systems represented by various organoid models. An important part of this advancement is the effective creation and maintenance of organoids, areas where AI, especially through machine learning algorithms, shows considerable promise. AI's ability to process and analyse amounts of omics data provides valuable feedback on organoid functionality and construction techniques. This leads to more effective and higher-quality organoid production, facilitating a smoother progression from experimental research to practical clinical use.</p>

Engineers at UC San Diego demonstrated for the first time that human brain organoids implanted in mice have established functional connectivity to the animals’ cortex.  Picture by: David Baillot | UC San Diego Jacobs School of Engineering | Flickr

Organoid intelligence is a pioneering field where scientists use human brain cells to improve computer functions.

Diving into Organoid Intelligence

<p id="isPasted">We are thrilled to invite engineers and innovators to the <strong id="isPasted">Pervasive AI Developer Contest</strong>, powered by AMD and Hackster. This is an exceptional opportunity for AI enthusiasts to showcase their skills in developing innovative AI projects that could revolutionize various sectors including data centers, client workstations, industrial gaming, and automotive.&nbsp;</p><p>Enter to win incredible cash prizes and gain recognition in the tech world!</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><hr><h2><strong>What is the Pervasive AI Developer Contest?</strong></h2><p>The Pervasive AI Developer Contest is an initiative by AMD to encourage and empower developers to create AI-driven innovations using AMD's cutting-edge technology. With a focus on various applications in the tech industry, this challenge is designed to explore and expand the horizons of AI capabilities.</p><h3><strong>Exciting Categories &amp; Generous Prizes</strong></h3><p>AMD has lined up three stimulating categories for this challenge: <strong>Generative AI, Robotics AI, and PC AI.</strong> Each category is an avenue to unleash your creativity and technical prowess. Amazing cash prizes await the winners, with a top prize of $10,000 USD in each category!</p><h3><strong>Special Awards</strong></h3><p>In addition to the main categories, AMD is offering a special award of $2,500 USD for the best project submitted by a university student. The challenge also has a “Women in Technology” award with the same cash prize, encouraging teams with a majority of female members to participate and showcase their talents.</p><h2><strong>How to Participate?</strong></h2><p>Enroll in the AMD Developer Program, apply for the challenge, and take a quiz to qualify for free hardware. This is your chance to work with AMD’s latest technology and potentially lead the next wave of AI innovations.</p><p><a href="https://wevlv.co/hackster-pervasive-ai" target="_blank" rel="noopener noreferrer" class="button-main-action">APPLY FOR HARDWARE</a></p><h2><strong>Key Dates to Remember</strong></h2><p>Don’t miss out! Make sure to mark these important dates on your calendar:</p><ul><li><strong>Contest begins:</strong> November 30, 2023 at 1:00 AM PST</li><li><strong>Applications for hardware close:</strong> February 29, 2024 at 11:59 PM PST</li><li><strong>Hardware recipients announced:</strong> March 15, 2024 at 11:00 AM PST</li><li><strong>Submissions close:</strong> June 30, 2024 at 11:59 PM PDT</li><li><strong>Winners announced by:</strong> July 15, 2024</li></ul><h2 id="isPasted">Prizes Galore</h2><p>This challenge offers $160,000 in total prizes! An expert panel of judges will evaluate the top 14 projects, focusing on their innovation, implementation, and overall impact on AI.</p><h3 id="isPasted">Hardware Availability</h3><p>There are up to <strong>700</strong> total pieces of free development hardware and cloud instances available from AMD in this year's contest. Each category has a specific amount of hardware allocated:</p><ul><li>Generative AI: <strong>up to 300</strong> MI210 Cloud Instances or Radeon PRO W7900 GPUs</li><li>Robotics AI : <strong>up to 200pcs</strong> KR260 Robotics Starter Kit</li><li>PC AI: <strong>up to 200pcs</strong> Ryzen AI Powered PC</li></ul><p><em>* If you already have access to any of the above hardware, you are welcome to use it.</em></p><h2>Engage and Collaborate</h2><p>By joining the AMD Pervasive AI Developer Contest, you not only get the opportunity to compete for multiple cash prizes, but you will be able to engage with AMD technical experts, connect with fellow developers, showcase your ability and innovations to a broader audience, and gain practical experience by working on real-world challenges.</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/3EMrWVr3oI4?&amp;t=38s&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p>

Fueling Groundbreaking Innovations - Win up to $160,000 in prizes!

AMD Pervasive AI Developer Contest

In this episode, we delve into the ever-evolving world of hybrid AI and generative AI models.

Podcast: The Prius of Artificial Intelligence: Hybrid AI Models

<p>This article originally appeared in <a href="https://news.mit.edu/2023/new-technique-helps-robots-pack-objects-tight-space-1017" target="_blank" rel="noopener noreferrer">MIT News</a> and has been republished with permission.</p><hr><p id="isPasted">Anyone who has ever tried to pack a family-sized amount of luggage into a sedan-sized trunk knows this is a hard problem. Robots struggle with dense packing tasks, too.</p><p>For the robot, solving the packing problem involves satisfying many constraints, such as stacking luggage so suitcases don’t topple out of the trunk, heavy objects aren’t placed on top of lighter ones, and collisions between the robotic arm and the car’s bumper are avoided.</p><p>Some traditional methods tackle this problem sequentially, guessing a partial solution that meets one constraint at a time and then checking to see if any other constraints were violated. With a long sequence of actions to take, and a pile of luggage to pack, this process can be impractically time consuming. &nbsp;&nbsp;</p><p>MIT researchers used a form of generative AI, called a diffusion model, to solve this problem more efficiently. Their method uses a collection of machine-learning models, each of which is trained to represent one specific type of constraint. These models are combined to generate global solutions to the packing problem, taking into account all constraints at once.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/mhtYU7IevtE?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><p id="isPasted">Their method was able to generate effective solutions faster than other techniques, and it produced a greater number of successful solutions in the same amount of time. Importantly, their technique was also able to solve problems with novel combinations of constraints and larger numbers of objects, that the models did not see during training.</p><p>Due to this generalizability, their technique can be used to teach robots how to understand and meet the overall constraints of packing problems, such as the importance of avoiding collisions or a desire for one object to be next to another object. Robots trained in this way could be applied to a wide array of complex tasks in diverse environments, from order fulfillment in a warehouse to organizing a bookshelf in someone’s home.</p><p>“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments. With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” says Zhutian Yang, an electrical engineering and computer science graduate student and lead author of a <a href="https://arxiv.org/pdf/2309.00966.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper on this new machine-learning technique</a>.</p><p>Her co-authors include MIT graduate students Jiayuan Mao and Yilun Du; Jiajun Wu, an assistant professor of computer science at Stanford University; Joshua B. Tenenbaum, a professor in MIT’s Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Tomás Lozano-Pérez, an MIT professor of computer science and engineering and a member of CSAIL; and senior author Leslie Kaelbling, the Panasonic Professor of Computer Science and Engineering at MIT and a member of CSAIL. The research will be presented at the Conference on Robot Learning.</p><h2><strong>Constraint complications</strong></h2><p>Continuous constraint satisfaction problems are particularly challenging for robots. These problems appear in multistep robot manipulation tasks, like packing items into a box or setting a dinner table. They often involve achieving a number of constraints, including geometric constraints, such as avoiding collisions between the robot arm and the environment; physical constraints, such as stacking objects so they are stable; and qualitative constraints, such as placing a spoon to the right of a knife.</p><p>There may be many constraints, and they vary across problems and environments depending on the geometry of objects and human-specified requirements.</p><p>To solve these problems efficiently, the MIT researchers developed a machine-learning technique called <a href="https://diffusion-ccsp.github.io/" target="_blank">Diffusion-CCSP</a>. Diffusion models learn to generate new data samples that resemble samples in a training dataset by iteratively refining their output.</p><p>To do this, diffusion models learn a procedure for making small improvements to a potential solution. Then, to solve a problem, they start with a random, very bad solution and then gradually improve it.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697557883397-ezgif-5-a827fde90a.gif" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Using generative AI models, MIT researchers created a technique that could enable robots to efficiently solve continuous constraint satisfaction problems, such as packing objects into a box while avoiding collisions, as shown in this simulation. Image: Courtesy of the researchers</span></span></span></p><p id="isPasted">For example, imagine randomly placing plates and utensils on a simulated table, allowing them to physically overlap. The collision-free constraints between objects will result in them nudging each other away, while qualitative constraints will drag the plate to the center, align the salad fork and dinner fork, etc.</p><p>Diffusion models are well-suited for this kind of continuous constraint-satisfaction problem because the influences from multiple models on the pose of one object can be composed to encourage the satisfaction of all constraints, Yang explains. By starting from a random initial guess each time, the models can obtain a diverse set of good solutions.</p><h2><strong>Working together</strong></h2><p>For Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints. In packing for instance, one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located.</p><p>Diffusion-CCSP learns a family of diffusion models, with one for each type of constraint. The models are trained together, so they share some knowledge, like the geometry of the objects to be packed.</p><p>The models then work together to find solutions, in this case locations for the objects to be placed, that jointly satisfy the constraints.</p><p>“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” she says.</p><p>Training individual models for each constraint type and then combining them to make predictions greatly reduces the amount of training data required, compared to other approaches.</p><p>However, training these models still requires a large amount of data that demonstrate solved problems. Humans would need to solve each problem with traditional slow methods, making the cost to generate such data prohibitive, Yang says.</p><p>Instead, the researchers reversed the process by coming up with solutions first. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.</p><p>“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.</p><p>Trained using these data, the diffusion models work together to determine locations objects should be placed by the robotic gripper that achieve the packing task while meeting all of the constraints.</p><p>They conducted feasibility studies, and then demonstrated Diffusion-CCSP with a real robot solving a number of difficult problems, including fitting 2D triangles into a box, packing 2D shapes with spatial relationship constraints, stacking 3D objects with stability constraints, and packing 3D objects with a robotic arm.</p><p>Their method outperformed other techniques in many experiments, generating a greater number of effective solutions that were both stable and collision-free.</p><p>In the future, Yang and her collaborators want to test Diffusion-CCSP in more complicated situations, such as with robots that can move around a room. They also want to enable Diffusion-CCSP to tackle problems in different domains without the need to be retrained on new data.</p><p>“Diffusion-CCSP is a machine-learning solution that builds on existing powerful generative models,” says Danfei Xu, an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology and a Research Scientist at NVIDIA AI, who was not involved with this work. “It can quickly generate solutions that simultaneously satisfy multiple constraints by composing known individual constraint models. Although it’s still in the early phases of development, the ongoing advancements in this approach hold the promise of enabling more efficient, safe, and reliable autonomous systems in various applications.”</p><p>This research was funded, in part, by the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research, the MIT-IBM Watson AI Lab, the MIT Quest for Intelligence, the Center for Brains, Minds, and Machines, Boston Dynamics Artificial Intelligence Institute, the Stanford Institute for Human-Centered Artificial Intelligence, Analog Devices, JPMorgan Chase and Co., and Salesforce.</p>

MIT researchers are using generative AI models to help robots more efficiently solve complex object manipulation problems, such as packing a box with different objects. Image: Courtesy of the researchers

Researchers coaxed a family of generative AI models to work together to solve multistep robot manipulation problems.

New technique helps robots pack objects into a tight space

<p id="isPasted">This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/50b8e0ea-1cdb-4b34-87e6-4867feefade3?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p>The full article will continue below.</p><hr><p><em>This is the second article in a short series. Read the<a href="https://www.wevolver.com/article/balancing-the-cloud-and-the-edge-a-close-look-at-enabling-hybrid-ai" target="_blank" rel="noopener noreferrer">&nbsp;first piec</a>e here.</em></p><h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">Generative AI is a rapidly advancing field of artificial intelligence (AI) focused on creating novel data assets like images, text, or music. Such AI models are trained using extensive datasets, enabling them to create new data resembling what they were trained on. The growth in generative AI's development owes itself to several key factors: availability of large datasets, improved computing hardware capabilities, and the evolution of novel machine learning (ML) algorithms and architectures.</p><p dir="ltr">Large language models–based chatbots like ChatGPT, which can generate coherent and contextually relevant text and even images, have gained the interest and creativity of researchers, businesses, and the public. Generative models have demonstrated remarkable capabilities across a range of applications, from natural language understanding and text generation to creative content creation and even virtual assistants. However, as these models grow in complexity and sophistication, they bring to the forefront an increasingly pressing issue in the form of scalability challenges that apply to both training and inference.&nbsp;</p><p dir="ltr">These challenges encompass the need for significant computational resources, the growing size of models, and increasing costs. Latency and real-time responses primarily relate to the inference phase, while ethical considerations, interoperability, privacy, energy consumption, and cost management are critical factors throughout the entire lifecycle of AI systems, including training and inference.&nbsp;</p><p dir="ltr">One promising solution arises in response to these challenges - on-device AI. Deploying AI models directly on user devices (like smartphones and tablets) offers multiple advantages. It reduces reliance on cloud resources, cutting computational and energy costs. It improves privacy by processing data locally, reducing the need to send sensitive information to the cloud. It also lowers latency, supports offline functionality, and aligns with sustainability goals by reducing energy consumption.</p><h2 dir="ltr">What is Hybrid AI?</h2><p dir="ltr">To properly define Hybrid AI, we need to introduce the terms on-device AI and hybrid AI. On-Device AI, also known as edge AI or local AI, refers to the deployment of AI algorithms and models directly on user devices (smartphones, tablets, IoT devices, autonomous vehicles, etc.), without heavy reliance on cloud-based infrastructure. It enables real-time processing and decision-making on the device itself, reducing the need for constant data transmission to remote servers. Hybrid AI represents a synergy between local devices and cloud infrastructure, strategically dividing AI computations to optimize both user experiences and resource efficiency.</p><p dir="ltr">In certain situations, AI processing predominantly occurs on the device, with the option to delegate tasks to the cloud as needed. In cloud-centric setups, devices will opportunistically offload AI workloads from the cloud, where possible and subject to their capabilities. Employing a hybrid AI architecture, or relying solely on on-device AI, yields numerous advantages including cost savings, improved energy efficiency, enhanced performance, heightened privacy and security, and personalized experiences on a global scale.</p><p dir="ltr">Within a hybrid AI framework, on-device AI processing delivers consistent performance that can rival or surpass cloud-based alternatives, particularly in scenarios characterized by congested cloud servers and network connectivity issues. Instances of high demand for generative AI queries in the cloud can result in lengthy queues and elevated latency, and in extreme cases, lead to service disruptions. These challenges can be mitigated by redistributing computational workloads to edge devices.</p><p dir="ltr">Furthermore, the presence of on-device processing capabilities in a hybrid AI architecture enables generative AI applications to function seamlessly in locations with limited or no connectivity. In a device-centric hybrid AI setup, the device serves as the central point of operation, with cloud resources primarily utilized for tasks that exceed the device's processing capacity. Given that many generative AI models can efficiently run on the device, edge devices can handle the bulk of the processing load. As the development of on-device AI continues, it holds significant potential to revolutionize AI service deployment and user experiences while maintaining a balance between performance and ethical considerations.</p><h2 dir="ltr">Unique Benefits of On-Device AI</h2><p dir="ltr"><a href="https://www.qualcomm.com/news/onq/2023/08/5-benefits-of-on-device-generative-ai" target="_blank">On-device AI offers several unique benefits</a>, including:</p><h3>Cost-efficiency&nbsp;</h3><p>By processing data locally, it reduces the need for extensive cloud infrastructure, which can be expensive to maintain and scale. On-device processing eliminates the necessity of sending data to the cloud, a process that can be both expensive and time-consuming. &nbsp;For instance, on-device AI for image generation, can be employed to generate high-quality images in real-time without relying on cloud-based servers. A mobile photography application equipped with generative AI can apply artistic filters, improve image quality, or create artistic effects directly on the user's device.&nbsp;</p><p>This ensures that users can quickly and seamlessly edit and generate images without uploading them to a remote server, preserving both privacy and reducing latency. Additionally, it allows for offline image generation, enabling users to create and edit images even when they're in locations with poor or no internet connectivity. This level of on-device image generation can significantly improve the user experience and provide powerful tools for content creation without compromising data privacy or speed.&nbsp;</p><h3>Energy Savings&nbsp;</h3><p>On-device AI mitigates the issue of energy consumption by performing computations locally. This is particularly crucial for devices with constrained battery life, like smartphones and wearables. Comparing the power efficiency of running generative AI models on edge devices versus to the cloud, edge devices offer superior performance per watt and reduced energy consumption, potentially resulting in environmental and cost benefits.&nbsp;</p><h3>Performance and Low Latency</h3><p>By processing tasks locally, it minimizes communication and computing delays associated with cloud-based AI. This is particularly important in real-time applications such as autonomous vehicles, medical devices, and interactive content, where even milliseconds of delay can have a significant impact on functionality and user experience.</p><h3>Enhanced Privacy and Security</h3><p>This approach reduces the risk of data breaches and unauthorized access during data transfer and cloud storage. Users can have greater confidence that their personal information remains on their devices, reinforcing privacy and security standards, especially for applications handling sensitive data, like financial transactions and medical information.</p><h3>Personalization and User Experience</h3><p>These models can adapt to the user's context and needs, improving user experiences by recommending relevant products or services based on their preferences and current circumstances. For example, a shopping application could use on-device AI to recommend products similar to past purchases or consider the user's location and weather conditions. Real-time personalization boosts relevance, engagement, and customer satisfaction.</p><h2 dir="ltr">Case Studies: On-Device AI in Action</h2><p dir="ltr">The following case studies illustrate the transformative potential of on-device AI across various domains, showcasing its benefits.</p><h3 dir="ltr" id="isPasted">Automotive</h3><p dir="ltr">In the automotive industry, AI-driven cockpits are revolutionizing the driving experience. These systems, like those in smartphones and PCs, are starting to employ in-vehicle digital assistants powered by generative AI to create highly personalized and hands-free interactions for drivers and passengers.&nbsp;</p><p dir="ltr">These digital assistants leverage personal and vehicle sensor data, including cameras and microphones, to improve the driving journey. Third-party services can also integrate seamlessly, improving navigation, offering proactive assistance, and enabling in-car commerce, such as ordering food. Furthermore, the cockpit experience is improved through individual recognition, customization of media content, and the rise of in-vehicle augmented reality. Vehicle maintenance becomes proactive as AI digital assistants analyze data to predict maintenance needs, to seamlessly communicate with users using natural language, and to provide precise repair guidance. This proactive approach not only enhances reliability but also effectively reduces costs.&nbsp;</p><p dir="ltr">Qualcomm, which leads the field in cockpit and in-vehicle infotainment solutions with numerous global automakers choosing Snapdragon® Cockpit Platforms. These platforms power digital cockpits, with many already in production or design stages. Partners include Honda, Mercedes, Renault, Volvo, and others. The Snapdragon Cockpit Platform's latest iteration focuses on superior in-vehicle user experiences, safety, comfort, and reliability, setting new standards for connected car digital cockpit solutions. Qualcomm's Snapdragon Ride™ Platform complements this, offering scalable automated driving SoC platforms for ADAS, featuring vision perception, parking, and path planning.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUyNzU0OTg0LXF1YWxjb21tIGNhYmluLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">&nbsp;Cockpit platforms are designed to transform in-vehicle experiences, supporting higher levels of compute and computing intelligence needed for advanced capabilities featured in next generation vehicles. Image credit: Qualcomm.</span></span></span></p><h3 dir="ltr" id="isPasted">Extended Reality</h3><p dir="ltr">For Extended Reality (XR), generative AI has the potential to democratize 3D content creation. In the near future, AI rendering tools will enable content creators to generate 3D objects and entire virtual worlds using various inputs like text, speech, images, or video. Furthermore, text-to-text models will facilitate lifelike conversations for avatars, transforming the way we interact with XR content. Despite the exciting possibilities, widespread adoption timelines remain uncertain, though significant progress is expected soon. Immersive XR experiences will benefit from text-to-image models for realistic textures, likely available on smartphones and XR devices within a year.&nbsp;</p><p dir="ltr">These capabilities will involve distributed processing, with headsets handling perception and rendering, while generative AI models run on paired smartphones or in the cloud. In a few years, text-to-3D and image-to-3D models will likely produce high-quality 3D objects, evolving to generate entire scenes and rooms. Eventually, XR may enable users to immerse themselves in virtual worlds constructed from their imagination. Virtual avatars will undergo a similar evolution, utilizing text-to-text models for natural conversations and text-to-image models for outfits. Over time, avatars may become increasingly photorealistic, fully animated, and mass-producible, driven by image-to-3D and encoder/decoder models. This progression promises to revolutionize how we create and experience immersive XR content, with continual advancements anticipated in the years ahead.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUzMTYyMjIyLUNhbm5lcy1YUi1WaXJ0dWFsLUJyaW5ncy1JbW1lcnNpdmUtQXJ0LXRvLUxpZmUtV2l0aC1WaXJ0dWFsLVJlYWxpdHkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Image credit: Cannes</span></span></span></p><h2 dir="ltr">Challenges and Solutions</h2><p dir="ltr">The technical challenges specific to hybrid AI are being actively addressed through ongoing research and innovation:</p><h3 dir="ltr">Model size</h3><p>Model size is a notable challenge, particularly on resource-constrained devices. As large models are challenging to store and run efficiently, a potential solution involves employing compression and quantization techniques to reduce model size while maintaining accuracy. <a href="https://stackoverflow.blog/2023/08/23/fitting-ai-models-in-your-pocket-with-quantization/" target="_blank">Qualcomm</a> is actively researching ways to enhance <a href="https://www.qualcomm.com/news/onq/2019/03/heres-why-quantization-matters-ai" target="_blank">quantization</a> techniques, specifically aiming to quantize 32-bit floating point parameters to 8-bit or 4-bit integers in neural networks while maintaining accuracy. The research encompasses various model quantization methods, such as Bayesian learning, adaptive rounding, and transformer quantization, as part of <a href="https://www.qualcomm.com/research/artificial-intelligence/ai-research" target="_blank">Qualcomm AI Research</a>'s ongoing efforts.&nbsp;</p><p>For instance, the Qualcomm AI Research team employed the <a href="https://www.forbes.com/sites/davealtavilla/2023/05/15/powering-ai-on-mobile-devices-requires-new-math-and-qualcomm-is-pioneering-it/?sh=54cb40c91a16" target="_blank">AIMET</a> tool (AI Model Efficiency Toolkit) to quantize the 32-bit floating point Stable Diffusion AI model into an 8-bit integer format while preserving almost the same accuracy level. By optimizing and scaling the model for INT8 precision operations instead of FP32, Qualcomm successfully enabled Stable Diffusion to run efficiently on handheld devices without internet connectivity, significantly reducing the model's size while maintaining accuracy and enhancing performance on low-power Snapdragon AI accelerators.</p><h3>Computational power</h3><p>Large AI models may place heavy computational demands, particularly on devices with restricted processing capabilities. A potential solution involves leveraging specialized hardware like AI accelerators to enhance AI model performance on such devices. In 2022, Snapdragon 8 Gen 2 provided groundbreaking AI, integrated across the entire system, powered by their fastest, most advanced Qualcomm AI Engine to date. Users can experience faster natural language processing with multi-language translation or have fun with AI cinematic video capture.&nbsp;</p><p>The latest Hexagon Processor introduced a dedicated power delivery system, which adapts power according to the workload. Special hardware improves group convolution, activation function acceleration, and the performance of the Hexagon Tensor Accelerator. Support for micro-tile inferencing and INT4 hardware acceleration provide even higher performance while reducing power and memory traffic. The transformer acceleration dramatically speeds up inference for multi-head attention that is used throughout generative AI, resulting in AI performance up to 4.35X on certain use cases with MobileBERT.&nbsp;</p><p dir="ltr">These ongoing research efforts are addressing the technical challenges of hybrid AI, with expectations to overcome them successfully as the technology advances and becomes more widely adopted and utilized.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">On-device AI plays a critical role in addressing the future scalability challenges of generative AI models. It has the potential to revolutionize numerous industries and significantly enhance user experiences. By enabling local processing and real-time personalization, on-device AI reduces the dependence on cloud resources, improves privacy and security as well as the efficiency of AI applications. This approach not only supports the responsible development of AI technologies but also opens new possibilities for scalable and personalized AI solutions across diverse sectors, from healthcare to entertainment, promising a brighter future for both businesses and users alike.</p><p>To stay up to date on the latest from Qualcomm, <a href="https://assets.qualcomm.com/mobile-computing-newsletter-sign-up.html" target="_blank">join the “What’s Next in AI and Computing” newsletter</a> and read their hybrid AI whitepapers (<a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 1</a>, <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-2-Qualcomm-is-uniquely-positioned-to-scale-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 2</a>)</p>

How on-device and cloud AI will foster the advancement and scalability of next-generation generative AI experiences

The Scalability Challenge: Focusing on Hybrid AI in the Future of Generative AI Models

In this episode, we discuss how the popularity of generative AI has led to the emergence of prompt engineering which begs the question: what is prompt engineering?

Generative AI

ORGANIZATIONS.

SHAPING THE INDUSTRY.

Berkeley Artificial Intelligence Research

Synera

Latest Posts