<p id="isPasted">We are thrilled to invite engineers and innovators to the <strong id="isPasted">Pervasive AI Developer Contest</strong>, powered by AMD and Hackster. This is an exceptional opportunity for AI enthusiasts to showcase their skills in developing innovative AI projects that could revolutionize various sectors including data centers, client workstations, industrial gaming, and automotive.&nbsp;</p><p>Enter to win incredible cash prizes and gain recognition in the tech world!</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><hr><h2><strong>What is the Pervasive AI Developer Contest?</strong></h2><p>The Pervasive AI Developer Contest is an initiative by AMD to encourage and empower developers to create AI-driven innovations using AMD's cutting-edge technology. With a focus on various applications in the tech industry, this challenge is designed to explore and expand the horizons of AI capabilities.</p><h3><strong>Exciting Categories &amp; Generous Prizes</strong></h3><p>AMD has lined up three stimulating categories for this challenge: <strong>Generative AI, Robotics AI, and PC AI.</strong> Each category is an avenue to unleash your creativity and technical prowess. Amazing cash prizes await the winners, with a top prize of $10,000 USD in each category!</p><h3><strong>Special Awards</strong></h3><p>In addition to the main categories, AMD is offering a special award of $2,500 USD for the best project submitted by a university student. The challenge also has a “Women in Technology” award with the same cash prize, encouraging teams with a majority of female members to participate and showcase their talents.</p><h2><strong>How to Participate?</strong></h2><p>Enroll in the AMD Developer Program, apply for the challenge, and take a quiz to qualify for free hardware. This is your chance to work with AMD’s latest technology and potentially lead the next wave of AI innovations.</p><p><a href="https://wevlv.co/hackster-pervasive-ai" target="_blank" rel="noopener noreferrer" class="button-main-action">APPLY FOR HARDWARE</a></p><h2><strong>Key Dates to Remember</strong></h2><p>Don’t miss out! Make sure to mark these important dates on your calendar:</p><ul><li><strong>Contest begins:</strong> November 30, 2023 at 1:00 AM PST</li><li><strong>Applications for hardware close:</strong> February 29, 2024 at 11:59 PM PST</li><li><strong>Hardware recipients announced:</strong> March 15, 2024 at 11:00 AM PST</li><li><strong>Submissions close:</strong> June 30, 2024 at 11:59 PM PDT</li><li><strong>Winners announced by:</strong> July 15, 2024</li></ul><h2 id="isPasted">Prizes Galore</h2><p>This challenge offers $160,000 in total prizes! An expert panel of judges will evaluate the top 14 projects, focusing on their innovation, implementation, and overall impact on AI.</p><h3 id="isPasted">Hardware Availability</h3><p>There are up to <strong>700</strong> total pieces of free development hardware and cloud instances available from AMD in this year's contest. Each category has a specific amount of hardware allocated:</p><ul><li>Generative AI: <strong>up to 300</strong> MI210 Cloud Instances or Radeon PRO W7900 GPUs</li><li>Robotics AI : <strong>up to 200pcs</strong> KR260 Robotics Starter Kit</li><li>PC AI: <strong>up to 200pcs</strong> Ryzen AI Powered PC</li></ul><p><em>* If you already have access to any of the above hardware, you are welcome to use it.</em></p><h2>Engage and Collaborate</h2><p>By joining the AMD Pervasive AI Developer Contest, you not only get the opportunity to compete for multiple cash prizes, but you will be able to engage with AMD technical experts, connect with fellow developers, showcase your ability and innovations to a broader audience, and gain practical experience by working on real-world challenges.</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/3EMrWVr3oI4?&amp;t=38s&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p>

Fueling Groundbreaking Innovations - Win up to $160,000 in prizes!

AMD Pervasive AI Developer Contest

In this episode, we delve into the ever-evolving world of hybrid AI and generative AI models.

Podcast: The Prius of Artificial Intelligence: Hybrid AI Models

<p>This article originally appeared in <a href="https://news.mit.edu/2023/new-technique-helps-robots-pack-objects-tight-space-1017" target="_blank" rel="noopener noreferrer">MIT News</a> and has been republished with permission.</p><hr><p id="isPasted">Anyone who has ever tried to pack a family-sized amount of luggage into a sedan-sized trunk knows this is a hard problem. Robots struggle with dense packing tasks, too.</p><p>For the robot, solving the packing problem involves satisfying many constraints, such as stacking luggage so suitcases don’t topple out of the trunk, heavy objects aren’t placed on top of lighter ones, and collisions between the robotic arm and the car’s bumper are avoided.</p><p>Some traditional methods tackle this problem sequentially, guessing a partial solution that meets one constraint at a time and then checking to see if any other constraints were violated. With a long sequence of actions to take, and a pile of luggage to pack, this process can be impractically time consuming. &nbsp;&nbsp;</p><p>MIT researchers used a form of generative AI, called a diffusion model, to solve this problem more efficiently. Their method uses a collection of machine-learning models, each of which is trained to represent one specific type of constraint. These models are combined to generate global solutions to the packing problem, taking into account all constraints at once.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/mhtYU7IevtE?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><p id="isPasted">Their method was able to generate effective solutions faster than other techniques, and it produced a greater number of successful solutions in the same amount of time. Importantly, their technique was also able to solve problems with novel combinations of constraints and larger numbers of objects, that the models did not see during training.</p><p>Due to this generalizability, their technique can be used to teach robots how to understand and meet the overall constraints of packing problems, such as the importance of avoiding collisions or a desire for one object to be next to another object. Robots trained in this way could be applied to a wide array of complex tasks in diverse environments, from order fulfillment in a warehouse to organizing a bookshelf in someone’s home.</p><p>“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments. With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” says Zhutian Yang, an electrical engineering and computer science graduate student and lead author of a <a href="https://arxiv.org/pdf/2309.00966.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper on this new machine-learning technique</a>.</p><p>Her co-authors include MIT graduate students Jiayuan Mao and Yilun Du; Jiajun Wu, an assistant professor of computer science at Stanford University; Joshua B. Tenenbaum, a professor in MIT’s Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Tomás Lozano-Pérez, an MIT professor of computer science and engineering and a member of CSAIL; and senior author Leslie Kaelbling, the Panasonic Professor of Computer Science and Engineering at MIT and a member of CSAIL. The research will be presented at the Conference on Robot Learning.</p><h2><strong>Constraint complications</strong></h2><p>Continuous constraint satisfaction problems are particularly challenging for robots. These problems appear in multistep robot manipulation tasks, like packing items into a box or setting a dinner table. They often involve achieving a number of constraints, including geometric constraints, such as avoiding collisions between the robot arm and the environment; physical constraints, such as stacking objects so they are stable; and qualitative constraints, such as placing a spoon to the right of a knife.</p><p>There may be many constraints, and they vary across problems and environments depending on the geometry of objects and human-specified requirements.</p><p>To solve these problems efficiently, the MIT researchers developed a machine-learning technique called <a href="https://diffusion-ccsp.github.io/" target="_blank">Diffusion-CCSP</a>. Diffusion models learn to generate new data samples that resemble samples in a training dataset by iteratively refining their output.</p><p>To do this, diffusion models learn a procedure for making small improvements to a potential solution. Then, to solve a problem, they start with a random, very bad solution and then gradually improve it.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697557883397-ezgif-5-a827fde90a.gif" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Using generative AI models, MIT researchers created a technique that could enable robots to efficiently solve continuous constraint satisfaction problems, such as packing objects into a box while avoiding collisions, as shown in this simulation. Image: Courtesy of the researchers</span></span></span></p><p id="isPasted">For example, imagine randomly placing plates and utensils on a simulated table, allowing them to physically overlap. The collision-free constraints between objects will result in them nudging each other away, while qualitative constraints will drag the plate to the center, align the salad fork and dinner fork, etc.</p><p>Diffusion models are well-suited for this kind of continuous constraint-satisfaction problem because the influences from multiple models on the pose of one object can be composed to encourage the satisfaction of all constraints, Yang explains. By starting from a random initial guess each time, the models can obtain a diverse set of good solutions.</p><h2><strong>Working together</strong></h2><p>For Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints. In packing for instance, one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located.</p><p>Diffusion-CCSP learns a family of diffusion models, with one for each type of constraint. The models are trained together, so they share some knowledge, like the geometry of the objects to be packed.</p><p>The models then work together to find solutions, in this case locations for the objects to be placed, that jointly satisfy the constraints.</p><p>“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” she says.</p><p>Training individual models for each constraint type and then combining them to make predictions greatly reduces the amount of training data required, compared to other approaches.</p><p>However, training these models still requires a large amount of data that demonstrate solved problems. Humans would need to solve each problem with traditional slow methods, making the cost to generate such data prohibitive, Yang says.</p><p>Instead, the researchers reversed the process by coming up with solutions first. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.</p><p>“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.</p><p>Trained using these data, the diffusion models work together to determine locations objects should be placed by the robotic gripper that achieve the packing task while meeting all of the constraints.</p><p>They conducted feasibility studies, and then demonstrated Diffusion-CCSP with a real robot solving a number of difficult problems, including fitting 2D triangles into a box, packing 2D shapes with spatial relationship constraints, stacking 3D objects with stability constraints, and packing 3D objects with a robotic arm.</p><p>Their method outperformed other techniques in many experiments, generating a greater number of effective solutions that were both stable and collision-free.</p><p>In the future, Yang and her collaborators want to test Diffusion-CCSP in more complicated situations, such as with robots that can move around a room. They also want to enable Diffusion-CCSP to tackle problems in different domains without the need to be retrained on new data.</p><p>“Diffusion-CCSP is a machine-learning solution that builds on existing powerful generative models,” says Danfei Xu, an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology and a Research Scientist at NVIDIA AI, who was not involved with this work. “It can quickly generate solutions that simultaneously satisfy multiple constraints by composing known individual constraint models. Although it’s still in the early phases of development, the ongoing advancements in this approach hold the promise of enabling more efficient, safe, and reliable autonomous systems in various applications.”</p><p>This research was funded, in part, by the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research, the MIT-IBM Watson AI Lab, the MIT Quest for Intelligence, the Center for Brains, Minds, and Machines, Boston Dynamics Artificial Intelligence Institute, the Stanford Institute for Human-Centered Artificial Intelligence, Analog Devices, JPMorgan Chase and Co., and Salesforce.</p>

MIT researchers are using generative AI models to help robots more efficiently solve complex object manipulation problems, such as packing a box with different objects. Image: Courtesy of the researchers

Researchers coaxed a family of generative AI models to work together to solve multistep robot manipulation problems.

New technique helps robots pack objects into a tight space

<p id="isPasted">This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/50b8e0ea-1cdb-4b34-87e6-4867feefade3?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p>The full article will continue below.</p><hr><p><em>This is the second article in a short series. Read the<a href="https://www.wevolver.com/article/balancing-the-cloud-and-the-edge-a-close-look-at-enabling-hybrid-ai" target="_blank" rel="noopener noreferrer">&nbsp;first piec</a>e here.</em></p><h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">Generative AI is a rapidly advancing field of artificial intelligence (AI) focused on creating novel data assets like images, text, or music. Such AI models are trained using extensive datasets, enabling them to create new data resembling what they were trained on. The growth in generative AI's development owes itself to several key factors: availability of large datasets, improved computing hardware capabilities, and the evolution of novel machine learning (ML) algorithms and architectures.</p><p dir="ltr">Large language models–based chatbots like ChatGPT, which can generate coherent and contextually relevant text and even images, have gained the interest and creativity of researchers, businesses, and the public. Generative models have demonstrated remarkable capabilities across a range of applications, from natural language understanding and text generation to creative content creation and even virtual assistants. However, as these models grow in complexity and sophistication, they bring to the forefront an increasingly pressing issue in the form of scalability challenges that apply to both training and inference.&nbsp;</p><p dir="ltr">These challenges encompass the need for significant computational resources, the growing size of models, and increasing costs. Latency and real-time responses primarily relate to the inference phase, while ethical considerations, interoperability, privacy, energy consumption, and cost management are critical factors throughout the entire lifecycle of AI systems, including training and inference.&nbsp;</p><p dir="ltr">One promising solution arises in response to these challenges - on-device AI. Deploying AI models directly on user devices (like smartphones and tablets) offers multiple advantages. It reduces reliance on cloud resources, cutting computational and energy costs. It improves privacy by processing data locally, reducing the need to send sensitive information to the cloud. It also lowers latency, supports offline functionality, and aligns with sustainability goals by reducing energy consumption.</p><h2 dir="ltr">What is Hybrid AI?</h2><p dir="ltr">To properly define Hybrid AI, we need to introduce the terms on-device AI and hybrid AI. On-Device AI, also known as edge AI or local AI, refers to the deployment of AI algorithms and models directly on user devices (smartphones, tablets, IoT devices, autonomous vehicles, etc.), without heavy reliance on cloud-based infrastructure. It enables real-time processing and decision-making on the device itself, reducing the need for constant data transmission to remote servers. Hybrid AI represents a synergy between local devices and cloud infrastructure, strategically dividing AI computations to optimize both user experiences and resource efficiency.</p><p dir="ltr">In certain situations, AI processing predominantly occurs on the device, with the option to delegate tasks to the cloud as needed. In cloud-centric setups, devices will opportunistically offload AI workloads from the cloud, where possible and subject to their capabilities. Employing a hybrid AI architecture, or relying solely on on-device AI, yields numerous advantages including cost savings, improved energy efficiency, enhanced performance, heightened privacy and security, and personalized experiences on a global scale.</p><p dir="ltr">Within a hybrid AI framework, on-device AI processing delivers consistent performance that can rival or surpass cloud-based alternatives, particularly in scenarios characterized by congested cloud servers and network connectivity issues. Instances of high demand for generative AI queries in the cloud can result in lengthy queues and elevated latency, and in extreme cases, lead to service disruptions. These challenges can be mitigated by redistributing computational workloads to edge devices.</p><p dir="ltr">Furthermore, the presence of on-device processing capabilities in a hybrid AI architecture enables generative AI applications to function seamlessly in locations with limited or no connectivity. In a device-centric hybrid AI setup, the device serves as the central point of operation, with cloud resources primarily utilized for tasks that exceed the device's processing capacity. Given that many generative AI models can efficiently run on the device, edge devices can handle the bulk of the processing load. As the development of on-device AI continues, it holds significant potential to revolutionize AI service deployment and user experiences while maintaining a balance between performance and ethical considerations.</p><h2 dir="ltr">Unique Benefits of On-Device AI</h2><p dir="ltr"><a href="https://www.qualcomm.com/news/onq/2023/08/5-benefits-of-on-device-generative-ai" target="_blank">On-device AI offers several unique benefits</a>, including:</p><h3>Cost-efficiency&nbsp;</h3><p>By processing data locally, it reduces the need for extensive cloud infrastructure, which can be expensive to maintain and scale. On-device processing eliminates the necessity of sending data to the cloud, a process that can be both expensive and time-consuming. &nbsp;For instance, on-device AI for image generation, can be employed to generate high-quality images in real-time without relying on cloud-based servers. A mobile photography application equipped with generative AI can apply artistic filters, improve image quality, or create artistic effects directly on the user's device.&nbsp;</p><p>This ensures that users can quickly and seamlessly edit and generate images without uploading them to a remote server, preserving both privacy and reducing latency. Additionally, it allows for offline image generation, enabling users to create and edit images even when they're in locations with poor or no internet connectivity. This level of on-device image generation can significantly improve the user experience and provide powerful tools for content creation without compromising data privacy or speed.&nbsp;</p><h3>Energy Savings&nbsp;</h3><p>On-device AI mitigates the issue of energy consumption by performing computations locally. This is particularly crucial for devices with constrained battery life, like smartphones and wearables. Comparing the power efficiency of running generative AI models on edge devices versus to the cloud, edge devices offer superior performance per watt and reduced energy consumption, potentially resulting in environmental and cost benefits.&nbsp;</p><h3>Performance and Low Latency</h3><p>By processing tasks locally, it minimizes communication and computing delays associated with cloud-based AI. This is particularly important in real-time applications such as autonomous vehicles, medical devices, and interactive content, where even milliseconds of delay can have a significant impact on functionality and user experience.</p><h3>Enhanced Privacy and Security</h3><p>This approach reduces the risk of data breaches and unauthorized access during data transfer and cloud storage. Users can have greater confidence that their personal information remains on their devices, reinforcing privacy and security standards, especially for applications handling sensitive data, like financial transactions and medical information.</p><h3>Personalization and User Experience</h3><p>These models can adapt to the user's context and needs, improving user experiences by recommending relevant products or services based on their preferences and current circumstances. For example, a shopping application could use on-device AI to recommend products similar to past purchases or consider the user's location and weather conditions. Real-time personalization boosts relevance, engagement, and customer satisfaction.</p><h2 dir="ltr">Case Studies: On-Device AI in Action</h2><p dir="ltr">The following case studies illustrate the transformative potential of on-device AI across various domains, showcasing its benefits.</p><h3 dir="ltr" id="isPasted">Automotive</h3><p dir="ltr">In the automotive industry, AI-driven cockpits are revolutionizing the driving experience. These systems, like those in smartphones and PCs, are starting to employ in-vehicle digital assistants powered by generative AI to create highly personalized and hands-free interactions for drivers and passengers.&nbsp;</p><p dir="ltr">These digital assistants leverage personal and vehicle sensor data, including cameras and microphones, to improve the driving journey. Third-party services can also integrate seamlessly, improving navigation, offering proactive assistance, and enabling in-car commerce, such as ordering food. Furthermore, the cockpit experience is improved through individual recognition, customization of media content, and the rise of in-vehicle augmented reality. Vehicle maintenance becomes proactive as AI digital assistants analyze data to predict maintenance needs, to seamlessly communicate with users using natural language, and to provide precise repair guidance. This proactive approach not only enhances reliability but also effectively reduces costs.&nbsp;</p><p dir="ltr">Qualcomm, which leads the field in cockpit and in-vehicle infotainment solutions with numerous global automakers choosing Snapdragon® Cockpit Platforms. These platforms power digital cockpits, with many already in production or design stages. Partners include Honda, Mercedes, Renault, Volvo, and others. The Snapdragon Cockpit Platform's latest iteration focuses on superior in-vehicle user experiences, safety, comfort, and reliability, setting new standards for connected car digital cockpit solutions. Qualcomm's Snapdragon Ride™ Platform complements this, offering scalable automated driving SoC platforms for ADAS, featuring vision perception, parking, and path planning.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUyNzU0OTg0LXF1YWxjb21tIGNhYmluLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">&nbsp;Cockpit platforms are designed to transform in-vehicle experiences, supporting higher levels of compute and computing intelligence needed for advanced capabilities featured in next generation vehicles. Image credit: Qualcomm.</span></span></span></p><h3 dir="ltr" id="isPasted">Extended Reality</h3><p dir="ltr">For Extended Reality (XR), generative AI has the potential to democratize 3D content creation. In the near future, AI rendering tools will enable content creators to generate 3D objects and entire virtual worlds using various inputs like text, speech, images, or video. Furthermore, text-to-text models will facilitate lifelike conversations for avatars, transforming the way we interact with XR content. Despite the exciting possibilities, widespread adoption timelines remain uncertain, though significant progress is expected soon. Immersive XR experiences will benefit from text-to-image models for realistic textures, likely available on smartphones and XR devices within a year.&nbsp;</p><p dir="ltr">These capabilities will involve distributed processing, with headsets handling perception and rendering, while generative AI models run on paired smartphones or in the cloud. In a few years, text-to-3D and image-to-3D models will likely produce high-quality 3D objects, evolving to generate entire scenes and rooms. Eventually, XR may enable users to immerse themselves in virtual worlds constructed from their imagination. Virtual avatars will undergo a similar evolution, utilizing text-to-text models for natural conversations and text-to-image models for outfits. Over time, avatars may become increasingly photorealistic, fully animated, and mass-producible, driven by image-to-3D and encoder/decoder models. This progression promises to revolutionize how we create and experience immersive XR content, with continual advancements anticipated in the years ahead.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUzMTYyMjIyLUNhbm5lcy1YUi1WaXJ0dWFsLUJyaW5ncy1JbW1lcnNpdmUtQXJ0LXRvLUxpZmUtV2l0aC1WaXJ0dWFsLVJlYWxpdHkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Image credit: Cannes</span></span></span></p><h2 dir="ltr">Challenges and Solutions</h2><p dir="ltr">The technical challenges specific to hybrid AI are being actively addressed through ongoing research and innovation:</p><h3 dir="ltr">Model size</h3><p>Model size is a notable challenge, particularly on resource-constrained devices. As large models are challenging to store and run efficiently, a potential solution involves employing compression and quantization techniques to reduce model size while maintaining accuracy. <a href="https://stackoverflow.blog/2023/08/23/fitting-ai-models-in-your-pocket-with-quantization/" target="_blank">Qualcomm</a> is actively researching ways to enhance <a href="https://www.qualcomm.com/news/onq/2019/03/heres-why-quantization-matters-ai" target="_blank">quantization</a> techniques, specifically aiming to quantize 32-bit floating point parameters to 8-bit or 4-bit integers in neural networks while maintaining accuracy. The research encompasses various model quantization methods, such as Bayesian learning, adaptive rounding, and transformer quantization, as part of <a href="https://www.qualcomm.com/research/artificial-intelligence/ai-research" target="_blank">Qualcomm AI Research</a>'s ongoing efforts.&nbsp;</p><p>For instance, the Qualcomm AI Research team employed the <a href="https://www.forbes.com/sites/davealtavilla/2023/05/15/powering-ai-on-mobile-devices-requires-new-math-and-qualcomm-is-pioneering-it/?sh=54cb40c91a16" target="_blank">AIMET</a> tool (AI Model Efficiency Toolkit) to quantize the 32-bit floating point Stable Diffusion AI model into an 8-bit integer format while preserving almost the same accuracy level. By optimizing and scaling the model for INT8 precision operations instead of FP32, Qualcomm successfully enabled Stable Diffusion to run efficiently on handheld devices without internet connectivity, significantly reducing the model's size while maintaining accuracy and enhancing performance on low-power Snapdragon AI accelerators.</p><h3>Computational power</h3><p>Large AI models may place heavy computational demands, particularly on devices with restricted processing capabilities. A potential solution involves leveraging specialized hardware like AI accelerators to enhance AI model performance on such devices. In 2022, Snapdragon 8 Gen 2 provided groundbreaking AI, integrated across the entire system, powered by their fastest, most advanced Qualcomm AI Engine to date. Users can experience faster natural language processing with multi-language translation or have fun with AI cinematic video capture.&nbsp;</p><p>The latest Hexagon Processor introduced a dedicated power delivery system, which adapts power according to the workload. Special hardware improves group convolution, activation function acceleration, and the performance of the Hexagon Tensor Accelerator. Support for micro-tile inferencing and INT4 hardware acceleration provide even higher performance while reducing power and memory traffic. The transformer acceleration dramatically speeds up inference for multi-head attention that is used throughout generative AI, resulting in AI performance up to 4.35X on certain use cases with MobileBERT.&nbsp;</p><p dir="ltr">These ongoing research efforts are addressing the technical challenges of hybrid AI, with expectations to overcome them successfully as the technology advances and becomes more widely adopted and utilized.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">On-device AI plays a critical role in addressing the future scalability challenges of generative AI models. It has the potential to revolutionize numerous industries and significantly enhance user experiences. By enabling local processing and real-time personalization, on-device AI reduces the dependence on cloud resources, improves privacy and security as well as the efficiency of AI applications. This approach not only supports the responsible development of AI technologies but also opens new possibilities for scalable and personalized AI solutions across diverse sectors, from healthcare to entertainment, promising a brighter future for both businesses and users alike.</p><p>To stay up to date on the latest from Qualcomm, <a href="https://assets.qualcomm.com/mobile-computing-newsletter-sign-up.html" target="_blank">join the “What’s Next in AI and Computing” newsletter</a> and read their hybrid AI whitepapers (<a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 1</a>, <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-2-Qualcomm-is-uniquely-positioned-to-scale-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 2</a>)</p>

How on-device and cloud AI will foster the advancement and scalability of next-generation generative AI experiences

The Scalability Challenge: Focusing on Hybrid AI in the Future of Generative AI Models

In this episode, we discuss how the popularity of generative AI has led to the emergence of prompt engineering which begs the question: what is prompt engineering?

Generative AI

ORGANIZATIONS. SHAPING THE INDUSTRY.

Synera

Latest Posts

AMD Pervasive AI Developer Contest

Podcast: The Prius of Artificial Intelligence: Hybrid AI Models

New technique helps robots pack objects into a tight space

The Scalability Challenge: Focusing on Hybrid AI in the Future of Generative AI Models

Podcast: Prompt Engineering: Latest Meme or The Next Big Thing?