<p dir="ltr" id="isPasted">In recent years, the rise of Generative Artificial Intelligence (AI) has been nothing short of revolutionary. This technology, capable of producing content that closely mimics human output, has paved the way for innovations across various fields. From writing convincingly real emails to creating impressive images, generative AI's capabilities are vast and transformative.</p><p dir="ltr">However, as with any technological advancement, there are two sides to the story. The same capabilities that offer numerous benefits also pose significant risks, particularly concerning cybersecurity. Specifically, one of the most concerning developments has been the enhancement of phishing schemes, making them more sophisticated and challenging to detect than ever before. This article explores the rise of generative AI, how it's changing the phishing landscape, and best practices to avoid becoming a victim.&nbsp;</p><h1 dir="ltr">The Evolution of Generative AI</h1><p dir="ltr">Generative AI is marked by its ability to learn from a vast dataset of existing digital content and generate new, unique outputs that can pass for human-created. This process involves complex algorithms and neural network models that analyze patterns, styles, and structures within the data they're fed. As a result, generative AI can produce a wide range of outputs, including textual content, images, music, and even synthetic voices. Its applications are far-reaching, offering potential benefits in areas such as content creation, entertainment, education, and beyond. <sup>[1]</sup></p><p dir="ltr">However, the power of generative AI extends beyond positive use cases. Its very capabilities, particularly in mimicking human communication styles and generating personalized content, have opened new avenues for cybercriminals. These advancements have led to a new generation of phishing attacks that are significantly more difficult to identify and combat.</p><h1 dir="ltr">The Transformation of Phishing Tactics</h1><p>Phishing schemes have long been a staple in the arsenal of cybercriminals. Traditionally, these attacks involved sending generic, often poorly written emails, attempting to lure recipients into divulging sensitive information or downloading malicious software. While effective to a degree, these attempts were easier to spot and avoid due to their lack of personalization and sophistication. <sup>[2]</sup></p><p>Enter generative AI, and the landscape of phishing has changed dramatically. With the ability to analyze and replicate communication styles, generative AI enables cybercriminals to craft highly convincing and personalized messages. These messages can mimic the tone, style, and even specific phrases used by individuals or organizations, making them far more likely to deceive recipients.</p><h1 dir="ltr">Understanding AI-Enhanced Phishing</h1><p>The core threat of AI-enhanced phishing is its personalization and realism. Generative AI can create emails, text messages, or social media messages that look and sound like they're from a trusted source, such as a colleague, friend, or a reputable organization. This level of personalization is achieved by analyzing publicly available data or previously breached information to tailor attacks to individual targets. <sup>[3]</sup></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 790px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE0Mzg4NjY3Mjg2LWN5YmVyMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"></span></span></span></p><p dir="ltr" id="isPasted">Phishing attacks begin with tricking users out of their important credentials. Image credit:&nbsp;<a href="https://www.valimail.com/guide-to-phishing/" target="_blank">Vailmail</a></p><p><br></p><p dir="ltr" id="isPasted">For instance, a generative AI-powered phishing email might reference recent transactions, use language that matches the supposed sender's usual tone, or mimic the layout and design of legitimate correspondence. With such attention to detail, these fraudulent messages are incredibly persuasive, increasing the likelihood that recipients will comply with their requests, whether that involves providing passwords, financial information, or other sensitive data.</p><h1 dir="ltr">The Signs of AI-Generated Phishing Attacks</h1><p>Despite their sophistication, AI-generated phishing attacks are not foolproof. There are still telltale signs that can help discerning individuals identify them. These include slight inconsistencies in language or format, unusual requests, or links to websites that, upon closer inspection, have subtle differences from legitimate addresses. Moreover, these messages might lack the personal touch or in-depth knowledge that genuine communications from friends or colleagues would have, relying instead on information that could be publicly accessed or inferred.</p><h1 dir="ltr">Strategies for Personal and Organizational Protection</h1><p dir="ltr">Naturally, the advancement of AI-enhanced phishing schemes necessitates stronger defensive measures, both at personal and organizational levels. Awareness is the first step; understanding that no one is immune to these attacks is crucial. From there, adopting a multi-layered approach to security can significantly mitigate risks. <sup>[4]</sup></p><h2 dir="ltr">For individuals:</h2><ul><li dir="ltr"><p dir="ltr">Be Skeptical of Unsolicited Communications: Always approach unexpected requests for information or action with caution, even if they appear to come from known contacts or organizations.</p></li><li dir="ltr"><p dir="ltr">Verify Independently:&nbsp;If a message asks for sensitive information or to perform a specific action, verify its authenticity through an independent channel. For example, if you receive an email from your bank asking for personal details, contact the bank directly using a phone number from their official website.</p></li><li dir="ltr"><p dir="ltr">Use Advanced Security Features:&nbsp;Enable two-factor authentication (2FA) wherever possible. This adds an extra verification step that can protect your accounts even if your password is compromised.</p></li><li dir="ltr"><p dir="ltr">Regularly Update Software:&nbsp;Ensure that your operating system, applications, and anti-virus software are up to date to protect against known vulnerabilities.</p></li></ul><h3>For organizations:</h3><ul><li dir="ltr"><p dir="ltr">Conduct Regular Training: Employees should be regularly trained on the latest cybersecurity threats and how to recognize phishing attempts. Simulated phishing exercises can be particularly effective.</p></li><li dir="ltr"><p dir="ltr">Implement Strict Information Handling Policies: Limit the amount of personal and sensitive information that can be shared or accessed based on job roles.</p></li><li dir="ltr"><p dir="ltr">Adopt Advanced Security Technologies: Utilize email filtering, intrusion detection systems, and AI-based security solutions that can identify and block phishing attempts.</p></li><li dir="ltr"><p dir="ltr">Establish Clear Reporting Procedures:&nbsp;Employees should know how to report suspected phishing attempts promptly, allowing IT security teams to respond quickly.</p></li></ul><h1 dir="ltr">The Role of Continuous Education</h1><p dir="ltr">As generative AI continues to evolve, so too will the tactics used by cybercriminals. For this reason, staying informed about the latest developments in AI and cybersecurity is essential. For individuals, this means staying abreast of the latest security advice and best practices. For organizations, it involves regular training sessions and updates on cybersecurity trends and threats. <sup>[5]</sup></p><p dir="ltr">Awareness campaigns, workshops, and online courses can play significant roles in educating the public and employees about the dangers of AI-enhanced phishing schemes and how to protect against them. Knowledge is power, and in the context of cybersecurity, it's also the best defense.</p><h1 dir="ltr">Looking Ahead: The Future of AI in Cybersecurity</h1><p dir="ltr">The dual-use nature of generative AI in cybersecurity presents a continuous game of cat and mouse between cybercriminals and defenders. As AI technologies become more sophisticated, so too do the tools and strategies to combat malicious use. Future developments in AI could offer more robust detection mechanisms, identifying and neutralizing phishing attempts before they reach their intended targets. Similarly, advancements in natural language processing could enable more effective filtering of phishing content, while machine learning models might predict and block new phishing tactics.<sup>[6]</sup></p><p dir="ltr">However, as technology advances, the complexity and novelty of phishing attacks will likely increase as well. A proactive and adaptive approach to cybersecurity, emphasizing technological solutions and human vigilance and education, will be necessary to combat these advances.</p><h1 dir="ltr">Conclusion</h1><p dir="ltr">The rise of generative AI has brought with it an era of enhanced phishing schemes, presenting new challenges in the realm of cybersecurity. While these sophisticated attacks pose significant risks, the combination of advanced security measures, continuous education, and heightened awareness provides a formidable defense. As we navigate this evolving landscape, staying informed, cautious, and prepared is essential for protecting personal and organizational integrity in the digital age. Through collective effort and the strategic application of technology, we can mitigate the threats posed by AI-enhanced phishing and secure our digital future.</p><h3 dir="ltr">References</h3><ol><li dir="ltr"><p dir="ltr"><a href="https://www.techtarget.com/searchenterpriseai/definition/generative-AI" target="_blank">https://www.techtarget.com/searchenterpriseai/definition/generative-AI</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://www.phishing.org/what-is-phishing" target="_blank">https://www.phishing.org/what-is-phishing</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://withpersona.com/blog/ai-phishing-attacks" target="_blank">https://withpersona.com/blog/ai-phishing-attacks</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://consumer.ftc.gov/articles/how-recognize-and-avoid-phishing-scams" target="_blank">https://consumer.ftc.gov/articles/how-recognize-and-avoid-phishing-scams</a></p></li><li dir="ltr"><p dir="ltr"><a href="https://www.cisa.gov/secure-our-world/teach-employees-avoid-phishing" target="_blank">https://www.cisa.gov/secure-our-world/teach-employees-avoid-phishing</a></p></li><li dir="ltr"><p dir="ltr">https://sennovate.com/the-role-of-artificial-intelligence-in-detecting-phishing-attacks/</p></li></ol>

Generative AI has opened up an entirely new world of cybersecurity threats. 


The Rise of Generative AI and Enhanced Phishing Schemes

<p id="isPasted">We are thrilled to invite engineers and innovators to the <strong id="isPasted">Pervasive AI Developer Contest</strong>, powered by AMD and Hackster. This is an exceptional opportunity for AI enthusiasts to showcase their skills in developing innovative AI projects that could revolutionize various sectors including data centers, client workstations, industrial gaming, and automotive.&nbsp;</p><p>Enter to win incredible cash prizes and gain recognition in the tech world!</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><hr><h2><strong>What is the Pervasive AI Developer Contest?</strong></h2><p>The Pervasive AI Developer Contest is an initiative by AMD to encourage and empower developers to create AI-driven innovations using AMD's cutting-edge technology. With a focus on various applications in the tech industry, this challenge is designed to explore and expand the horizons of AI capabilities.</p><h3><strong>Exciting Categories &amp; Generous Prizes</strong></h3><p>AMD has lined up three stimulating categories for this challenge: <strong>Generative AI, Robotics AI, and PC AI.</strong> Each category is an avenue to unleash your creativity and technical prowess. Amazing cash prizes await the winners, with a top prize of $10,000 USD in each category!</p><h3><strong>Special Awards</strong></h3><p>In addition to the main categories, AMD is offering a special award of $2,500 USD for the best project submitted by a university student. The challenge also has a “Women in Technology” award with the same cash prize, encouraging teams with a majority of female members to participate and showcase their talents.</p><h2><strong>How to Participate?</strong></h2><p>Enroll in the AMD Developer Program, apply for the challenge, and take a quiz to qualify for free hardware. This is your chance to work with AMD’s latest technology and potentially lead the next wave of AI innovations.</p><p><a href="https://wevlv.co/hackster-pervasive-ai" target="_blank" rel="noopener noreferrer" class="button-main-action">APPLY FOR HARDWARE</a></p><h2><strong>Key Dates to Remember</strong></h2><p>Don’t miss out! Make sure to mark these important dates on your calendar:</p><ul><li><strong>Contest begins:</strong> November 30, 2023 at 1:00 AM PST</li><li><strong>Applications for hardware close:</strong> February 29, 2024 at 11:59 PM PST</li><li><strong>Hardware recipients announced:</strong> March 15, 2024 at 11:00 AM PST</li><li><strong>Submissions close:</strong> June 30, 2024 at 11:59 PM PDT</li><li><strong>Winners announced by:</strong> July 15, 2024</li></ul><h2 id="isPasted">Prizes Galore</h2><p>This challenge offers $160,000 in total prizes! An expert panel of judges will evaluate the top 14 projects, focusing on their innovation, implementation, and overall impact on AI.</p><h3 id="isPasted">Hardware Availability</h3><p>There are up to <strong>700</strong> total pieces of free development hardware and cloud instances available from AMD in this year's contest. Each category has a specific amount of hardware allocated:</p><ul><li>Generative AI: <strong>up to 300</strong> MI210 Cloud Instances or Radeon PRO W7900 GPUs</li><li>Robotics AI : <strong>up to 200pcs</strong> KR260 Robotics Starter Kit</li><li>PC AI: <strong>up to 200pcs</strong> Ryzen AI Powered PC</li></ul><p><em>* If you already have access to any of the above hardware, you are welcome to use it.</em></p><h2>Engage and Collaborate</h2><p>By joining the AMD Pervasive AI Developer Contest, you not only get the opportunity to compete for multiple cash prizes, but you will be able to engage with AMD technical experts, connect with fellow developers, showcase your ability and innovations to a broader audience, and gain practical experience by working on real-world challenges.</p><p><a href="https://www.hackster.io/contests/amd2023/?utm_source=avnet&utm_campaign=amd2023" target="_blank" rel="noopener noreferrer" class="button-main-action"><strong id="isPasted">LEARN MORE</strong></a></p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/3EMrWVr3oI4?&amp;t=38s&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p>

Fueling Groundbreaking Innovations - Win up to $160,000 in prizes!

AMD Pervasive AI Developer Contest

In this episode, we delve into the ever-evolving world of hybrid AI and generative AI models.

Podcast: The Prius of Artificial Intelligence: Hybrid AI Models

<p>This article originally appeared in <a href="https://news.mit.edu/2023/new-technique-helps-robots-pack-objects-tight-space-1017" target="_blank" rel="noopener noreferrer">MIT News</a> and has been republished with permission.</p><hr><p id="isPasted">Anyone who has ever tried to pack a family-sized amount of luggage into a sedan-sized trunk knows this is a hard problem. Robots struggle with dense packing tasks, too.</p><p>For the robot, solving the packing problem involves satisfying many constraints, such as stacking luggage so suitcases don’t topple out of the trunk, heavy objects aren’t placed on top of lighter ones, and collisions between the robotic arm and the car’s bumper are avoided.</p><p>Some traditional methods tackle this problem sequentially, guessing a partial solution that meets one constraint at a time and then checking to see if any other constraints were violated. With a long sequence of actions to take, and a pile of luggage to pack, this process can be impractically time consuming. &nbsp;&nbsp;</p><p>MIT researchers used a form of generative AI, called a diffusion model, to solve this problem more efficiently. Their method uses a collection of machine-learning models, each of which is trained to represent one specific type of constraint. These models are combined to generate global solutions to the packing problem, taking into account all constraints at once.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/mhtYU7IevtE?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><p id="isPasted">Their method was able to generate effective solutions faster than other techniques, and it produced a greater number of successful solutions in the same amount of time. Importantly, their technique was also able to solve problems with novel combinations of constraints and larger numbers of objects, that the models did not see during training.</p><p>Due to this generalizability, their technique can be used to teach robots how to understand and meet the overall constraints of packing problems, such as the importance of avoiding collisions or a desire for one object to be next to another object. Robots trained in this way could be applied to a wide array of complex tasks in diverse environments, from order fulfillment in a warehouse to organizing a bookshelf in someone’s home.</p><p>“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments. With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” says Zhutian Yang, an electrical engineering and computer science graduate student and lead author of a <a href="https://arxiv.org/pdf/2309.00966.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper on this new machine-learning technique</a>.</p><p>Her co-authors include MIT graduate students Jiayuan Mao and Yilun Du; Jiajun Wu, an assistant professor of computer science at Stanford University; Joshua B. Tenenbaum, a professor in MIT’s Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Tomás Lozano-Pérez, an MIT professor of computer science and engineering and a member of CSAIL; and senior author Leslie Kaelbling, the Panasonic Professor of Computer Science and Engineering at MIT and a member of CSAIL. The research will be presented at the Conference on Robot Learning.</p><h2><strong>Constraint complications</strong></h2><p>Continuous constraint satisfaction problems are particularly challenging for robots. These problems appear in multistep robot manipulation tasks, like packing items into a box or setting a dinner table. They often involve achieving a number of constraints, including geometric constraints, such as avoiding collisions between the robot arm and the environment; physical constraints, such as stacking objects so they are stable; and qualitative constraints, such as placing a spoon to the right of a knife.</p><p>There may be many constraints, and they vary across problems and environments depending on the geometry of objects and human-specified requirements.</p><p>To solve these problems efficiently, the MIT researchers developed a machine-learning technique called <a href="https://diffusion-ccsp.github.io/" target="_blank">Diffusion-CCSP</a>. Diffusion models learn to generate new data samples that resemble samples in a training dataset by iteratively refining their output.</p><p>To do this, diffusion models learn a procedure for making small improvements to a potential solution. Then, to solve a problem, they start with a random, very bad solution and then gradually improve it.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697557883397-ezgif-5-a827fde90a.gif" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Using generative AI models, MIT researchers created a technique that could enable robots to efficiently solve continuous constraint satisfaction problems, such as packing objects into a box while avoiding collisions, as shown in this simulation. Image: Courtesy of the researchers</span></span></span></p><p id="isPasted">For example, imagine randomly placing plates and utensils on a simulated table, allowing them to physically overlap. The collision-free constraints between objects will result in them nudging each other away, while qualitative constraints will drag the plate to the center, align the salad fork and dinner fork, etc.</p><p>Diffusion models are well-suited for this kind of continuous constraint-satisfaction problem because the influences from multiple models on the pose of one object can be composed to encourage the satisfaction of all constraints, Yang explains. By starting from a random initial guess each time, the models can obtain a diverse set of good solutions.</p><h2><strong>Working together</strong></h2><p>For Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints. In packing for instance, one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located.</p><p>Diffusion-CCSP learns a family of diffusion models, with one for each type of constraint. The models are trained together, so they share some knowledge, like the geometry of the objects to be packed.</p><p>The models then work together to find solutions, in this case locations for the objects to be placed, that jointly satisfy the constraints.</p><p>“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” she says.</p><p>Training individual models for each constraint type and then combining them to make predictions greatly reduces the amount of training data required, compared to other approaches.</p><p>However, training these models still requires a large amount of data that demonstrate solved problems. Humans would need to solve each problem with traditional slow methods, making the cost to generate such data prohibitive, Yang says.</p><p>Instead, the researchers reversed the process by coming up with solutions first. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.</p><p>“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.</p><p>Trained using these data, the diffusion models work together to determine locations objects should be placed by the robotic gripper that achieve the packing task while meeting all of the constraints.</p><p>They conducted feasibility studies, and then demonstrated Diffusion-CCSP with a real robot solving a number of difficult problems, including fitting 2D triangles into a box, packing 2D shapes with spatial relationship constraints, stacking 3D objects with stability constraints, and packing 3D objects with a robotic arm.</p><p>Their method outperformed other techniques in many experiments, generating a greater number of effective solutions that were both stable and collision-free.</p><p>In the future, Yang and her collaborators want to test Diffusion-CCSP in more complicated situations, such as with robots that can move around a room. They also want to enable Diffusion-CCSP to tackle problems in different domains without the need to be retrained on new data.</p><p>“Diffusion-CCSP is a machine-learning solution that builds on existing powerful generative models,” says Danfei Xu, an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology and a Research Scientist at NVIDIA AI, who was not involved with this work. “It can quickly generate solutions that simultaneously satisfy multiple constraints by composing known individual constraint models. Although it’s still in the early phases of development, the ongoing advancements in this approach hold the promise of enabling more efficient, safe, and reliable autonomous systems in various applications.”</p><p>This research was funded, in part, by the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research, the MIT-IBM Watson AI Lab, the MIT Quest for Intelligence, the Center for Brains, Minds, and Machines, Boston Dynamics Artificial Intelligence Institute, the Stanford Institute for Human-Centered Artificial Intelligence, Analog Devices, JPMorgan Chase and Co., and Salesforce.</p>

MIT researchers are using generative AI models to help robots more efficiently solve complex object manipulation problems, such as packing a box with different objects. Image: Courtesy of the researchers

Researchers coaxed a family of generative AI models to work together to solve multistep robot manipulation problems.

New technique helps robots pack objects into a tight space

<p id="isPasted">This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/50b8e0ea-1cdb-4b34-87e6-4867feefade3?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p>The full article will continue below.</p><hr><p><em>This is the second article in a short series. Read the<a href="https://www.wevolver.com/article/balancing-the-cloud-and-the-edge-a-close-look-at-enabling-hybrid-ai" target="_blank" rel="noopener noreferrer">&nbsp;first piec</a>e here.</em></p><h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">Generative AI is a rapidly advancing field of artificial intelligence (AI) focused on creating novel data assets like images, text, or music. Such AI models are trained using extensive datasets, enabling them to create new data resembling what they were trained on. The growth in generative AI's development owes itself to several key factors: availability of large datasets, improved computing hardware capabilities, and the evolution of novel machine learning (ML) algorithms and architectures.</p><p dir="ltr">Large language models–based chatbots like ChatGPT, which can generate coherent and contextually relevant text and even images, have gained the interest and creativity of researchers, businesses, and the public. Generative models have demonstrated remarkable capabilities across a range of applications, from natural language understanding and text generation to creative content creation and even virtual assistants. However, as these models grow in complexity and sophistication, they bring to the forefront an increasingly pressing issue in the form of scalability challenges that apply to both training and inference.&nbsp;</p><p dir="ltr">These challenges encompass the need for significant computational resources, the growing size of models, and increasing costs. Latency and real-time responses primarily relate to the inference phase, while ethical considerations, interoperability, privacy, energy consumption, and cost management are critical factors throughout the entire lifecycle of AI systems, including training and inference.&nbsp;</p><p dir="ltr">One promising solution arises in response to these challenges - on-device AI. Deploying AI models directly on user devices (like smartphones and tablets) offers multiple advantages. It reduces reliance on cloud resources, cutting computational and energy costs. It improves privacy by processing data locally, reducing the need to send sensitive information to the cloud. It also lowers latency, supports offline functionality, and aligns with sustainability goals by reducing energy consumption.</p><h2 dir="ltr">What is Hybrid AI?</h2><p dir="ltr">To properly define Hybrid AI, we need to introduce the terms on-device AI and hybrid AI. On-Device AI, also known as edge AI or local AI, refers to the deployment of AI algorithms and models directly on user devices (smartphones, tablets, IoT devices, autonomous vehicles, etc.), without heavy reliance on cloud-based infrastructure. It enables real-time processing and decision-making on the device itself, reducing the need for constant data transmission to remote servers. Hybrid AI represents a synergy between local devices and cloud infrastructure, strategically dividing AI computations to optimize both user experiences and resource efficiency.</p><p dir="ltr">In certain situations, AI processing predominantly occurs on the device, with the option to delegate tasks to the cloud as needed. In cloud-centric setups, devices will opportunistically offload AI workloads from the cloud, where possible and subject to their capabilities. Employing a hybrid AI architecture, or relying solely on on-device AI, yields numerous advantages including cost savings, improved energy efficiency, enhanced performance, heightened privacy and security, and personalized experiences on a global scale.</p><p dir="ltr">Within a hybrid AI framework, on-device AI processing delivers consistent performance that can rival or surpass cloud-based alternatives, particularly in scenarios characterized by congested cloud servers and network connectivity issues. Instances of high demand for generative AI queries in the cloud can result in lengthy queues and elevated latency, and in extreme cases, lead to service disruptions. These challenges can be mitigated by redistributing computational workloads to edge devices.</p><p dir="ltr">Furthermore, the presence of on-device processing capabilities in a hybrid AI architecture enables generative AI applications to function seamlessly in locations with limited or no connectivity. In a device-centric hybrid AI setup, the device serves as the central point of operation, with cloud resources primarily utilized for tasks that exceed the device's processing capacity. Given that many generative AI models can efficiently run on the device, edge devices can handle the bulk of the processing load. As the development of on-device AI continues, it holds significant potential to revolutionize AI service deployment and user experiences while maintaining a balance between performance and ethical considerations.</p><h2 dir="ltr">Unique Benefits of On-Device AI</h2><p dir="ltr"><a href="https://www.qualcomm.com/news/onq/2023/08/5-benefits-of-on-device-generative-ai" target="_blank">On-device AI offers several unique benefits</a>, including:</p><h3>Cost-efficiency&nbsp;</h3><p>By processing data locally, it reduces the need for extensive cloud infrastructure, which can be expensive to maintain and scale. On-device processing eliminates the necessity of sending data to the cloud, a process that can be both expensive and time-consuming. &nbsp;For instance, on-device AI for image generation, can be employed to generate high-quality images in real-time without relying on cloud-based servers. A mobile photography application equipped with generative AI can apply artistic filters, improve image quality, or create artistic effects directly on the user's device.&nbsp;</p><p>This ensures that users can quickly and seamlessly edit and generate images without uploading them to a remote server, preserving both privacy and reducing latency. Additionally, it allows for offline image generation, enabling users to create and edit images even when they're in locations with poor or no internet connectivity. This level of on-device image generation can significantly improve the user experience and provide powerful tools for content creation without compromising data privacy or speed.&nbsp;</p><h3>Energy Savings&nbsp;</h3><p>On-device AI mitigates the issue of energy consumption by performing computations locally. This is particularly crucial for devices with constrained battery life, like smartphones and wearables. Comparing the power efficiency of running generative AI models on edge devices versus to the cloud, edge devices offer superior performance per watt and reduced energy consumption, potentially resulting in environmental and cost benefits.&nbsp;</p><h3>Performance and Low Latency</h3><p>By processing tasks locally, it minimizes communication and computing delays associated with cloud-based AI. This is particularly important in real-time applications such as autonomous vehicles, medical devices, and interactive content, where even milliseconds of delay can have a significant impact on functionality and user experience.</p><h3>Enhanced Privacy and Security</h3><p>This approach reduces the risk of data breaches and unauthorized access during data transfer and cloud storage. Users can have greater confidence that their personal information remains on their devices, reinforcing privacy and security standards, especially for applications handling sensitive data, like financial transactions and medical information.</p><h3>Personalization and User Experience</h3><p>These models can adapt to the user's context and needs, improving user experiences by recommending relevant products or services based on their preferences and current circumstances. For example, a shopping application could use on-device AI to recommend products similar to past purchases or consider the user's location and weather conditions. Real-time personalization boosts relevance, engagement, and customer satisfaction.</p><h2 dir="ltr">Case Studies: On-Device AI in Action</h2><p dir="ltr">The following case studies illustrate the transformative potential of on-device AI across various domains, showcasing its benefits.</p><h3 dir="ltr" id="isPasted">Automotive</h3><p dir="ltr">In the automotive industry, AI-driven cockpits are revolutionizing the driving experience. These systems, like those in smartphones and PCs, are starting to employ in-vehicle digital assistants powered by generative AI to create highly personalized and hands-free interactions for drivers and passengers.&nbsp;</p><p dir="ltr">These digital assistants leverage personal and vehicle sensor data, including cameras and microphones, to improve the driving journey. Third-party services can also integrate seamlessly, improving navigation, offering proactive assistance, and enabling in-car commerce, such as ordering food. Furthermore, the cockpit experience is improved through individual recognition, customization of media content, and the rise of in-vehicle augmented reality. Vehicle maintenance becomes proactive as AI digital assistants analyze data to predict maintenance needs, to seamlessly communicate with users using natural language, and to provide precise repair guidance. This proactive approach not only enhances reliability but also effectively reduces costs.&nbsp;</p><p dir="ltr">Qualcomm, which leads the field in cockpit and in-vehicle infotainment solutions with numerous global automakers choosing Snapdragon® Cockpit Platforms. These platforms power digital cockpits, with many already in production or design stages. Partners include Honda, Mercedes, Renault, Volvo, and others. The Snapdragon Cockpit Platform's latest iteration focuses on superior in-vehicle user experiences, safety, comfort, and reliability, setting new standards for connected car digital cockpit solutions. Qualcomm's Snapdragon Ride™ Platform complements this, offering scalable automated driving SoC platforms for ADAS, featuring vision perception, parking, and path planning.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUyNzU0OTg0LXF1YWxjb21tIGNhYmluLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">&nbsp;Cockpit platforms are designed to transform in-vehicle experiences, supporting higher levels of compute and computing intelligence needed for advanced capabilities featured in next generation vehicles. Image credit: Qualcomm.</span></span></span></p><h3 dir="ltr" id="isPasted">Extended Reality</h3><p dir="ltr">For Extended Reality (XR), generative AI has the potential to democratize 3D content creation. In the near future, AI rendering tools will enable content creators to generate 3D objects and entire virtual worlds using various inputs like text, speech, images, or video. Furthermore, text-to-text models will facilitate lifelike conversations for avatars, transforming the way we interact with XR content. Despite the exciting possibilities, widespread adoption timelines remain uncertain, though significant progress is expected soon. Immersive XR experiences will benefit from text-to-image models for realistic textures, likely available on smartphones and XR devices within a year.&nbsp;</p><p dir="ltr">These capabilities will involve distributed processing, with headsets handling perception and rendering, while generative AI models run on paired smartphones or in the cloud. In a few years, text-to-3D and image-to-3D models will likely produce high-quality 3D objects, evolving to generate entire scenes and rooms. Eventually, XR may enable users to immerse themselves in virtual worlds constructed from their imagination. Virtual avatars will undergo a similar evolution, utilizing text-to-text models for natural conversations and text-to-image models for outfits. Over time, avatars may become increasingly photorealistic, fully animated, and mass-producible, driven by image-to-3D and encoder/decoder models. This progression promises to revolutionize how we create and experience immersive XR content, with continual advancements anticipated in the years ahead.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjUzMTYyMjIyLUNhbm5lcy1YUi1WaXJ0dWFsLUJyaW5ncy1JbW1lcnNpdmUtQXJ0LXRvLUxpZmUtV2l0aC1WaXJ0dWFsLVJlYWxpdHkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Image credit: Cannes</span></span></span></p><h2 dir="ltr">Challenges and Solutions</h2><p dir="ltr">The technical challenges specific to hybrid AI are being actively addressed through ongoing research and innovation:</p><h3 dir="ltr">Model size</h3><p>Model size is a notable challenge, particularly on resource-constrained devices. As large models are challenging to store and run efficiently, a potential solution involves employing compression and quantization techniques to reduce model size while maintaining accuracy. <a href="https://stackoverflow.blog/2023/08/23/fitting-ai-models-in-your-pocket-with-quantization/" target="_blank">Qualcomm</a> is actively researching ways to enhance <a href="https://www.qualcomm.com/news/onq/2019/03/heres-why-quantization-matters-ai" target="_blank">quantization</a> techniques, specifically aiming to quantize 32-bit floating point parameters to 8-bit or 4-bit integers in neural networks while maintaining accuracy. The research encompasses various model quantization methods, such as Bayesian learning, adaptive rounding, and transformer quantization, as part of <a href="https://www.qualcomm.com/research/artificial-intelligence/ai-research" target="_blank">Qualcomm AI Research</a>'s ongoing efforts.&nbsp;</p><p>For instance, the Qualcomm AI Research team employed the <a href="https://www.forbes.com/sites/davealtavilla/2023/05/15/powering-ai-on-mobile-devices-requires-new-math-and-qualcomm-is-pioneering-it/?sh=54cb40c91a16" target="_blank">AIMET</a> tool (AI Model Efficiency Toolkit) to quantize the 32-bit floating point Stable Diffusion AI model into an 8-bit integer format while preserving almost the same accuracy level. By optimizing and scaling the model for INT8 precision operations instead of FP32, Qualcomm successfully enabled Stable Diffusion to run efficiently on handheld devices without internet connectivity, significantly reducing the model's size while maintaining accuracy and enhancing performance on low-power Snapdragon AI accelerators.</p><h3>Computational power</h3><p>Large AI models may place heavy computational demands, particularly on devices with restricted processing capabilities. A potential solution involves leveraging specialized hardware like AI accelerators to enhance AI model performance on such devices. In 2022, Snapdragon 8 Gen 2 provided groundbreaking AI, integrated across the entire system, powered by their fastest, most advanced Qualcomm AI Engine to date. Users can experience faster natural language processing with multi-language translation or have fun with AI cinematic video capture.&nbsp;</p><p>The latest Hexagon Processor introduced a dedicated power delivery system, which adapts power according to the workload. Special hardware improves group convolution, activation function acceleration, and the performance of the Hexagon Tensor Accelerator. Support for micro-tile inferencing and INT4 hardware acceleration provide even higher performance while reducing power and memory traffic. The transformer acceleration dramatically speeds up inference for multi-head attention that is used throughout generative AI, resulting in AI performance up to 4.35X on certain use cases with MobileBERT.&nbsp;</p><p dir="ltr">These ongoing research efforts are addressing the technical challenges of hybrid AI, with expectations to overcome them successfully as the technology advances and becomes more widely adopted and utilized.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">On-device AI plays a critical role in addressing the future scalability challenges of generative AI models. It has the potential to revolutionize numerous industries and significantly enhance user experiences. By enabling local processing and real-time personalization, on-device AI reduces the dependence on cloud resources, improves privacy and security as well as the efficiency of AI applications. This approach not only supports the responsible development of AI technologies but also opens new possibilities for scalable and personalized AI solutions across diverse sectors, from healthcare to entertainment, promising a brighter future for both businesses and users alike.</p><p>To stay up to date on the latest from Qualcomm, <a href="https://assets.qualcomm.com/mobile-computing-newsletter-sign-up.html" target="_blank">join the “What’s Next in AI and Computing” newsletter</a> and read their hybrid AI whitepapers (<a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 1</a>, <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-2-Qualcomm-is-uniquely-positioned-to-scale-hybrid-AI.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">part 2</a>)</p>

How on-device and cloud AI will foster the advancement and scalability of next-generation generative AI experiences

The Scalability Challenge: Focusing on Hybrid AI in the Future of Generative AI Models

In this episode, we discuss how the popularity of generative AI has led to the emergence of prompt engineering which begs the question: what is prompt engineering?

Podcast: Prompt Engineering: Latest Meme or The Next Big Thing?

Caracol

nTopology-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

nTopology-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

nTopology-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

nTopology-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

nTopology-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Arduino Pro

Georgia Institute of Technology-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Georgia Institute of Technology-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Georgia Institute of Technology-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Georgia Institute of Technology-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Georgia Institute of Technology-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

generative ai

Generative AI

Profiles