To build AI systems that can collaborate effectively with humans, it helps to have a good model of human behavior to start with. But humans tend to behave suboptimally when making decisions.This irrationality, which is especially difficult to model, often boils down to computational constraints. A human can’t spend decades thinking about the ideal solution to a single problem.Researchers at MIT and the University of Washington developed a way to model the behavior of an agent, whether human or machine, that accounts for the unknown computational constraints that may hamper the agent’s problem-solving abilities.Their model can automatically infer an agent’s computational constraints by seeing just a few traces of their previous actions. The result, an agent’s so-called “inference budget,” can be used to predict that agent’s future behavior.In a new paper, the researchers demonstrate how their method can be used to infer someone’s navigation goals from prior routes and to predict players’ subsequent moves in chess matches. Their technique matches or outperforms another popular method for modeling this type of decision-making.Ultimately, this work could help scientists teach AI systems how humans behave, which could enable these systems to respond better to their human collaborators. Being able to understand a human’s behavior, and then to infer their goals from that behavior, could make an AI assistant much more useful, says Athul Paul Jacob, an electrical engineering and computer science (EECS) graduate student and lead author of a&nbsp;<a href="https://openreview.net/pdf?id=W3VsHuga3j" target="_blank">paper on this technique</a>.“If we know that a human is about to make a mistake, having seen how they have behaved before, the AI agent could step in and offer a better way to do it. Or the agent could adapt to the weaknesses that its human collaborators have. Being able to model human behavior is an important step toward building an AI agent that can actually help that human,” he says.Jacob wrote the paper with Abhishek Gupta, assistant professor at the University of Washington, and senior author Jacob Andreas, associate professor in EECS and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research will be presented at the International Conference on Learning Representations.<h2>Modeling behavior</h2>Researchers have been building computational models of human behavior for decades. Many prior approaches try to account for suboptimal decision-making by adding noise to the model. Instead of the agent always choosing the correct option, the model might have that agent make the correct choice 95 percent of the time.However, these methods can fail to capture the fact that humans do not always&nbsp;behave suboptimally in the same way.Others at MIT have also&nbsp;<a href="https://news.mit.edu/2020/building-machines-better-understand-human-goals-1214" target="_blank">studied more effective ways</a> to plan and infer goals in the face of suboptimal decision-making.To build their model, Jacob and his collaborators drew inspiration from prior studies of chess players. They noticed that players took less time to think before acting when making simple moves and that stronger players tended to spend more time planning than weaker ones in challenging matches.“At the end of the day, we saw that the depth of the planning, or how long someone thinks about the problem, is a really good proxy of how humans behave,” Jacob says.They built a framework that could infer an agent’s depth of planning from prior actions and use that information to model the agent’s decision-making process.The first step in their method involves running an algorithm for a set amount of time to solve the problem being studied. For instance, if they are studying a chess match, they might let the chess-playing algorithm run for a certain number of steps. At the end, the researchers can see the decisions the algorithm made at each step.Their model compares these decisions to the behaviors of an agent solving the same problem. It will align the agent’s decisions with the algorithm’s decisions and identify the step where the agent stopped planning.From this, the model can determine the agent’s inference budget, or how long that agent will plan for this problem. It can use the inference budget to predict how that agent would react when solving a similar problem.<h2>An interpretable solution</h2>This method can be very efficient because the researchers can access the full set of decisions made by the problem-solving algorithm without doing any extra work. This framework could also be applied to any problem that can be solved with a particular class of algorithms.“For me, the most striking thing was the fact that this inference budget is very interpretable. It is saying tougher problems require more planning or being a strong player means planning for longer. When we first set out to do this, we didn’t think that our algorithm would be able to pick up on those behaviors naturally,” Jacob says.The researchers tested their approach in three different modeling tasks: inferring navigation goals from previous routes, guessing someone’s communicative intent from their verbal cues, and predicting subsequent moves in human-human chess matches.Their method either matched or outperformed a popular alternative in each experiment. Moreover, the researchers saw that their model of human behavior matched up well with measures of player skill (in chess matches) and task difficulty.Moving forward, the researchers want to use this approach to model the planning process in other domains, such as reinforcement learning (a trial-and-error method commonly used in robotics). In the long run, they intend to keep building on this work toward the larger goal of developing more effective AI collaborators.This work was supported, in part, by the MIT Schwarzman College of Computing Artificial Intelligence for Augmentation and Productivity program and the National Science Foundation.

MIT and other researchers developed a framework that models irrational or suboptimal behavior of a human or AI agent, based on their computational constraints. Their technique can help predict an agent’s future actions, for instance, in chess matches. Image: iStock

A new technique can be used to predict the actions of human or AI agents who behave suboptimally while working toward unknown goals.

To build a better AI helper, start by modeling the irrational behavior of humans

Researchers have developed a new training tool to help artificial intelligence (AI) programs better account for the fact that humans don’t always tell the truth when providing personal information. The new tool was developed for use in contexts when humans have an economic incentive to lie, such as applying for a mortgage or trying to lower their insurance premiums.“AI programs are used in a wide variety of business contexts, such as helping to determine how large of a mortgage an individual can afford, or what an individual’s insurance premiums should be,” says Mehmet Caner, co-author of a paper on the work. “These AI programs generally use mathematical algorithms driven solely by statistics to do their forecasting. But the problem is that this approach creates incentives for people to lie, so that they can get a mortgage, lower their insurance premiums, and so on.“We wanted to see if there was some way to adjust AI algorithms in order to account for these economic incentives to lie,” says Caner, who is the Thurman-Raytheon Distinguished Professor of Economics in North Carolina State University’s Poole College of Management.To address this challenge, the researchers developed a new set of training parameters that can be used to inform how the AI teaches itself to make predictions. Specifically, the new training parameters focus on recognizing and accounting for a human user’s economic incentives. In other words, the AI trains itself to recognize circumstances in which a human user might lie to improve their outcomes.In proof-of-concept simulations, the modified AI was better able to detect inaccurate information from users.“This effectively reduces a user’s incentive to lie when submitting information,” Caner says. “However, small lies can still go undetected. We need to do some additional work to better understand where the threshold is between a ‘small lie’ and a ‘big lie.’”The researchers are making the new AI training parameters publicly available, so that AI developers can experiment with them.“This work shows we can improve AI programs to reduce economic incentives for humans to lie,” Caner says. “At some point, if we make the AI clever enough, we may be able to eliminate those incentives altogether.”The paper, “<a href="https://www.tandfonline.com/doi/full/10.1080/07350015.2024.2316102" data-type="link" data-id="https://www.tandfonline.com/doi/full/10.1080/07350015.2024.2316102" target="_blank" rel="noreferrer noopener">Should Humans Lie to Machines? The Incentive Compatibility of Lasso and GLM Structured Sparsity Estimators</a>,” is published in the Journal of Business &amp; Economic Statistics. The paper was co-authored by Kfir Eliaz of Tel-Aviv University and the University of Utah.<hr>“Should Humans Lie to Machines? The Incentive Compatibility of Lasso and GLM Structured Sparsity Estimators”Authors: Mehmet Caner, North Carolina State University; and Kfir Eliaz, Tel-Aviv University and the University of UtahPublished: March 12,&nbsp;Journal of Business &amp; Economic StatisticsDOI: 10.1080/07350015.2024.2316102

Researchers have developed a new training tool to help artificial intelligence (AI) programs better account for the fact that humans don’t always tell the truth when providing personal information.

New Technique Helps AI Tell When Humans Are Lying

With 70% of U.S. companies expected to adopt AI by 2030, organizations need to understand and address the key challenges of employee-AI interaction. To shine light on the future of AI in the workplace, <a href="https://engineering.cmu.edu/directory/bios/mccomb-christopher.html" target="_blank">Chris McComb</a> and <a href="https://engineering.cmu.edu/directory/bios/cagan-jonathan.html" target="_blank">Jon Cagan</a> put human and AI-human “hybrid" teams to the test. In a head-to-head matchup, researchers studied which team could better design a fleet of delivery drones under dynamic design constraints.“We compared teams of five humans against hybrid teams of three humans and two AI teammates,” explained McComb, associate professor of mechanical engineering. “We wanted to understand failure modes in human-AI teams, so midway through the experiment we added some drastic changes that the AI agents couldn’t see.”&nbsp;The study was conducted on the online platform, HyForm, developed by researchers at CMU and Penn State University, so that while teams worked to maximize the profit of their drone delivery service, researchers could monitor both the teams’ communications and the success of their fleets.At the end of the day, the human-AI hybrid teams performed just as well as the human teams, demonstrating the ability of hybrid teams to adapt to unexpected events.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1707828962738-0130-em-human-ai-teams.png" style="width: 766px;" class="fr-fic fr-dib">HyForm, an online platform, enabled teams to work remotely on their drone delivery serviceResearchers found that overall, the human teams communicated significantly more than the hybrid teams until the unplanned-to-the-user constraints were introduced. Communication within the human-AI teams nearly tripled from that point forward. Much of this increase was due to communication between humans and their AI teammates, indicating that some human members of the team took on the role of “AI handler” to keep the AI agents on track.&nbsp;Despite this success, members of the hybrid teams felt that they did not operate cohesively or effectively.“It’s important for AI adopters to see that the flexibility of human-AI teams can come from the human side of the partnership,” said Cagan, department head and professor of <a href="https://www.meche.engineering.cmu.edu/index.html" rel="noopener" target="_blank">mechanical engineering</a>. “We don’t need to develop a perfectly adaptive AI partner.”<blockquote>Humans are an essential part of human + AI teaming. Future engineers need to be trained to work effectively with AI agents.<cite>Chris McComb<cite id="isPasted">Associate Professor<cite>, Mechanical Engineering</cite></cite> </cite></blockquote>“The bottom line here is that humans are an essential part of human + AI teaming, and future engineers need to be trained to work effectively with AI agents,” emphasized McComb.

New research from Carnegie Mellon University’s Department of Mechanical Engineering and the Human+AI Design Initiative underlines the adage “teamwork makes the dream work,” especially when it comes to human-AI collaboration.

Head-to-head: Human vs. AI-human teams

<h2 dir="ltr" id="isPasted">Challenge Overview</h2>The Build2gether Challenge was designed to address gaps in assistive technologies, focusing on enhancing the daily lives of individuals with disabilities. Participants were called upon to develop cutting-edge solutions across three specific themes: gaming and travelling for people with mobility impairments, and swimming for people with visual impairments.&nbsp;<h2 dir="ltr">Spotlight on the Ultimate Winner: Bob Hammell</h2>Among the various outstanding submissions, Bob Hammell's projects stood out, earning him the Inclusive Innovation Challenge Ultimate Award. Hammell presented three distinct projects, each targeting a different theme of the challenge:<ul><li dir="ltr">Underwater Sound Beacons for Visually Impaired Swimmers: An innovative solution utilizing audio cues to help visually impaired swimmers navigate safely in pools. <img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAzODYxMDEzOTgyLTE3MDM4NjEwMTM5ODIucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="602" height="451" class="fr-fic fr-dii">Image credit: Bob Hammell</li><li dir="ltr">Voice-Enabled Video Game Controller: A controller designed for gamers with limited mobility, allowing voice commands to serve as game inputs. <img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAzODYxMDEzODEwLTE3MDM4NjEwMTM4MTAucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="602" height="451" class="fr-fic fr-dii">Image credit: Bob Hammell</li><li dir="ltr">Interactive Display for Finding Wheelchair-Accessible Places: A touch-enabled display aiding mobility-impaired individuals in locating wheelchair-accessible facilities while travelling. <img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAzODYxMDEzOTAxLTE3MDM4NjEwMTM5MDEucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" width="602" height="451" class="fr-fic fr-dii">Image credit: Bob Hammell</li></ul>Each project demonstrated a deep understanding of the challenges faced by individuals with disabilities and offered practical, innovative solutions.<h2 dir="ltr">More Winners and Awards</h2>In addition to Hammell's achievements, the Build2gether Challenge recognized numerous other contributions across its varied themes. These awards highlighted the diverse range of innovations, each contributing significantly to the field of assistive technologies. <a href="https://www.hackster.io/contests/buildtogether" target="_blank" class="button-main-action">Learn more</a><h2 dir="ltr">Discover More</h2>The Build2gether Inclusive Innovation Challenge has set a new benchmark in technological advancements for disability support. For a complete overview of the challenge, the winners, and their groundbreaking projects, interested individuals are encouraged to <a href="https://www.hackster.io/contests/buildtogether" target="_blank">visit this page&nbsp;</a>for detailed information and insights into these remarkable innovations.

The Build2gether Inclusive Innovation Challenge has successfully concluded with the announcement of its winners. This challenge represented a collaborative effort to push the boundaries of innovation in creating solutions for people with disabilities.


Build2gether Inclusive Innovation Challenge: Winners Announced

This article provides a detailed overview of the Modbus TCP protocol fundamentals, communication, data representation, security, daily life applications, and integration.

Modbus TCP: A Comprehensive Guide to the Protocol and Its Applications

The language model ChatGPT is widely known by people in the tech community and the general public. Through a chatbot, one can interact with these language models directly without needing any programming skills. Instead of using code, the user can input commands and questions with natural language.Sometimes this interaction works well, and the desired result appears on-screen. However, at other times the language model does not understand the command, and the generated output can turn out unexpectedly or even unsatisfactory. What people usually do as a response is that they follow up with another query. ChatGPT will then try to correct its mistakes and adapt its answer. This way of using a language model is messy and random, and it can take quite some time to get the preferred outcome.To solve this problem, ETH researchers now developed a new programming language and open-source platform called LMQL (Language Model Query Language). Utilizing LMQL allows a user to interact with large language models like ChatGPT in a more elevated and controlled way. This programming language enables a novel way of programming and is a new form of computer-human interaction because the user can directly talk to and instruct the computer.LMQL is the first language that combines natural and programming languages&rsquo; power to interact with these large language models. For simple queries, it is sufficient to guide ChatGPT using natural language. However, for more complex and specific tasks, such as creating a database or analyzing data, it is essential to instruct the language model precisely. Therefore, the formalism of programming languages is needed to guide the language model with formal constructs to ensure the user gets the desired output. Martin Vechev, Professor of Computer Science and one of the creators, clarifies: &ldquo;Essentially, it is a much more concise way to get what you want. Decreasing the necessary exchanges with the language model also reduces the costs of interacting with the model, which can be quite expensive. Using LMQL increases the chances of getting the desired output. It sometimes even makes it possible to get a result you would have never gotten otherwise because you can formulate your query more accurately.&rdquo;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1682690936421-image.imageformat.1286.1170780518.png" style="width: 766px;" class="fr-fic fr-dib">LMQL allows the user to combine natural and programming languages when formulating a query. (Image: SRI Lab / ETH Zurich)&nbsp;<h2 id="isPasted">A key advantage</h2>The datasets that large language models are trained with and based on are so massive that the user cannot control and comprehend what happens internally within the model. Because of that, these models can sometimes produce unexpected or controversial outputs. According to the researchers, one of the substantial problems for people using these large language models is that they cannot understand why a certain result was produced and how to prevent it.LMQL allows its user to express safety constraints which can help guide the model in the right direction and try to steer it away from unwanted or unexpected outputs. &ldquo;Using LMQL, you can restrict your language model to strictly follow a specific framework you designed. This allows you to better control how the language model behaves. Of course, full guaranteed prevention of bad behaviour is still very hard to achieve, but LMQL is one step in this direction&rdquo;, explains Luca Beurer-Kellner, one of the researchers. For example, LMQL allows the user to exclude specific words or prevents the model from going in certain directions of reasoning.<h2>Transparency is key</h2>Many companies develop their large language models behind closed doors. Due to that, the large language models become intransparent, and their reasoning behind a particular output is incomprehensible to the user. According to the researchers, to counterbalance this, academia must produce open-source tools like LMQL that are transparent, accessible, and adaptable for people. Marc Fischer, who also developed the new programming language, further declares: &ldquo;I think for both a technical and non-technical crowd, it is crucial to have this open-source to see what is going on rather than LMQL being a magical black box. Especially since the research on language models is moving at an insane pace, it is crucial that LMQL simultaneously offers transparency and enables fast development.&rdquo;<h2>A helpful tool even for less experienced users</h2>LMQL is a declarative, SQL-like language from a syntactic point of view. Therefore, it is a very accessible language requiring less expertise to achieve the desired results. The new programming language can also function as an innovative tool for researchers of various disciplines. &ldquo;If you are not too invested in coding or may not have the time to code because it is not a core part of your work, then LMQL makes it much more accessible to interact with large language models in a precise, yet easy way&rdquo;, explains Beurer-Kellner.Additionally, it can function as a helpful base for advanced users and an expert community because you can add different programming constructs to the natural language query. Technical programmer users can use LMQL as a building block and build their own programs on top to interact with large language models.A vast interest and interdisciplinary community is already beginning to form around the new programming language. The researchers declare LMQL a long-term project and plan several follow-ups and papers. They were also accepted to present their work in June at the ACM PLDI, one of the top international conferences on programming language design and implementation.

The interaction with large language models is cheaper and more reliable through LMQL. (Image: SRI Lab / ETH Zurich)

Researchers from ETH Zurich created a new open-​source platform and programming language called LMQL. This programming language makes it easier, cheaper, and safer to interact with large language models like ChatGPT.

A new form of human-computer interaction

In this episode, we talk about how perfectly tuning the curiosity of an AI results in more efficient and robust models. &nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/06e95808-1e8a-4962-a0ee-f1bd3d7bf323?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe><hr>This podcast is sponsored by <a href="https://www2.mouser.com/" id="isPasted" rel="nofollow" target="_blank">Mouser Electronics</a>. &nbsp; &nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1670339698469-Mouser-sponsor-1-a.jpg" style="width: 766px;" class="fr-fic fr-dib"><hr><h3 id="isPasted">EPISODE NOTES</h3>(5:20) -&nbsp;<a href="https://www.wevolver.com/article/ensuring-ai-works-with-the-right-dose-of-curiosity" target="_blank">Ensuring AI Works With The Right Dose of Curiosity</a>This episode was brought to you by Mouser, our favorite place to get electronic components for any project. Click <a href="https://resources.mouser.com/artificial-intelligence/ai-can-code-now" target="_blank">HERE</a> to learn more about the copilot and GPT AI models discussed in this episode!<hr>As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.To learn more about this show, please visit our<a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">&nbsp;shows page</a>. By following the page, you will get automatic updates by email when a new show is published. Be sure to give us a follow and review on<a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" rel="nofollow" target="_blank">&nbsp;Apple&nbsp;</a>podcasts,<a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" rel="nofollow" target="_blank">&nbsp;Spotify</a>, and most of your favorite podcast platforms!

In this episode, we talk about how perfectly tuning the curiosity of an AI results in more efficient and robust models. 

Podcast: More Curious AI To Save The World

Article #5 of the Enabling IoT Series. MEMS microphones can deliver high-quality sound in an ultra-small form-factor. 


High SNR microphones are transforming laptops into all-round communication hubs

This article was discussed in our <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1670426207448000&usg=AOvVaw1x-b-kkAmH54ptZfmv08qg" href="https://www.wevolver.com/profile/the.next.byte" id="isPasted" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/06e95808-1e8a-4962-a0ee-f1bd3d7bf323?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>The full article will continue below.<hr>This article was first published on <a href="https://www.csail.mit.edu/news/ensuring-ai-works-right-dose-curiosity" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>. &nbsp;<hr>It&rsquo;s a dilemma as old as time. Friday night has rolled around, and you&rsquo;re trying to pick a restaurant for dinner. Should you visit your most beloved watering hole or try a new establishment, in the hopes of discovering something superior? Potentially, but that curiosity comes with a risk: If you explore the new option, the food could be worse. On the flip side, if you stick with what you know works well, you won&#39;t grow out of your narrow pathway. &nbsp;Curiosity drives artificial intelligence to explore the world, now in boundless use cases &mdash; autonomous navigation, robotic decision-making, optimizing health outcomes, and more. Machines, in some cases, use &ldquo;reinforcement learning&rdquo; to accomplish a goal, where an AI agent iteratively learns from being rewarded for good behavior and punished for bad. Just like the dilemma faced by humans in selecting a restaurant, these agents also struggle with balancing the time spent discovering better actions (exploration) and the time spent taking actions that led to high rewards in the past (exploitation). Too much curiosity can distract the agent from making good decisions, while too little means the agent will never discover good decisions.In the pursuit of making AI agents with just the right dose of curiosity, researchers from MIT&rsquo;s Improbable AI Laboratory and Computer Science and Artificial Intelligence Laboratory (CSAIL) created an algorithm that overcomes the problem of AI being too &ldquo;curious&rdquo; and getting distracted by a given task. Their algorithm automatically increases curiosity when it&#39;s needed, and suppresses it if the agent gets enough supervision from the environment to know what to do.When tested on over 60 video games, the algorithm was able to succeed at both hard and easy exploration tasks, where previous algorithms have only been able to tackle only a hard or easy domain alone. With this method, AI agents use fewer data for learning decision-making rules that maximize incentives. &nbsp;&ldquo;If you master the exploration-exploitation trade-off well, you can learn the right decision-making rules faster &mdash; and anything less will require lots of data, which could mean suboptimal medical treatments, lesser profits for websites, and robots that don&#39;t learn to do the right thing,&rdquo; says Pulkit Agrawal, an assistant professor of electrical engineering and computer science (EECS) at MIT, director of the Improbable AI Lab, and CSAIL affiliate who supervised the research. &ldquo;Imagine a website trying to figure out the design or layout of its content that will maximize sales. If one doesn&rsquo;t perform exploration-exploitation well, converging to the right website design or the right website layout will take a long time, which means profit loss. Or in a health care setting, like with Covid-19, there may be a sequence of decisions that need to be made to treat a patient, and if you want to use decision-making algorithms, they need to learn quickly and efficiently &mdash; you don&#39;t want a suboptimal solution when treating a large number of patients. We hope that this work will apply to real-world problems of that nature.&rdquo;&nbsp;It&rsquo;s hard to encompass the nuances of curiosity&rsquo;s psychological underpinnings; the underlying neural correlates of challenge-seeking behavior are a poorly understood phenomenon. Attempts to categorize the behavior have spanned studies that dived deeply into studying our impulses, deprivation sensitivities, and social and stress tolerances.&nbsp;With reinforcement learning, this process is &ldquo;pruned&rdquo; emotionally and stripped down to the bare bones, but it&rsquo;s complicated on the technical side. Essentially, the agent should only be curious when there&rsquo;s not enough supervision available to try out different things, and if there is supervision, it must adjust curiosity and lower it.&nbsp;Since a large subset of gaming is little agents running around fantastical environments looking for rewards and performing a long sequence of actions to achieve some goal, it seemed like the logical test bed for the researchers&rsquo; algorithm. In experiments, researchers divided games like &ldquo;Mario Kart&rdquo; and &ldquo;Montezuma&rsquo;s Revenge&rdquo; into two different buckets: one where supervision was sparse, meaning the agent had less guidance, which were considered &ldquo;hard&rdquo; exploration games, and a second where supervision was more dense, or the &ldquo;easy&rdquo; exploration games.&nbsp;Suppose in &ldquo;Mario Kart,&rdquo; for example, you only remove all rewards so you don&rsquo;t know when an enemy eliminates you. You&rsquo;re not given any reward when you collect a coin or jump over pipes. The agent is only told in the end how well it did. This would be a case of sparse supervision. Algorithms that incentivize curiosity do really well in this scenario.&nbsp;But now, suppose the agent is provided dense supervision &mdash; a reward for jumping over pipes, collecting coins, and eliminating enemies. Here, an algorithm without curiosity performs really well because it gets rewarded often. But if you instead take the algorithm that also uses curiosity, it learns slowly. This is because the curious agent might attempt to run fast in different ways, dance around, go to every part of the game screen &mdash; things that are interesting, but do not help the agent succeed at the game. The team&rsquo;s algorithm, however, consistently performed well, irrespective of what environment it was in.&nbsp;Future work might involve circling back to the exploration that&rsquo;s delighted and plagued psychologists for years: an appropriate metric for curiosity &mdash; no one really knows the right way to mathematically define curiosity.&nbsp;&ldquo;Getting consistent good performance on a novel problem is extremely challenging &mdash; so by improving exploration algorithms, we can save your effort on tuning an algorithm for your problems of interest, says Zhang-Wei Hong, an EECS PhD student, CSAIL affiliate, and co-lead author along with Eric Chen &rsquo;20, MEng &rsquo;21 on a new paper about the work. &ldquo;We need curiosity to solve extremely challenging problems, but on some problems it can hurt performance. We propose an algorithm that removes the burden of tuning the balance of exploration and exploitation. Previously what took, for instance, a week to successfully solve the problem, with this new algorithm, we can get satisfactory results in a few hours.&rdquo;&ldquo;One of the greatest challenges for current AI and cognitive science is how to balance exploration and exploitation &mdash; the search for information versus the search for reward. Children do this seamlessly, but it is challenging computationally,&rdquo; notes Alison Gopnik, professor of psychology and affiliate professor of philosophy at the University of California at Berkeley, who was not involved with the project. &ldquo;This paper uses impressive new techniques to accomplish this automatically, designing an agent that can systematically balance curiosity about the world and the desire for reward, [thus taking] another step towards making AI agents (almost) as smart as children.&rdquo;&ldquo;Intrinsic rewards like curiosity are fundamental to guiding agents to discover useful diverse behaviors, but this shouldn&rsquo;t come at the cost of doing well at the given task. This is an important problem in AI, and the paper provides a way to balance that trade-off,&rdquo; adds Deepak Pathak, an assistant professor at Carnegie Mellon University, who was also not involved in the work. &ldquo;It would be interesting to see how such methods scale beyond games to real-world robotic agents.&rdquo;Chen, Hong, and Agrawal wrote the paper alongside Joni Pajarinen, assistant professor at Aalto University and research leader at the Intelligent Autonomous Systems Group at TU Darmstadt. The research was supported, in part, by the MIT-IBM Watson AI Lab, DARPA Machine Common Sense Program, the Army Research Office by the United States Air Force Research Laboratory, and the United States Air Force Artificial Intelligence Accelerator. The paper will be presented at Neural Information and Processing Systems (NeurIPS) 2022.<hr>&quot;Reprinted with permission of <a href="https://www.csail.mit.edu/news/ensuring-ai-works-right-dose-curiosity" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>&quot;&nbsp;

Curiosity drives artificial intelligence to explore the world, now in boundless use cases — autonomous navigation, robotic decision-making, optimizing health outcomes, and more.

Ensuring AI works with the right dose of curiosity

Matroid

RAD Lab-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

RAD Lab-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

RAD Lab-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

RAD Lab-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

RAD Lab-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Matroid-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Matroid-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Matroid-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Matroid-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Matroid-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Xsens-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Movella-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Movella-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Movella-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Movella-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Movella-Thu Oct 05 2023 18:51:53 GMT+0300 (Eastern European Summer Time)

Movella-Thu Oct 05 2023 18:51:58 GMT+0300 (Eastern European Summer Time)

Movella-Mon Oct 30 2023 14:34:24 GMT+0200 (Eastern European Standard Time)

Movella-Tue Oct 31 2023 23:59:59 GMT+0200 (Eastern European Standard Time)

human computer interaction

Human-Computer Interaction

Profiles