We are tasking our computers with processing ever-increasing amounts of data to speed up drug discovery, improve weather and climate predictions, train artificial intelligence, and much more. To keep up with this demand, we need faster, more energy-efficient computer memory than ever before.Researchers at Stanford have demonstrated that a new material may make phase-change memory – which relies on switching between high and low resistance states to create the ones and zeroes of computer data – an improved option for future AI and data-centric systems. Their scalable technology, as detailed recently in <a href="https://www.nature.com/articles/s41467-023-42792-4" data-extlink="" target="_blank">Nature Communications</a>, is fast, low-power, stable, long-lasting, and can be fabricated at temperatures compatible with commercial manufacturing.“We are not just improving on a single metric, such as endurance or speed; we are improving several metrics simultaneously,” said <a href="https://profiles.stanford.edu/epop?releaseVersion=10.5.2" data-extlink="" target="_blank">Eric Pop</a>, the Pease-Ye Professor of Electrical Engineering and professor, by courtesy, of materials science and engineering at Stanford. “This is the most realistic, industry-friendly thing we’ve built in this sphere. I’d like to think of it as a step towards a universal memory.”<h2>A faster nonvolatile memory</h2>Today’s computers store and process data in separate locations. Volatile memory – which is fast but disappears when your computer turns off – handles the processing, while nonvolatile memory – which isn’t as fast but can hold information without constant power input – takes care of the long-term data storage. Shifting information between these two locations can cause bottlenecks while the processor waits for large amounts of data to be retrieved.“It takes a lot of energy to shuttle data back and forth, especially with today’s computing workloads,” said <a href="https://profiles.stanford.edu/xiangjin-wu" data-extlink="" target="_blank">Xiangjin Wu</a>, co-lead author on the paper and a doctoral candidate co-advised by Pop and <a href="https://profiles.stanford.edu/philip-wong" data-extlink="" target="_blank">Philip Wong</a>, the Willard R. and Inez Kerr Bell Professor in the School of Engineering. “With this type of memory, we’re really hoping to bring the memory and processing closer together, ultimately into one device, so that it uses less energy and time.”There are many technical hurdles to achieving an effective, commercially viable universal memory capable of both long-term storage and fast, low-power processing without sacrificing other metrics, but the new phase change memory developed in Pop’s lab is as close as anyone has come so far with this technology. The researchers hope that it will inspire further development and adoption as a universal memory.The memory relies on GST467, an alloy of four parts germanium, six parts antimony, and seven parts tellurium, which was developed by collaborators at the University of Maryland. Pop and his colleagues found ways to sandwich the alloy between several other nanometer-thin materials in a superlattice, a layered structure they’ve previously used to achieve good nonvolatile memory results.“The unique composition of GST467 gives it a particularly fast switching speed,” said <a href="https://profiles.stanford.edu/asir" data-extlink="" target="_blank">Asir Intisar Khan</a>, who earned his doctorate in Pop’s lab and is co-lead author on the paper. “Integrating it within the superlattice structure in nanoscale devices enables low switching energy, gives us good endurance, very good stability, and makes it nonvolatile – it can retain its state for 10 years or longer.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1709684449864-computer_memory_1.png.png" style="width: 100%px;" class="fr-fic fr-dib">Cross-sections of phase-change memory devices in the high- and low-resistance states. The diameter of the bottom electrode is ~40 nanometers. Arrows mark some of the van der Waals (vdW) interfaces, which form between layers of the superlattice materials. The superlattice is disrupted and reformed between the high- and low-resistance states. | Image courtesy of the Pop Lab<h2 id="isPasted">Setting a new bar</h2>The GST467 superlattice clears several important benchmarks. Phase change memory can sometimes drift over time – essentially the value of the ones and zeros can slowly shift – but their tests show that this memory is extremely stable. It also operates at below 1 volt, which is the goal for low-power technology, and is significantly faster than a typical solid-state drive.“A few other types of nonvolatile memory can be a bit faster, but they operate at higher voltage or higher power,” Pop said. “With all these computing technologies, there are tradeoffs between speed and energy. The fact that we’re switching at a few tens of nanoseconds while operating below one volt is a big deal.”The superlattice also packs a good amount of memory cells into a small space. The researchers have shrunk the memory cells down to 40 nanometers in diameter – less than half the size of a coronavirus. That’s not quite as dense as it could be, but the researchers are exploring ways to compensate by stacking the memory in vertical layers, which is possible thanks to the superlattice’s low fabrication temperature and the techniques used to create it.“The fabrication temperature is well below what you need,” Pop said. “People are talking about stacking memory in thousands of layers to increase density. This type of memory can enable such future 3D layering.”Pop is a member of&nbsp;<a href="https://systemx.stanford.edu/" data-extlink="" target="_blank">Stanford SystemX Alliance</a>&nbsp;and an affiliate of&nbsp;<a href="https://slac.stanford.edu/" data-extlink="" target="_blank">SLAC</a>&nbsp;and the&nbsp;<a href="https://energy.stanford.edu/" data-extlink="" target="_blank">Precourt Institute for Energy</a>. Wong is a professor of electrical engineering, a member of&nbsp;<a href="http://biox.stanford.edu/" data-extlink="" target="_blank">Stanford Bio-X</a>, the&nbsp;<a href="https://neuroinstitute.stanford.edu/" data-extlink="" target="_blank">Wu Tsai Neurosciences Institute</a>, and an affiliate of the&nbsp;<a href="https://energy.stanford.edu/" data-extlink="" target="_blank">Precourt Institute for Energy</a>.Additional co-authors are from Taiwan Semiconductor Manufacturing Company, the National Institute of Standards and Technology, Theiss Research Inc, University of Maryland, and Tianjin University.

Researchers at Stanford have demonstrated that a new material may make phase-change memory an improved option for future AI and data-centric systems. | Image courtesy of Vitalii Pasichnyk/iStock

Stanford researchers have developed a new phase-change memory that could help computers process large amounts of data faster and more efficiently.

New candidate for universal memory is fast, low-power, stable, and long-lasting

In this episode, we talk about how a driven undergraduate student from Columbia University mapped fingerprints from different fingers of the same person - which was widely known to be impossible by the experts in the field - using publicly available data and machine learning.

Podcast: Matching "Unmatchable" Fingerprints

Capping lines in modern factories are machines of incredible speed and precision, capable of installing thousands of caps on bottles and containers every hour. Based in Canelli, Italy, <a href="https://www.arol.com/" target="_blank" rel="noopener noreferrer">AROL</a> is a leading provider of innovative and custom capping and closure system machines, with over 25,000 pieces of equipment currently installed around the world. With production lines processing up to 100,000 bottles per hour, maintaining their machines’ speed and ongoing reliability is critical to operations. In other words, any outage or unplanned downtime would be detrimental to meeting production quotas and costly to their clients.<h3>Adding Brains to Speed for Improved Productivity</h3>To support diagnostic, prognostic and predictive maintenance, they decided to draw on their experience to develop a smart solution able to gather data from industrial equipment and the surrounding environment, creating a machine monitoring system based on remote wireless sensors that could be powered by battery or 24 V. The project faced specific challenges in the fact sensor nodes had to be mounted in rotating capping turrets, fitting in a small space and yet always remaining connected to a communication unit at the base of the line. What’s more, they had to perform part of the data processing on the edge, in order to not overload the central system. And of course, the upgraded capping machine had to be made smart without costing sensibly more. <iframe id="67e601e88697e927ea2b0551" width="250" class="specs-iframe" height="440" src="https://wevolver.com/iframe/specs/portenta-machine-control?iframe=true" frameborder="0" scrolling="no"></iframe><h3>An Innovative System for Predictive Maintenance</h3>To accelerate the process, AROL relied on a cost-effective, open-source system based on a combination of Arduino Pro’s Nicla Sense ME and Portenta Machine Control. This allowed their engineers to work with advanced sensors and high computational power, while still leveraging the ease of off-the-shelf components. With the support of a problem-solving and forward-thinking partner, the company quickly developed and tested a working solution: “The Arduino team was very proactive in understanding our requirements and providing a business-oriented solution with a customer-focus approach,” commented Denis Ruffino, R&amp;D technical director at AROL.AROL fitted each of the 15 capping turrets in one of their machines with a compact <a href="https://www.arduino.cc/pro/hardware-nicla-sense-me/" target="_blank">Nicla Sense ME</a>, providing vibration and temperature detection thanks to state-of-the-art, onboard Bosch Sensortec sensors – as well as the computational power to process part of the data directly at the level of the sensing nodes. At the base of the machine, one <a href="https://www.arduino.cc/pro/portenta-machine-control" target="_blank">Portenta Machine Control</a> was easily mounted on DIN rail to concentrate communication with all the sensors via Bluetooth®.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4NDQ4MzMxNTIxLVBNQ19sb3cuanBlZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width:300px" class="fr-fic fr-dib" />Having chosen an open-source solution, AROL is poised to scale up the idea with limited refactoring – and further upgrade their advanced capping machines, raising the bar of innovation for their industry.

Seeking a way to make their high-speed production equipment even more reliable and productive, AROL recently decided to develop an innovative solution to enable data monitoring and predictive maintenance, upgrading their offering with an Industry 4.0 approach.

AROL Tops Capping Machines with Smart Maintenance

Behrooz Tahmasebi — an MIT PhD student in the Department of Electrical Engineering and Computer Science (EECS) and an affiliate of the Computer Science and Artificial Intelligence Laboratory (CSAIL) — was taking a mathematics course on differential equations in late 2021 when a glimmer of inspiration struck. In that class, he learned for the first time about Weyl’s law, which had been formulated 110 years earlier by the German mathematician Hermann Weyl. Tahmasebi realized it might have some relevance to the computer science problem he was then wrestling with, even though the connection appeared — on the surface — to be thin, at best. Weyl’s law, he says, provides a formula that measures the complexity of the spectral information, or data, contained within the fundamental frequencies of a drum head or guitar string.Tahmasebi was, at the same time, thinking about measuring the complexity of the input data to a neural network, wondering whether that complexity could be reduced by taking into account some of the symmetries inherent to the dataset. Such a reduction, in turn, could facilitate — as well as speed up — machine learning processes.Weyl’s law, conceived about a century before the boom in machine learning, had traditionally been applied to very different physical situations — such as those concerning the vibrations of a string or the spectrum of electromagnetic (black-body) radiation given off by a heated object. Nevertheless, Tahmasebi believed that a customized version of that law might help with the machine learning problem he was pursuing. And if the approach panned out, the payoff could be considerable.He spoke with his advisor, Stefanie Jegelka — an associate professor in EECS and affiliate of CSAIL and the MIT Institute for Data, Systems, and Society — who believed the idea was definitely worth looking into. As Tahmasebi saw it, Weyl’s law had to do with gauging the complexity of data, and so did this project. But Weyl’s law, in its original form, said nothing about symmetry.He and Jegelka have now succeeded in modifying Weyl’s law so that symmetry can be factored into the assessment of a dataset’s complexity. “To the best of my knowledge,” Tahmasebi says, “this is the first time Weyl’s law has been used to determine how machine learning can be enhanced by symmetry.”The&nbsp;<a href="https://arxiv.org/abs/2303.14269" target="_blank">paper</a> he and Jegelka wrote earned a “Spotlight” designation when it was presented at the December 2023 conference on Neural Information Processing Systems — widely regarded as the world’s top conference on machine learning.This work, comments Soledad Villar, an applied mathematician at Johns Hopkins University, “shows that models that satisfy the symmetries of the problem are not only correct but also can produce predictions with smaller errors, using a small amount of training points. [This] is especially important in scientific domains, like computational chemistry, where training data can be scarce.”In their paper, Tahmasebi and Jegelka explored the ways in which symmetries, or so-called “invariances,” could benefit machine learning. Suppose, for example, the goal of a particular computer run is to pick out every image that contains the numeral 3. That task can be a lot easier, and go a lot quicker, if the algorithm can identify the 3 regardless of where it is placed in the box — whether it’s exactly in the center or off to the side — and whether it is pointed right-side up, upside down, or oriented at a random angle. An algorithm equipped with the latter capability can take advantage of the symmetries of translation and rotations, meaning that a 3, or any other object, is not changed in itself by altering its position or by rotating it around an arbitrary axis. It is said to be invariant to those shifts. The same logic can be applied to algorithms charged with identifying dogs or cats. A dog is a dog is a dog, one might say, irrespective of how it is embedded within an image.&nbsp;The point of the entire exercise, the authors explain, is to exploit a dataset’s intrinsic symmetries in order to reduce the complexity of machine learning tasks. That, in turn, can lead to a reduction in the amount of data needed for learning. Concretely, the new work answers the question: How many fewer data are needed to train a machine learning model if the data contain symmetries?There are two ways of achieving a gain, or benefit, by capitalizing on the symmetries present. The first has to do with the size of the sample to be looked at. Let’s imagine that you are charged, for instance, with analyzing an image that has mirror symmetry — the right side being an exact replica, or mirror image, of the left. In that case, you don’t have to look at every pixel; you can get all the information you need from half of the image — a factor of two improvement. If, on the other hand, the image can be partitioned into 10 identical parts, you can get a factor of 10 improvement. This kind of boosting effect is linear.To take another example, imagine you are sifting through a dataset, trying to find sequences of blocks that have seven different colors — black, blue, green, purple, red, white, and yellow. Your job becomes much easier if you don’t care about the order in which the blocks are arranged. If the order mattered, there would be 5,040 different combinations to look for. But if all you care about are sequences of blocks in which all seven colors appear, then you have reduced the number of things — or sequences — you are searching for from 5,040 to just one.Tahmasebi and Jegelka discovered that it is possible to achieve a different kind of gain — one that is exponential — that can be reaped for symmetries that operate over many dimensions. This advantage is related to the notion that the complexity of a learning task grows exponentially with the dimensionality of the data space. Making use of a multidimensional symmetry can therefore yield a disproportionately large return. “This is a new contribution that is basically telling us that symmetries of higher dimension are more important because they can give us an exponential gain,” Tahmasebi says.&nbsp;The NeurIPS 2023 paper that he wrote with Jegelka contains two theorems that were proved mathematically. “The first theorem shows that an improvement in sample complexity is achievable with the general algorithm we provide,” Tahmasebi says. The second theorem complements the first, he added, “showing that this is the best possible gain you can get; nothing else is achievable.”He and Jegelka have provided a formula that predicts the gain one can obtain from a particular symmetry in a given application. A virtue of this formula is its generality, Tahmasebi notes. “It works for any symmetry and any input space.” It works not only for symmetries that are known today, but it could also be applied in the future to symmetries that are yet to be discovered. The latter prospect is not too farfetched to consider, given that the search for new symmetries has long been a major thrust in physics. That suggests that, as more symmetries are found, the methodology introduced by Tahmasebi and Jegelka should only get better over time.According to Haggai Maron, a computer scientist at Technion (the Israel Institute of Technology) and NVIDIA who was not involved in the work, the approach presented in the paper “diverges substantially from related previous works, adopting a geometric perspective and employing tools from differential geometry. This theoretical contribution lends mathematical support to the emerging subfield of ‘Geometric Deep Learning,’ which has applications in graph learning, 3D data, and more. The paper helps establish a theoretical basis to guide further developments in this rapidly expanding research area.”

New MIT research provides a theoretical proof for a phenomenon observed in practice: that encoding symmetries in the machine learning model helps the model learn with fewer data. Image: Alex Shipps/MIT CSAIL

Exploiting the symmetry within datasets, MIT researchers show, can decrease the amount of data needed for training neural networks.

How symmetry can come to the aid of machine learning

<section id="isPasted">Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers at Texas A&amp;M University achieved when combining machine learning, neural networks and novel compression tactics in a new project advancing reservoir production forecasts.Why is the research important? Because advancing forecasts improves the availability and accuracy of information oil and gas companies use to make sound financial and operating decisions."Production forecasting can be done several ways, but everything depends on it," said researcher Mohammad Elkady. "Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce."&nbsp;Elkady and fellow researcher Veena Kumar created a flexible method they say could apply to understanding and evaluating any complex reservoir. Both are graduate students in the Harold Vance Department of Petroleum Engineering and work under the guidance of their advisor, Dr. Siddharth Misra, the Ted H. Smith, Jr. '75 and Max R. Vordenbaum '73 DVG Associate Professor.</section><blockquote>Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce.<cite>Mohammad Elkady</cite></blockquote><section>Misra and Elkady introduced the work in the paper, "<a href="https://doi.org/10.2118/216935-MS" rel="noopener" target="_blank">Ultrafast Multiphase Production Forecasting for Large Gas Condensate Shale Reservoirs</a>," during the Abu Dhabi International Progressive Energy Congress in Oct 2023.&nbsp;Misra also presented the work in the paper,&nbsp;"Rapid Production Forecasting for Hydraulically Fractured Wells in Large, Heterogeneous Shale Reservoir," at the Society of Petroleum Engineers’ Workshop in New Orleans, Louisiana in Nov. 2023.<h2>Production forecasting history</h2>To make a profit, oil and gas companies weigh equipment, human resources, and other costs against the profitable hydrocarbons they can produce from subsurface reservoirs. Such decisions would be easy if petroleum engineers could peek inside the earth and see what's there. Since the subsurface is opaque, it's tricky to accurately predict oil recovery from reservoirs as conditions change over time.In the past, engineers used two main methods for forecasting. The first and oldest plots a well's oil recovery rates and declines over time, and that decline curve analysis can help estimate the recoveries in nearby or similar wells. The second is newer and uses computational resources to create a visual map of a reservoir by compressing the massive amounts of data gathered from surface and subsurface sensors and simulating the information into simplified renderings.<h2>Refining data to enhance forecasting</h2>This research is not the first to use machine learning or even neural networks to get a more accurate forecast, but it is the first to combine those processes with geomodel data compression. Elkady said data compression isn't new, as it's been used successfully in areas like image and video analysis. High-resolution videos must be compressed when shared around the world on restrictive bandwidths, yet it must be done selectively to keep the subjects true to the original imaging. Though the application in petroleum is novel, data compression makes sense for similar reasons."Geological data usually has a lot of data, and some may not be as important as others," said Elkady. "That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data."</section><blockquote>That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data.<cite>Mohammad Elkady</cite></blockquote><section>Elkady and Kumar began by creating 4,000 virtual homogenous or uniformly structured reservoirs containing 88,000 grid cells. Each cell had three randomly sampled geological properties: permeability, porosity, water saturation, fracture space, etc. The total data generated for each reservoir equaled over a quarter of a million points. The researchers ran these original reservoirs through a simulator for the most accurate production forecasts possible.The researchers then compressed the original data using two different methods simultaneously to crunch the numbers down with a 50,000 to 1 ratio. After training in Python, they taught the neural network to do production forecast workflows of the compressed reservoir model. Results showed the workflow was much faster than the simulator – under a second for every 700 seconds the simulator took – and had an error rate of 3% when compared to simulated forecasts.Next, the researchers used simulated data guided by Aramco that mimicked actual reservoir information to create about 3,000 virtual reservoirs that were heterogeneous or complex in structure and content yet still had the same number of grid cells.“We used geostatistical tools to generate the geology to be as realistic as we could," said Elkady. "The Aramco team supervised the generation of these datasets to make sure that, from their experience, this looks like a real reservoir."When that set of comparisons between workflow and simulator forecasts was completed, the researchers added more parameters, starting with six or seven. Eventually, they included human-related parameters, such as when operators choose how much gas they wish to produce despite what the reservoir is capable of, a complex concept for neural networks to factor in.</section><blockquote>Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations.<cite>Veena Kumar</cite></blockquote><section>"Initially, the amount of accuracy to error was very bad when we added human variations to the data set," Kumar said. "I am proud of getting the error percentage down, but we're looking into the reason for the initial higher percentages."The model-building stage – including reservoir parameter definitions, geomodel compressions, and prediction parameter factoring – required about one hour, depending on the data complexity. Once the models were trained and tuned, the time ratio for generating forecasts from the models remained less than one second for the researcher’s workflow to every 700 seconds, or about 11 minutes, when using a commercial software simulator.<h2>Future direction</h2>Currently, the work is expanding the number of parameters, refining the techniques and input factors, and testing methods to reduce the training times needed to learn the software. After that, the researchers want to test it out on actual reservoir data, ideally a reservoir where some production has already been completed.They are optimistic that the work can be applied effectively in many different formations."The goal of refining the multivariate forecast model for long-term projections, potentially over a decade, is one of the approaches we're excited to try out," said Kumar. "Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations."</section>

Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers achieved when combining ML, neural networks and novel compression in a new project advancing reservoir production forecasts.

Ultrafast Reservoir Forecasting Boasts Speed and Accuracy

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/2ae057ae-41e1-49a0-b36e-b08e932ee203?dark=false" class="fr-draggable" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-10="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr><section id="isPasted">From “Law and Order” to “CSI,” not to mention real life, investigators have used fingerprints as the gold standard for linking criminals to a crime. But if a perpetrator leaves prints from different fingers in two different crime scenes, these scenes are very difficult to link, and the trace can go cold.It’s a well-accepted fact in the forensics community that fingerprints of different fingers of the same person--”intra-person fingerprints”--are unique, and therefore unmatchable.&nbsp;</section><section><h2>Research led by Columbia Engineering undergraduate</h2>A team led by <a href="http://engineering.columbia.edu/" rel="nofollow" target="_blank">Columbia Engineering</a> undergraduate senior Gabe Guo challenged this widely held presumption. Guo, who had no prior knowledge of forensics, found a public U.S. government database of some 60,000 fingerprints and fed them in pairs into an artificial intelligence-based system known as a deep contrastive network. Sometimes the pairs belonged to the same person (but different fingers), and sometimes they belonged to different people.&nbsp;</section><section><h2>AI has potential to greatly improve forensic accuracy</h2><iframe width="640" height="360" src="https://www.youtube.com/embed/s5esfRbBc18?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-10="true" id="833121476" title="AI Learns to Correlate a Person's Unique Fingerprints">&nbsp;</iframe>Credit: Gabe Guo and Aniv Ray/Columbia Engineering<section id="isPasted">Over time, the AI system, which the team designed by modifying a state-of-the-art framework, got better at telling when seemingly unique fingerprints belonged to the same person and when they didn’t. The accuracy for a single pair reached 77%. When multiple pairs were presented, the accuracy shot significantly higher, potentially increasing current forensic efficiency by more than tenfold. The project, a collaboration between Hod Lipson’s <a href="http://creativemachineslab.com/" rel="nofollow" target="_blank">Creative Machines lab</a> at Columbia Engineering and Wenyao Xu’s<a href="https://cse.buffalo.edu/~wenyaoxu/" rel="nofollow" target="_blank">&nbsp;Embedded Sensors and Computing lab</a> at University at Buffalo, SUNY, was <a href="https://www.science.org/doi/10.1126/sciadv.adi0329" rel="nofollow" target="_blank">published today in Science Advances.</a></section><section><h2>Study findings challenge–and surprise–forensics community</h2>Once the team verified their results, they quickly sent the findings to a well-established forensics journal, only to receive a rejection a few months later. The anonymous expert reviewer and editor concluded that “It is well known that every fingerprint is unique,” and therefore it would not be possible to detect similarities even if the fingerprints came from the same person.&nbsp;The team did not give up. They doubled down on the lead, fed their AI system even more data, and the system kept improving. Aware of the forensics community's skepticism, the team opted to submit their manuscript to a more general audience. The paper was rejected again, but <a href="https://www.engineering.columbia.edu/faculty/hod-lipson" rel="nofollow" target="_blank">Lipson</a>, who is the James and Sally Scapa Professor of Innovation in the Department of <a href="https://www.me.columbia.edu/" rel="nofollow" target="_blank">Mechanical Engineering</a> and co-director of the <a href="https://make.columbia.edu/" rel="nofollow" target="_blank">Makerspace Facility,&nbsp;</a>appealed. “I don’t normally argue editorial decisions, but this finding was too important to ignore,” he said. “If this information tips the balance, then I imagine that cold cases could be revived, and even that innocent people could be acquitted.”&nbsp;While the system’s accuracy is not sufficient to officially decide a case, it can help prioritize leads in ambiguous situations. After more back and forth, the paper was finally accepted for publication by Science Advances.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2MTAzNjczMzM2LWFpLWRpc2NvdmVycy10aGF0LW5vdC1ldmVyeS1maW5nZXJwcmludC1pcy11bmlxdWUuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">AI discovers a new way to compare fingerprints that seem different, but actually belong to different fingers of the same person. In contrast with traditional forensics, this AI relies mostly on the curvature of the swirls at the center of the fingerprint, as shown by the heatmap. Credit: Gabe Guo, Columbia Engineering; Midjourney generated silhouette.</section><section><h2>Unveiled: a new kind of forensic marker to precisely capture fingerprints</h2>One of the sticking points was the following question: What alternative information was the AI actually using that has evaded decades of forensic analysis? After careful visualizations of the AI system’s decision process, the team concluded that the AI was using a new kind of forensic marker.&nbsp;“The AI was not using ‘minutiae,’ which are the branchings and endpoints in fingerprint ridges – the patterns used in traditional fingerprint comparison,” said Guo, who began the study as a first-year student at Columbia Engineering in 2021. “Instead, it was using something else, related to the angles and curvatures of the swirls and loops in the center of the fingerprint.”&nbsp;Columbia Engineering senior Aniv Ray and PhD student Judah Goldfeder, who helped analyze the data, noted that their results are just the beginning. “Just imagine how well this will perform once it’s trained on millions, instead of thousands of fingerprints,” said Ray.</section><section><h2>A need for broader datasets</h2>The team is aware of potential biases in the data. The authors present evidence that indicates that the AI performs similarly across genders and races, where samples were available. However, they note, more careful validation needs to be done using datasets with broader coverage if this technique is to be used in practice.</section><section><h2>Transformative potential of AI in a well-established field</h2></section><section>This discovery is an example of more surprising things to come from AI, notes Lipson. “Many people think that AI cannot really make new discoveries–that it just regurgitates knowledge,” he said. “But this research is an example of how even a fairly simple AI, given a fairly plain dataset that the research community has had lying around for years, can provide insights that have eluded experts for decades.”&nbsp;He added, “Even more exciting is the fact that an undergraduate student, with no background in forensics whatsoever, can use AI to successfully challenge a widely held belief of an entire field. We are about to experience an explosion of AI-led scientific discovery by non-experts, and the expert community, including academia, needs to get ready.”</section></section>

PHOTO CREDIT: MARCO-MARCIL MONTOTO, COLUMBIA ENGINEERING, GENERATED WITH DALL-E

Columbia engineers have built a new AI that shatters a long-held belief in forensics–that fingerprints from different fingers of the same person are unique. It turns out they are similar, only we’ve been comparing fingerprints the wrong way!

AI Discovers That Not Every Fingerprint Is Unique

At least half of veterans with spinal cord injuries will develop sores on their skin from the unrelieved pressure of sitting for long periods of time in a wheelchair. It’s a constant worry, because these skin ulcers can greatly limit patients’ mobility.“Pressure injuries directly impact the veteran’s quality of life, because the medical provider will order the veteran to bed rest for weeks and potentially months,” said Kim House, a physician and medical director of the Spinal Cord Injury Clinic at the Atlanta Veterans Administration Healthcare System. “At every clinic visit, I provide education for pressure injury prevention.”House could one day have a new tool to offer her patients, thanks to researchers in the Georgia Tech College of Engineering, and wheelchair-bound veterans are just the beginning.Materials engineers are developing new fabric sensors and a customized wheelchair system that assesses and automatically eases pressure at contact points to prevent injuries from developing in the first place.“We have three key issues happening: First, continuous pressure. Second, moisture, because when you're sitting in the same spot, you tend to sweat and generate moisture. And third is shear. When you try to move somebody, the skin shears. That perfect combination is what causes pressure injuries,” said <a href="https://mse.gatech.edu/people/sundaresan-jayaraman" target="_blank">Sundaresan Jayaraman</a>, professor in the <a href="https://mse.gatech.edu/" target="_blank">School of Materials Science and Engineering</a> (MSE). “We believe we have a solution to the perfect storm of pressure, moisture and shear, which means the user’s quality of life is going to get better.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1701347606566-ezgif-3-3425e1aacc.jpg" style="width: 766px;" class="fr-fic fr-dib">Sundaresan Jayaraman (left) looks at pressure data from fabric sensors he developed with Sungmee Park, who is seated in their prototype wheelchair system. (Photo: Candler Hobbs)With Principal Research Scientist&nbsp;<a href="https://mse.gatech.edu/people/sungmee-park" target="_blank">Sungmee Park</a>, Jayaraman has designed fabric with a network of embedded sensors that covers the seat of a wheelchair. Conductive material is woven into the textile, and it’s washable without degrading the sensors.Data about pressure and moisture from the sensors feeds into a processing unit that uses artificial intelligence algorithms to identify trouble spots in real time and selectively raise or lower a series of actuators under the wheelchair seat to relieve pressure. That has the advantage of eliminating any shearing forces on the skin that come from sliding against the seat.Meanwhile, a series of fans activates to eliminate moisture. A companion smartphone app developed in the lab allows users to override the system to maintain comfort and stability —&nbsp;particularly important for people with spinal cord injuries who may not be able to correct their body position on their own.“Each person moves differently. This system gathers all the pressure and moisture data and also accounts for different body compositions, styles, and weights,” Park said. “We feed all of that big data into the AI algorithms so each person can have an intervention system that works independently. It would be a totally automatic system.”The team’s idea has advanced in a&nbsp;<a href="https://healthylongevitychallenge.org/winners/enhancing-health-longevity-of-individuals-with-limited-mobility/" target="_blank">National Academy of Medicine healthy longevity competition</a>, receiving one of 20 Catalyst Awards out of more than 500 ideas. For House, who earned a bachelor’s in mechanical engineering at Tech in 1995, the potential is powerful: “A wheelchair system that will assist veterans in decreasing risk for pressure injury will allow veterans to become less dependent on caregivers for pressure reduction strategies and allow for increased independence and self-determination.”Park and Jayaraman will work over the next year to “ruggedize” their system for real-world use. They’re hoping to work with House to collect feedback and ideas from wheelchair users with spinal cord injuries to improve their design.“With interactive input from users, we can intervene or modify if we need to and improve the prototype so we can move toward actual clinical testing for commercialization,” Park said.The researchers are focused initially on wheelchairs, but they see larger possibilities for their system. Hospital-acquired pressure injuries cost U.S. hospitals&nbsp;<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7948545/" target="_blank">more than $26 billion a year</a>, by some estimates. Avoiding them requires vigilance from medical staff, who must regularly help patients shift their position in bed. The sensor fabric could be deployed on hospital beds or in neonatal intensive care units to cue staff when a patient needs to be moved to relieve a buildup of pressure and moisture.Eventually, the ideas behind the system even could be extended to help office workers, software developers, or gamers who sit for long periods of time. Data from the sensor fabric might trigger reminders that encourage them to get up and counteract the negative health effects of too much sitting.“The theme of our work always has been, what can we do as engineers — by being innovative and creative, by bringing engineering problem-solving — to make an impact on society and people’s lives?” Jayaraman said. “We look forward to the day where this technology becomes a reality and people in wheelchairs don’t have to worry — and pressure injuries really become a scourge of the past.”

A customized wheelchair system — including fabric sensors, actuators, and fans — designed to prevent pressure injuries. (Photo: Candler Hobbs)

MSE researchers are using a Catalyst Award from the National Academy of Medicine to develop a pressure-relieving sensor system that could also be used in hospital beds.

Sensor Fabric, Big Data Could Help End Pressure Injuries for Wheelchair Users

This article is based on an article titled, <a href="https://cambrian-ai.com/wp-content/uploads/2023/08/BrainChip-Sees-Gold-Final_2.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">BrainChip Sees AI Gold in Sequential Data Analysis at the Edge</a>, from the Cambrian AI website.Amidst the whirlwind of excitement surrounding Large Language Model (LLM) generative AI, it's easy to lose sight of other AI domains that have seemingly dissolved into the shadows cast by ChatGPT's prominence. One underappreciated domain is the analysis of sequential data streams, which includes monitoring fluctuating stock prices and interpreting video feeds.&nbsp;BrainChip has identified this niche — the adept handling of sequential data — as a pivotal niche for deploying its Akida technology. Akida stands out for its prowess in Event-Based Neuromorphic computing, adeptly handling various neural network architectures like Vision Transformers (ViT), Convolutional Neural Networks (CNN), Temporal Evolution Network (TENN), and Recurrent Neural Networks (RNN).&nbsp;This article explores the need for efficient edge AI for time-series data.<h2 dir="ltr">Sequential data analysis</h2>Sequential analysis refers to the process of analyzing and extracting insights from data that is collected and organized in chronological order. This data type typically involves measurements or observations taken at regular intervals over time. Time-series analysis techniques aim to understand data patterns, trends, and dependencies and make predictions or forecasts based on historical patterns.A few use cases of sequential data analysis include:1. Financial Analysis: Time-series analysis is extensively used in finance to study stock market trends, analyze economic indicators, and forecast future market conditions. It helps model asset prices, trends, risk assessment, and portfolio optimization.2. Demand Forecasting: Sequential analysis is crucial in demand forecasting for retail, supply chain management, and manufacturing industries. By analyzing historical sales or demand data, businesses can predict future demand patterns and optimize their production, inventory, and supply chain accordingly.3. Predictive Maintenance: Predictive maintenance can monitor equipment and machinery in real-time. Analyzing sensor data and historical patterns can help detect anomalies and predict potential failures, enabling proactive maintenance and minimizing downtime.As well as applications in Energy Consumption Analysis and IoT Sensor Data Analysis.The market size for applying AI in time-series data analysis is continually growing as organizations recognize the value of extracting insights and making accurate predictions from temporal data. While specific market size figures for this realm are not readily available, the broader AI market, including applications in time-series analysis, is expected to grow substantially. According to a report by Grand View Research, the global AI market size was valued at USD 62.35 billion in 2020 and is projected to expand at a compound annual growth rate (CAGR) of 40.2% from 2021 to 2028. This growth encompasses various AI applications, including time-series analysis, across multiple industries.<h1 dir="ltr">Introducing the Dimension of Time to Neural Networks</h1>Traditional CNNs have been around for 30+ years and combine multiple hidden layers trained in a supervised manner. These are sequential and hence referred to as feed-forward neural networks. Bi-directional networks, also called RNNs, invented at the turn of the century, added capability for more complex learning, such as language modeling. But for applications to time series, a machine learning engineer needed a combination of CNNs and a temporal network for spatial-temporal analysis. While academia developed networks that did temporal convolution, they have yet to be power efficient or easy to train to make it to the far Edge.<h1 dir="ltr">Temporal Event-based Neural Networks</h1>Temporal Artificial Neural Networks (TENNs). This approach simplifies training and reduces model size while maintaining accuracy, resulting in improved performance and efficiency for Edge AI devices.Using TENNs for data analysis offers several advantages, such as the ability to learn the temporal structure of the data, which is crucial for tasks such as forecasting and anomaly detection. TENNs can make predictions for future time steps, and they are capable of learning from large datasets of time series data. Overall, these features make TENNs a valuable tool for data analysis in various domains.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529042393-image3.png" style="width: 300px;" class="fr-fic fr-dib">Traditional time series analysis applies to all kinds of signals. Image credit: Cambrian AIBeyond traditional Edge time series applications, BrainChip suggests that TENNs could minimize or eliminate the need for DSP filtering of raw audio signals and vital signs in health monitoring. Thus, offering compact solutions for wearable, hearing, and implantable devices with minimal power consumption is a significant advance for preventative healthcare.Additionally, TENNs also excel at treating streaming inputs as a time series of frames, performing 3D convolutions composed of a temporal convolution on the time axis and a spatial convolution on the XY axis. This is efficiently achieved to improve the detection of higher-resolution video objects in low-power scenarios.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529079641-image2.png" style="width: 300px;" class="fr-fic fr-dib">Treating streaming inputs as a 3D time series of frames with a highly efficient 3D convolution. Image credit: Cambrian AI.TENNs offer developers the flexibility to configure them in either buffered temporal convolution or recurrent modes, enabling them to adapt the network to their specific application requirements. Furthermore, TENNs can be efficiently trained on parallel hardware, such as GPUs and TPUs in the cloud, similar to convolutional networks, while retaining the compactness of RNNs for efficient inference at the edge. This approach helps to minimize the exponentially growing cost of training, which is a constant concern.<h1 dir="ltr">Akida 2nd&nbsp;Generation Processor and TENNs</h1>The BrainChip Akida processor is a digital portable processor IP inspired by the energy-efficient functionality of the human brain. Unlike traditional neuromorphic approaches, which are analog, it utilizes fully digital technology and offers a range of capabilities, including image classification, semantic segmentation, odor recognition, and time-series analysis. It supports various neural network architectures, including TENNs.The Akida processor utilizes highly parallel event-based neural processing cores, merging neuromorphic processing with native support for traditional convolutional capabilities and functions, along with hardware support for TENN networks. Its neuromorphic processing cores communicate using sparse, asynchronous events, making it ideal for efficient time-series data analysis and managing high-speed, asynchronous, and continuous data streams.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529133867-image4.png" style="width: 300px;" class="fr-fic fr-dib">The TENN operation may combine transform from convolutions in training to recurrence operation for inference. Image credit: Cambrian AI.BrainChip’s Akida allows the analysis of vision, video, and three-dimensional data as a time series, improving video object detection. Akida's support for spatial-temporal convolutions enhances speed and reduces energy consumption. This ability is possible because the processor features a high-speed, low-power digital design optimized for edge computing applications, offering real-time processing and low-latency analysis. It can handle structured and unstructured data, learning and recognizing patterns from streaming data, making it suitable for time-series data analysis.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699529192601-image4.png" style="width: 300px;" class="fr-fic fr-dib">The foundations of the Akida architecture. Image credit: Cambrian AI.Additionally, Akida includes a traditional CNN accelerator, as well as TENN and ViT logic, providing a comprehensive solution for sequential processing. The Akida processor is particularly effective for real-time data classification, anomaly detection, and predictive analytics. Consequently, the second-generation Akida processor, with TENN support, extends its efficiency and accelerated hardware solutions to multi-dimensional applications like video object detection and vision using event-based processing paradigms.<h1 dir="ltr">Conclusion</h1>The rapid expansion of the AI field has primarily focused on image processing and LLMs, leaving a significant gap in the development of sequential data analysis. BrainChip's Akida technology, featuring TENNs, simplifies training, reduce model size, and enhance performance and efficiency in time series data analysis.&nbsp;This innovative approach can be applied in different industries, including finance, demand forecasting, predictive maintenance, energy consumption analysis, and IoT data analysis, meeting the growing demand for effective AI solutions in these domains. The Akida processor extends its efficiency to multi-dimensional applications improving real-time data analysis, predictive analytics, and video object detection for event-based processing.

How BrainChip's Akida technology is revolutionizing time series data analysis.


The huge potential of sequential data analysis at the edge

"Hey, Alexa, play the latest Taylor Swift album.” Smart speakers offer amazing convenience – from playing your favorite tunes to re-ordering toilet paper – with only a simple voice command. But that convenience can come with a steep cost in privacy that many consumers aren’t even aware they’re paying.We’ve all had the uncanny experience of searching for something on the internet and then suddenly ads for that very thing are popping up everywhere we look online. It’s no coincidence, says Umar Iqbal, assistant professor of computer science &amp; engineering in the McKelvey School of Engineering at Washington University in St. Louis.“My collaborators and I uncovered that Amazon uses smart speaker interaction data to infer user interests and then uses those interests to target personalized ads to the user,” Iqbal said. “That’s something that Amazon was not upfront about before our research.”The team presented its work Oct. 26 at the ACM Internet Measurement Conference in Montreal, where they received the best paper award. They aim to provide visibility into what information is captured by smart speakers, how it is shared with other parties, and how it is used by such parties, allowing consumers to better understand the privacy risks of these devices and the impact of data sharing on people's online experiences.To crack open the black box around smart devices and the data they capture, the research team built an auditing framework to measure the collection, usage and sharing of Amazon Echo interaction data. First, they created several personas with interests in specific categories and one control persona. Each persona interacted with a different Echo device, then the researchers measured data collection by intercepting network traffic and inferred data usage by observing ads targeted to each persona on the web and on Echo devices.The team reported that as many as 41 advertisers sync or share their cookies – which are typically linked to personal information – with Amazon, and then those advertisers further sync their cookies with 247 other third parties, including advertising services.They also found that Amazon did not clearly disclose that users’ smart speaker interactions are used for profiling them for the purposes of ad targeting. Specifically, Amazon's general privacy policy and Alexa-specific privacy disclosures did not mention that smart speaker interactions are used for ad targeting. However, after their work’s preprint was released and Amazon was made aware, Amazon updated the Alexa Privacy Hub and Alexa Device FAQs to include that Alexa Echo interaction data is used for ad targeting.“Unfortunately, surveillance is the business model of the internet,” Iqbal said. “The issues we identified in our study seem to be part of the design of the smart speaker ecosystem, and the purpose of our study is to bring public transparency. In fact, after our work's preprint was released, Amazon updated its disclosure to include that it uses smart speaker interaction data for ad targeting.”&nbsp;“Consumer protection government agencies, such as the Federal Trade Commission in the U.S. and the European Consumer Organization in the EU have also shown significant interest in our findings,” Iqbal added.Whether interventions by lawmakers or consumer-protection agencies, including recent lawsuits against Amazon by the FTC and consumers themselves, will be successful remains to be seen. Regardless, Iqbal says it’s important for consumers to be aware of just how much data they’re giving away when they invite smart devices into their homes and how that information might be used.<hr>Iqbal U, Bahrami PN, Trimananda R, Cui H, Gamero-Garrido A, Dubois D, Choffnes D, Markopoulou A, Roesner F, and Shafiq Z. Tracking, profiling, and ad targeting in the Amazon smart speaker ecosystem. ACM Internet Measurement Conference (IMC), Oct. 24-26, 2023.&nbsp;<a href="https://umariqbal.com/papers/alexa-echos-imc2023.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">https://umariqbal.com/papers/alexa-echos-imc2023.pdf</a>This work is supported in part by the National Science Foundation (CNS-1956393, CNS-1955227, CNS-2103439, CNS-2114230, CNS-1909020), the Computing Research Association for the CIFellows 2021 Project (CNS-2127309), the Northeastern University Future Faculty Fellowship (2021), and the Consumer Reports Digital Lab.

The Amazon Echo smart speaker and Alexa assistant offer convenience in exchange for your data. Umar Iqbal is bringing transparency to how Amazon uses smart speaker interactions for targeted advertising. (Photo by Jonathan Borba on Unsplash)

Umar Iqbal and collaborators developed a framework to measure data collection, sharing and use by smart speaker platforms

Your smart speaker data is used in ways that you might not expect

To get a sense of what <a href="http://groups.csail.mit.edu/hcie/files/research-projects/structcode/2023-SCF-StructCode-paper.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">StructCode</a> is all about, says Mustafa Doğa Doğan, think of Superman. Not the “faster than a speeding bullet” and “more powerful than a locomotive” version, but a Superman, or Superwoman, who sees the world differently from ordinary mortals — someone who can look around a room and glean all kinds of information about ordinary objects that is not apparent to people with less penetrating faculties.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1698219535969-StructCode5.png" style="width: 766px;" class="fr-fic fr-dib">“The goal is to change the way we interact with objects,” says MIT PhD student Mustafa Doğa Doğan, “by embedding information into objects in ways that can be readily accessed.” Image courtesy of the researchers/MIT CSAIL.That, in a nutshell, is “the high-level idea behind StructCode,” explains Doğan, a PhD student in electrical engineering and computer science at MIT and an affiliate of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “The goal is to change the way we interact with objects” — to make those interactions more meaningful and more meaning-laden — “by embedding information into objects in ways that can be readily accessed.”StructCode grew out of an effort called InfraredTags, which Doğan and other colleagues introduced in 2022. That work, as well as the current project, was carried out in the laboratory of MIT Associate Professor Stefanie Mueller — Doğan’s advisor, who has taken part in both projects. In last year’s approach, “invisible” tags — that can only be seen with cameras capable of detecting infrared light — were used to reveal information about physical objects. The drawback there was that many cameras cannot perceive infrared light. Moreover, the method for fabricating these objects and affixing the tags to their surfaces relied on 3D printers, which tend to be very slow and often can only make objects that are small.<iframe width="640" height="360" src="https://www.youtube.com/embed/uZo7SyZV-gM?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>StructCode, at least in its original version, relies on objects produced with laser-cutting techniques that can be manufactured within minutes, rather than the hours it might take on a 3D printer. Information can be extracted from these objects, moreover, with the RGB cameras that are commonly found in smartphones; the ability to operate in the infrared range of the spectrum is not required.In their initial demonstrations of the idea, the MIT-led team decided to construct their objects out of wood, making pieces such as furniture, picture frames, flowerpots, or toys that are well suited to laser-cut fabrication. A key question that had to be resolved was this: How can information be stored in a way that is unobtrusive and durable, as compared to externally-attached bar codes and QR codes, and also will not undermine an object’s structural integrity?The solution that the team has come up with, for now, is to rely on joints, which are ubiquitous in wooden objects made out of more than one component. Perhaps the most familiar is the finger joint, which has a kind of zigzag pattern whereby two wooden pieces adjoin at right angles such that every protruding “finger” along the joint of the first piece fits into a corresponding “gap” in the joint of the second piece and, similarly, every gap in the joint of the first piece is filled with a finger from the second.“Joints have these repeating features, which are like repeating bits,” Dogan says. To create a code, the researchers slightly vary the length of the gaps or fingers. A standard size length is accorded a 1. A slightly shorter length is assigned a 0, and a slightly longer length is assigned a 2. The encoding scheme is based on the sequence of these numbers, or bits, that can be observed along a joint. For every string of four bits, there are 81 (34) possible variations.The team also demonstrated ways of encoding messages in “living hinges” — a kind of joint that is made by taking a flat, rigid piece of material and making it bendable by cutting a series of parallel, vertical lines. As with the finger joints, the distance between these lines can be varied: 1 being the standard length, 0 being a slightly shorter length, and 2 being slightly longer. And in this way, a code can be assembled from an object that contains a living hinge.The idea is described in a paper, “<a href="http://groups.csail.mit.edu/hcie/files/research-projects/structcode/2023-SCF-StructCode-paper.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">StructCode: Leveraging Fabrication Artifacts to Store Data in Laser-Cut Objects</a>,” that was presented this month at the 2023 ACM Symposium on Computational Fabrication in New York City. Doğan, the paper’s first author, is joined by Mueller and four coauthors — recent MIT alumna Grace Tang ’23, MNG ’23; MIT undergraduate Richard Qi; University of California at Berkeley graduate student Vivian Hsinyueh Chan; and Cornell University Assistant Professor Thijs Roumen.“In the realm of materials and design, there is often an inclination to associate novelty and innovation with entirely new materials or manufacturing techniques,” notes Elvin Karana, a professor of materials innovation and design at the Delft University of Technology. One of the things that impresses Karana most about StructCode is that it provides a novel means of storing data by “applying a commonly used technique like laser cutting and a material as ubiquitous as wood.”The idea for StructCode, adds University of Colorado computer scientist Ellen Yi-Luen Do, “is “simple, elegant, and totally makes sense. It’s like having the Rosetta Stone to help decipher Egyptian hieroglyphs.”Patrick Baudisch, a computer scientist at the Hasso Plattner Institute in Germany, views StructCode as “a great step forward for personal fabrication.&nbsp;It takes a key piece of functionality&nbsp;that's only offered today for mass-produced goods and brings it to custom objects."Here, in brief, is how it works: First, a laser cutter — guided by a model created via StructCode — fabricates an object into which encoded information has been embedded. After downloading a StructCode app, an user can decode the hidden message by pointing a cellphone camera at the object, which can (aided by StructCode software) detect subtle variations in length found in an object’s outward-facing joints or living hinges.The process is even easier if the user is equipped with augmented reality glasses, Doğan says. “In that case, you don’t need to point a camera. The information comes up automatically.” And that can give people more of the “superpowers” that the designers of StructCode hope to confer.“The object doesn’t need to contain a lot of information,” Doğan adds. “Just enough — in the form of, say, URLs — to direct people to places they can find out what they need to know.”Users might be sent to a website where they can obtain information about the object — how to care for it, and perhaps eventually how to disassemble it and recycle (or safely dispose of) its contents. A flowerpot that was made with living hinges might inform a user, based on records that are maintained online, as to when the plant inside the pot was last watered and when it needs to be watered again. Children examining a toy crocodile could, through StructCode, learn scientific details about various parts of the animal’s anatomy. A picture frame made with finger joints modified by StructCode could help people find out about the painting inside the frame and about the person (or persons) who created the artwork — perhaps linking to a video of an artist talking about this work directly.“This technique could pave the way for new applications, such as interactive museum exhibits,” says Raf Ramakers, a computer scientist at Hasselt University in Belgium. “It holds the potential for broadening the scope of how we perceive and interact with everyday objects” — which is precisely the goal that motivates the work of Doğan and his colleagues.But StructCode is not the end of the line, as far as Doğan and his collaborators are concerned. The same general approach could be adapted to other manufacturing techniques besides laser cutting, and information storage doesn't have to be confined to the joints of wooden objects. Data could be represented, for instance, in the texture of leather, within the pattern of woven or knitted pieces, or concealed by other means within an image. Doğan is excited by the breadth of available options and by the fact that their “explorations into this new realm of possibilities, designed to make objects and our world more interactive, are just beginning.”

StructCode modifies fabrication features like finger joints and living hinges in laser-cut objects, allowing for the addition of information such as labels and instructions without adding extra parts or materials. It also includes a robust tag decoding system and ensures the structural integrity of the laser-cut objects is maintained. Image courtesy of the researchers/MIT CSAIL.

StructCode, developed by MIT CSAIL researchers, encodes machine-readable data in laser-cut objects by modifying their fabrication features.

A new way to integrate data with physical objects

In machine learning, the quality and relevance of data play a pivotal role in determining the success of a project. As the saying goes, "Data is the new oil," but just like oil, raw data needs refining to be useful. Refining, organizing, and ensuring data quality is known as dataset curation. This step is paramount, especially in the machine learning domain, where the accuracy and efficiency of models heavily depend on the data they are trained on.&nbsp;As AI integration across industries accelerates, the significance of data curation becomes even more pronounced. A well-curated dataset ensures optimal model performance and mitigates risks associated with biased or inaccurate predictions; for companies like Edge Impulse, which emphasize the importance of on-device machine learning, the efficiency and reliability of datasets are crucial. This article delves into the best practices in dataset curation, highlighting strategies and considerations that can make or break a machine learning project[1].<iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> <h2 dir="ltr">Tailoring Best Practices Based on Data Type</h2>While there are general best practices that apply universally to dataset curation, the nuances of different types of data - sensor, audio, and picture/image - necessitate specific considerations. We will explore whether a one-size-fits-all approach is feasible or if tailored strategies are required for each data type.<h3 dir="ltr">Standard Practices in Dataset Curation</h3>Dataset curation is a meticulous process that ensures data quality, relevance, and usability. The foundational principles that apply across data types include:Versioning and Backing up Files: It's crucial to have a system to track different document or software project versions. Platforms like <a href="https://github.com/" target="_blank">GitHub</a>, <a href="https://osf.io/" target="_blank">Open Science Framework (OSF)</a>, and LabArchives@Brown offer versioning for projects. Moreover, backing up data locally and remotely safeguards against data loss due to unforeseen circumstances.[1]Contextualizing Your Data: Describing data in a standardized manner is essential for comprehension. Data glossaries, dictionaries, README files, and codebooks provide context. This includes column headings, measurement units, value types, and data collection methods.Choosing Formats for Re-Use: Proprietary data from instruments might require licenses for access. It's beneficial to convert such data to open or commonly used formats, ensuring broader accessibility.<h3 dir="ltr">Data-Specific Best Practices</h3>Sensor Data: Time-series data, inherent in sensors, requires careful handling to maintain its sequential integrity. Dealing with noise and ensuring the data's real-world relevance are also paramount.Audio Data: Sound quality assurance is vital, given the variations in audio formats. It's also essential to consider the diverse acoustic environments from which audio data might originate.Image Data: Image quality control is crucial, especially when dealing with different resolutions and formats. Diverse visual contexts, such as lighting and perspective, should also be considered.While there are overarching best practices in dataset curation, the specific nature of the data type in question demands tailored strategies to ensure optimal results.<h2 dir="ltr">Addressing Challenges in Dataset Curation</h2>Dataset curation is integral to the success of machine-learning projects. However, the process has its challenges. Addressing these challenges head-on is crucial for ensuring the quality and reliability of the curated datasets.<h3 dir="ltr">Imbalanced Datasets</h3>One of the most common challenges in dataset curation is dealing with imbalanced datasets. An imbalanced dataset is one where certain data classes are overrepresented while others are underrepresented. This can lead to biased models that perform well on the overrepresented classes but poorly on the underrepresented ones.For instance, in a machine learning project aimed at recognizing different sounds, the model might need help recognizing the latter if the dataset has thousands of urban sounds but only a handful of rural sounds. Addressing this requires techniques like oversampling the minority class, undersampling the majority class, or using synthetic data generation methods.<h3 dir="ltr">Data Privacy and Ethics</h3>Another significant challenge is ensuring data privacy and adhering to ethical standards during dataset curation. With the increasing awareness and regulations around data privacy, such as <a href="https://gdpr-info.eu/" target="_blank">GDPR</a>, it's imperative to handle data carefully, ensuring that personal and sensitive information is anonymized or removed.A recent article from <a href="https://www.nature.com/articles/s43588-020-00015-6" target="_blank">Nature Computational Science</a> highlighted the challenges faced during data curation related to pandemic data.[3] The article emphasized the importance of global data curation and standardization efforts to ensure rapid data integration and dissemination. It also highlighted the ethical, legal, and privacy issues hindering open data sharing. Such challenges are not exclusive to pandemic data but are relevant to any data curation effort.Moreover, ethical considerations extend beyond privacy. For instance, ensuring that the data doesn't reinforce existing biases or lead to discrimination against specific populations based on gender, age, or location.&nbsp;<h2 dir="ltr">Tips and Tricks for Effective Dataset Curation</h2>Dataset curation is a crucial step in the machine learning pipeline. Properly curated datasets ensure the accuracy of models and streamline the training process. Here are some actionable tips to enhance the dataset curation process:1. Plan for Accuracy at the Source: It's more efficient to validate data at its origin than to assess its accuracy later. For instance, asking users to validate their data or using sampling and auditing can help estimate accuracy levels.2. Annotate and Label: Annotating and labeling datasets during curation can simplify data management and troubleshooting. This can include adding timestamps or event locations. However, ensure that metadata is accurate to prevent inaccuracies during data transformation.3. Prioritize Security and Privacy: Large datasets can be vulnerable to breaches. Security practices like encryption, de-identification, and a robust data governance model can mitigate risks. Edge Impulse's emphasis on on-device processing also underscores the importance of data privacy.4. Anticipate Future Needs: Begin the data curation process with a clear vision of the end goal. Track how analytics and machine learning applications utilize datasets and refine the data aggregation process accordingly.5. Balance Data Governance with Agility: Striking a balance between data governance and business agility is crucial. Centralized platforms can help users identify relevant data and processes for their analytics or machine learning projects.6.&nbsp;Identify Business Needs: Ensure the curated data aligns with business objectives. Engage business stakeholders to understand their requirements early and avoid curating irrelevant data.<h3 dir="ltr">Data-Specific Best Practices</h3>1. Sensor Data: Consider the data collection frequency and noise potential for time-series data. Ensure that the data reflects real-world scenarios and is relevant to the problem.2. Audio Data: Ensure sound quality by considering factors like background noise and audio format compatibility. Diverse acoustic environments can introduce variability, so curate data that captures this diversity.3. Image Data: Image quality is paramount. Consider factors like resolution, format, and the context in which images are captured. Diverse visual contexts can provide a more comprehensive dataset for training models.By adhering to these best practices, organizations can curate datasets that are not only high-quality but also tailored to their specific needs, ensuring the success of their machine-learning projects.<h2 dir="ltr">Leveraging Edge Impulse for Dataset Curation</h2>In machine learning, dataset curation is a pivotal step that can make or break the performance of a model. Edge Impulse, a leading development platform for machine learning on edge devices, offers tools that streamline and enhance the dataset curation process.<h3 dir="ltr">Edge Impulse's Data Collection Tools</h3><a href="https://edgeimpulse.com/" target="_blank" rel="noopener noreferrer">Edge Impulse</a> understands the importance of diverse and high-quality data. The platform allows developers to gather data from many sources, be it their sensor hardware, public datasets, or data generated through simulations or synthetic data generation.[5] This flexibility ensures the dataset is comprehensive and relevant to the specific application.<h2 dir="ltr">Data Visualization and Analysis with Edge Impulse</h2>Once data is collected, understanding and analyzing it becomes crucial. Edge Impulse provides advanced data analysis tools that help quickly detect data quality issues, such as visual representation of outliers in a dataset, or mislabeled data. Whether you're working with sensor data, vision, or audio, the platform offers tailored tools that ensure your dataset is of the highest quality, ready to be fed into a machine-learning model.[5]<h2 dir="ltr">Conclusion</h2>Dataset curation is the backbone of any successful machine-learning project. The quality and relevance of the data directly influence the model's performance. With platforms like Edge Impulse, the process of dataset curation becomes more streamlined and efficient. By leveraging the right tools and adhering to best practices, one can ensure that their machine learning models are built on a solid foundation of high-quality data.<iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> <h2 dir="ltr">References</h2><ol><li dir="ltr"><a href="https://www.artiba.org/blog/the-essentials-of-machine-learning-data-curation" target="_blank">The Essentials of Machine Learning Data Curation</a></li><li dir="ltr"><a href="https://library.brown.edu/info/data_management_curation/" target="_blank">Brown University Library - Data Curation Best Practices</a></li><li dir="ltr"><a href="https://www.nature.com/articles/s43588-020-00015-6" target="_blank">Data curation during a pandemic and lessons learned from COVID-19 - Nature Computational Science</a></li><li dir="ltr"><a href="https://www.techtarget.com/searchdatamanagement/feature/8-tips-to-improve-the-data-curation-process" target="_blank">8 tips to improve the data curation process | TechTarget</a></li><li dir="ltr"><a href="https://www.edgeimpulse.com/" target="_blank">Edge Impulse</a></li></ol>

Unlocking AI Potential Through Strategic Dataset Curation: A Comprehensive Guide for Enhancing Machine Learning Models


Refining AI's Foundation: Mastering Dataset Curation for Machine Learning

This article originally appeared in&nbsp;<a href="https://www.caltech.edu/about/news/artificial-intelligence-is-key-to-better-climate-models-say-researchers" target="_blank" rel="noopener noreferrer">Caltech</a> and has been republished with permission.<hr>Scientists have become pretty good at predicting the weather, at least up to 10 days in the future. But making accurate, longer-term predictions about how climate is changing remains challenging. The ability to predict climate change is important because it can help guide decisions about insurance coverage in fire- or flood-prone areas; it can also help architects design safer and more comfortable buildings for occupants in a warmer future.Our ability to model climate change has been hampered by the enormous amount of computing power required to simulate all facets of climate, says <a href="https://www.eas.caltech.edu/people/tapio" target="_blank">Tapio Schneider</a>, the Theodore Y. Wu Professor of Environmental Science and Engineering and a senior research scientist at JPL, which Caltech manages for NASA. To make a global climate model accurate, it needs to capture small-scale processes, such as those controlling droplet formation in clouds, over the entire planet. Simulating this huge range of relevant processes is currently impossible.Instead, climate models calculate how temperature, winds, and ocean currents evolve on a mesh with a horizontal resolution around 50–100 kilometers. Smaller-scale processes, such as those in low-lying marine clouds that help shape the climate, are represented by rough approximations called parameterizations. These parameterizations often lack a strong foundation in both scientific principles and data.Recently, a group of climate scientists proposed a massively scaled climate-modeling project as a solution. The project would consist of a series of centers running supercomputers that are staffed by 200–300 scientists each. The goal would be to create climate models with a resolution of 1 kilometer.Schneider says that might prove difficult."Running climate models is enormously computationally expensive," Schneider says. "If you want to make the computational mesh finer by a factor of 10 in the horizontal plane, you need a thousand times the computational power."Schneider says a project of that nature would strain our current computational abilities, and it would not be able to represent the small-scale processes necessary for accurate modeling. It would further concentrate resources among developed nations better able to fund such centers, thereby leaving out developing nations, he says.In <a href="https://www.nature.com/articles/s41558-023-01769-3" target="_blank">a paper</a> recently published in the journal Nature Climate Change, Schneider and 13 other scientists from around the world propose to develop models with a moderately high resolution (tens of kilometers), providing a stronger scientific foundation for parameterizations of smaller-scale processes, using machine learning to fill in the gaps. This, the researchers say, will create robust models without the need for gargantuan computing centers."We cannot possibly resolve the marine clouds over the tropical oceans globally anytime soon because it will require a computer about a hundred billion times faster than the fastest one we have," Schneider says. "What we can do is simulate these clouds in small areas and generate data. Those data can also inform a coarser climate model."Schneider says work on the kinds of AI tools needed to improve the coarser models is happening right now, some of it at Caltech. This technology could do for climate models what better data has done for weather models."Weather forecasts have become quite good over the last few decades. It's a bit of an unsung success story of science," he says. "The primary reason they've become quite good is that they use data much more extensively than they used to. In some ways, we want to achieve the same thing for climate prediction. I think that can be achieved by using AI tools to harness the extensive data about Earth we already have."Andrew Stuart, Bren Professor of Computing and Mathematical Sciences and a co-author of the proposal, has lent his expertise to Schneider's effort, helping build the tools necessary for the project. Stuart says one set of tools aims to make climate models less computationally expensive to run.This can be done by creating parameterizations for coarser models, which Stuart describes as "excellent sketches" of actual physical processes, such as the behavior of individual clouds. In essence, they capture the main features needed to predict the effects that small-scale processes will have on larger-scale processes in a global model simulation. Because they are simplified versions of actual events, they have to be tuned to behave like their real-life counterparts.Even coarser models with parameterizations are expensive to run in terms of computing time and power, and the tuning process means they need to be run repeatedly until the researchers succeed in tuning them to produce reliable and realistic outputs."The primary computational challenge is to perform the tuning with the smallest number of costly model evaluations," Stuart says. "We do that by building on ideas that go back to 1960 and an engineer named Rudolf Kálmán; the ideas have now evolved into what are called ensemble Kalman filters. They allow us to make predictions and calibration with the minimal number of evaluations. They are used in weather forecasting, and we've repurposed them."The second set of tools Stuart has worked on in the climate arena fall under the umbrella of what might be popularly referred to as A.I., and more specifically, machine learning. He says machine learning can help quantify some of the uncertainty inherent in climate modeling and generate new data at scales that are hard to measure."The models have an enormous amount of uncertainty in them," Stuart says. "This is where machine learning enters."The machine learning used by the team has similarities to image generators like OpenAI's DALL-E. In the case of DALL-E, the machine-learning algorithm is trained on a huge set of sample images. So if you ask it, for example, to generate a picture of a man holding an ice cream cone at sunset, it can.The algorithms used in climate science are instead trained to generate pieces—hundreds or thousands of them—that can be plugged into bigger simulations to fill in the missing gaps. Just like DALL-E can conjure up images of apples when asked (because it has learned to identify them), the algorithms used in this research can generate models of processes important to climate modeling, such as droplet formation in clouds.By putting simulated processes of that sort together with real-world data, researchers hope to create realistic predictions about Earth's rapidly changing climate, Schneider says."We can use the physical laws as far as we can, given the limited computational power at our disposal," he says. "What we cannot simulate, we can learn from data."

The butterfly effect—the idea that a butterfly flapping its wings in South America could change the course of a tornado in Texas—has been a popular way to illustrate the difficulty of making predictions about the behavior of chaotic systems like the weather. Because such systems are extremely sensitive to their initial conditions, even the tiniest of changes can send their behavior careening off into unexpected directions.

Artificial Intelligence is Key to Better Climate Models, Say Researchers

This article originally appeared in <a href="https://news.engin.umich.edu/2023/10/choosing-exoskeleton-settings-like-a-pandora-radio-station/" target="_blank" rel="noopener noreferrer">Michigan Engineering</a> and has been republished with permission.<hr>Taking inspiration from music streaming services, a team of engineers at the University of Michigan, Google and Georgia Tech has designed the simplest way for users to program their own exoskeleton assistance settings.Of course, what’s simple for the users is more complex underneath, as a machine learning algorithm repeatedly offers pairs of assistance profiles that are most likely to be comfortable for the wearer. The user then selects one of these two, and the predictor offers another assistance profile that it believes might be better. This approach enables users to set the exoskeleton assistance based on their preferences using a very simple interface, conducive to implementing on a smartwatch or phone.“It’s essentially like Pandora music,” said <a href="https://me.engin.umich.edu/people/faculty/elliott-rouse/" target="_blank" rel="noreferrer noopener">Elliott Rouse</a>, U-M associate professor of robotics and mechanical engineering and corresponding author of the study in Science Robotics. “You give it feedback, a thumbs up or thumbs down, and it curates a radio station based on your feedback. This is a similar idea, but it’s with exoskeleton assistance settings. In both cases, we are creating a model of the user’s preferences and using this model to optimize the user’s experience.”<iframe width="640" height="360" src="https://www.youtube.com/embed/OJ2ApTzZU2w?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;</iframe>The team tested the approach with 14 participants, each wearing a pair of ankle exoskeletons as they walked at a steady pace of about 2.3 miles per hour. The volunteers could take as much time as they wanted between choices, although they were limited to 50 choices. Most participants were choosing the same assistance profile repeatedly by the 45th decision.&nbsp;After 50 rounds, the experimental team began testing the users to see whether the final assistance profile was truly the best—pairing it against 10 randomly generated (but plausible) profiles. On average, participants chose the settings suggested by the algorithm about nine out of 10 times, which highlights the accuracy of the proposed approach.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697684045536-ExoChoice-content1.jpg" style="width: 766px;" class="fr-fic fr-dib">Ellie Wilson, a PhD student in mechanical engineering, demonstrates a new control strategy that customizes assistance provided by a pair of ankle exoskeletons. A machine learning algorithm presents her with two assistance profiles that it has selected based on previous data. She chooses one, and it presents another for comparison. After about 45 choices, most users have found an assistance profile that they choose repeatedly. Photo: Brenda Ahearn/Michigan Engineering“By using clever algorithms and a touch of AI, our system figures out what users want with easy yes-or-no questions,” said Ung Hee Lee, a recent U-M doctoral graduate from mechanical engineering and first author of the study, now at the robotics company Nuro. “I’m excited that this approach will make wearable robots comfortable and easy to use, bringing them closer to becoming a normal part of our day-to-day life.”The control algorithm manages four exoskeleton settings: how much assistance to give (peak torque), how long to go between peaks (timing), and how the exoskeleton both ramps up and reduces the assistance on either side of each peak. This assistance approach is based on how our calf muscle adds force to propel us forward in each step.Rouse reports that few groups are enabling users to set their own exoskeleton settings.“In most cases, controllers are tuned based on biomechanical or physiological results. The researchers are adjusting the settings on their laptops, minimizing the user’s metabolic rate. Right now, that’s the gold standard for exoskeleton assessment and control,” Rouse said.“I think our field overemphasizes testing with metabolic rate. People are actually very insensitive to changes in their own metabolic rate, so we’re developing exoskeletons to do something that people can’t actually perceive.”&nbsp;In contrast, user preference approaches not only focus on what users can perceive but also enable them to prioritize qualities that they feel are valuable.The study builds on the team’s previous effort to enable users to apply their own settings to an ankle exoskeleton. In that study, users had a touchscreen grid that put the level of assistance on one axis and the timing of the assistance on another. Users tried different points on the grid until they found one that worked well for them.&nbsp;Once users had discovered what was comfortable, over the course of a couple of hours, they were then able to find their settings on the grid within a couple of minutes. The new study cuts down that longer period of discovering which settings feel best as well as offering two new parameters: how the assistance ramps up and down.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697684067237-ExoChoice-content3-1024x683.jpg" style="width: 766px;" class="fr-fic fr-dib">A close-up of the exoskeleton worn by Ellie Wilson. Photo: Brenda Ahearn/Michigan EngineeringThe data from that earlier study were used to feed the machine learning predictor. An evolutionary algorithm produces variations based on the assistance profiles that those earlier users preferred, and then the predictor—a neural network—ranked those assistance profiles. With each choice the users made, new potential assistance profiles were generated, ranked and presented to the user alongside their previous choice.&nbsp;The study was funded by X, Google’s “Moonshot Factory,” Robotics at Google (now Google Deepmind), and the D. Dan and Betty Kahn Foundation. The concept is currently licensed by Alphabet spinoff <a href="https://www.skipwithjoy.com/" target="_blank" rel="noreferrer noopener">Skip Innovations</a>.

Using a simple and convenient touchscreen interface, the algorithm learns the assistance preferences of the wearer.

Choosing exoskeleton settings like a Pandora radio station

Training robust AI models can be challenging when using crowdsourced or publicly available datasets. The most useful neural networks are ones that replace expert knowledge to solve real problems. One example would be a visual inspection model for the factory floor.<h2>Human level performance</h2>Human performance comes into tiers: general or expert. An expert is someone who has spent a minimum amount of time or has received practical training in the field.The accuracy of your AI model is directly correlated to the quality of data used to train it. When using general datasets for training, the model usually surpasses the knowledge level embedded in the training samples, but it remains well below expert-level performance. A computer is more attentive than an operator performing the same inspection task, so the model trained will probably reach an accuracy that is marginally better than the human labelled dataset.If you think it's a good idea to use services like Amazon Mechanical Turk to label your dataset, let me ask you a question. Would you take a person and after reading a quick instruction manual get them to work inside the factory?That does not sound like a great idea. But it turns out that is what every one of us is doing when leveraging services like these. The bulk of your dataset can be generated in this manner, but is it helping your model?<h2>Practical considerations</h2>Tricky samples may not make it to the dataset because someone was not able to understand your labeling instructions, and that sample may get discarded. While human-level performance is achievable with large crowd labeled datasets, the truth is that the easiest samples will be labeled first.I had seen numerous occasions in which turks were simply targeting the lowest hanging fruit by submitting no labels for the empty samples. In that case, it is a first come first rewarded economy. The difficult samples may end up unlabeled.<h2>Model hardening</h2>Synthetic data allows the possibility to make the task at hand harder than it will be in real life.Variance or noise is good for your model, while generating samples programmatically can make the model more resilient to unseen inputs. Human labeled data is often curated and less noisy than the real inputs.This may work out, in the beginning, to quickly get started with your model. But believe me, you don't want a crowd trained model to be put in production across a factory floor without verifying that its performance surpasses expert level or that it gets as closely as possible to those expectations.<h2>Methods for generating data</h2>There are several methods you can leverage when generating synthetic data.<h3>Simulator data</h3>Use a simulator to generate new data. Custom shaders can provide RGB+D and segmentation labels. Microsoft is now focusing on aerospace-grade data generation because it is open source, AirSim 3D simulator project obtained promising results hence a lot of companies were using it for internal purposes.<iframe width="640" height="360" src="https://www.youtube.com/embed/b59sxe-M-bg?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe><h3 id="isPasted">Synthetic augmentation</h3>Simple computer vision enhancements can work as well. Using the OpenCVs inpaint function the price was rewritten as per the examples shown bellow. A mixed approach was implemented here, having the human labelers define the polygon in which the price was contained.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1697620602389-synthetic_inpaint_example-768x376.jpg" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">Development focus</h2>Validating crowd samples is going to take a similar amount of effort as developing a dataset generator.This is the most essential part of AI and a lot of people get it wrong. If the dataset is poorly curated and not enough time is spent at this stage, your project may end up failing. The dataset should be one of your main priorities, and if you have enough quality data, you can train almost every model to perfection.<h2>Conclusion</h2>The most successful models deployed in production use a mix of the synthetic and real data to achieve great results.<a href="https://www.linkedin.com/company/adapta-robotics" target="_blank">Let us know</a> what other techniques you are using to give your model that extra edge in performance.

Training robust AI models can be challenging when using crowdsourced or publicly available datasets. The most useful neural networks are ones that replace expert knowledge to solve real problems. One example would be a visual inspection model for the factory floor.

Synthetic datasets-Or why you can't train robust models with generic data

Sensor and actuator networks are systems that collect and transmit data from a physical environment for digital processing. Data is collected at sensors and actuators in the field, then digitized at the sensor or actuator. The digitized data can be used for a variety of applications, and the method for data processing depends on the application

big data

Profiles