The High-Temperature Carbon-Fiber 3D Printing Challenge invited engineers and designers to submit ideas on how to leverage cutting-edge materials from Mitsubishi Chemical Advanced Materials. Participants in the challenge were asked to incorporate the strength and properties of the KyronMax&reg; carbon-fiber 3D printing filament with the flexibility and sustainability of 3D printing in their design.We are composing a&nbsp;series of articles featuring the finalists and winners of the challenge.In this article we showcase the work of <a href="https://www.linkedin.com/in/robinkocher/" target="_blank">Robin Kocher</a> from Human Earth Industries an upcoming player in the micro-engineered suits domain.<h2 dir="ltr">Suits for the future</h2>The specialized suits designed by Kocher, aims to protect against harsh weather conditions. The suits are primarily intended for providing support during disaster relief, extreme sports, and various outdoor activities. They have multiple layers which can be customized based on the activity.&quot;We visualize the company in two parts. The primary focus behind Human Earth Industries is facilitating a change in society, imparting a sense of awareness in people regarding the 17 sustainable development goals, and spreading the word of change. The second part is the engineering aspect of it, where we aim to revolutionize the fashion technology industry,&quot; said Kocher. The idea for developing the suits was first conceptualized when Robin was still a student. As a gaming enthusiast, he wondered how the advanced suits in video games would be like in the real world. &quot;My interest in video games and science fiction helped me design the fabrics for the prototype. The ideas stemmed mainly from publicly available research,&quot; he added. &nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY5NTc0MzQxNDM4LWltYWdlMS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">The suit imagined by Human Earth Industries. Image credit: Human Earth Industries<h2 dir="ltr" id="isPasted">Sustainability is the top priority</h2>The suit has a customizable design where one can alter various functionalities. It would be possible to insert a layer that can measure the heart rate, blood oxygen, and ambient conditions. Additionally, another layer with a pathfinder module and smart ventilation (heating and cooling), primarily geared towards disaster relief activities would also be incorporated. &quot;The material that we use for the suits are not particularly environmentally friendly. I want to make products that can be used for a long time and the suits are designed to last a lifetime. I want people to pass on the suits to the next generations. This is one of my approaches to sustainability. The components would be made from bio-compostable products in the future,&quot; said Kocher.<h2 dir="ltr">Diversifying functionalities with carbon-fiber</h2>The early-stage startup wants to engineer their specialized suits for various operations. For instance, Robin has deep-dived into the topic and has identified the risks involved for horseback riders.He is developing a prototype of a safety vest with a carbon fiber layer that protects the riders from potentially severe falls. This vest can also be purposed for other activities such as skiing and motorbiking.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY5NTc0MzkzODMxLWltYWdlMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><h2 dir="ltr" id="isPasted">MCAM&rsquo;s Growth Garage challenge pushed design ideation</h2>When we asked Robin, what the most challenging criteria of the submission was, he said: &quot;The criteria pertaining to the technical requirements was quite impressive, especially when we had to work with high-temperature requirements. When we started designing the product, we had to make sure that it satisfied the sustainability as well the high temperature criterion. It helped me visualize the incorporation of the carbon fiber layer in the specialized suits. The challenge was quite an enjoyable experience for me.&quot;Kocher is impressed by Mitsubishi Chemical&rsquo;s focus on sustainability and conscientious practices. It was also one of the reasons that compelled him to enter the challenge. &quot;As a self-financed startup, it is always nice to receive support from organizations that can help us take our product to the next level, in terms of design, manufacturing and product upgrades. I would love to take part in the new challenges and create an awareness of thriving in the industry by consciously contributing back to the environment and society,&quot; he said.Follow <a href="https://www.linkedin.com/showcase/growth-garage-mcam/" target="_blank">Growth Garage</a>&#39;s&nbsp;LinkedIn page for more interesting stories about 3D printing entrepreneurs in various fields.<h2 id="isPasted">The Composite Engineering Challenge</h2>We are inviting innovators, entrepreneurs, and early adopters at universities, (pre-) startups, and scaleups that need fiber-reinforced composites to realize their prototype, part, product, or application.&nbsp;We welcome projects, programs, and companies in the field of new mobility (EV, e-motor, battery casing, etc.), autonomous (delivery) vehicles, people movers, robotics, aerospace, space, bicycles, UAV, AAV, bionics (wheelchairs, exoskeletons, prosthetics), alternative energy (production, storage, transport), self-sensing or self-healing, sport &amp; leisure, alternative (low carbon footprint) fibers and more!The winning entry will be rewarded with a partnership with Growth Garage,&nbsp;Business Incubator of the MCG Advanced Materials Division.&nbsp;This partnership package is valued at $25,000 and will be tailored to the winning team&rsquo;s needs according to their current business and product development stage.&nbsp;<a class="button-main-action" href="https://wevlv.co/homepage-banner" rel="nofollow" target="_blank">READ MORE ABOUT THE CHALLENGE HERE.</a><table cellpadding="0" cellspacing="0"><tbody><tr><td valign="top"><h2>Key Dates</h2><table><tbody><tr><td>Composite Challenge Launch&nbsp;</td><td>Monday, October 24th, 2022</td></tr><tr><td>Challenge Close&nbsp;</td><td>Sunday 22 January 2023&nbsp; </td></tr><tr><td>Finalists Announced&nbsp;</td><td>Wednesday 1 February 2023&nbsp; </td></tr><tr><td>Wevolver Community Vote&nbsp;</td><td>From 1-12 February 2023&nbsp; </td></tr><tr><td>Winners announcement&nbsp;</td><td>Tuesday 21 February 2023&nbsp;</td></tr></tbody></table></td></tr></tbody></table><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY5NTc0NDUyMDMzLWV5SmlkV05yWlhRaU9pSjNaWFp2YkhabGNpMXdjbTlxWldOMExXbHRZV2RsY3lJc0ltdGxlU0k2SW1aeWIyRnNZUzh4TmpZMk5UazRNakEyTlRnMExVMURSeUJzYjJkdklHSmhibTVsY2k1cWNHY2lMQ0psWkdsMGN5STZleUp5WlhOcGVtVWlPbnNpZDJsa2RHZ2lPamsxTUN3aVptbDBJam9pWTI5MlpYSWlmWDE5LndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">

Carbon fiber fashion aims to assist humans to endure extreme weather and activities. 


Augmenting human endurance with carbon fiber suits

This article was first published on <a href="https://news.mit.edu/2022/system-helps-severely-motor-impaired-individuals-type-more-quickly-accurately-0405" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>. &nbsp;<hr>In 1995, French fashion magazine editor Jean-Dominique Bauby suffered a seizure while driving a car, which left him with a condition known as locked-in syndrome, a neurological disease in which the patient is completely paralyzed and can only move muscles that control the eyes.Bauby, who had signed a book contract shortly before his accident, wrote the memoir &ldquo;The Diving Bell and the Butterfly&rdquo; using a dictation system in which his speech therapist recited the alphabet and he would blink when she said the correct letter. They wrote the 130-page book one blink at a time.Technology has come a long way since Bauby&rsquo;s accident. Many individuals with severe motor impairments caused by locked-in syndrome, cerebral palsy, amyotrophic lateral sclerosis, or other conditions can communicate using computer interfaces where they select letters or words in an onscreen grid by activating a single switch, often by pressing a button, releasing a puff of air, or blinking.But these row-column scanning systems are very rigid, and, similar to the technique used by Bauby&rsquo;s speech therapist, they highlight each option one at a time, making them frustratingly slow for some users. And they are not suitable for tasks where options can&rsquo;t be arranged in a grid, like drawing, browsing the web, or gaming.A more flexible system being developed by researchers at MIT places individual selection indicators next to each option on a computer screen. The indicators can be placed anywhere &mdash; next to anything someone might click with a mouse &mdash; so a user does not need to cycle through a grid of choices to make selections. The system, called Nomon, incorporates probabilistic reasoning to learn how users make selections, and then adjusts the interface to improve their speed and accuracy.Participants in a user study were able to type faster using Nomon than with a row-column scanning system. The users also performed better on a picture selection task, demonstrating how Nomon could be used for more than typing.&ldquo;It is so cool and exciting to be able to develop software that has the potential to really help people. Being able to find those signals and turn them into communication as we are used to it is a really interesting problem,&rdquo; says senior author Tamara Broderick, an associate professor in the MIT Department of Electrical Engineering and Computer Science (EECS) and a member of the Laboratory for Information and Decision Systems and the Institute for Data, Systems, and Society.Joining Broderick on the paper are lead author Nicholas Bonaker, an EECS graduate student; Emli-Mari Nel, head of innovation and machine learning at Averly and a visiting lecturer at the University of Witwatersrand in South Africa; and Keith Vertanen, an associate professor at Michigan Tech. The research is being presented at the ACM Conference on Human Factors in Computing Systems.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649330429689-MIT-Single-Switch-02-press.jpg" style="width: 766px;" class="fr-fic fr-dib">In the Nomon interface, a small analog clock is placed next to every option the user can select. (A gnomon is the part of a sundial that casts a shadow.) Here is an example of the interface being used as a keyboard. Image courtesy of the researchers.<h2>On the clock</h2>In the Nomon interface, a small analog clock is placed next to every option the user can select. (A gnomon is the part of a sundial that casts a shadow.) The user looks at one option and then clicks their switch when that clock&rsquo;s hand passes a red &ldquo;noon&rdquo; line. After each click, the system changes the phases of the clocks to separate the most probable next targets. The user clicks repeatedly until their target is selected.When used as a keyboard, Nomon&rsquo;s machine-learning algorithms try to guess the next word based on previous words and each new letter as the user makes selections.Broderick developed a simplified version of Nomon several years ago but decided to revisit it to make the system easier for motor-impaired individuals to use. She enlisted the help of then-undergraduate Bonaker to redesign the interface.They first consulted nonprofit organizations that work with motor-impaired individuals, as well as a motor-impaired switch user, to gather feedback on the Nomon design.Then they designed a user study that would better represent the abilities of motor-impaired individuals. They wanted to make sure to thoroughly vet the system before using much of the valuable time of motor-impaired users, so they first tested on non-switch users, Broderick explains.<h2>Switching up the switch</h2>To gather more representative data, Bonaker devised a webcam-based switch that was harder to use than simply clicking a key. The non-switch users had to lean their bodies to one side of the screen and then back to the other side to register a click.&ldquo;And they have to do this at precisely the right time, so it really slows them down. We did some empirical studies which showed that they were much closer to the response times of motor-impaired individuals,&rdquo; Broderick says.They ran a 10-session user study with 13 non-switch participants and one single-switch user with an advanced form of spinal muscular dystrophy. In the first nine sessions, participants used Nomon and a row-column scanning interface for 20 minutes each to perform text entry, and in the 10th session they used the two systems for a picture selection task.Non-switch users typed 15 percent faster using Nomon, while the motor-impaired user typed even faster than the non-switch users. When typing unfamiliar words, the users were 20 percent faster overall and made half as many errors. In their final session, they were able to complete the picture selection task 36 percent faster using Nomon.&ldquo;Nomon is much more forgiving than row-column scanning. With row-column scanning, even if you are just slightly off, now you&rsquo;ve chosen B instead of A and that&rsquo;s an error,&rdquo; Broderick says.<h2>Adapting to noisy clicks</h2>With its probabilistic reasoning, Nomon incorporates everything it knows about where a user is likely to click to make the process faster, easier, and less error-prone. For instance, if the user selects &ldquo;Q,&rdquo; Nomon will make it as easy as possible for the user to select &ldquo;U&rdquo; next.Nomon also learns how&nbsp;a user clicks. So, if the user always clicks a little after the clock&rsquo;s hand strikes noon, the system adapts to that in real time. It also adapts to noisiness. If a user&rsquo;s click is often off the mark, the system requires extra clicks to ensure accuracy.This probabilistic reasoning makes Nomon powerful but also requires a higher click-load than row-column scanning systems. Clicking multiple times can be a trying task for severely motor-impaired users.Broderick hopes to reduce the click-load by incorporating gaze tracking into Nomon, which would give the system more robust information about what a user might choose next based on which part of the screen they are looking at. The researchers also want to find a better way to automatically adjust the clock speeds to help users be more accurate and efficient.They are working on a new series of studies in which they plan to partner with more motor-impaired users.&ldquo;So far, the feedback from motor-impaired users has been invaluable to us; we&rsquo;re very grateful to the motor-impaired user who commented on our initial interface and the separate motor-impaired user who participated in our study. We&rsquo;re currently extending our study to work with a bigger and more diverse group of our target population. With their help, we&rsquo;re already making further improvements to our interface and working to better understand the performance of Nomon,&rdquo; she says.&ldquo;Nonspeaking individuals with motor disabilities are currently not provided with efficient communication solutions for interacting with either speaking partners or computer systems. This &lsquo;communication gap&rsquo; is a known unresolved problem in human-computer interaction, and so far there are no good solutions. This paper demonstrates that a highly creative approach underpinned by a statistical model can provide tangible performance gains to the users who need it the most: nonspeaking individuals reliant on a single switch to communicate,&rdquo; says Per Ola Kristensson, professor of interactive systems engineering at Cambridge University, who was not involved with this research. &ldquo;The paper also demonstrates the value of complementing insights from computational experiments with the involvement of end-users and other stakeholders in the design process. I find this a highly creative and important paper in an area where it is notoriously difficult to make significant progress.&rdquo;<hr>&quot;Reprinted with permission of <a href="https://news.mit.edu/2022/system-helps-severely-motor-impaired-individuals-type-more-quickly-accurately-0405" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>&quot;&nbsp;

Researchers at MIT and elsewhere are developing a system that enables severely motor-impaired individuals who communicate using a single switch to do so faster and with more accuracy. Their system is more flexible than many common interfaces, enabling it to be used for tasks like drawing, gaming, or surfing the web. Image: Clock figures courtesy of the researchers, photo via iStockphoto

For individuals who communicate using a single switch, a new interface learns how they make selections, and then self-adjusts accordingly.

System helps severely motor-impaired individuals type more quickly and accurately

The advancements in computer vision, pattern recognition and signal processing have made visual speech recognition possible for a variety of applications, such as human-computer interaction, speaker recognition, talking heads, sign language recognition and video surveillance. Visual speech recognition deals with several technological breakthroughs, like image processing, artificial intelligence, object detection, and statistical modeling only uses visual signals of the lip movement that produce speech without relying on the audio [1]. In the first generation of VSR, the research could not meet the high accuracy demands of the industry due to factors concerning the size of the audio-visual dataset with limited vocabulary and hand-crafted visual features that are not optimal for VSR model training. &nbsp;The introduction of deep learning and neural networks has made visual speech recognition models more robust than ever before while achieving higher accuracy in unstructured real-life situations. Most recent innovations allow the models to perform much better due to the availability of larger datasets and training models with a huge amount of data. A group of researchers affiliated with the Department of Computing at Imperial College London and Meta AI proposed a novel approach to add prediction-based auxiliary tasks to the VSR model and outperform state-of-the-art methods trained on the publicly available data [2].<h2 dir="ltr">Looking to Listen</h2>In the research paper, “Visual Speech Recognition for Multiple Languages in the Wild,” the team carried out work with the aim to reduce the word error rate (WER) through appropriate data augmentation and hyperparameter optimization of the existing architecture while working on the publicly available datasets that are much smaller in size. Through this, the team points out the fact that the optimization of the models can be brought in through carefully designing the models and not only increasing the size of the training data. Also, the proposed novel method for visual speech recognition not only evaluates English as a sole language but also on Mandarin, Spanish, Italian, French and Portuguese and achieves state-of-the-art performance on all the languages.&nbsp;“We optimize hyperparameters and improve the language model with the aim of squeezing extra performance out of the model,” the team explains. Then, “we introduce time-masking which is a temporal augmentation method and is commonly used in ASR models. Finally, we use a VSR model with auxiliary tasks where the model jointly performs visual speech recognition and prediction of audio and visual representations extracted from pre-trained VSR and ASR models.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1647384806878-Visual+Speech+Recognition+Results+on+the+LSR2+Dataset.png" style="width: 800px;" class="fr-fic fr-dib" alt="Visual Speech Recognition Results on the LSR2 Dataset">Visual Speech Recognition Results on the LSR2 Dataset [Image Source: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 374.688px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>Hyperparameter optimization aims to improve the performance of the model by adjusting the values of the parameters that are employed to control the training process and the model architecture. There are several hyperparameters taken into consideration including initial learning rate, learning rate decay parameter, number of layers, size of layers, dropout rate, and the loss function that was performed on the LRS2 dataset achieving reduction in WER on the validation set. The main hyperparameter that had a significant impact on the performance and which might not have been exhaustively optimized was the batch size. The team observed that reduction WER was achieved more when the batch size increased from 8 to 16. Due to GPU memory constraints, the team recognized that certain hyperparameters could not be increased further, but they would bring down the word error rate.&nbsp;<h2 dir="ltr">State-of-the-art performance</h2>The team decided to use all the open-source datasets available for public use, namely LRS2 (an audio-visual English dataset from BBC programs), LRS3 (an audio-visual English dataset taken from TED and TEDx talks), CMLR (a Mandarin dataset collected from Chinese news program), CMU-MOSEAS (contains multiple languages from YouTube videos), Multilingual TEDx (multilingual corpus taken from TEDx talks), and AVSpeech (dataset with over 4700 hours of video in multiple languages). To measure performance, the word error rate is a common performance metric in speech recognition, with an alternative of a character error rate that measures how close the predicted and target character sequences are.&nbsp;For the multilingual TEDxSpanish dataset, the approach results in a 5.6% absolute reduction in the WER and a further reduction of 4.2% can be achieved by using additional training data. While for the Italian corpus, the approach results in an absolute drop of 5.6% in the WER, the same as that for Spanish, but a further reduction of up to 8% for additional training data. “Results on the Multilingual TEDxFrench dataset are shown as a result of a 9.4 % absolute reduction in the WER and a further reduction of 7.6 % can be achieved by using additional training data.” [More details on the additional training data are available in the research article]<iframe width="640" height="360" src="https://www.youtube.com/embed/FIau-6JA9Po?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe>The proposed novel method for visual speech recognition performs visual speech recognition and prediction of audio and visual representation on a publicly available dataset to outperform state-of-the-art methods. The research work was published on Cornell University’s research sharing platform, arXiv under <a href="https://arxiv.org/abs/2202.13084" target="_blank">open access terms</a>. The researchers have provided pre-trained networks and testing code on the authors’ <a href="https://github.com/mpc001/Visual_Speech_Recognition_for_Multiple_Languages" target="_blank">GitHub repository</a> for community access.<h2 dir="ltr">References</h2>[1] Ahmad B. A. Hassanat: Visual Speech Recognition. DOI arXiv: <a href="https://arxiv.org/abs/1409.1411" target="_blank">1409.1411 [cs.CV]</a>.[2] Pingchuan Ma, Stavros Petridis, Maja Pantic: Visual Speech Recognition for Multiple Languages in the Wild. DOI arXiv: <a href="https://arxiv.org/abs/2202.13084" target="_blank">2202.13084 [cs.CV]</a>.&nbsp;

Visual Speech Recognition on a Multilingual Dataset that outperforms the state-of-the-art methods [Image Source: Research Paper]

A novel approach to visual speech recognition that carefully designs the VSR model rather than increasing the size of the training data.


Research suggests improvements in visual speech recognition on a multilingual dataset

<a href="https://brooksrehab.org/" target="_blank">Brooks Rehabilitation</a> recently announced that the MagTrack study, a collaborative research endeavor with the&nbsp;<a href="https://www.ece.gatech.edu/" target="_blank">Georgia Tech School of Electrical and Computer Engineering</a>, has been successfully completed. Feedback from the Brooks clinical team and its patients has allowed Georgia Tech engineers to transform their early research prototype into a user-ready version that was tested by more than 17 power wheelchair users living with tetraplegia — a form of paralysis caused by spinal cord injury that affects the arms, hands, trunk, legs, and pelvic organs.The collaboration between the Brooks and Georgia Tech teams has created a path to a first-of-its-kind, innovative application for individuals living with disabilities. The teams — comprised of physicians, clinical therapists, and engineers — brought together multidisciplinary expertise in advanced science, technology, and clinical rehabilitation.“We met with the Georgia Tech team years ago, when we first heard of the research breakthroughs they were achieving for wheelchair users. Brooks is constantly looking for technology that is useful for our patient population living with spinal cord injuries and mobility impairments. To see where the MagTrack project has advanced even just since the early stages of this study is incredible,” said Geneva Tonuzi, medical director of the spinal cord injury program, the spinal cord injury and related disorders day treatment program, and Cyberdyne HAL Therapy at Brooks Rehabilitation.As a result of this engineering-clinical collaboration,&nbsp;<a href="https://magtrack.ece.gatech.edu/" target="_blank">MagTrack</a> was created as a cutting-edge assistive technology that enables power wheelchair users to control their connected devices (e.g., smartphone, computer) and drive their power wheelchairs using an alternative, multimodal controller. In addition, the assistive device is designed to be wearable, wireless, and adaptable to the user’s specific condition.The MagTrack study is earning the praise of patients and scientists alike and has been&nbsp;<a href="https://ieeexplore.ieee.org/document/9540352" target="_blank">published</a> in&nbsp;<a href="https://www.embs.org/tbme/" target="_blank">IEEE Transactions on Biomedical Engineering</a>. From the beginning, the MagTrack studies have tested the performance of the Head-Tongue Controller (HTC), an earlier version of the MagTrack technology, on its ability to perform complex human-machine interactions that will enhance users’ quality of life. The MagTrack’s HTC allows the user to perform a variety of complex tasks in a single controller through the use of tongue and head movements, which are detected by eyewear and a tiny tracer that is temporarily glued onto the tongue using Glustitch’s&nbsp;<a href="https://periacryl.com/" target="_blank">PeryAcryl</a> bio-compatible adhesive. Target-specific commands are generated from these motions using advanced data processing and machine learning models. This combination of input modalities allows the user to perform a variety of daily functions with customizable control, from performing complex computer tasks (e.g., mouse navigation, scrolling, drag-and-drop) to completing advanced driving maneuvers when connected to a power wheelchair.In the latest study, researchers connected the MagTrack technology to a single power wheelchair donated by Quantum Rehab, and recruited 17 patient volunteers from Brooks Rehabilitation to test the functionality and usability of the device by completing a set of both simple and advanced driving tasks. Results showed that new users of MagTrack can complete these tasks as fast, and sometimes even faster, with the MagTrack’s HTC rather than their personal, alternative controller. Since the study session lasted less than 3 hours — and in a power wheelchair that wasn’t their own — it is anticipated that participants would be more proficient, and thus perform better with MagTrack, if they were given more time to familiarize themselves with its multimodal capabilities and using their own power wheelchair."Working with all of the participants has been very rewarding,” said Jesse Milliken, speech-language pathologist in the spinal cord injury program at&nbsp;<a href="https://brooksrehab.org/" target="_blank">Brooks Rehabilitation Hospital</a>. “Each patient who came in was someone who has been directly affected by a spinal cord injury and who can truly benefit from this technology. It was amazing to see how their faces lit up when they saw they were able to control their wheelchair with such ease and comfort. They all said they can see this improving their day-to-day lives if it were available to them. It's been such an honor to be a part of this process and see the work and thought process behind such advanced technology." Patients from the&nbsp;<a href="https://brooksrehab.org/" target="_blank">Brooks</a> spinal cord injury program who have participated in phase three of the study call the advancement “exciting” and a “great system that can be used for so many things.” After experiencing the technology themselves, they believe it will “touch the lives of those who are able to use it.”To date, the head array and sip-and-puff are the most common alternative controllers recommended by physical therapists to individuals living with tetraplegia, while specialized switches and joystick technology are available for those with mobility in their upper extremities. These technologies were developed many decades ago for the basic need of controlling a power wheelchair. Since then, a lack of innovation in this field has hindered these assistive technologies from adapting to today’s technology. Furthermore, they are affixed to the wheelchair, which becomes inaccessible once the user is transferred to a bed, a couch, or any location away from the wheelchair. Therefore, there is a growing need for this population to have access to new, alternative controllers that will enable them to be active members of an interconnected digital world.“The trajectory of the MagTrack study shows an unprecedented possibility for the advancement of independent function as well as mobility for electric wheelchair users. Our team and partners are energized and motivated by the recent patient trials to continue to push this technology and its capabilities as far as possible. This technology can significantly improve people’s lives. We will continue to work to see these advances in assistive technology come to life,” said Georgia Tech’s Omer T. Inan, director of the Inan Research Lab, Linda J. and Mark C. Smith Chair in Bioscience and Bioengineering, associate professor in the School of Electrical and Computer Engineering, and adjunct associate professor in the Wallace H. Coulter Department of Biomedical Engineering.“MagTrack is an innovative assistive technology aimed for those living with physical paralysis to have access to more complex human-machine interactions, which will facilitate the control of more devices in their everyday life that they cannot easily use otherwise,” said Nordine Sebkhi, postdoctoral researcher at the Inan Research Lab in the School of Electrical and Computer Engineering. “The development of our wearable alternative controller eliminates the need for having multiple assistive technologies, replacing them with a single multimodal and integrated system.” Sebkhi is the co-creator of MagTrack and the technical lead in the development of this assistive technology.As a result of these studies,&nbsp;<a href="https://magtrack.ece.gatech.edu/" target="_blank">MagTrack</a> has been refined to offer a fully integrated, all-in-one experience so that a user can seamlessly switch between driving their wheelchair and controlling connected devices in their surroundings (e.g., smartphone, computer, automated door opener, smart TV). The system can be used anywhere since it is wearable, and its built-in wireless connectivity facilitates portability.The team at Georgia Tech is already working on a new version of&nbsp;<a href="https://magtrack.ece.gatech.edu/" target="_blank">MagTrack</a> that is not only more inconspicuous, but also includes detection of facial gestures that significantly augments its control capabilities. Thanks to a grant from the&nbsp;<a href="https://gra.org/" target="_blank">Georgia Research Alliance</a>, this new version of MagTrack will be tested in a focus group at the&nbsp;<a href="https://www.shepherd.org/" target="_blank">Shepherd Center</a> and in simulated testing in a home-like environment. In the coming year, the team plans to make MagTrack available to early adopters for at-home validation testing to further improve the technology before pursuing commercialization.This successful study is only a start since, at its core, MagTrack is a new type of body motion tracking. The Georgia Tech team is working on various designs of MagTrack to be used as a wearable articulograph for motor speech disorders, as a hand and joint tracking system for physical rehabilitation, and even as a finger tracking for VR/AR applications. The MagTrack team will be partnering with the&nbsp;<a href="https://gcmiatl.com/" target="_blank">Global Center for Medical Innovation</a> to assist in regulatory strategy and project planning to transition the technology from the lab to the market.

A squint is an example of a facial gesture that MagTrack can detect and issue a discrete command to control a connected device.

Clinical work begins with MagTrack, a cutting-edge assistive technology that enables power wheelchair users to control their connected devices and drive their power wheelchairs using an alternative, multimodal controller.

MagTrack Technology Opens Doors for Independent Operation of Smartphones, Computers, and Other Devices for Wheelchair Users

Training robots to learn new skills has been a complex process restricting the work for programmers and roboticists. Scientists who study human-robot interaction often focus on easing the robot training process for it to learn new tasks in an unstructured environment. Today, most of the techniques for teaching robots are limited to the use of remote hardware or imitating pre-recorded demonstrations. The idea of teaching robots through a self-learning mechanism by watching humans do the task— allows the deployment of these robots in a new setting. As fancy as it sounds, there are certain challenges faced by the vision-based learning process as there exists a clear embodiment gap between the human demonstrator and the robotic hardware that executes the task.&nbsp;As reported by the existing research [1,2], even though humans and robots are capable of performing the same task, the differences in the physique can make easy tasks complex or even impossible to do. A team of researchers from Stanford University, Robotics at Google and the University of California at Berkeley collaborated to apply well-established reporting of human-robot interaction to focus on teaching robots from third-person observations of demonstrations [3]. The researchers identified the opportunity to investigate the scope of self-supervised methods for vision-based learning techniques while considering the differences in human-robot embodiments, like shape, action and end-effector dynamics.&nbsp;<h2 dir="ltr">Robots See, Robots Do</h2>In the paper, “XIRL: Cross-embodiment Inverse Reinforcement Learning,” the researchers carried out work to solve the challenges of embodiment through self-supervised mechanisms of high-level task objectives from videos. Rather than focusing on the way humans perform tasks and relating it to robot training, the proposition summarizes the knowledge in the form of a reward function that is invariant to the embodiment differences. This means it is now possible to learn vision-based reward functions from cross-embodiment demonstration videos that are robust to the differences mentioned earlier.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1646257426964-Cross-Embodiment+Inverse+Reinforcement+Learning.gif" style="width: 800px;" class="fr-fic fr-dib" alt="Cross-Embodiment Inverse Reinforcement Learning">XIRL self-supervises reward functions using temporal cycle consistency (TCC), then uses them for reinforcement learning to learn new skills from third-person demonstrations [Image Source: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 429.75px; height: 63px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>“The learned rewards can be used together with reinforcement learning (RL) to teach the task to agents with new physical embodiments through trial and error,” the team explains in a <a href="https://ai.googleblog.com/2022/02/robot-see-robot-do.html" rel="noopener noreferrer" target="_blank">Google AI blog post</a>. “Our approach is general and scales autonomously with data — the more embodiment diversity presented in the videos, the more invariant and robust the reward functions become.”Even in the presence of embodiment differences, there still exist visual cues that indicate the task objective. The fundamental idea behind the XIRL is to discover the key moments in the videos of different lengths and clusters to encode task progression. The work carried out in XIRL leverages the Temporal Cycle Consistency (TCC) [4] that aligns the video accurately while learning the visual cues for understanding the task objective without requiring any ground-truth correspondences. The reward function in the learning process is the negative Euclidean distance between the current observation and the goal observation in the learned embedding space. This reward function is then fed into a standard Markov decision process and uses a reinforcement learning algorithm to learn the third-person demonstrated behavior.&nbsp;<h2 dir="ltr">Transfer policies from humans to robots in simulation</h2>Along with the cross-embodiment inverse reinforcement learning label-free framework, the team proposes a cross-embodiment imitation learning benchmark, X-MAGICAL that features several tried agents with different embodiments performing the same defined task including one thousand expert demonstrations for a single agent. “The goal of X-MAGICAL is to test how well imitation or reward learning techniques can adapt to systematic embodiment gaps between the demonstrator and the learner,” the team notes. “These differences in execution speeds and state-action trajectories pose challenges for current LfD techniques, and the ability to generalize across embodiments is precisely what this benchmark evaluates.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1646257571549-Comparison+of+XIRL+with+baselines+using+the+simulated+State+Pusher+environment.png" style="width: 800px;" class="fr-fic fr-dib" alt="Comparison of XIRL with baselines using the simulated State Pusher environment">Comparison of XIRL with baselines using the simulated State Pusher environment [Image Source: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 477.5px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>The results show that the XIRL significantly outperforms the alternative methods on both the X-MAGICAL benchmarks and the human-to-robot transfer benchmark. The team demonstrates the effectiveness of the proposed methodology by using third-person demonstrations on the State Pusher [5] setting where the reward function is first learned on real-world human demonstration and then used to teach a Sawyer arm to perform the task in simulation. The XIRL methodology trained on real-world demonstration (purple) reaches 80% of the total performance around 85% faster than the reinforcement learning with videos (RLV) as baseline (orange).<h2 dir="ltr">Conclusion on XIRL be used for learning RL policies</h2>In the proposed approach to tackle the cross-embodiment imitation problems, the XIRL learns an embodiment invariant reward function technique using a temporal cycle consistency objective. Compared to baseline alternatives, XIRL reward functions are significantly more efficient. While the experimental results are quite promising, the team plans to show policy learning on a real robot post-COVID pandemic.&nbsp;This research article was published on Cornell University’s research sharing platform, arXiv under <a href="https://arxiv.org/pdf/2106.03911.pdf" rel="noopener noreferrer" target="_blank">open access terms</a>.<h2 dir="ltr">References</h2>[1] Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, Sergey Levine: Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning. DOI <a href="https://arxiv.org/abs/1703.02949" target="_blank">arXiv:1703.02949 [cs.AI]</a>.[2] Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine: Time-Contrastive Networks: Self-Supervised Learning from Video. DOI <a href="https://arxiv.org/abs/1704.06888v3" target="_blank">arXiv:1704.06888 [cs.CV]</a>.[3] Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, Debidatta Dwibedi: XIRL: Cross-embodiment Inverse Reinforcement Learning. DOI <a href="https://arxiv.org/abs/2106.03911" target="_blank">arXiv:2106.03911 [cs.RO]</a>.[4] D. Dwibedi, Y. Aytar, J. Tompson, P. Sermanet, and A. Zisserman. Temporal cycle-consistency learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.[5] K. Schmeckpeper, O. Rybkin, K. Daniilidis, S. Levine, and C. Finn. Reinforcement learning with videos: Combining offline observations with interaction, 2020

Real-world-demo cross-embodiment setting [Image Source: Research Paper]

Human-robot interaction focuses on teaching robots from third-person observations of demonstrations to make it invariant and robust to the embodiment differences.

Vision-based learning technique for robots during training

Voice control has emerged as one of the most popular approaches for hands-free interaction with technology, from dictating short messages to a smartphone to telling a home assistant system to turn the heating up. It has one big drawback, however: The fact you have to voice your intentions aloud, which brings with it privacy and etiquette issues for public use.SpeeChin, a necklace-style wearable developed at Cornell University with colleagues from McGill and Zhejiang Universities, aims to solve that problem — by ensuring you never have to utter a single word out loud, using silent speech recognition (SSR) derived from infrared camera footage.<h1>Why a necklace?</h1>“There are two questions: First, why a necklace? And second, why silent speech,” corresponding author Chen Zhang, assistant professor at Cornell, says of the project. "We feel a necklace is a form factor that people are used to, as opposed to ear-mounted devices, which may not be as comfortable.“As far as silent speech, people may think, ‘I already have a speech recognition device on my phone.’ But you need to vocalize sound for those, and that may not always be socially appropriate, or the person may not be able to vocalize speech.”Silent speech recognition systems which track facial movements associated with speech without the need for the user to make any actual noise — simply mouthing the word is enough. Existing systems have typically relied on a front-facing camera or sensors fitted to the head, face, or even inside the mouth, none of which are particularly well-suited to frequent use.<iframe src="https://www.youtube.com/embed/pDf86dSPYYI?&amp;wmode=opaque&amp;rel=0" allowfullscreen="" class="fr-draggable" width="640" height="360" frameborder="0"></iframe> SpeeChin, by contrast, uses a single infrared camera module plus infrared LEDs for lighting — and points it up at the user’s chin from a 3D-printed necklace worn on a small silver chain around the neck. The images thus captured are processed, then fed through a deep-learning system which uses changes in the shape of the user’s neck and face to determine which of a series of pre-set command words are being silently uttered.“We first tried using a standard RGB [visible light] camera,” the team explains of its prototypes. “However, we found it difficult to segment the user’s head from different backgrounds. We considered using a thermal camera or depth camera [but these] are too large to be considered ‘minimally obtrusive’ or the image resolution is too low. We settled on an IR camera due to its convenient filter system for background segmentation, small size, and relatively high image resolution.”Using an infrared camera system brings an additional advantage, too: Fitted with filters operating at 650nm and 850nm, the device is unable to capture background details easily — addressing a big privacy issue with previous camera-based silent speech recognition implementations.<h1>Learning the words</h1>Images from the necklace camera, which is connected to a low-cost off-the-shelf Raspberry Pi single-board computer, are pre-processed to remove noise caused by changes in the device’s angle and positioning — minimized through the use of a weighted base and a free-sliding chain — and from clothes or lights entering the frame.Once processed, the images are split up based on an utterance detection algorithm, designed to cut actual utterances out of general movement, and then fed into the SpeechNet end-to-end deep learning model. Based on a combination of both convolutional and recurrent neural network (CNN and RNN) technologies, the complete chain includes a global average pooling step and a long short-term memory (LSTM) block plus a final linear layer which outputs the detected speech command.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1646240571681-speechin2.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="An image showing the SpeeChin wearable with silver chain, LEDs, lens, 3D printed case, weight-balancing coin, and FPC cable to a Raspberry Pi. The data flow is also shown: footage capture, pre-processing, passing through the SpeechNet model, and recognition of silent speech commands.">The SpeeChin sits around the wearer's neck to pick up on unvocalized silent speech commands, with surprisingly accurate results in ideal conditions. Trained on an augmented dataset — boosted by splitting captured footage’s 60 frames per second into two samples at 30 frames per second plus randomized affine transformation to simulate translation and rotation — the model was used across two primary studies: One with 10 participants testing 54 English-language commands, and another with a further 10 participants using 44 Mandarin commands.The commands chosen for both languages ranged from digits and smartphone-related interaction tasks like call answering and media playback to punctuation, text handling, and common voice assistant wake-phrases like “Alexa” and “Hey, Siri.” The results are impressive: The English-language test completed with a 90.5 per cent accuracy, while the Mandarin test hit 91.6 per cent.A third test tried out “non-words” in an effort to determine the SpeeChin system’s ability to extend to the ability to recognize individual phonemes and thus greatly expand its recognizable vocabulary. Performance here was weaker, but showed enough promise — particularly in being able to distinguish between phonemes involving the tongue, despite it being hidden from the camera’s view — for the team to consider implementing a phoneme-based variant in the future.<h1>Work in progress</h1>SpeeChin isn’t quite ready to replace audibly-spoken voice command technology just yet, however. The team admitted to weak user-independent performance — 54.4 per cent and 61.2 per cent in English and Mandarin tests respectively, with a high standard deviation — which could be improved by training a user-adaptive model.A bigger issue is in the system’s use in mobile settings, where it would arguably be of most use. In a six-participant study, users were given the freedom to walk around while being fed instructions on commands to use via speakers: Differences in users’ walking styles and head movement dropped the accuracy to 72.3 and 65.5 per cent for English and Mandarin respectively in a reduced test set of just 10 phrases each — an issue the team suggests could be corrected through a change in the necklace’s design to reduce movement, stronger post-processing, the collection of a larger-scale training dataset, or the introduction of an activation gesture to trigger recognition.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1646240694251-speechin3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="Three images captured by the SpeeChin camera: the first with no sunlight, showing a clear view of the users; the second under strong sunlight showing a more muddied view with background noise. The third image shows a higher-performance IR camera, the Leap Motion, delivering better results.">The team has proposed a move to a different infrared camera platform to address problems with strong sunlight. Other areas to be improved include the prototype’s power consumption, which draws 5.4W excluding the deep-learning system running on a considerably more powerful external server, the camera’s sensitivity to strong sunlight, and the influence of clothing and hair which can occlude the user.The team’s work has been published under closed-access terms in the <cite><a href="https://dl.acm.org/doi/10.1145/3494987" target="_blank">Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies</a></cite>.<h1>Reference</h1>Ruidong Zhang, Mingyang Chen, Benjamin Steeper, Yaxuan Li, Zihan Yan, Yizhuo Chen, Songyun Tao, Tuochao Chen, Hyunchul Lim, and Cheng Zhang: <cite>SpeeChin: A Smart Necklace for Silent Speech Recognition, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Vol. 5 Iss. 4</cite>. DOI <a href="https://doi.org/10.1145/3494987" target="_blank">10.1145/3494987</a>.

The SpeeChin prototype is worn around the neck to capture facial movements via an upwards-angled infrared camera.

Designed as a necklace-style wearable in a 3D-printed case, SpeeChin is capable of recognizing 54 English and 44 Chinese voice commands — even if the wearer never says anything out loud.

SpeeChin aims to boost privacy by using silent speech recognition for smartphone control

As technological advances come through, humans and robots are working more closely together to increase the productivity of industries and the quality of manufactured products, resulting in improved efficiency and growth. The research interest is to design humanoid robots that can become co-workers more than mere tools. From industrial robots to domestic and service robots, roboticists and computer engineers are witnessing the need for human-robot collaboration. For deploying robotic systems to work alongside human resources, it is important for humans to trust the abilities of their functionalities and willingness to use the robot as a co-worker.&nbsp;Human-robot interaction has been widely studied in the past decade, evaluating robotic failures and varying severity in terms of impactful consequences. However, the existing research evaluation was restricted to artificial settings which lacked real-life external parameters, resulting in an inaccurate response. Personal relevance (PeR), defined as the level of involvement with an object (in this case, Robot) has been impacting the user preferences and perceptions of the robots. Celsi and Olson found that PeR affects human perception [1] which is why a group of scientists from the Ben-Gurion University of the Negev, in Israel [2] speculated that this will affect robotic failures’ perception.&nbsp;<h2 dir="ltr">The Evaluation</h2>In the research article, “Is it personal? The impact of personally relevant robotic failures (PeRFs) on humans' trust, likeability, and willingness to use the robot,” the team carried out a study to evaluate the factors that affect how humans trust the robots– without affecting the productivity. The paper primarily focuses on the impact of personally relevant robotic failures (PeRFs) which would reduce the trust in robots and robots’ likeability and willingness to use more than failures that are not personal to the user.&nbsp;While the existing research failed to evaluate the factors in real-life scenarios, the team carried out a series of three experiments in the experimental environment, designed to simulate a realistic laundry sorting setup. To investigate the role of PeR in human-robot interaction, the three different laboratory experiments included one with damaging to property, the second with causing financial losses, followed by the first-person versus third-person failure scenarios. For the collaborative task of laundry sorting, a total of 132 participants engaged with the robot with the results indicating the impact of PeRFs on perceptions of the robot differed across the experimental setups.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ1Njg4MDQ4OTY2LVJvYm90aWMtSHVtYW4gSW50ZXJhY3Rpb24ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 800px;" class="fr-fic fr-dib" alt="Summary of Experiment 1 to check user preference after robotic failures">Experiment 1: The interaction between the level of personal relevance and the session in terms of trust and willingness to use. [Image Credit: Research Paper]<div is-sapling-overlay="true" style="pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 477.7px; height: 63px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: &quot;PT Serif&quot;, serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;"></div><div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 477.5px; height: 63px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>Specifically, in the first experiment, the team aims to evaluate the impact of a failure that may cause damage to participants and examine the interaction between PeR and perceived failure severity. The severity differed from being high when the robot dropped a clothing object into the trash can instead of the laundry bin while the severity dropped when the robot dropped the clothing item onto the floor. The second experiment of financial loss due to delays caused by robotic failures reported that the failures with PeR can negatively impact humans’ trust in robots. The results from experiment two suggest that the perceived trust before failure was more than trust after failure, indicating that the failures decreased the user’s trust.&nbsp;Finally, in the last experimental setup, the failures were wrong gender identification of the participant or the experimenter. The conversation between the robot and the participant was in Hebrew, a gendered language that addresses the other person specifying the person’s gender. In high PeR, the robot incorrectly identified the participant’s gender while the lower PeR suggested that the robot incorrectly identified the experimenter’s gender. Surprising in this result, there was no difference between trust before failure and trust after failure, while no difference between LWtU before failure and after failure. This indicated that the manipulation of the interaction PeRF had no impact on the participant.&nbsp;The findings of the research article indicate that the robot’s failure changed human perception and impacted the extent to which users are ready to trust a robot for its designed functionality. The willingness to use had been affected by the robot’s failure which sparked an interesting observation fueling future studies. This research article was published on Cornell University’s research sharing platform, arXiv under <a href="https://arxiv.org/abs/2201.05322" target="_blank">open access terms</a>. &nbsp;&nbsp;<h2 dir="ltr">Reference</h2>[1] R. L. Celsi and J. C. Olson, “The Role of Involvement in Attention and Comprehension Processes,” JOURNAL OF CONSUMER RESEARCH, vol. 15, pp. 210–224, 1988.[2] Romi Gideoni, Shanee Honig, Tal Oron-Gilad: Is it personal? The impact of personally relevant robotic failures (PeRFs) on humans' trust, likeability, and willingness to use the robot. DOI <a href="https://arxiv.org/abs/2201.05322" target="_blank">arXiv:220105322</a> [cs.RO]

The robotic-human interaction experimentation– Laundry Sorting Workstation [Image Credit: Research Paper]

Impact of personally relevant robotic failures (PeRFs) which would reduce trust in robots and robots’ likeability and willingness to use more than failures that are not personal to the user. 


Did personally relevant robotic failures affect human perception?

The images made headlines around the world in late 2018. David Mzee, who had been left paralyzed by a partial spinal cord injury suffered in a sports accident, got up from his wheelchair and began to walk with the help of a walker. This was the first proof that Courtine and Bloch’s system – which uses electrical stimulation to reactivate spinal neurons – could work effectively in patients.<iframe src="https://www.youtube.com/embed/4wUADfnCMdc?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 767px; height: 437px;"></iframe>Fast forward three years, and a new milestone has just been reached. The research team led by both Courtine, a professor at EPFL, and Bloch, a professor and neurosurgeon at CHUV, has enhanced their system with more sophisticated implants controlled by artificial-intelligence software. These implants can stimulate the region of the spinal cord that activates the trunk and leg muscles. Thanks to this new technology, three patients with complete spinal cord injury were able to walk again outside the lab. “Our stimulation algorithms are still based on imitating nature,” says Courtine. “And our new, soft implanted leads are designed to be placed underneath the vertebrae, directly on the spinal cord. They can modulate the neurons regulating specific muscle groups. By controlling these implants, we can activate the spinal cord like the brain would do naturally to have the patient stand, walk, swim or ride a bike, for example.”<h2>Activating motor sequences at the push of a button</h2>On a cold, snowy day last December, Michel Roccati – an Italian man who became paralyzed after a motorcycle accident four years earlier – braved the icy wind to try out the system outdoors, in central Lausanne. He had recently undergone the surgical procedure in which Bloch placed the new, implanted lead on his spinal cord. Scientists from the Courtine and Bloch’s .NeuroRestore research center were with him, helping to prepare the demonstration. They attached two small remote controls to Michel’s walker and connected them wirelessly to a tablet that forwards the signals to a pacemaker in Michel’s abdomen. The pacemaker in turn relays the signals to the implanted spinal lead that stimulates specific neurons, causing Michel to move. When he was ready, Michel grasped the walker and set off. He pressed the button on the right side of the walker with the firm intention of taking a step forward with his left leg. His left foot rose like magic and fell to the ground a few centimeters ahead. He then did the same thing with the button on his left side, and his right foot moved forward. He was walking! “The first few steps were incredible – a dream come true!” he says. “I’ve been through some pretty intense training in the past few months, and I’ve set myself a series of goals. For instance, I can now go up and down stairs, and I hope to be able to walk one kilometer by this spring.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644544468002-ezgif-5-e5d9eb5e93.jpg" style="width: 766px;" class="fr-fic fr-dib">Michel Roccati stands up and walks in Lausanne. © EPFL / Alain Herzog 2021&nbsp;Two other patients have also successfully tested the new system, which is described in an <a href="https://www.nature.com/articles/s41591-021-01663-5" rel="noopener" target="_blank">article appearing today in Nature Medicine</a>. “Our breakthrough here is the longer, wider implanted leads with electrodes arranged in a way that corresponds exactly to the spinal nerve roots,” says Bloch. “That gives us precise control over the neurons regulating specific muscles.” Ultimately, it allows for greater selectivity and accuracy in controlling the motor sequences for a given activity.<h2>One day is all it takes</h2>Extensive training is obviously necessary for patients to get comfortable using the device. But the pace and scope of rehabilitation is amazing. “All three patients were able to stand, walk, pedal, swim and control their torso movements in just one day, after their implants were activated!” says Courtine. “That’s thanks to the specific stimulation programs we wrote for each type of activity. Patients can select the desired activity on the tablet, and the corresponding protocols are relayed to the pacemaker in the abdomen.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644544498412-ezgif-5-2264b1550c.jpg" style="width: 766px;" class="fr-fic fr-dib">Thanks to his implant, Michel Roccati can now stand to enjoy a drink. © EPFL / Alain Herzog 2021&nbsp;While the progress achievable in a single day is astonishing, the gains after several months are even more impressive. The three patients followed a training regimen based on the stimulation programs and were able to regain muscle mass, move around more independently, and take part in social activities like having a drink standing at a bar. What’s more, because the technology is miniaturized, the patients can perform their training exercises outdoors and not only inside a lab.“This study further demonstrates the benefits of our approach,” says Courtine. “We’re now working with <a href="http://onwd.com/" rel="noopener" target="_blank">ONWARD Medical</a>, which recently listed on Euronext, to turn our discoveries into genuine treatments that can improve the lives of thousands of people around the world.”<hr>ReferencesActivity-dependent spinal cord neuromodulation rapidly restores trunk and leg motor functions after complete paralysis, Nature Medicine, February 7 2022, DOI: https://doi.org/10.1038/s41591-021-01663-5

A system developed by Grégoire Courtine and Jocelyne Bloch now enables patients with a complete spinal cord injury to stand, walk and even perform recreational activities like swimming, cycling and canoeing. 

New implant offers promise for the paralyzed

In this episode, we talk about how a new wearable device can combat fatigue and how monitoring BAC via sweat can be a new use case for wearables. As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/c866dfbc-964d-4828-b95e-edf2ff87e8da?dark=false" class="fr-draggable"></iframe>&nbsp;<hr>This podcast is sponsored by <a href="https://www2.mouser.com/" id="isPasted" rel="nofollow" target="_blank">Mouser Electronics</a>.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644321153399-Mouser-sponsor-1-a.jpg" style="width: 766px;" class="fr-fic fr-dib"><hr><h3 id="isPasted">EPISODE NOTES</h3>(3:00) - <a href="https://www.wevolver.com/article/energizing-the-brain-combating-worker-fatigue-using-wearable-neurotechnology/" target="_blank">Energizing The Brain🧠⚡️</a>(14:35) - <a href="https://www.wevolver.com/article/can-wearable-technology-predict-the-negative-consequences-of-drinking/" target="_blank">Wearables For Alcohol Consumption⌚️🍻</a>Episode 56 was brought to you by Mouser Electronics, Farbod &amp; Daniel’s favorite electronics distributor. Click <a href="https://www.mouser.com/applications/article-iot-wearable-devices/" target="_blank">here&nbsp;</a>to read the article giving an overview of how wearable electronics are transitioning away from being peripheral devices.<hr>About the podcast:Every day, some of the most innovative universities, companies, and individual technology developers share their knowledge on Wevolver. To ensure we can also provide this knowledge for the growing group of podcast listeners, we started a collaboration with two young engineers, Daniel Scott Mitchell &amp; Farbod Moghaddam who discuss the most interesting content in this podcast series.&nbsp;To learn more about this show, please visit the <a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">shows page</a>. By following the page, you will get automatic updates by email when a new show is published.Be sure to give us a follow and review on <a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" rel="nofollow" target="_blank">Apple</a> podcasts, <a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" rel="nofollow" target="_blank">Spotify</a>, and most of your favorite podcast platforms!

Out of the counterbalance of anodal stimulation, sham stimulation and no stimulation, the team found that by exciting neuronal activity through anodal stimulation, task performance under fatigue improved, while it decreased under the other two sessions. | Image: Getty Images

human-machine interaction (hmi)