Wearable haptic devices, which provide touch-based feedback, can provide more realistic experiences in virtual reality, assist with rehabilitation, and create new opportunities for silent communication. Currently, most of these devices rely on vibration, as pressure-based haptics have typically required users to wear stiff exoskeletons or other bulky structures.Now, researchers at Stanford Engineering have designed a comfortable, flexible knit sleeve, called Haptiknit, that can provide realistic pressure-based haptic feedback. Their design, published Dec. 18 in Science Robotics, shows that pressure may be more effective than vibration for some applications and is the first step toward a new category of haptic devices.“A device like this opens up a lot of new possibilities for user interfaces – how we experience virtual environments, how we experience remote communication,” said&nbsp;<a href="https://profiles.stanford.edu/allison-okamura" data-extlink="">Allison Okamura</a>, the Richard W. Weiland Professor of Engineering at Stanford and senior author on the paper. “It’s much more lightweight, wearable, and comfortable.”<h2>A knit solution</h2>Okamura and her colleagues designed a battery-powered pneumatic system with pressure actuators that were essentially small, inflatable pouches that could be rapidly filled with air. But they needed a way to hold those pouches against the skin without using a clunky exoskeleton.“If you put air into a balloon next to your skin but don’t anchor it there, it’s going to expand in all directions,” said Cosima du Pasquier, a postdoctoral researcher at Stanford and first author on the paper. “You’re going to waste most of the inflation potential.”Du Pasquier, who makes clothes as a hobby, realized that knit fabric might hold the answer. The researchers designed a soft textile that would be stiff in some areas – creating an inflexible backing to hold the pressure actuators against the skin – and flexible where needed to allow for movement and comfort.They worked with a team at MIT’s Self-Assembly Lab to manufacture the Haptiknit sleeve prototype on a knitting machine, with space for eight actuators arranged in two rows. Most of the sleeve was knit from nylon and cotton, but the areas backing each actuator also included a thermoplastic fiber. Once the knitting was done, the researchers used heat to melt the thermoplastic fibers and cause them to harden, stiffening those areas.<iframe width="640" height="360" src="https://www.youtube.com/embed/il4lICuOIAc?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>Demo video of the “Knit Haptic Pneumatic Sleeve” | Skylar Tibbits, MIT Self-Assembly Lab“A challenge in the field of soft robotics is how do you stick together something hard and something soft – they tend to delaminate,” Okamura said. “But putting these fibers into knitting makes a totally seamless transition from parts that are hard to parts that are soft, because it’s one continuous fabric.”The researchers tested the Haptiknit prototype with 32 users. They found that people could more accurately discern the location of individual touches from the pressure actuators than from a similarly arrayed vibrational device. Okamura and her colleagues also tried inflating the actuators consecutively at different speeds to try to create the feeling of a pleasant stroke, as opposed to discrete touches (or the potentially unpleasant feeling of a spider crawling up one’s arm). In general, participants found faster, more overlapping indentation mimicked the feeling of a continuous stroke – the opposite of what the researchers had found for vibration.The third test was to see if participants could recognize six emotions – attention, gratitude, happiness, calming, love, and sadness – conveyed through the pressure signals. The researchers used touch patterns established in previous research and found that participants generally guessed correctly at a higher-than-chance rate, although the gestures for “calming” and “love” were easily confused.<h2>More comfortable haptics</h2>Overall, participants found the Haptiknit sensations to be equally or more pleasant than those caused by vibrations. They also described the prototype sleeve as comfortable and easy to use, which is promising for more long-term uses.“What was particularly interesting was that there was a correlation between whether someone had tried a haptic device before and how highly they rated the comfort of our sleeve,” du Pasquier said. “Essentially, if they had tried out other haptic devices, they scored our sleeves much higher.”With more lightweight, comfortable haptic devices, the researchers are envisioning new opportunities for conveying information through touch. Haptiknit could be used to aid in navigation, military communication, and even dog training, du Pasquier said.Okamura, du Pasquier, and their colleagues are working on refining and optimizing their knitting patterns, as well as creating larger-scale devices – perhaps even a full suit – for virtual reality interactions. They also hope to incorporate this work into assistive devices that can help people move or aid in rehabilitation.“We can use this to start testing how people actually interpret and respond to this type of haptic information,” Okamura said. “Whether the purpose is entertainment, communication, training, or physical assistance, this really brings these wearable devices toward things that people might actually want to use in their everyday lives.”&nbsp;<hr>Okamura, a professor of mechanical engineering, is a member of<a href="http://biox.stanford.edu/" data-extlink="">&nbsp;Stanford Bio-X</a>, the<a href="https://humanperformance.stanford.edu/" data-extlink="">&nbsp;Wu Tsai Human Performance Alliance</a>&nbsp;and the<a href="https://neuroscience.stanford.edu/" data-extlink="">&nbsp;Wu Tsai Neurosciences Institute</a>, a faculty affiliate of the<a href="https://hai.stanford.edu/" data-extlink="">&nbsp;Institute for Human-Centered Artificial Intelligence</a>, a science fellow at the&nbsp;<a href="https://www.hoover.org/" data-extlink="">Hoover Institution</a>, an executive committee member for the&nbsp;<a href="https://src.stanford.edu/" data-extlink="">Stanford Robotics Center</a>, and a faculty member of the&nbsp;<a href="https://centerfordesignresearch.stanford.edu/" data-extlink="">Center for Design Research</a>.Additional Stanford co-authors of this research are visiting students Ian Scholl and Liana Tilton.Other co-authors of this work are from the Massachusetts Institute of Technology and the University of Houston.

Haptiknit sleeve worn on the arm, including the control system on the upper arm. | Susan Williams, MIT Self-Assembly Lab

Researchers at Stanford Engineering have developed a lightweight, comfortable knit sleeve that uses pressure-based haptics to simulate touch, opening up new possibilities for wearable devices.

New knit haptic sleeve simulates realistic touch

How can virtual reality (VR) be experienced haptically, i.e., through the sense of touch? This is one of the fundamental questions that modern VR research is investigating. Computer scientist André Zenner, who is based in Saarbrücken, Germany, has come a significant step closer to answering this question in his doctoral thesis – by inventing new devices and developing software-based techniques inspired by human perception. He has now been awarded the prestigious "Best Dissertation Award" at the world's leading VR conference.The award-winning work is about how physical props (technical term: "proxies") can be used to make objects in virtual environments tangible. "Of course, you can't have a proxy for every virtual object, then the approach wouldn't be scalable. In my dissertation, I, therefore, thought about what devices could look like that could be used to simulate the physical properties of several different virtual objects as effectively as possible," explains André Zenner, who completed his doctorate at the Saarbrücken Graduate School of Computer Science at Saarland University and is now conducting research at Saarland University and the German Research Center for Artificial Intelligence.This resulted in the prototypes for two special VR controllers, "Shifty" and "Drag:on." VR controllers are devices that can be held in the user’s hand to control or manipulate objects in virtual reality using tracking technology.<a target="_blank"></a><header><h2>Shifty</h2></header>"Shifty" is a tubular controller in which a movable weight is installed. The weight can be moved along the lengthwise axis by a motor, changing the center of gravity and inertia of the rod. "In combination with corresponding visualizations in virtual reality, Shifty can be used to create the illusion that a virtual object is getting longer or heavier," explains André Zenner. In experiments, he was able to prove that objects are perceived as lighter or smaller when the weight is close to the user's hand and that, coupled with the corresponding visual input; they are perceived as longer and heavier the further the weight in the rod moves away from the user. "This is mainly due to changes in the inertia of the controller, as the overall weight does not change," explains André Zenner. The research and development department of gaming giant Sony is already experimenting with this concept and cites André Zenner's work in the development of new VR controllers.<a target="_blank"></a><header><h2>Drag:on</h2></header>The second controller, "Drag:on," consists of two flamenco fans that can be unfolded using servomotors, thus increasing the air resistance of the controller. This means that the further the fans are unfolded, the more force the user has to exert to move the controller through the air. "Coupled with the right visual stimuli, Drag:on can be used to create the impression that the user is holding a small shovel or a large paddle, for example, or that they are pushing a heavy trolley or are twisting a knob that is difficult to turn," explains André Zenner. Both controllers are basic research and so-called "proof of concepts." This means that the prototypes can be used to show in user experiments that different controller states can improve the perception of different VR objects. Still, specific products using this technology are not yet available on the market.<a target="_blank"></a><header><h2>Similarity and colocation problem</h2></header>With the controllers, the Saarbrücken-based computer scientist first addressed the so-called ‘similarity problem.’ The aim here is to ensure that virtual and real objects feel as similar as possible. In the second part of his work, he dealt with the so-called ‘colocation problem,’ i.e., the question of how the proxy can be spatially located in real life where the user sees it in virtual reality. This is particularly challenging as the controllers act as proxies for different virtual objects. Consequently, the user must be given the illusion that they are reaching for various objects, although in reality, they will always grasp the same proxy.To achieve this, the researcher made use of the already established method of "hand redirection." As the name suggests, this involves redirecting the movement of the hand in virtual reality so that the user thinks they are reaching to the left, for example, even though they are actually stretching their hand forward. "We conducted experiments to investigate the point at which users realize that their hand has been redirected. Our results showed that this point was reached quickly, so we thought about how we could better conceal the hand redirection," says André Zenner. The solution: he tricked the brain by only redirecting the hand when the brain was blind to visual changes - namely during blinking. Together with a student under his supervision, he developed the appropriate software and used the eye trackers built into many VR headsets. In control studies, the team was then able to show that their new controllers, in combination with hand redirection algorithms, led to more convincing VR perceptions than previously possible.This research achievement has now been recognized internationally. The work entitled "Advancing Proxy-Based Haptic Feedback in Virtual Reality" was awarded the "IEEE VGTC Virtual Reality Best Dissertation Award" in mid-March at the "IEEE Conference on Virtual Reality and 3D User Interfaces", which took place this year in Orlando, Florida. According to the IEEE, the award is presented each year to the author of the most outstanding dissertation defended in the previous two calendar years in the research fields of virtual and augmented reality. A total of 16 dissertations were submitted to the 2024 edition of the conference and reviewed by an international committee of experts. The thesis was supervised by Prof. Dr. Antonio Krüger, CEO of the German Research Center for Artificial Intelligence and Head of the Ubiquitous Media Lab at Saarland University.<hr>Original Publication:&nbsp;Zenner, A. (2022). Advancing Proxy-Based Haptic Feedback in Virtual Reality. Universität des Saarlandes. <a href="https://dx.doi.org/10.22028/D291-37879" target="_blank" rel="noreferrer">doi:10.22028/D291-37879</a>

How can virtual reality (VR) be experienced haptically, i.e., through the sense of touch? This is one of the fundamental questions that modern VR research is investigating.

Tricking the brain: New dimensions of haptics in virtual reality

Accessibility is an undeniably important subject, yet one which is all-too often overlooked in the development of new technologies. Despite the march of progress, the majority of legally blind individuals rely on a simple white cane for navigation — something researchers Manuel Zahn and Armaghan Ahmad Khan are looking to change.The pair, working in the Center for Digital Technology and Management at the Technical University of Munich and in partnership with FRAMOS GmbH, have developed a system which combines a depth-sensing 3D camera with a wearable haptic feedback sleeve — providing, they say, navigation and obstacle avoidance capabilities entirely hands-free, and at a fidelity and range which far exceeds a cane.<h1>Computer vision for the blind</h1>The fully-wearable system proposed by Zahn and Khan is built around an Intel RealSense D415 depth-sensing camera, housed in a 3D-printed pair of glasses worn over the user’s eyes. This is linked to a single-board computer, an AAEON Up Board, which handles processing the data from the camera, and a battery pack for portability.The RealSense Camera is designed around an active infrared stereo imaging system, projecting beams of invisible infrared light and measuring the depth of its surroundings. Compared with a system based on visible light cameras, it has one major advantage: It can operate in absolute darkness, allowing its user to navigate without difficulty.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261002540-blind2.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="Two images: the left image shows an Intel RealSense depth-sensing camera mounted in a 3D printed pair of glasses, in place of the lenses; the right image shows a person wearing the camera system.">An off-the-shelf depth-sensing camera combined with a 3D printed frame acts as the input to a hands-free navigation system for the blind. Determining the depth of the wearer’s surroundings is only part of the system’s function, however: That information has to be communicated to the user in some way. Eschewing spoken or otherwise audible feedback, as likely to take away from the user’s ability to easily hear their surroundings, the researchers developed a wearable haptic feedback sleeve made from 25 vibration motors sewn onto fabric and linked to an Adafruit Feather 32U4 microcontroller development board.With this, the system is complete: A depth-sensing camera which can map the wearer’s surroundings onto the 25-“pixel” haptic feedback system, providing a physical response when obstacles are near.<h1>Novel navigation</h1>Exactly what feedback is provided depends on where the user is navigating, the pair explain. For indoor navigation, the system provides vibration of nearby obstacles and walls — the location of the vibration mapped to the direction of the obstacle and the intensity mapped to its proximity. This way, a user can navigate with only the wearable system.An outdoor mode switches from a standalone navigation device to an accessory, designed for use with the traditional cane: The obstacle detection range is extended to beyond two meters (around six and a half feet), providing information about obstacles and other objects outside the range of a traditional cane.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261012136-blind3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a custom-built haptic feedback sleeve, comprised of a grid of 25 vibration motors sewn onto fabric. Wires are seen leading to a control box and a battery pack.">Designed to keep the user's hands free, this haptic feedback sleeve proved easy to use. To prove the concept, the pair set up a navigation route which included walkways, doors, and various obstacles — and which had to be traversed in absolute darkness. Impressively, after a period of familiarization with the vibration feedback system, the five volunteers were able to complete the obstacle course perfectly on their very first try.In subsequent attempts, the time taken to traverse the course reduced — showing increasing familiarity with and trust in the system: “Overall the averages for the runs decreased from 320 seconds in the first run to 148 seconds in the third run, two weeks later,” the researchers note — while admitting that one participant did become disoriented for a short period in their last run with the system.The results encourage, with Zahn and Khan pointing to the system’s low cost and lightweight nature as making it of interest for future development — as too is the fact it does not interfere with the wearer’s hearing and leaves both hands free, either for using a traditional navigation aid like cane or for any other task.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261019783-blind4.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a narrow hall with a grid of colored boxes overlaid; an image to the right shows how each color maps to a strength of vibration on the haptic feedback sleeve.">Varying strength and positional haptic feedback turn obstacles into a feeling on the wearer's arm. “Further developments could include the implementation of an object recognition, based on the RGB [visible light] image of the camera,” the pair add. “This would allow the system to artificially generate bordered paths, solving the problem of orientation in a large open space.”The only sticking point: Intel, the manufacturer of the camera platform used for the prototype, <a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" rel="noopener noreferrer" target="_blank">recently confirmed</a> it was to discontinue its RealSense platform — though, at the time of writing, it continued to provide the RealSense D415 for sale.Zahn and Khan's paper is available <a href="https://arxiv.org/abs/2201.04453" rel="noopener noreferrer" target="_blank">on Cornell's arXiv.org preprint server</a>.<h1>References</h1>Manuel Zahn, Armaghan Ahmad Khan: <cite>Obstacle avoidance for blind people using a 3D camera and a haptic feedback sleeve</cite>. DOI <a href="https://arxiv.org/abs/2201.04453" target="_blank">arXiv:2201.04453</a> [cs.HC].Dylan Martin, CRN: Why Pat Gelsinger Decided to Wind Down Intel's RealSense Unit: Exclusive.&nbsp;<a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" id="isPasted" rel="noopener noreferrer" target="_blank">https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive</a>

Using a depth-sensing camera, this wearable navigation platform can alert its wearer to obstacles via haptic feedback.

Built around a custom-made haptic feedback sleeve and an Intel RealSense camera in a 3D-printed glasses housing, this navigation system lets its user find their way even in absolute darkness.

A wearable navigation system, featuring a 3D camera, offers a hands-free navigation solution for the blind

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&amp;amp;-gas, environments.In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video will be showed during the international conference ICRA 2021, on May 31st.With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot&nbsp;combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (&gt; 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (&lt; 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.<hr>Further informationThe research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).&nbsp;Website: <a href="https://advr.iit.it/projects/inail-scc/teleoperazione" target="_blank">https://advr.iit.it/projects/inail-scc/teleoperazione</a>

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation.

haptic interfaces