Accessibility is an undeniably important subject, yet one which is all-too often overlooked in the development of new technologies. Despite the march of progress, the majority of legally blind individuals rely on a simple white cane for navigation — something researchers Manuel Zahn and Armaghan Ahmad Khan are looking to change.The pair, working in the Center for Digital Technology and Management at the Technical University of Munich and in partnership with FRAMOS GmbH, have developed a system which combines a depth-sensing 3D camera with a wearable haptic feedback sleeve — providing, they say, navigation and obstacle avoidance capabilities entirely hands-free, and at a fidelity and range which far exceeds a cane.<h1>Computer vision for the blind</h1>The fully-wearable system proposed by Zahn and Khan is built around an Intel RealSense D415 depth-sensing camera, housed in a 3D-printed pair of glasses worn over the user’s eyes. This is linked to a single-board computer, an AAEON Up Board, which handles processing the data from the camera, and a battery pack for portability.The RealSense Camera is designed around an active infrared stereo imaging system, projecting beams of invisible infrared light and measuring the depth of its surroundings. Compared with a system based on visible light cameras, it has one major advantage: It can operate in absolute darkness, allowing its user to navigate without difficulty.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261002540-blind2.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="Two images: the left image shows an Intel RealSense depth-sensing camera mounted in a 3D printed pair of glasses, in place of the lenses; the right image shows a person wearing the camera system.">An off-the-shelf depth-sensing camera combined with a 3D printed frame acts as the input to a hands-free navigation system for the blind. Determining the depth of the wearer’s surroundings is only part of the system’s function, however: That information has to be communicated to the user in some way. Eschewing spoken or otherwise audible feedback, as likely to take away from the user’s ability to easily hear their surroundings, the researchers developed a wearable haptic feedback sleeve made from 25 vibration motors sewn onto fabric and linked to an Adafruit Feather 32U4 microcontroller development board.With this, the system is complete: A depth-sensing camera which can map the wearer’s surroundings onto the 25-“pixel” haptic feedback system, providing a physical response when obstacles are near.<h1>Novel navigation</h1>Exactly what feedback is provided depends on where the user is navigating, the pair explain. For indoor navigation, the system provides vibration of nearby obstacles and walls — the location of the vibration mapped to the direction of the obstacle and the intensity mapped to its proximity. This way, a user can navigate with only the wearable system.An outdoor mode switches from a standalone navigation device to an accessory, designed for use with the traditional cane: The obstacle detection range is extended to beyond two meters (around six and a half feet), providing information about obstacles and other objects outside the range of a traditional cane.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261012136-blind3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a custom-built haptic feedback sleeve, comprised of a grid of 25 vibration motors sewn onto fabric. Wires are seen leading to a control box and a battery pack.">Designed to keep the user's hands free, this haptic feedback sleeve proved easy to use. To prove the concept, the pair set up a navigation route which included walkways, doors, and various obstacles — and which had to be traversed in absolute darkness. Impressively, after a period of familiarization with the vibration feedback system, the five volunteers were able to complete the obstacle course perfectly on their very first try.In subsequent attempts, the time taken to traverse the course reduced — showing increasing familiarity with and trust in the system: “Overall the averages for the runs decreased from 320 seconds in the first run to 148 seconds in the third run, two weeks later,” the researchers note — while admitting that one participant did become disoriented for a short period in their last run with the system.The results encourage, with Zahn and Khan pointing to the system’s low cost and lightweight nature as making it of interest for future development — as too is the fact it does not interfere with the wearer’s hearing and leaves both hands free, either for using a traditional navigation aid like cane or for any other task.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261019783-blind4.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a narrow hall with a grid of colored boxes overlaid; an image to the right shows how each color maps to a strength of vibration on the haptic feedback sleeve.">Varying strength and positional haptic feedback turn obstacles into a feeling on the wearer's arm. “Further developments could include the implementation of an object recognition, based on the RGB [visible light] image of the camera,” the pair add. “This would allow the system to artificially generate bordered paths, solving the problem of orientation in a large open space.”The only sticking point: Intel, the manufacturer of the camera platform used for the prototype, <a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" rel="noopener noreferrer" target="_blank">recently confirmed</a> it was to discontinue its RealSense platform — though, at the time of writing, it continued to provide the RealSense D415 for sale.Zahn and Khan's paper is available <a href="https://arxiv.org/abs/2201.04453" rel="noopener noreferrer" target="_blank">on Cornell's arXiv.org preprint server</a>.<h1>References</h1>Manuel Zahn, Armaghan Ahmad Khan: <cite>Obstacle avoidance for blind people using a 3D camera and a haptic feedback sleeve</cite>. DOI <a href="https://arxiv.org/abs/2201.04453" target="_blank">arXiv:2201.04453</a> [cs.HC].Dylan Martin, CRN: Why Pat Gelsinger Decided to Wind Down Intel's RealSense Unit: Exclusive.&nbsp;<a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" id="isPasted" rel="noopener noreferrer" target="_blank">https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive</a>

Using a depth-sensing camera, this wearable navigation platform can alert its wearer to obstacles via haptic feedback.

Built around a custom-made haptic feedback sleeve and an Intel RealSense camera in a 3D-printed glasses housing, this navigation system lets its user find their way even in absolute darkness.

A wearable navigation system, featuring a 3D camera, offers a hands-free navigation solution for the blind

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&amp;amp;-gas, environments.In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video will be showed during the international conference ICRA 2021, on May 31st.With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot&nbsp;combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (&gt; 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (&lt; 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.<hr>Further informationThe research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).&nbsp;Website: <a href="https://advr.iit.it/projects/inail-scc/teleoperazione" target="_blank">https://advr.iit.it/projects/inail-scc/teleoperazione</a>

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation.

The new IIT's "Robot Teleoperativo" (teleoperation robot)

READY Robotics

RAD Lab-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

RAD Lab-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

RAD Lab-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

RAD Lab-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

RAD Lab-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

RAD Lab

haptic interfaces

haptic interfaces

Profiles