ETH student Alexander Bayer was inspired to upgrade the traditional white cane by a blind classmate in high school. Together with three other ETH students, he is working in the Student Project House on the &ldquo;NextGuide&rdquo; project, a smart cane for the blind and visually impaired equipped with an integrated camera and the ability to provide haptic feedback. The cane not only indicates the direction in which an individual should walk to avoid obstacles but also communicates via vibrations whether they are standing in front of a door, pedestrian crossing or staircase.&nbsp;

ETH students have developed a smart cane featuring an integrated camera that detects its surroundings and a tactile pointer that helps blind and visually impaired individuals navigate their way safely.

The smart cane for the blind and visually impaired

Prof. Becky Peterson led a team that has developed a scalable, manufacturable method for developing thin film transistors (TFTs) that operate at the lowest possible voltage. This is particularly important for TFT integration with today&rsquo;s silicon complementary metal-oxide semiconductors (CMOS), which are used in the vast majority of integrated circuits.&ldquo;We&rsquo;re essentially developing a less complicated device that operates at lower voltage,&rdquo; said ECE PhD student Tonglin (Tanya) Newsom, who is first author on the paper. &ldquo;With this steep sub-threshold swing device, we could significantly lower the energy dissipation of our circuitry, meaning there&rsquo;s less energy loss. This could help everyone who uses electronic devices.&rdquo;TFTs enable the operation of modern displays, acting as switches that control the light at each individual pixel. Switching efficiently between the on and off states allows for lower voltage operation, and results in a more energy-efficient system. One of the key manufacturing challenges to achieving a highly efficient switch is the requirement of a super clean interface between the different material layers within the TFT. A clean interface means electrons can flow in the channel to turn the pixel &ldquo;on&rdquo; or &ldquo;off&rdquo; without getting trapped.&ldquo;The key technology that we were trying to develop is an ultra-clean interface between the semiconductor and the gate insulator, so the switching process between the on and off states would be very sharp,&rdquo; Peterson said. &ldquo;We were able to achieve switching at the fastest rate possible, according to fundamental physical limits at room temperature.&rdquo;To do this, Peterson&rsquo;s group partnered with Mechanical Engineering Professor<a href="https://dasgupta.engin.umich.edu/" target="_blank">&nbsp;</a><a href="https://dasgupta.engin.umich.edu/" target="_blank">Neil Dasgupta</a>, with whom they developed an atomic layer deposition technology for zinc tin oxide, which is a wide bandgap semiconductor that can be used for electronic and energy devices, such as TFTs, versatile sensors, and solar cells. Peterson&rsquo;s team took this technology one step further by directly integrating the atomic layer deposition processes for two different key parts of the transistor &ndash; the gate capacitor and the semiconductor channel.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1674827663044-EDL-chip-content.jpg" style="width: 766px;" class="fr-fic fr-dib">Microchip containing thin film transistors having record sub-threshold slope, made using the in situ atomic layer deposition process. Photo by Silvia Cardarelli, Michigan ECE.&nbsp;&ldquo;We get the best results by depositing the two layers back-to-back within the same tool, without breaking vacuum,&rdquo; Peterson said. &ldquo;This approach demonstrates a straightforward method to achieve optimal TFT performance.&rdquo;Peterson&rsquo;s team focused on amorphous oxide semiconductors, which are a category of materials that are commercialized in displays and enable us to individually control pixels. They&rsquo;re important for achieving low power operation, high pixel density screens, touch screens, and haptic displays, which generate tactile effects &ndash; such as vibrations &ndash; for a variety of applications, including wearables and AR/VR devices.&ldquo;I truly hope more people will become interested in doing this kind of fundamental science and engineering,&rdquo; Newsom said &ldquo;because this work is not just about discovering new possibilities &ndash; it&rsquo;s about improving technology to help the world.&rdquo;&quot;This work is not just about discovering new possibilities &ndash; it&rsquo;s about improving technology to help the world.&quot; Tanya Newsom<hr>The research, &ldquo;<a href="https://ieeexplore.ieee.org/document/9937198" target="_blank">59.9 mV&middot;dec-1 Subthreshold Swing Achieved in Zinc Tin Oxide TFTs With In Situ Atomic Layer Deposited Al2O3 Gate Insulator</a>,&rdquo; was selected as an Editor&rsquo;s Pick in IEEE Electron Device Letters. In addition to Newsom, Peterson, and Dasgupta, co-authors include ECE PhD alum Christopher Allemang and Mechanical Engineering PhD student Tae H. Cho.Fabrication work was accomplished in the<a href="https://lnf.umich.edu/" target="_blank">&nbsp;</a><a href="https://lnf.umich.edu/" target="_blank">Lurie Nanofabrication Facility</a>.

ECE PhD student Tonglin (Tanya) Newsom in the lab of Prof. Becky (R.L.) Peterson, measuring the electrical characteristics of thin film transistors. Photo by Silvia Cardarelli, Michigan ECE.

Led by Prof. Becky Peterson, the research focuses on a category of materials important for low power logic operations, high pixel density screens, touch screens, and haptic displays.

Scalable method to manufacture thin film transistors achieves ultra-clean interface for high performance, low-voltage device operation

Accessibility is an undeniably important subject, yet one which is all-too often overlooked in the development of new technologies. Despite the march of progress, the majority of legally blind individuals rely on a simple white cane for navigation — something researchers Manuel Zahn and Armaghan Ahmad Khan are looking to change.The pair, working in the Center for Digital Technology and Management at the Technical University of Munich and in partnership with FRAMOS GmbH, have developed a system which combines a depth-sensing 3D camera with a wearable haptic feedback sleeve — providing, they say, navigation and obstacle avoidance capabilities entirely hands-free, and at a fidelity and range which far exceeds a cane.<h1>Computer vision for the blind</h1>The fully-wearable system proposed by Zahn and Khan is built around an Intel RealSense D415 depth-sensing camera, housed in a 3D-printed pair of glasses worn over the user’s eyes. This is linked to a single-board computer, an AAEON Up Board, which handles processing the data from the camera, and a battery pack for portability.The RealSense Camera is designed around an active infrared stereo imaging system, projecting beams of invisible infrared light and measuring the depth of its surroundings. Compared with a system based on visible light cameras, it has one major advantage: It can operate in absolute darkness, allowing its user to navigate without difficulty.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261002540-blind2.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="Two images: the left image shows an Intel RealSense depth-sensing camera mounted in a 3D printed pair of glasses, in place of the lenses; the right image shows a person wearing the camera system.">An off-the-shelf depth-sensing camera combined with a 3D printed frame acts as the input to a hands-free navigation system for the blind. Determining the depth of the wearer’s surroundings is only part of the system’s function, however: That information has to be communicated to the user in some way. Eschewing spoken or otherwise audible feedback, as likely to take away from the user’s ability to easily hear their surroundings, the researchers developed a wearable haptic feedback sleeve made from 25 vibration motors sewn onto fabric and linked to an Adafruit Feather 32U4 microcontroller development board.With this, the system is complete: A depth-sensing camera which can map the wearer’s surroundings onto the 25-“pixel” haptic feedback system, providing a physical response when obstacles are near.<h1>Novel navigation</h1>Exactly what feedback is provided depends on where the user is navigating, the pair explain. For indoor navigation, the system provides vibration of nearby obstacles and walls — the location of the vibration mapped to the direction of the obstacle and the intensity mapped to its proximity. This way, a user can navigate with only the wearable system.An outdoor mode switches from a standalone navigation device to an accessory, designed for use with the traditional cane: The obstacle detection range is extended to beyond two meters (around six and a half feet), providing information about obstacles and other objects outside the range of a traditional cane.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261012136-blind3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a custom-built haptic feedback sleeve, comprised of a grid of 25 vibration motors sewn onto fabric. Wires are seen leading to a control box and a battery pack.">Designed to keep the user's hands free, this haptic feedback sleeve proved easy to use. To prove the concept, the pair set up a navigation route which included walkways, doors, and various obstacles — and which had to be traversed in absolute darkness. Impressively, after a period of familiarization with the vibration feedback system, the five volunteers were able to complete the obstacle course perfectly on their very first try.In subsequent attempts, the time taken to traverse the course reduced — showing increasing familiarity with and trust in the system: “Overall the averages for the runs decreased from 320 seconds in the first run to 148 seconds in the third run, two weeks later,” the researchers note — while admitting that one participant did become disoriented for a short period in their last run with the system.The results encourage, with Zahn and Khan pointing to the system’s low cost and lightweight nature as making it of interest for future development — as too is the fact it does not interfere with the wearer’s hearing and leaves both hands free, either for using a traditional navigation aid like cane or for any other task.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261019783-blind4.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a narrow hall with a grid of colored boxes overlaid; an image to the right shows how each color maps to a strength of vibration on the haptic feedback sleeve.">Varying strength and positional haptic feedback turn obstacles into a feeling on the wearer's arm. “Further developments could include the implementation of an object recognition, based on the RGB [visible light] image of the camera,” the pair add. “This would allow the system to artificially generate bordered paths, solving the problem of orientation in a large open space.”The only sticking point: Intel, the manufacturer of the camera platform used for the prototype, <a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" rel="noopener noreferrer" target="_blank">recently confirmed</a> it was to discontinue its RealSense platform — though, at the time of writing, it continued to provide the RealSense D415 for sale.Zahn and Khan's paper is available <a href="https://arxiv.org/abs/2201.04453" rel="noopener noreferrer" target="_blank">on Cornell's arXiv.org preprint server</a>.<h1>References</h1>Manuel Zahn, Armaghan Ahmad Khan: <cite>Obstacle avoidance for blind people using a 3D camera and a haptic feedback sleeve</cite>. DOI <a href="https://arxiv.org/abs/2201.04453" target="_blank">arXiv:2201.04453</a> [cs.HC].Dylan Martin, CRN: Why Pat Gelsinger Decided to Wind Down Intel's RealSense Unit: Exclusive.&nbsp;<a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" id="isPasted" rel="noopener noreferrer" target="_blank">https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive</a>

Using a depth-sensing camera, this wearable navigation platform can alert its wearer to obstacles via haptic feedback.

Built around a custom-made haptic feedback sleeve and an Intel RealSense camera in a 3D-printed glasses housing, this navigation system lets its user find their way even in absolute darkness.

A wearable navigation system, featuring a 3D camera, offers a hands-free navigation solution for the blind

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&amp;amp;-gas, environments.In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video will be showed during the international conference ICRA 2021, on May 31st.With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot&nbsp;combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (&gt; 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (&lt; 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.<hr>Further informationThe research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).&nbsp;Website: <a href="https://advr.iit.it/projects/inail-scc/teleoperazione" target="_blank">https://advr.iit.it/projects/inail-scc/teleoperazione</a>

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation.

haptics