Skin-integrated CL-HMI: Electronic skin as wireless human-machine interfaces for robotic VR

Overview

The Skin-Intergated Closed-Loop Human-Machine Interface (CL-HMI) provides accurate sensing as well as feedback. The electronics are securely interfaced with the whole body for wireless motion capturing and haptic feedback via Bluetooth, Wi-Fi, and the Internet. The combination of a virtual reality headset connected to the robot's eyes and a skin-integrated CL-HMI would result in intelligent robotic VR, a cutting-edge technology.

Problem / Solution

Human-machine interfaces (HMIs) in teleoperated robots have enormous potential in a variety of fields, including immersive gaming, prosthesis or exoskeleton control, and extreme environment operation. However, current HMIs are primarily focused on machine control and lack adequate user feedback. These HMIs are frequently bulky and rigid, limiting their function, wearability, and comfortability.

Design

Human motion is converted into an electrical signal or data by the sensors. The MCU processes this data and wirelessly transmits it to the robot. Based on the received signal, the connected robot moves. The pressure sensors on the robot serve as feedback signals to control the vibration intensity of the haptic actuators, allowing the user to receive haptic feedback.

 Skin-Integrated CL-HMI Design

The skin-intergrated CL-HMI is a multilayer device that functions as the control panel. In the CL-HMI patches, seven bending sensors for sensing and five actuators for feedback are connected to the general purpose inputs/outputs (GPIOs) and analog-to-digital converters (ADCs) of the MCU. 

 Sensors

All functional components of the sensors are sandwiched between two layers of skin-tone elastomeric silicon, which also serves as the soft adhesive interface to the skin. The copper (Cu) traces supported by polyimide interconnect a collection of chip-scale integrated circuits and sensing components and serve as a wireless power transmitting antenna. The circuits include resistors, capacitors, a Bluetooth module, a microcontroller unit (MCU), and bridges. While the sensing component includes the self-developed soft sensors and actuators. Tiny buckled Cu wires connect all these components. The Cu traces are patterned in a filamentary serpentine structure, resulting in the system’s stretchable feature.

The CL-HMI sensor detects body motion accurately by measuring resistance changes at different bending angles. These bending sensors are mounted on various joints to capture body movement. These sensors also serve as pressure sensors on robots for detecting robotic activities. A piezoresistive thin film on top of interdigital electrodes serves as the functional component. 

 Actuators

The haptic feedback system is provided by skin-integrated vibratory actuators that are based on Lorentz forces. The vibration amplitude of the actuator is controlled by frequency.A neodymium magnet is attached to the center of a thin disk of polyethylene terephthalate (PET) with a semicircular slit. The PET disk is placed on top of a resin ring, with the glued magnet inside the ring. A copper coil located at the bottom of the ring makes the magnet vibrate when the pulse width modulation (PWM) current passes through it. The frequency of the PWM current is adjusted based on feedback from controlled robots. Five vibration levels with different frequencies (20, 50, 83, 166, and 250 Hz) are chosen to give haptic feedback that ranges from weak to strong.

 Battery

The skin-integrated CL-HMI with full-load operation is powered for over 68 minutes by a 4.2 V rechargeable Lithium-ion battery. The surrounding Cu coils in series with a 14-pF capacitor with a resonant frequency of 13.56 MHz can charge the battery wirelessly even when bent, stretched, and twisted.

 Adhesiveness, Stretchability, Flexibility 

The CL-HMI with the highly adhesive elastomeric layer can be tightly interfaced to the skin via van der Waals forces. The adhesive duration of a mounted CL-HMI system under continuous bending is 8 hours, while the bending sensors remain adhesive for 3.35 to 5.87 hours. The maximum equivalent strains in the copper layer on twisted, bent, and stretched devices are less than the elastic limit and copper's fracture strain, allowing for smooth and firm integration onto the curved surfaces of human skin.

 Wireless control

The CL-HMI can transmit and receive data by three different means: Bluetooth, Wi-Fi, and the Internet. Bluetooth has the shortest range of 1-5m but with the most rapid response time of under 4µs. On the other hand, Wi-Fi has a range of hundreds of meters under the condition of sharing the same network, and an increased response time of 350µs. Lastly, the Internet offers unlimited range as long as it's connected to the internet with a response time of between 30.2 and 47.8 ms.