Closed-loop human-machine interface that allows for wireless control with haptic feedbacks

Skin-integrated CL-HMI: Electronic skin as wireless human-machine interfaces for robotic VR

<h3>Overview</h3>The Skin-Intergated Closed-Loop Human-Machine Interface (CL-HMI) provides accurate sensing as well as feedback. The electronics are securely interfaced with the whole body for wireless motion capturing and haptic feedback via Bluetooth, Wi-Fi, and the Internet. The combination of a virtual reality headset connected to the robot's eyes and a skin-integrated CL-HMI would result in intelligent robotic VR, a cutting-edge technology. <h3>Problem / Solution</h3>Human-machine interfaces (HMIs) in teleoperated robots have enormous potential in a variety of fields, including immersive gaming, prosthesis or exoskeleton control, and extreme environment operation. However, current HMIs are primarily focused on machine control and lack adequate user feedback. These HMIs are frequently bulky and rigid, limiting their function, wearability, and comfortability.<h3>Design</h3>Human motion is converted into an electrical signal or data by the sensors. The MCU processes this data and wirelessly transmits it to the robot. Based on the received signal, the connected robot moves. The pressure sensors on the robot serve as feedback signals to control the vibration intensity of the haptic actuators, allowing the user to receive haptic feedback.&nbsp;Skin-Integrated CL-HMI DesignThe skin-intergrated CL-HMI is a multilayer device that functions as the control panel. In the CL-HMI patches, seven bending sensors for sensing&nbsp;and five actuators for feedback&nbsp;are connected&nbsp;to the general purpose inputs/outputs (GPIOs) and&nbsp;analog-to-digital converters (ADCs) of the MCU.&nbsp;&nbsp;SensorsAll functional components of the sensors are sandwiched between two layers of skin-tone elastomeric silicon, which also serves as the soft adhesive interface to the skin. The copper (Cu) traces supported by polyimide interconnect a collection of chip-scale integrated circuits and sensing components and serve as a wireless power transmitting antenna. The circuits include resistors, capacitors, a Bluetooth module, a microcontroller unit (MCU), and bridges. While the sensing component includes the self-developed soft sensors and actuators. Tiny buckled Cu wires connect all these components. The Cu traces are patterned in a filamentary serpentine structure, resulting in the system’s stretchable feature.The CL-HMI sensor detects body motion accurately by measuring resistance changes at different bending angles. These bending sensors are mounted on various joints to capture body movement. These sensors also serve as pressure sensors on robots for&nbsp;detecting robotic activities. A piezoresistive thin film on top of interdigital electrodes serves as the functional component.&nbsp;&nbsp;ActuatorsThe haptic feedback system is provided by skin-integrated vibratory actuators that are based on Lorentz forces. The vibration amplitude of the actuator is controlled by frequency.A neodymium magnet is attached to the center of a thin disk of polyethylene terephthalate (PET) with a semicircular slit. The PET disk is placed on top of a resin ring, with the glued&nbsp;magnet&nbsp;inside the ring. A copper coil located at the bottom of the ring makes the magnet vibrate when the pulse width modulation (PWM) current passes through it. The frequency of the PWM current is adjusted based on feedback from controlled robots. Five vibration levels with different frequencies (20, 50, 83, 166, and 250 Hz) are chosen to give haptic feedback that ranges from weak to strong.&nbsp;BatteryThe skin-integrated CL-HMI with full-load operation is powered for over 68 minutes by a 4.2 V rechargeable Lithium-ion&nbsp;battery. The surrounding Cu coils in series with a 14-pF capacitor with a resonant frequency of 13.56 MHz can charge the battery wirelessly even when bent, stretched, and twisted.&nbsp;Adhesiveness, Stretchability, Flexibility&nbsp;The CL-HMI with the highly adhesive elastomeric layer can be tightly interfaced to the skin via van der Waals forces. The adhesive duration of a mounted CL-HMI system under continuous bending is 8 hours, while the bending sensors remain adhesive for 3.35 to 5.87 hours. The maximum equivalent strains in the copper layer on twisted, bent, and stretched devices are less than the elastic limit and copper's&nbsp;fracture strain, allowing for smooth and firm integration onto the curved surfaces of human skin.&nbsp;Wireless controlThe CL-HMI can transmit and receive data by three different means: Bluetooth, Wi-Fi, and the Internet. Bluetooth has the shortest range of 1-5m but with the most rapid response time of under 4µs. On the other hand, Wi-Fi has a range of hundreds of meters under the condition of sharing the same network, and an increased response time of 350µs. Lastly, the Internet offers unlimited range as long as it's connected to the internet with a response time of between 30.2 and 47.8 ms.

A research paper describing the challenge, design, and outcome of the research.

Electronic skin as wireless human-machine interfaces for robotic VR

Skin-integrated electronics usher in a new age of human-machine interface for robotic virtual reality

Skin-Integrated CL-HMI – Control Panel

Elastomeric Silicone 

Copper (Cu) traces dimensions 

Polyimide (PI) thickness

Battery

Wireless Charging

Permanent Magnet 

Polyethylene Terephthalate (PET) 

Resin ring 

Bottom Copper coil

Pressure sensor sensitivity

Five selected vibration levels

A closed-loop human-machine interface that allows for wireless robot control with haptic feedbacks. 

Skin-Integrated CL-HMI – Control Panel	57 x 39 x 0.8	mm
Elastomeric Silicone	0.4 mm, ~145 kPa
	Material: polydimethylsiloxane (PDMS)
Copper (Cu) traces dimensions	Width: 200-1000 μm; Thickness: 6 μm
Polyimide (PI) thickness	12 μm
Battery	lithium-ion; rechargeable