A wireless wristband that detects and predicts keyboard typing inputs on any flat surface.

TapType: Ten-finger text entry on everyday surfaces via Bayesian inference

<h3>Problem / Solution</h3>The new types on the keyboards of the latest portable gadgets are often shrunk or sparse to fit their screen sizes. This reduces the comfort, accuracy, and speed of typing compared to a regular-sized keyboard.TapType is a mobile text entry system that supports full-size touch typing on flat surfaces without the actual QWERTY keyboard. The sensors of the two wristbands register the taps of typing through inertial sensors, and send them to the backend. The system predicts the most likely character sequences by using a Bayesian neural network classifier and an n-gram language model. An average of 19 words per minute with a character error rate of 0.6% is achieved after 30 minutes of training.<h3>Design</h3>HardwareTapType is composed of a silicone wristband, two accelerometers, and a mainboard with a system on a chip, Bluetooth Low Energy (BLE), and a 3V coin cell battery.The electronics are cast in the flexible wrist strap made from TFC elastic silicone. The silicone wrist strap firmly and comfortably attaches to the wearer’s skin, and helps couple the tap events to the accelerometers.&nbsp;The identification of tapping fingers from mechanical vibrations is recorded using two accelerometer sensors placed near the ulna and radius. The dual-inertial sensor design features two ultra-low power three-axis accelerometers (BMA456) for increased precision by providing higher resolution of 16-bit, ±2G, and higher sampling rates of 1600 Hz. The accelerometers are attached at the ends of the wrist strap, with the mainboard in the middle.&nbsp;The mainboard contains the system on a chip (DA14695), Bluetooth Low Energy (BLE), and a 3V coin cell battery (CR2032). The accelerometers measure the vibrations of the typing hands and send the data to the system on a chip. Then, the data is sent to a backend via the BLE antenna. The backend predicts the typed character. TapType consumes 10-15 mW and lasts approximately 30 hours on a 3V coin cell battery.&nbsp;Bayesian classifier and Text Entry DecoderThe apparatus captures representative accelerometer signals for typing input in order to collect samples for the TapType System's learning-based approach. All hand motions, including touch and tap events, are&nbsp;included in the continuous stream of accelerations. Ground-truth touch events are recorded to label&nbsp;sensor data&nbsp;with the correct finger identity. Four mounted cameras record the motions of participants’ fingertips in midair before and after the type event. The reconstructed finger movements are combined with the recorded events and coordinates to obtain records of finger identities, touch locations, and all IMU streams. Spurious inputs that refer to recorded events in the IMU streams that do not correspond to a touch event are identified. The setup allows for supervised training of the Bayesian classifiers.&nbsp;A dataset of a large number of sentences from different sources are used as training data. The n-gram language model is built using Witten-Bell smoothing for characters, and modified Kneser-Ney smoothing for word sequences.&nbsp;The text entry decoder predicts the most likely character sequence by cross analyzing the finger probabilities from the Bayesian neural network tap classifier and characters’ prior probabilities from the n-gram language model.&nbsp;

A research paper describing the challenge, design, and outcome of the research

Accelerometers Model

Accelerometers Features

Accelerometers Resolution

Accelerometers sampling rates

System-on-a-chip Model

System on a chip Features

Battery 

Wrist Strap Material

A wireless wristband that detects typing inputs on any flat surface, then sends data to a backend to predict character outputs. 

Accelerometers Model	Two BMA456, Bosch Sensortec
Accelerometers Features	ultra-low power; three-axis
Accelerometers Resolution	16-bit, ±2G
Accelerometers sampling rates	1600 Hz
System-on-a-chip Model	DA14695, ARM Cortex M33
System on a chip Features	Dialog Semiconductor