Innovators Update S2E1: A Glove That Bridges The Gap Between Reality and Digitized Worlds

author avatar

25 Mar, 2021

Andrew Spielberg is a PhD student at MIT's Computer Science and Artificial Intelligence Laboratory where he develops tools to automate both rigid and soft robot design.

Inspired by the quest to bridge the gap between reality and digitized worlds, MIT CSAIL PhD student Andrew Spielberg realized the solution was at his fingertips, by way of “Memglove”, a wearable smart glove that can detect your hand pose, distinguish between more than 30 different household objects, and potentially transform AR, VR, and video games. It can even tell which letter of the alphabet a user is writing, just based on their hand pose (so far it can estimate up to J, at 85% accuracy). 

While it seems like smart devices are becoming ubiquitous, with smart watches, bracelets, and rings gracing countless wrists and fingers, the glove -- which can be made with off the shelf components for as little as $60 -- was originally spurred by an idea to use textiles as sensors (think e-textiles for smart clothing or fabrics). 

“We wanted to take an ambitious approach to textile-based sensing in terms of simplicity and capability,” says Spielberg. “One of my co-researchers developed a new fiber-based pressure sensing technique, and another had a novel idea for resistance-based sensing, so we thought, could we combine these two sensor modalities to sense many different types of stimuli? We were  surprised with just how many things we could measure with the combined sensors - temperature, force, stiffness, conductibility, heart rate, grasped object classes, and even handwriting.” 

Two types of sensors let the glove feel and think: resistive and pressure. The resistive sensors - a small number of electrodes placed on a fully conductive glove - give a lot of information with sparse input.   As the glove is stretched and bent, the resistance changes at different points in the glove. This change in resistance over the glove is manifested differently over each pair of electrodes, and that information allows hand pose to be accurately reconstructed.

While the architecture sounds fancy, the glove is cheap: the materials are all off the shelf, and it’s all textile based. The glove substrate is a few dollars, and the cost of the wires, electrodes, and pressure tubes all combine to only a few dollars. The most expensive part of the device is the electronics, such as the pressure transducers.  

Now, while it may be cheap, don’t let the price fool you on its capabilities. A big limiting factor in many AR/VR applications is the ability to realistically interact with your environment. Typically, clunky controllers are used as input for interaction, with need for some type of utensil in the virtual world. 

“A major challenge right now is having robots that can perform adroit manipulation. And a big limitation in that, is creating manipulators that have tactile feedback, and understand what they’re grasping,” says Spielberg. “We think by improving the sensing side of robotic manipulators, we can greatly improve the control side as well.” 

Spielberg worked on MemGlove alongside CSAIL postdoc Josie Hughes,  MIT undergraduate Mark Chounlakone, designer Gloria Chang and MIT professors Wojciech Matusik and Daniela Rus.

Prior to joining MIT, Speilberg received his MEng in Computer Science and his BS in Computer Science and Engineering Physics at Cornell University, and then spent two years working as a research engineer in the Asymmetric Operations Department at the Johns Hopkins University Applied Physics Lab (JHU/APL).

Spielberg currently works on the intersection of fabrication and robotics, where he’s creating tools to enable users to quickly design, program, and easily fabricate their own personal robots using data-driven tools. Spielberg has worked on projects in developing fast, differentiable simulators, powerful optimization and learning algorithms for automated robot design, and novel digital fabrication techniques. He is strongly interested in democratizing fabrication, both in form and function, and computational creativity. His work falls in the intersection of computer graphics, machine learning, and robotics.

This video was produced by Rachel Gordon from MIT CSAIL and Richard Hulskes from Wevolver.

More by Rachel Gordon

Communications and Media Relations Manager at CSAIL, MIT’s Computer Science and Artificial Intelligence Laboratory pioneers research in computing that improves the way people work, play, and learn.

Wevolver 2022