Dexterous Robotic Hands Part 3: Shadow Gloves - The Final Piece of the Robotics Puzzle
02 Oct, 2023
In this third part of the series, we will take a deeper look at the development of the Shadow Hand and Glove, the technology behind the glove, and the potential impact of this system on the field of robotics.
Introduction
In Part 2 of this series, we examined the Shadow Dexterous Hand, its remarkable technology and versatility, and we presented real world examples of the role it has played in groundbreaking robotics research. The engineering advancements required to design and create a truly dexterous robotic hand also come with the challenge of controlling it. Shadow Robot has tackled this challenge with an innovative solution—the Shadow Glove.
Why Create Dexterous Robotic Hands?
The problem of developing a truly dexterous robotic hand has challenged engineers for decades, and Shadow Robot’s Dexterous Hand has brought researchers closer to reaching this goal. The Shadow Hand is designed to replicate the human hand as closely as possible, in size, shape, and abilities. Most grippers and tooling on the end of robotic arms are designed to perform one specific task, or can only grab and hold one type of object, so they are limited in their ability to interact with different objects and in variable environments. The Shadow Dexterous Hand reaches beyond that standard into the realm of object handling, fine manipulation, and machine learning.
In the words of Shadow Robot, the development of the hand encompasses their vision to create robots that can work in the world in the same way that a human can: with the ability to learn and adapt. From that standpoint, the company determined that the hand must be able to maximize the types of objects that the robot could interact with. According to Shadow Robot, a truly versatile robot should have the same kind of hands as humans, so that they can work with the same tools, objects, and in the same environments that humans can work in.
Developing the Shadow Glove
Researchers have explored numerous approaches to controlling dexterous robotic hands, including traditional automation, classical robot programming, and reinforcement learning. In this section, we will explore the challenge of controlling dexterous robotic hands using traditional approaches, and introduce Shadow Robot’s innovative Shadow Glove solution.
Approach 1: Automation
The first approach to dexterous hand control is automation, where the hand is programmed with a series of positions, and moves from one pose to the other. This method of control works well in very specific settings, where the environment is precisely set up and nothing changes. In some use cases, this may be all the control that is required. However, there is no room at all for human-like adaptation to change.
Approach 2: Classical Robot Programming
The second approach to dexterous hand control is classical robot programming and motion planning. In this approach, a vision system identifies the location of an object, then a grasp planner calculates the optimal gripping technique. A motion planner then coordinates the hand’s movement to precisely execute the grasp. Throughout this process, sensors provide feedback so that the system can make decisions at each stage. This methodology works in scenarios with clearly defined objectives, like “pick up one of ten objects and put it in a specific place”. However, as the complexity of the environment and task increases, the required complexity of the system and software also quickly increases, making this approach impractical for a truly versatile system.
Approach 3: Reinforcement Learning
A third approach to dexterous hand control is to use AI programming toolkits that are available today. This method relies on reinforcement learning and simulation tools to train robotic dexterous hands, and requires a model that accurately represents the complexities and dynamics of the real world. This method can work well, but requires complex software and a large amount of data and computational resources. Also, any discrepancies between the simulation model and the real world can lead to a performance gap.
Approach 4: Human Demonstration with the Shadow Glove
The fourth approach for dexterous hand control is human demonstration, where the robot replicates the movement of the operator. This is the approach that Shadow Robot applied with the Shadow Glove. The operator wears the Shadow Glove, which tracks the operator’s movement, and the Shadow Hand reproduces the movement. This method has an extremely intuitive user interface, which makes it easy for the operator to learn how to operate the system and perform complex tasks.
Additionally, with this approach, there are two ways to use the data captured from the glove:
- Perform a task directly. The operator directly controls the robotic hand for a specific task. This allows for real-time control and immediate execution.
- Capture the data from the glove and use it to train learning systems. With this method the captured data is used to develop models that can learn from the demonstrated movements. These trained systems can then control the robotic hand and perform the same task autonomously.
In both cases, the human transfers dexterous skills to the robot hand, allowing the robot to perform tasks that require a high level of dexterity and precision. With its capabilities and intuitive operation, the Shadow Glove’s approach to use human demonstration for dexterous hand control has evident advantages over traditional methods.
A Deeper Look at the Technology Behind the Shadow Glove
The Shadow Glove is a wearable technology that has captured the attention of researchers for its advanced features. The sections below explore the technology behind the Shadow Glove, including its sensing and motion tracking capabilities, integration with the Robot Operating System (ROS), real-time control, hand calibration, and feedback customization [1].
Sensing and Motion Tracking
According to Shadow Robot, the motion tracking technology in the glove relies on two different tracking systems. The first is a system developed by Polhemus that consists of a signal-emitting source and multiple antennas that receive the signal. In the case of the Shadow Glove, the source is placed on the palm, while the antennae are positioned on the fingertips. By analyzing the angle and strength of the signal received by the antennae, it is possible to determine the precise location of the fingertips in relation to the palm. This system provides information about the palm relative to the fingertips, resulting in a model with a total of 30 degrees of freedom.
The second tracking system on the glove determines the position of the palm in space. This tracker relies on the HTC Vive, and produces a model with an additional six degrees of freedom. When this data is combined with the data from the first tracker, the system can extrapolate the position of the fingertips in space in relation to the palm, providing a complete picture of the location of the hand in space.
Integration with ROS
Like the Shadow Hand, the Shadow Glove is compatible with the open-source ROS framework. This means that the hand’s software can easily connect to other systems, like the Shadow Hand itself. This modularity opens possibilities for integration with existing systems in both research and industrial settings.
Real-Time Control
Response time is a critical area of development for the Shadow Glove. Quick response is required for accurate and intuitive control of the robot hand, and any pronounced latency makes control through human demonstration extremely difficult. To that end, Shadow Robot has coupled the Shadow Hand and Shadow Glove so that they communicate almost instantaneously. According to the company, the glove sends signals that relay its position at a rate of 960 Hz, which is nearly once per millisecond. The data is immediately converted to movement control signals for both the hand and arm.
One of the biggest latency challenges for Shadow Robot has been remote operation, where the operator uses the glove in one location on a network, and the hand is in a different location on a separate network. However, the Shadow Glove’s fast signal speed and the increasing speed of network communication have minimized this problem. In one remote test, the company measured less than 200 milliseconds of latency from London to San Francisco.
Hand-Size Calibration
Another major innovation of the Shadow Glove is its ability to calibrate to a widely variable range of human hand sizes and physical abilities. Similar wearable controllers are often made using traditional gloves with fixed fingertips, meaning that some people cannot use the controller because they cannot wear the glove. Instead, Shadow Robot created a glove with flexible fingertip attachments, a design that will fit nearly every hand size and accommodate operators who cannot wear a traditional glove. Moreover, the Shadow Glove’s relative fingertip positioning technology means that the glove is instantly calibrated to the operator’s hand size. 
Customizable Haptic Feedback
In addition to its adaptability, the Shadow Glove’s open-frame design also makes it highly customizable. The glove can accommodate custom feedback mechanisms, and when combined with tactile sensors on the Shadow Hand, such as the BioTac sensors discussed in Part 2, the system can provide a full sensory experience for the operator, offering even more precise control of the hand.
Shadow Robot and the Future of Robotics
The Shadow Glove and Shadow Hand have the potential to shape the future of robotics and automation. Both have already been used in groundbreaking research and prototype development, and further studies could lead to the development of dexterous robotic systems that can perform tasks in a variety of industries. For instance, robots equipped with the Shadow Hand could be used in manufacturing plants to assemble products with greater precision, speed, and versatility, or in healthcare settings to perform delicate surgeries remotely and with greater accuracy.
Moreover, the Shadow Glove makes it possible for humans to remotely operate robots in hazardous or hard-to-reach environments. This could impact industries like space exploration, mining, oil and gas exploration, and disaster response, where humans face significant risks. Additionally, the data captured by the Shadow Glove can be used to train machine learning systems, meaning that robots can learn from human actions and continuously improve their performance over time. The Shadow Glove and Shadow Hand have the potential to transform the way we think about robotics and automation, and could lead to a safer, more efficient future for many industries.
Conclusion
The development of the Shadow Glove and Shadow Hand represents a significant breakthrough in the field of robotics. By enabling humans to control robots with greater precision and dexterity, this technology has the potential to change a whole range of industries, from manufacturing, to healthcare, to space exploration, and disaster response. The Shadow Hand’s advanced dexterity combined with the motion tracking capabilities of the Shadow Glove make it possible to replicate human actions and perform amazing tasks.
The integration of the Shadow Hand and Glove with ROS and other software platforms mean that this technology is highly adaptable and customizable for a range of applications. The Shadow Hand and Glove represent a major step forward in the development of truly dexterous robotic systems. As the field of robotics continues to evolve, it is clear that the Shadow Hand and Glove will continue to play a role in shaping the future of research and industry.
This is the final installment of our three-part series on dexterous robotic hands. If you haven’t read the first two parts, we encourage you to take a look.
- Part 1 presents the history of dexterous hands, the technology behind them, and introduces the Shadow Dexterous Hand.
- Part 2 is a deeper dive into the Shadow Dexterous Hand, including its technology, custom integrations, and case studies.
For more information about the Shadow Robot Company, please visit https://www.shadowrobot.com/