What is Human-in-the-Loop for Machine Learning?

Computers are incredibly fast, accurate and stupid; humans are incredibly slow, inaccurate and brilliant; together they are powerful beyond imagination hence the concept human-in-the-Loop.

author avatar

08 Jun, 2019

What is Human-in-the-Loop for Machine Learning?

Computers are incredibly fast, accurate and stupid; humans are incredibly slow, inaccurate and brilliant; together they are powerful beyond imagination hence the concept human-in-the-Loop.

Given that there have been huge advances in the development and accuracy of machine-driven systems, they still tend to fall short of the desired accuracy rates. This is the philosophy behind the concept of Human-in-the-Loop for Machine Learning.

Human-in-the-Loop (HITL)

This concept leverages both human and machine intelligence to create machine learning models. In this approach, humans are directly involved in training, tuning and testing data for a particular ML algorithm.

The intention being, to use a trained crowd or general human population to correct inaccuracies in machine predictions thereby increasing accuracy, which results in higher quality of results.

Research suggests that a variant of Pareto’s 80:20 rule is consistent with most accurate machine learning systems to date, with 80% AI-driven, 19% human input and 1% randomness.

HITL = SML + AL

Wondering what was the weird formula you just read? What the formula essentially delivers is that HITL is the combination of Supervised Machine Learning (SML) and Active Learning (AL).

Supervised ML, curated (labeled) data sets used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data.

In Active Learning, the data is taken, trained, tuned, tested and more data is fed back into the algorithm to make it smarter, more confident, and more accurate. This approach–especially feeding data back into a classifier is called active learning.

A combination of AI and Human Intelligence gives rise to an extremely high level of accuracy and intelligence (Super Intelligence). This combination is powerful beyond imagination.

When does HITL come into play?

  1. When the cost of errors is too much. An ML algorithm can have absolutely no margin for error. Any room for error leads to dire situations.
  2. When there are Class Imbalances. There are many situations where the thing you are looking for is quite rare, machines cannot answer this question with a high level of confidence. Humans can help resolve matters and, in doing so, retrain the ML model.
  3. When there’s little data available at present. For example, classification of social media posts, for a new business in its early stages, by machines might not be a viable option due to the scarcity of data. Humans will make much better judgments in the early stages, but, over time, machines can learn and can take over the task.

Potential of HITL in Machine Learning applications

  • Traffic cameras that automatically detect lane violations.
  • Fitness applications that automatically log your calorie count from pictures of the food you eat. You don’t have to input the amount and type of food anymore.
  • Security cameras that annotate the root cause of motion sensor triggers (e.g. whether it was an animal, human, falling leaves, a car driving by, etc.) and react accordingly. It also helps decrease the frequency of false alarms.
  • Text messaging apps that transcribe voice to text with high accuracy. With a HITL, it would be easier to transcribe voices carrying a particular jargon or slang.

08 Jun, 2019

I’m a full-stack marketer with 5 years of experience in creating and optimizing digital campaigns seeking to apply actionable strategies and marketing experience to a management level position. Ready to assist businesses leverage digital media with digital strategic planning and measures to achieve ... learn more

Stay Informed, Be Inspired

Wevolver’s free newsletter delivers the highlights of our award winning articles weekly to your inbox.