What does “Human-in-the-Loop” mean for computer vision?
Why does “human-in-the-loop” matter?
Balancing efficiency and human oversight
Creating a symbiotic relationship

The essential role of “human-in-the-loop” in AI & ML

13^th December 2024

What does “Human-in-the-Loop” mean for computer vision?
Why does “human-in-the-loop” matter?
Balancing efficiency and human oversight
Creating a symbiotic relationship

Question on screen: What does human in the loop mean?

Humans play a massive role in making AI fit for purpose. They help people design systems that actually provide a good service and do not essentially make mistakes because there is a human reviewing what is coming from the AI. So actually putting a human in the loop makes a lot of sense.

Question on screen: Is it fair that Amazon faced backlash for using humans to verify transactions?

It did not surprise me at all to find out that Amazon’s Just Walk Out technology actually used humans in the loop to help make the service as good as it was. The challenge they tried to solve using the Just Walk Out technology was a very, very difficult challenge. If you imagine how many different scenarios are involved in a shopper selecting products and then wanting to take them out of a store, it’s very, very hard to design a system that’s foolproof.

You’re always going to make mistakes. Having the system with humans in the loop to look for the most difficult situations and correct those mistakes means that the overall experience would have been better with those humans in the loop. I think the challenge that they had was the way that this information came out. It wasn’t upfront. It was kind of hidden away and of course that was the wrong thing to do. I mean, just not saying that we actually have reviewers looking for circumstances actually gave the wrong impression to people.

They thought, oh, this is completely automated. Isn’t it amazing? I think we just have to accept that AI needs sometimes help to make the right decisions.

Question on screen: How can human involvement be balanced with efficiency?

Actually, when we put a human in the loop, we put them into operational circumstances where it is natural for a human to be involved, not somehow engineer it and shoehorn a human into making a decision on behalf of some kind of AI system.

So a good example is in a security situation where you’re looking for some kind of suspicious behavior, say someone’s trying to conceal a product that they’ve taken off a shelf. You would send a video clip to someone just to review that and say, did we really see what we think we’ve seen? And if it is true, they can follow up on it. If it is not true, they can just say, no, it’s not true.

In many cases, there’s a symbiotic relationship between AI systems and human beings. As long as it’s engineered that way, then human beings and AI can work very well together to improve many, many things in the world.

As artificial intelligence (AI) becomes increasingly embedded in everyday processes, the concept of “human in-the-loop” (HITL) has grown ever more important. Human-in-the-loop can be applied to any deep learning artificial intelligence (AI) project including computer vision, natural language processing (NLP) and transcription.

What does “Human-in-the-Loop” mean for computer vision?

Computer vision systems are designed to perform tasks that replicate human vision and how humans make sense of what they see. The systems take input from a camera, push that through AI use cases and then alert in some form. For instance, at self-checkout, a computer vision system could detect if an item hasn’t been scanned, prompting either an alert to the shopper to self-correct (nudge) or notifying an assistant (blocked checkout). The loop is then closed when the shopper or employee validates the alert.

Human-in-the-loop is the deliberate integration of human oversight in computer vision systems to enhance accuracy, transparency, and reliability.

Why does “human-in-the-loop” matter?

In retail, computer vision AI systems operate in complex environments where excessive alerts or miscalculations can lead to frustration among shoppers or employees, or worse distrust in the system itself. Human oversight trains the system and enables the models to learn what is ‘true’ and what is ‘false’.

For instance, Amazon’s Just Walk Out technology, designed to enable seamless, cashier-less shopping, faced criticism when it was revealed that humans were involved in annotating video images and validating some transactions, which Amazon themselves noted as being necessary to continuously improve the underlying machine learning models.

The issue in this case wasn’t human oversight itself, but rather the lack of transparency on how humans were being used in the loop, which led to confusion over the extent of automation involved and even speculation that AI was not in use.

Balancing efficiency and human oversight

The goal of human-in-the-loop is not to undermine automation but to strike a balance where human judgment complements AI’s speed and efficiency. In retail security, for instance, AI can flag suspicious activities, such as concealment of products. A human can then review a video clip to confirm or dismiss the alert, ensuring that false positives don’t disrupt the shopping experience.

Creating a symbiotic relationship

Rather than seeing human-in-the-loop as a limitation, it should be viewed as an intelligent collaboration. Computer vision AI systems perform repetitive, data-driven tasks with unmatched speed, while humans add contextual understanding to refine those results. This partnership improves accuracy and trust, making systems more responsive to real-world complexity. By recognizing and incorporating this symbiotic relationship, businesses can deploy computer vision and AI solutions that genuinely serve their purpose and users.