Retail Analytics with Visual Recognition & Behavior Detection AI

Overview

The client is a top printing company in Japan that also operates in information communications, lifestyle and industrial supplies, and electronics. For an upcoming technology solution, they wanted to develop a behavior detection AI. This AI aims to detect and analyze in-store customer behavior to identify buying intent. 

Key Features

  • Detect customers’ actions through 3D camera images
  • Analyze patterns of actions 
  • Identify potential customers with buying intent and alert store staff

Technical Stacks

  • Skeleton Extraction
  • CNN
  • LSTM
  • SVM
  • Yolo
  • Deepsort

Project Overview

  • Team: 2 people
  • Market: Japan
gemvietnam

Background

For years, market research companies have been placing researchers in the stores to conduct shopper behavior research. The researchers will record shopper data based on a predefined set of criteria. Then they’ll have to log the data into databases manually. This process is time-consuming and runs risks of human errors.

Meanwhile, most retail stores already have video surveillance (CCTV) cameras for security purposes. We can put this video content into further use with an AI that can identify and analyze objects in the footage. Then, it turns the image data into actionable information analysis and reports customer insights for better decision making. In real-time, it can also alert store staff of high-intent customers. The staff will then assist and support the customer to close the deal. Hence, the AI can improve conversion rate and shopper experience in any brick-and-mortar store. 

 

Challenges

When we started, the client provided us with an extensive data set of video records to work with, thanks to continuous monitoring. Challenges arose as we had to identify the actual customers from the footage and distinguish them from store staff. Then we need to interpret and code unstructured shopper behavior into the AI system.

 

Solutions

First, we started with training our system to analyze the data collected through 3D cameras. Our AI then determined the human body frame and the customers’ location to identify people from the footage. It differentiated between customers and staff by analyzing their clothes.  Then, we started to code the behavior patterns in a customer buying journey for the AI to recognize, such as:

  • Approaching a product
  • Touching and examining a product
  • Putting back or choosing a product
  • Standing and discussing in groups

From these behavior patterns, the AI learned to recognize the buying intent of the customer. With the insights from the AI, it could alert staff where buying assistance was needed, suggesting a personalized shopping experience.  With adequate data, the AI could continue to develop the model through unsupervised learning.

 

Impacts

The behavioral detection AI has been implemented in a few stores in the market and brings impressive results. We are working with the client to continue developing the AI and expanding it to more retailers.