Behavior Analytics in Retail - Accuracy

Definition and Best Practices in Auditing Accuracy in People Counting (Footfall Traffic), Queue

Management, and In-Store Behavior Measurement Solutions

Why should we care about Accuracy?

Accuracy is

the single most important factor in Behavior Analytics

Technologies, such as video, thermal or wireless

Return On Investmentfrom improving sales

conversion and optimization of labor

depends on the accuracy of measuring the

behaviors (activities) of the staff and customers

Increase Conversion by 1%Sales up 3%+

Why 80% Accuracy is Not Enough?

• This system under-counts randomly between 5% to 20%, therefore the Sales Conversion is higher than the actual conversion

• The system’s data distorts the sales opportunities

• Accuracy errors can accumulate….

System’s Conversion

ActualConversion

High Accuracy, Better Store

Performance, More Sales!

The Accuracy Rateis the percentage of system

counts to actual behavior, i.e. 95%

Accuracy for Line Counts

refers to the number of people crossing a line, per

period of timeFor example

95% (or 105%) accuracyper 30 minutes

for the raw data of Arrivals

Accuracy for Queues refers to the behavior of people waiting (standing and moving)

inside a specific area, per period of timeSuch as

95% accuracy for calculating the average number of people

waiting for service, in the last 10 minutes

The Accuracy Rateis defined by

Data ConsistencyData Granularity

and Behavior Anomalies

Data consistency refers to the consistency of the

error, over-count or under-count but not both, and avoiding

inaccuracy spikes and troughs

Data Consistency - Pass

• Compare Arrivals Data with 2 video clips.

• Each video clip runs 15 minutes, for a total of 30 minutes auditing time, and actual count of 100 people entering a store.

• If the system counted 100 people and the auditor viewed the video clips and manually counted 100 people entering the store during the 30 minute period, then the accuracy rate is 100%.

Data Consistency - Pass

• If in each of the 15 minute videos, the system’s data is 50, and the auditor saw 50 people, then the Accuracy Rate is 100%, for each of the 15 minute periods.

• 100% correlation for the 30 minute period

• 100% correlation for each 15 minute period

Audit Result – Pass!

Data Consistency - Fail

• If in the first 15 minutes, the system counted 75 arrivals, but the auditor saw only 25 people enter the store, then the accuracy is 0%.

• If in the 2nd clip, the system output is 25 arrivals, but the auditor counted 75 people in the video clip, then again the accuracy is 0%

Data Consistency – Fail For 30 minute period

total counts are the same for system (75+25=100)

and actual (25+75=100) but

inconsistency of the 15 minute counts fails the audit

The System canOver-Count or Under-

Count not Both!

Data Granularity refers to the time period

of raw data such as the number of people

entering the storeevery 15 minutes

Data Granularity in Arrivals

• Arrivals: An audit of the number of people entering a Bricks-and-Mortar store includes 12 segments of 15 minute video clips

• 3 Hours Accuracy Rate compares the system’s 354 people to the actual arrivals of 352, for accuracy of 99%

99% Accura

• Over-Count: At the 4pm segment, the system’s data of Arrivals was higher than the Actual Arrivals by 22 people (55 versus 34)

• The 15 Minute Segment of 4pm to 4:15pm has the over-count variance of 64%

64% Over

• Under-Count: At 5pm, the Arrivals of the system was lower than the Actual Arrivals (11 versus 28)

• The 15 minute segment of 5pm to 5:15pm has the under-count variance of 60%

60% Under

-Count

Data Granularity in ArrivalsFor 3 hour audit, the Accuracy Rate is 99%

In fact, the audit is a failure, since the system over-counts and under-counts, in the required 15

minute segments

Under-Count

Over-Count

Best Practices in Granularity

are30 Minutes for Sales Conversion,

15 Minutes for Labor Optimization,

5 Minutes for Queue Management,

2 Minutes for Predictive Scheduling

Counting Employees

• Traffic counters capture the behaviors of all people, including employees.

• For many retailers, the behavior of employees is consistent enough to be part of the demand trend.

• For example, if the employees enter the stores before shifts and exit when they are done for the day.

• For example, if the staff uses a Back Door.• For example, if the staff uses the Exit Door.

Why Not Factor?When we factor, we define the

behavior of employeesas constant

and consistent for every period of time and for every traffic level

Why Not Factor?

• In Factoring, we estimate the percentage of staff movements and deduct it from the actual count.

• For example, 10% factoring will take out 10 out of 100 arrivals in the peak hour, and 2 out of 20 people at the closing hour.

Factoring omits a constant percent of traffic out of total demand,

in all time periods, and therefore is

not recommended!

Many retailers, can ignore the bias in

Counting Employees and focus on the trends

Behavior Anomalies - Guards & Greeters

• In luxury stores, it is typical to post greeters, who also serve as guards, at the main entrance.

• In addition to security duties, the guard also opens and closes the door for customers, especially in high-street stores located in cold climates.

• Since the movements of the guard may not be a complete entry or exiting behaviors, most premium counting technologies can identify and discard the guard’s counts.

Behavior Anomalies – Racks & Displays

• Common challenge to accurate counts occurs when a display or a rack is positioned close to the entrance and customers hover around the counting zone.

• For this behavior, the counts can be adjusted by separating between passing and standing behaviors.

• By identifying the length and direction of motion, a technology can pin point a customer who walks by the entrance area and a person who actually enters or exits.

Behavior Anomaliespose challenges to the accuracy of the count, and the ability to

deal with such behaviors is a key differentiator among vendors

Best Practices in Auditing Compare

System Counts to Feeds For example

Arrivals Raw Data to Video Clip

Best Practices in Auditing

No Manual Counting

No Clickers

Seriously!

Best Practices in Auditing Compare

Different Time Periods, Traffic Levels,

and Changes in the Store’s Environment

Best Practices in Auditing

Define Requirements for

Line (Door) Counting Queue Management

and Service Time

Auditing Bi-Directionality

Audit how many people cross the

line per period of time for each direction - Entry and Exit

Auditing Time Segment

Audit for the minimal time period required for the solution

such as 15 minute segments

for Labor Optimization

Auditing Traffic LevelsFor statistical purposes,

an acceptable level of traffic todetermine the accuracy rate of the audit should cover traffic

levels ofno less than 100 people

Auditing Queuesrequire special parameters

includingWaiting or Waiting Time

Time ThresholdDropping Error

Auditing QueuesKey Performance Metrics are

Per Period of TimeWaiting – Average Number of

people waiting in line for service

Waiting Time – Average Waiting Time in the queue

Auditing #Waiting

• Data Table: We use a system that tracks individuals, each with their own random Identification Number (ID) and Waiting Time.

• 5 Minute Segment: Summary for the 5 minute segment is 19 people each, and the error is 0%.

• System = 19 people (ID 1016 = 0)

• Actual = 19 people (ID 1011 = 0)

Auditing Time Threshold

• Waiting Behavior: ID 1016 is discounted by the system because the real person stood in the queue for only 4 seconds

• Time Threshold: This event is not considered as an error because Waiting in Queue has a 5 seconds Time Threshold

• 4 seconds is Passing Behavior and therefore is discarded from the count

Auditing Dropping Error

• Dropping Error: The system captured images of ID 1010 and ID 1011 incorrectly. The video clip showed an image of 1 person, but the system counted the single individual as two separate people

• Typically, this error occurs when a person stands in a spot for too long without moving

Auditing #Waiting

• Calculate the Accuracy Rate: Discounting the Threshold event of ID 1016, we have 18 correct counts out of the actual count of 19 people

• 18/19= 95% Accuracy Rate.

95% is a superior Accuracy Rate for a 5 minute audit of the Queue Length (Waiting

metric)

Auditing Waiting Time

• Waiting Time: For simplicity, we audit the average of the 20 recorded events

• System’s Average Waiting Time is 1:62 minutes

• Actual Average Waiting Time (manually measured from the video clip) is 1:78 minutes

• 5 minute audit period has an Accuracy Rate of 91%

Auditing Waiting TimeCorrectly

Capturing Waiting Time may be the most challenging

component in counting technologies

Proceed with caution!

Auditing Queues85% is a Good Accuracy Rate for Waiting

Waiting Time ModelsSuch as

Serve 90% of Customers in less than 3 minutes

Are cutting-edge techniques for Workforce Optimization

Auditing Service Time

• Service Time refers to how long people stand inside a defined virtual area (i.e. Zone)

• Shopping Unit: The business function classifies the occupants of the zone as a single buying transaction

• Occupied / Not Occupied: This means the number of people inside the zone can be either zero (none) or 1 (occupied), and as long as the zone is occupied, the system continues to count the Service Time

Auditing Wireless TechnologyWi-Fi and Bluetooth Technologies

depend on active features in the Customer’s Smartphone

thereforeAccuracy relates only

to customers captured in the sample

The Accuracy Statement – Technology Vendor

• Solutions: The company performed absolute accuracy tests for all device configurations, including counting, queuing, detection, service and tracking.

• Granularity: Tests include granularity of audit per individual track and per 15 minute averages

• Behaviors: individual and group behavior; adults and children height differentiation; and extreme and fluctuating ambient and light environments

• Accuracy Rate: consistent accuracy of 98%.

The Accuracy Statement – Value Added Reseller

• Local Environment: The accuracy of the sensor in use depends on the quality of the installation and on the anomalies of behavior.

• Installation Standards: For maximum accuracy, the company uses common auditing parameters and certified installers.

• Accuracy Rate: For typical installations, minimum consistent accuracy level is 95%.

Managing for Accuracy

• While Accuracy is the single most important factor in identifying a solution, system, and vendor, there are other aspects to consider in the decision.

• Each thread of the project comes from a different company—Technology Vendor, Installation Company, Value Added Reseller, and the Retailer itself.

• Follow Best Practices for conducting transparent audits, and promote communication across all levels.

Best Practices in Managing for Accuracy

The Retailer’s Request for Proposal (RFP)

should specify auditing requirements, and ask for the

vendor’s guidelines in defining the Accuracy Rate

Our mission is to nurture, train, and educate a community of behavior analytics professionals

Ronny Max is the author of “Behavior Analytics in Retail” (October 2013), and the founder of Silicon Waves, a consultancy specializing in Behavior Analytics, including People Counting, Queue Management and Schedule to Demand.

Website: BehaviorAnalyticsRetail.com

Behavior Analytics in Retail - Accuracy

Data & Analytics

Transcript of Behavior Analytics in Retail - Accuracy

Accuracy Matters – Using Text Analytics to Drive the Cisco Customer Experience

Discovering Digital Library User Behavior with Google Analytics

Behavior Analytics Social Media Mining. 2 Measures and Metrics 2 Social Media Mining Behavior Analytics Examples of Behavior Analytics What motivates.

UEBA™ User Entity Behavior Analytics - Secure360 · User Entity Behavior Analytics Aristotle Insight Sergeant Laboratories Presented by: Ken M. Shaurette, CISSP, CISA, CISM, CRISC

Leveraging Advanced Analytics to Drive Customer Behavior in the Airline ... · PDF fileLeveraging Advanced Analytics to Drive Customer Behavior in the Airline Industry • Cognizant

Visual Analytics of User Behavior

User behavior analytics - ManageEngine · strengthen your detection systems to identify malicious activity within your network. User behavior analytics (UBA) is your best bet for

Understand In-Store Shopper Behavior With Precise Location Analytics

Empathic accuracy and accommodative behavior among newly married … · 2010. 2. 1. · Empathic accuracy and accommodative behavior among newly married couples SHELLEY DEAN KILPATRICK,a

The Critical Infrastructure Attack Surface - LogRhythm · 2016/12/5 · User and Entity Behavior Analytics Endpoint Behavior Analytics Network Behavior Analytics Healthcare Snapshot

Safety & Behavior Analytics Demo - Dataflix · 2019. 3. 25. · Safety & Behavior Analytics Demo. Geo & Demographics. Driver Behavior. Gender Analysis. Crash Analysis. Let’s Talk

CyberPROOF's c-AssurPeople User Behavior Analytics - Major ... · CyberPROOF's c-AssurPeople User Behavior Analytics ... Major US online retailer enhances visibility into user ...

Understanding Behavior Flow Reports In Google Analytics

Accuracy of concrete strength prediction behavior in ...

User behavior analytics

Boost Your Text Analytics Accuracy - MeaningCloud Webinar

Search Accuracy Analytics White Paper

Analyzing Website Behavior with Google Analytics

Behavior Analytics in Retail - Service Intensity (Spanish)

EFFECTIVELY TARGETING ADVANCED THREATS · 2017-06-26 · User & Entity Behavior THREAT VISIBILITY Analytics ANALYTICS AND INVESTIGATION. ... User & Entity Behavior THREAT VISIBILITY