Antoine Zammit Elias Terman

VP of MarketingVP of Technology

Don’t Let Bad Bots

Deflate Your Conversion

Rates and Brand

+28 More Brands!

Good Bots, Bad Bots, and Human Traffic

The Open Web Application Security Project (OWASP) is an important standards body in the application security community. Their annual top

10 threats list is the basis for many web application security programs. They are now expanding their scope to include automated threats -

bots.

SUBSET OF THREATS NAME DEFINING CHARACTERISTICS

Account Aggregation

Account Creation

Credential Cracking

Credential Stuffing

Use by an intermediary application that collects together multiple accounts and interacts on their behalf

Create multiple account for subsequent misuse

Identify valid login credentials by trying different values for username and/or passwords

Mass log in attempts to verify the validity of stolen username/password pairs

Carding

Card Cracking

Cashing Out

Multiple payment authorisation attempts used to verify the validity of bulk stolen payment card data

Identify missing start/expiry dates and security codes for stolen payment card data by trying different values

Buy goods or obtain cash utilising validated stolen payment card or other user account data

Footprinting

Vulnerability Scanning

Fingerprinting

Probe and explore application to identify its constituents and properties

Crawl and fuzz application to identify weaknesses and possible vulnerabilities

Elicit information about the supporting software and framework types and versions

OTHER

Ad Fraud

CAPTCHA Bypass

Denial of Service

Expediting

Scalping

Scraping

Skewing

Sniping

Spamming

Token Cracking

False clicks and fraudulent display of web placed advertisements

Solve anti-automation tests

Target resources of the application and database servers, or individual user accounts, to achieve denial of service

Perform actions to hasten progress of usually slow, tedious or time-consuming actions

Obtain limited-availability and/or preferred goods/services by unfair methods

Collect application content and/or other data for use elsewhere

Repeated link clicks, page requests or form submissions intended to alter some metric

Last minute bid or offer for goods or services

Malicious or questionable information addition that appears in public or private content, databases or user messages

Mass enumeration of coupon numbers, voucher codes, discount tokens, etc

100% OF OWASP AUTOMATED THREATS (BOTS) TARGET TRAVEL INDUSTRY

PAYMENT

CARDHOLDER

DATA

ACCOUNT

CREDENTIALS

VULNERABILITY

IDENTIFICATION

This work is licensed under the Creative Commons Share-Alike License for OWASP Automated Threat Handbook Web Applications by Distil Networks

The bad bot landscape

How bad bots impact the travel industry

Web/screen scraping and spinning (hoarding)

Increased GDS pull costs

Decreased SEO, slowdowns, and downtime

Account takeover, credit card fraud, and points fraud

Skewed conversion metrics and look-to-book ratios

WMPH Vacations Case Study

Q&A

Agenda

Advanced Persistent Bots

Basic scripts running

in command line

Headless browsers,

advanced scripts,

Cycle IPs and User

Agents

Real browser

automation, malware

APBs

75%

More Bad Bots Claim to Be Mobile

The amount of bad bots claiming to be

mobile browsers jumped 42.78% in 2016

Mobile App Tools Used by Bot Operators

Mobile Device Farms Mobile Device Emulators Debugging Software

Mobile device emulators that mimic

human users

Testing systems that mimic human users

on mobile devices (e.g. AWS Device

Farm, Google Firebase Testing Lab)

Debugging software used for

tampering with SDKs/reverse

engineering the app

About Distil Networks

Industry Expertise

● Invented the category

● The recognized leader

● 70 airline customers

The Most Effective Technology

● Wider: Web, API, and Mobile

● Deeper: Catch more bots

● Smarter: Without impacting users

Vigilant and Dedicated Partner

● Not A Solution, Your Solution

● Unprecedented access

● An extension of your team

Bot Defense as Adaptable and Vigilant as the Threat Itself

Travel Industry Leaders Rely on Distil...

True or False?

You have good visibility and control

over unwanted website traffic and

transactions.

Poll

Question

You’ve Been Scraped

OWASP AUTOMATED THREAT: SCRAPING

Scraper Bot Sophistication

CompetitorsContent Theft

Competitive Intel

Price Scraping

AggregatorsStart-ups

Unauthorized Middlemen

Hackers / FraudstersContent for Fake Pages

Search EnginesGoogle

Bing

Yahoo

Baidu

Who is behind Web Scraping?

What Kind of Data is Being Scraped?

Customer data

Pricing info

Editorial content

GDS API pulls

SEO strategies

Booking engine inputs

Spinning (Hoarding) by Unauthorized Middlemen

Middlemen using mobile device emulators to continuously

hold seats in the airline booking engine, but not buying

Resell on a secondary market once a buyer is found

Monetary damage:

➔ Empty seats on planes

➔ Loss of add-on sales like upgrades, travel insurance,

etc. (about $20 to $40 of additional revenue per sale

for airlines*)

AIRLINE

CUSTOMER USE CASE

Spinning via

Mobile App

Emulators

Source: http://www.eyefortravel.com/mobile-and-technology/scraping-single-biggest-threat-travel-industry*

Application Denial of Service

OWASP AUTOMATED THREAT: DENIAL OF

SERVICE

Denial of Service Bot

Sophistication

DDoS vs. Application Denial of Service

Application Denial of Service

Attacks the application directly

Hard to spot because it won’t show up

as an anomaly on your firewall and

may not impact load balancer

DDoS

Attacks the ISP hosting your

application

Easier to spot because it floods

upstream infrastructure to point where

packets never arrive at the web server

Account Takeover, Credit Card

Fraud, and Loyalty Points Fraud

Bad Bots Love Login Pages

OWASP AUTOMATED THREATS:

CREDENTIAL CRACKING, CREDENTIAL STUFFING

Account Takeover Bot

Sophistication

How Credential Stuffing Works

Over 1 billion

usernames, passwords

combinations exist in the

wild

Credential stuffing exploits

our propensity to reuse

passwords across multiple

sites.

Account Based Fraud

OWASP AUTOMATED THREATS:

CARDING, CARD CRACKING, CASHING OUT

Account Exploitation Bot

Sophistication

Travel Rewards Fraud

Dark Web listings that indicate typical price

ranges for airline and hotel loyalty accounts:

Airline loyalty accounts: $3.20 - $208

Hotel loyalty accounts: $1.50 - $45

Source: http://blog.cxloyalty.com/the-cost-of-loyalty-accounts-on-the-dark-web-how-to-protect-members

72 percent of loyalty program managers say they

have experienced an instance of loyalty program

fraud firsthand

Skewed Analytics and Look-to-Book Ratios

OWASP AUTOMATED THREAT: SKEWING

Sophistication level of bots

that skew analytics

Sophisticated Bots Appear as Human in Analytic Data

53% of bots able to load external Assets (e.g. JavaScript)

These bots will skew marketing tools such as (Google

Analytics, A/B testing, conversion tracking, etc.)

Skewed Analytics Leads to Misinformed Business Decisions

Inaccurate analytic data results in

Poor funnel analysis & optimization

Poor conversion rates

Inaccurate KPI tracking

Skewed look-to-book ratios

Difficulty in planning server expansion

The bad bot problem I'm most

concerned about:

A. Web scraping

B. Account-based fraud

C. Skewed analytics / look-to-book

D. Slowdowns and downtime

Poll

Question

About WMPH Vacations

At a Glance

Founded 2004 / 140 employees

More than 600,000 clients booked

9 corporate brands

30 websites

Award-Winning Mobile App

Reservation systems serve both direct

customers and 45 agents

Private label solutions

WMPH Technology Stack

30 different web properties

Mobile iCruise App for IOS & Android

Standardized web application stack

Employee Intranet

10 Virtual Servers on AWS

Cloud-based Phone System using 8x8 technology

Entire company is now over 90% cloud-based

API calls into everything from small cruise lines to

large Global Distribution Systems

WMPH Bot Challenges

Bad Bot Challenges

Aggressive web scraping caused site

slowdowns

API scraping almost took a cruise partner

offline

Constant barrage of SQL injection attack

attempts caused lots of noise in logs

Spam on cruise inquiry forms polluted

backend systems

Bots skewed conversion metrics

Tried Several Approaches to Solve the Problem...

Put CAPTCHAs on Forms Looked for Patterns Blocked IPs in AWS ELB

Creates a poor user experience Bots appear human in logs Defeated by distributed IP attacks

Defeated by advanced bots Labor intensive Defeated by low and slow crawlers

Defeated by CAPTCHA farms Distributed attacks hard to pinpoint Defeated by peer-to-peer / proxies

Reduces conversions rates Reactive in nature Reactive in nature

WMPH Vacations Selection Criteria

Bot Detection and Mitigation Solution Requirements

Block web scrapers without impacting human visitors or

good bots like Googlebot

Increase website availability and speed

Simple setup

Little or no maintenance; “self-optimizing” solution

Protect APIs powering our websites and mobile apps

Protect our web and mobile API servers

Fingerprint device

Verify browser

Verify device

Verify human

Verify Mobile Device ID

Verify mobile app

Verify device

Verify human

Stop bot operators (using mobile device farms,

device emulators, etc.) from accessing the API

servers that power our mobile apps

Prevent scrapers from hitting our

APIs through our website or by going

directly to our API servers

WMPH Results with Distil

40% increase in response times; no slowdowns

since deploying Distil

Improved partner relationships

Leads up 100% – No more spam – Only serving

CAPTCHAs to bots

Conversion rates up 22%

Self-tuning, proactive approach saving 20 hours

per month

Protecting login of company intranet

iCruise.com Traffic Overview

iCruise.com Traffic Overview

iCruise.com Traffic Overview

iCruise.com Click Fraud Report

Best Practices and Lessons Learned

IT and marketing need to partner on solving

the bad bot problem.

Review the Distil logs daily.

Blacklist aggressive bot IP numbers

Report aggressive IPs to their respective

IPSs. Follow up, and follow up, and follow

up.

Distil support will give you a list of urls being

hit by the bad bots. This will help you

determine what they are trying to do.

Don’t whitelist your office IP right away.

www.distilnetworks.com/trial/

Offer Ends: October 31st at 5PM

Two Months of Free Service + Traffic Analysis

Antoine Zammit Elias Terman

VP of MarketingVP of Technology