Datamine Competition Analysis, for Retailers
Transcript of Datamine Competition Analysis, for Retailers
www.datamine.gr
Company profile
datamine ltdDatamine was founded in January '05 and is based in Athens, Greece. Having as primary activity the
design & implementation of integrated business solutions using cutting edge I.T. and data modeling
techniques, Datamine is dedicated in redefining the role of information technology within the modern,
customer-centric enterprise through innovative data-driven applications
Solutions & ServicesDatamine provides state-of-the-art I.T. solutions with embedded Statistical & Data Mining logic, answering
the demanding management information needs of competitive corporations. Solutions range from
analytical CRM, data modeling & business intelligence to loyalty, campaign management and executive
information systems
Human ResourcesDatamine’s team demonstrates sound scientific background and extensive professional experience in
statistical analysis, data mining, software architecture, data warehousing and B.I. within telecom and
banking environments
Major Customers Partners
www.datamine.gr
Contents
Market Intelligence SystemAn overview of the solution
Business Value Long term benefits of the solution
Architecture overviewThe key components of the solution
Market Search & Query engineDescription of functionality on market search, browsing and querying
Crawler specificationCrawler execution planning, editions/ variations depending on embedded processing model
Rule Engine & AlertingDescription and examples on the rule engine
ReportingHigh-level description of reporting capabilities
Implementation roadmapPackaging, task summary and estimated effort
www.datamine.gr
Market Intelligence System – Overview
Datamine’s Market Intelligence Solution
Market
Intelligence
database
The
web
Management, Business Analysts
Market
Synch
engine
Crawler
engine
Market Manager
Monitoring, analysis and administration
Integrated systems
Online retailer
Intranet applications
Owned Portals, e-
commerce applications
Obtain Market overview & insight, trends, competitors
pricing policies, KPIs
Define business rules, alerts, synchronization settings
The proposed solution performs the following set of tasks:
1. Automatically collects product price information from a configurable set of web sites
2. Maintains a historical market database
3. Enables powerful price comparisons, reporting and analytics
4. Provides advanced tools such as alerting, conditional synchronization of external systems
5. Provides a framework for management & administration (execution settings, control and monitoring)
www.datamine.gr
Benefits
A wider range of benefits result from the proper operation of the system:
- Market responsiveness: Become more competitive on a day-to-day level – flexible to
instantly response to selected price-related triggers.
- Monitor the market: Minimize the effort required to monitor the market, standardize
monitoring process, eliminate related sources of errors, missing info. KPIs, targets, baselines
and limits may ‘automate’ the monitoring process
- Obtain market insight: Analyze & understand the market through reporting and analytics,
custom analysis and queries
- Understand competition: Monitor and analyze competitors pricing strategies. Examine
trends, cycling behaviors, special offerings
- Market database: smooth operation of the system will build a rich, historical market database
with a wealth of information useful in several marketing activities
www.datamine.gr
Market Intelligence System – Overview
Crawler
Raw input
handler
Market
Intelligence
database1 2
5
Web Services/
API
Market Manager
Monitoring, analysis, adminCustom Apps
The
web
Text Modeler
Text structure, &
classification
Reporting
Services
Advanced
Analytics
Request
handler
Search &
Compare
functionality
Administration
Execution
settings,
configuration
manager
3
4
8
6
7
Retailer -
application
layer
Datamine’s
Market
Intelligence
infrastructure
Product
catalogue
Synchronizer
in/outbound
synchronization
Retailer
Intranet
applications
portals, e-
commerce
applications
Integrated
systemsData sources
www.datamine.gr
Market Search & Query Engine
Powerful search & query engine against the underlying market database. Enables efficient market
query and navigation. Authorized users will be able to search products and pricing info:
1. Across providers (competitors) for user defined time intervals
2. Search using structured product information, categories, price limits, product hierarchies
3. Search using tags as automatically generated and maintained by text modeler component,
on descriptive product information (free text)
Examples:
- Get all products in category <laptop computers> for which <Competitor A> is on average
more than <5%> cheaper within the last <6-months>
- Get all products in category <home cinema> with at least 3 price reduction events during the
last 12 months
- Get competitors with average price in category <electronics> between <150> and <300>.
Drill through for each competitor to examine the detailed product listing.
- Get detailed price change history for products with code like ‘XXXX%’ for <competitor A>
Authorized users can manage Market Queries (save, edit, status change)
Export functionality in MS Excel or flat files is provided
www.datamine.gr
Crawler Execution planning
Execution manager enables the configuration of parameters which define how and when crawler is
executed. Configuration is feasible in several levels. The most significant are:
- List of target sites: define the targets to by analyzed (set of web-sites, portals etc)
- List of target products: define the set of primary products for which to gather competition
data (by reference and/or through keywords)
- Frequency of execution: define how often and under what conditions the process is
executed
- Logging options/ level of detail: define the level of detailed to be logged in the database
- Alerting options: define rules in order to trigger execution alerts to subscribed users
Specialized reporting for monitoring the smooth execution of the crawler, is included in the solution.
This involves a set of action analyzers, providing detailed information on the performance &
effectiveness of the crawler
www.datamine.gr
Crawler: Key-Characteristics
The proposed crawler component ensures performance, reliability and value maximization of
the information collected. The most significant characteristics of the crawler engine are:
- Price validation against private or category pricing history, with alerting if price is outside ± n
standard deviations. This alerts for erroneous data collected and prevents from loading into
the database.
- Reference matching: product names and categories against a valid and rated product
catalogue and owned metadata as generated by the system itself
- Similarity measures: similarity-based matching enabling efficient handling of product name
variations etc.
- Product/ category matching score: indicating if the product is matched and with what
confidence. information on the level of product matching achieved will be available for
reporting, searching, alerting and publishing procedures
- Detailed history of products, prices and information retrieved for each competitor (target site)
- Reporting and analysis of product identification performance
www.datamine.gr
Crawler editions
Three major crawler variations are offered :
Target-Optimized Crawler: this takes advantage of the specific structure of each target site in order
to perform fast, accurate data extraction (with minimized navigation attempts, post processing and
noise reduction operations)
- Benefits: performance, implementation cost, minimized volume of data transferred/ handled
- Limitations: less flexible – must be pre-configured for each specific target site. In case of
target site structural changes, re-configuration must be performed.
Unattended Crawler: this variation required limited initial information on the structure of the target
site. The crawler identifies the pieces of information to be extracted through a full-reference search
(exhaustive site search until the product identification for reference product listing is complete)
- Benefits: flexible – no need to pre-configure site. In case of target site structural changes, re-
configuration is not needed.
- Limitations: performance downgrade, increased implementation cost, larger volumes of data
transferred/ handled.
Hybrid Crawler: this combines the benefits of the two versions. Supports but does not require initial
target pre-configuration.
www.datamine.gr
Rule engine & Alerting
Enables the definition & management of custom-alerts based on price-related triggers. Authorized
user may define rules to alert marketing when:
- Order changing price modification: a change in the price of a registered product, setting a
competitor in a different rank based on price offered.
- Distance changing price modification: a change in the price of a registered product altering
the distance between your offering and market’s best
- Sudden, significant price change : Price for any registered product is significantly changed
by a competitor> indicates a special offering> can be also verified through text modeller
automated text mining
- Massive product price change: competitor A changed prices in a large set of products in
category A, B or C
Alert execution monitoring is feasible through specialized reporting
www.datamine.gr
Reporting & Analytics
Efficiently monitor the market across competitors, product hierarchies, against time, through
powerful, standardized parameterized reporting:
- Market trends report: time series of price aggregates, per product categories, competitor
price statistics against time dimension
- Competitor pricing strategy analyzer: detailed price variations, across products, against
time, for the selected competitor
- Benchmarking reports: comparison figures on competitor positioning, against the product
hierarchy
Dynamic reporting (a drag-n-drop model) can be delivered via OLAP components. This advanced
mode enables power users to define specific, specialized view on the underlying market. Advanced
features such as drill down and drill through are also supported in this option.
Report export in any standard office format is supported for both static & dynamic reporting
Integration with an existing data mart/ warehouse or Business Intelligence infrastructure is offered as
an additional package of services
www.datamine.gr
Crawler & IP tracking: proposed solution
Web
Crawler Host
Server
Target Web
Server
ISP Activity Logging
Web Proxy 1
Web Proxy 2
Web Proxy 4
Web Proxy 3
Web Proxy 5
Web Proxy 6
Web Proxy N
Encrypted
connection
(Active SSH Tunnel)
George Krasadakis Head of Engineering & Product [email protected]
datamine ltdDecision Support Systems
22 Ethnikis Antistasis avenue15232 Chalandri
Athens, Greece
T: (+30) 210 6899960F: (+30) 210 6899968
[email protected]://www.datamine.gr
Contact