Enhancing Data QualityEnhancing Data Quality to Improve Customer Datato Improve Customer Data

The issues facing you The issues facing you &&

What you can do about itWhat you can do about it

Phil Corcoran – NS&IPhil Corcoran – NS&I

WARNING POOR DATA WILL DAMAGE YOUR WEALTH !

22%

20%

15%

21%

23%

75%

78%

82%

97%

93%

75%

71%2%

1%

1%

3%

7%

Not particularly important Not at all importantFairly important Very important

Q608 Q608 How important are the following factors to you when contacting a savings How important are the following factors to you when contacting a savings and investments provider?and investments provider?

Replies to application letter quickly

Letters and information that is clear and easy to understandLetters that answer all my questions

Acknowledges receipt of correspondence and cheques

Letters that show understanding of issues raised

Figures on letters are accurate

Details such as name and address are accurate

Scale: Very important = +2, Not at all important = -2

The Eight Truths of MarketingThe Eight Truths of Marketing

1. You will never have perfect customer data.1. You will never have perfect customer data. 2. You will never analyse all your customer data.2. You will never analyse all your customer data. 3. You will never control every customer interaction.3. You will never control every customer interaction. 4. You will never have enough in-house marketing expertise.4. You will never have enough in-house marketing expertise. 5. You will never be content with your in-house IS expertise.5. You will never be content with your in-house IS expertise. 6. You will never achieve the vision of one-to-one marketing.6. You will never achieve the vision of one-to-one marketing. 7. You will never have a centralised marketing dashboard.7. You will never have a centralised marketing dashboard. 8. You will never be immune from legislation.8. You will never be immune from legislation.

Source - Gartner GroupSource - Gartner Group

Approximately 13% of the UK population move house Approximately 13% of the UK population move house each year - each year - Office of Population Censuses & Surveys (OPCS)Office of Population Censuses & Surveys (OPCS)

Approximately 11% of addresses are mailed incorrectly Approximately 11% of addresses are mailed incorrectly each year - each year - Direct Mail Information Service (DMIS)Direct Mail Information Service (DMIS)

45% of the UK population believe that a mis-spelt name 45% of the UK population believe that a mis-spelt name or address is an indication of ‘junk mail’ – or address is an indication of ‘junk mail’ – DMISDMIS

What Does the Problem Look Like?

P CorcoranSarckCrowborough HillCrowboroughEast SussexTN6 2HH

P CochraneSarckCrowborough HillCrowboroughEast SussexTN6 2HH

P. CockramSarawakCrowborough HillCrowboroughEast SussexTN6 2HH

P CorcoranSarckCrowborough Hill

E. SussexTN6 2HH

So Many Problems with DataSo Many Problems with Data

The Perceived ProblemsThe Perceived Problems

Duplications Mis-matches

Missing Values

8 Hr.Data Runs

Don’t know enough Don’t know enough about customerabout customer

1 Month Delivery of Data

Suppressions

Not

A Segmented

Customer

Different

Customer I.D’s

Waiting for the data

Multiple Tables

“If you can keep your head when all around you are losing theirs……..”Kipling

Different Formats

in data

Different Network

Can’t do our own file merges

Low levels of match toGeodemographics

Effect of Poor Quality DataEffect of Poor Quality Data

Duplicated recordsDuplicated records Multiple mailings to same person / householdMultiple mailings to same person / household Increased mails costs Increased mails costs Increased customer annoyanceIncreased customer annoyance Inability to apply stops to all accounts in one goInability to apply stops to all accounts in one go Poor customer intelligence – poor targeting –low ROIPoor customer intelligence – poor targeting –low ROI Restricted ability to overlay external dataRestricted ability to overlay external data Restricted ability to dedupe cold lists from your own dataRestricted ability to dedupe cold lists from your own data Potential breach of Data Protection Act (4Potential breach of Data Protection Act (4thth principle -clean data) principle -clean data)

What’s Caused the Problems?What’s Caused the Problems? Legacy Data Legacy Data Data from a bygone age when a database was just information collected along the way.Data from a bygone age when a database was just information collected along the way. Typical comments :-Typical comments :-

““We never thought we’d need the full name”We never thought we’d need the full name”““We didn’t like to ask people how old they are”We didn’t like to ask people how old they are”““We’ve always showed returned mails as opt-outs”We’ve always showed returned mails as opt-outs”

Etc.Etc. Poor Data Entry procedures Poor Data Entry procedures

Different standards / versions on different systemsDifferent standards / versions on different systems

No validation on Data CaptureNo validation on Data Capture

No measures (metrics) kept on data quality No measures (metrics) kept on data quality

No attempt to establish a Single View of the Customer No attempt to establish a Single View of the Customer

Why Fix It?Why Fix It?Financial Benefits:Financial Benefits: No wasted mailings to undeliverable addressesNo wasted mailings to undeliverable addresses More Efficient Mailings – no duplicates, only one per householdMore Efficient Mailings – no duplicates, only one per household Maximised mailsort discountsMaximised mailsort discounts Better targeting by being able to add insight to more recordsBetter targeting by being able to add insight to more records Improved ROIImproved ROI

Non- Financial:Non- Financial: Faster processing of your data warehouseFaster processing of your data warehouse More confidence in analysisMore confidence in analysis Ability to apply opt-outs / opt-ins to a single recordAbility to apply opt-outs / opt-ins to a single record Better matching of external dataBetter matching of external data Improved customer experienceImproved customer experience

Financial Benefit ExampleFinancial Benefit Example

On a mailing size of 100,000On a mailing size of 100,000

Pack cost of 50pPack cost of 50p

Duplication level of 5%Duplication level of 5%

Undeliverable addresses at 5%Undeliverable addresses at 5%

Assuming 4 mailings per yearAssuming 4 mailings per year

4 x 50p x 10% x 100k = £20,000 wasted4 x 50p x 10% x 100k = £20,000 wasted

But …..even better….But …..even better….

…….. if we use that £20k more effectively in the mailing.. if we use that £20k more effectively in the mailing

At a pack cost of 50p gives 40,000 extra mailsAt a pack cost of 50p gives 40,000 extra mails

At a response rate of 0.5% At a response rate of 0.5%

40,000 x 0.5% = extra 200 responses p.a.40,000 x 0.5% = extra 200 responses p.a.

Where do you Start ?Where do you Start ?An External Data AuditAn External Data Audit

Will look at:Will look at: Duplication LevelsDuplication Levels Propagation of the data within fieldsPropagation of the data within fields Erroneous data in fields Erroneous data in fields (e.g. numerics in name field)(e.g. numerics in name field)

Address formats – (do they match PAF ?)Address formats – (do they match PAF ?) Validation of occupancyValidation of occupancy Salacious names Salacious names

What does it cost – often it costs nothing ! What does it cost – often it costs nothing !

What do we have ?What do we have ?

A boat with a hole in it – full of water and more water coming in.

Step 1Step 1

A Data AuditA Data Audit

to benchmark your start pointto benchmark your start point

Sort Internal Problems FirstSort Internal Problems First

Roger Rabbit, Michael Road, The Boat Ashore, D00 DAHWoger Rabbit, Michael Road, The Boat Ashore, D00 DAHWodger Rabbit, Michael Road, The Boat Ashore, D00 DAHRodger Rabbit, Micheal Road, The Boat Ashore, D00 DAH

By correcting data errors before external referencing you will reduce matching problems

Step 2Step 2

Plug the hole Plug the hole

• Ensure only clean data gets onto your database - use Ensure only clean data gets onto your database - use address validation on entry e.g. QASaddress validation on entry e.g. QAS• Ensure front-end staff aren’t allowed free-format fieldsEnsure front-end staff aren’t allowed free-format fields for key datafor key data• Ensure application records can’t be completed without Ensure application records can’t be completed without all data being valid.all data being valid.• Regular MI checks on data quality – consider Regular MI checks on data quality – consider establishing Quality Gates to control data entryestablishing Quality Gates to control data entry

Step 3Step 3Empty the waterEmpty the water

Invite tenders in order to acquire the services of a good Data Bureau to clean your Invite tenders in order to acquire the services of a good Data Bureau to clean your databasedatabase

Evaluate using a scoring method for subjects:- Evaluate using a scoring method for subjects:- e.g. i) can they cope with your size of database, will they sub-contracte.g. i) can they cope with your size of database, will they sub-contract ii) did they spot all the problems with your data (compare to the others) ii) did they spot all the problems with your data (compare to the others) iii) do they appear knowledgeable on the subject and take the time to explain their iii) do they appear knowledgeable on the subject and take the time to explain their thoughtsthoughts iv) do they feel comfortable to work with iv) do they feel comfortable to work with v) did they field a good team to answer questions (remember this is their best crew)v) did they field a good team to answer questions (remember this is their best crew) vi) Can they validate occupancy or otherwise add value to your data (how manyvi) Can they validate occupancy or otherwise add value to your data (how many sources have they access to?) sources have they access to?) vii) Did they come up with a one-off solution or an ongoing proposal vii) Did they come up with a one-off solution or an ongoing proposal viii) What’s the cost / timescales – second / third year costs?viii) What’s the cost / timescales – second / third year costs? ix) Are they a member of the DMA and therefore bound by its Code of Practice? ix) Are they a member of the DMA and therefore bound by its Code of Practice?

In-House or External ?In-House or External ?In-House solutionIn-House solution Has greatest knowledge of your own dataHas greatest knowledge of your own data May (but not necessarily) be cheaper solutionMay (but not necessarily) be cheaper solution May be able to notice some knock-on effects (snags or May be able to notice some knock-on effects (snags or

benefits) which an external supplier may notbenefits) which an external supplier may not

External solutionExternal solution Will have access to external validation sources (ER / Will have access to external validation sources (ER /

PAF / Suppression files etc.)PAF / Suppression files etc.) Will have wider reaching expertise Will have wider reaching expertise May have an off-the-shelf solution which can save May have an off-the-shelf solution which can save

development costsdevelopment costs Suffer less from scope-creepSuffer less from scope-creep Are usually more accountable for delivery dates and Are usually more accountable for delivery dates and

solutionsolution

EquifaxEquifax Euro DirectEuro Direct ExperianExperianElectoral RollElectoral Roll Electoral RollElectoral Roll Electoral RollElectoral Roll

Insight Insight (360m credit accounts, 130m live)(360m credit accounts, 130m live)

PAFPAF(standard postal address file)(standard postal address file)

PAFPAF(standard postal address file)(standard postal address file)

Search Enquiry DatabaseSearch Enquiry Database(>100m searches conducted in last yr)(>100m searches conducted in last yr)

SHARESHARE(closed user group – a/c behavr)(closed user group – a/c behavr)

Credit ActivesCredit Actives

CCJsCCJs(6m records)(6m records)

CCJsCCJs(6m records)(6m records)

CCJsCCJs(6m records)(6m records)

Alias FileAlias File(name changes)(name changes)

Investors DatabaseInvestors Database Directors DatabaseDirectors Database

Locate DatabaseLocate Database(home movers over time)(home movers over time)

BT OSISBT OSIS BT OSISBT OSIS

Equifax Alias fileEquifax Alias file(previous name information)(previous name information)

BAIsBAIs(Bankruptcies, Administrations & (Bankruptcies, Administrations &

Insolvencies)Insolvencies)

BAIsBAIs

Equifax Biographic Details Equifax Biographic Details DatabaseDatabase

UK InvestorsUK Investors Credit A/C Previous SearchesCredit A/C Previous Searches

Halo Deceased DatabaseHalo Deceased Database Directors at HomeDirectors at Home Gone Away FilesGone Away Files

Sanctions FileSanctions File Absolute MoversAbsolute Movers

Persona FilePersona File Experian Mortality FileExperian Mortality File

Gone Away Suppression Gone Away Suppression FileFile

Forwarding Address fileForwarding Address file

Deceased IndicatorsDeceased Indicators 200m records unique to 200m records unique to experianexperian

Issues to Tackle early Issues to Tackle early Consider an internal ‘cleanup’ of the data before trying to Consider an internal ‘cleanup’ of the data before trying to

carry out close matching e.g. remove spurious characterscarry out close matching e.g. remove spurious characters

Standardise the quality of addresses you hold – Standardise the quality of addresses you hold – benchmark against PAF benchmark against PAF

If you wish to hold ‘cherished addresses’ consider holding If you wish to hold ‘cherished addresses’ consider holding them in a separate field so you can use the primary one in them in a separate field so you can use the primary one in the matching process.the matching process.

Depending upon use of data (operational or marketing) Depending upon use of data (operational or marketing) establish tight matching rules (operational matching will establish tight matching rules (operational matching will be much tighter than marketing)be much tighter than marketing)

Consider customer validation/verification of any address Consider customer validation/verification of any address you ‘recover’ e.g. from NCOA – i.e. write to them to verify.you ‘recover’ e.g. from NCOA – i.e. write to them to verify.

An Example of an External ExerciseAn Example of an External Exercise

2. Enhancement /Align2. Enhancement /Align

4. Negative Verification

4. Negative Verification

3. Positive Verification

3. Positive Verification1. Data Audit1. Data Audit

6. MarketingData Append

6. MarketingData Append

5. Single Mktng. Customer View

5. Single Mktng. Customer View

7. Segmentation7. Segmentation 8. Final DataIntegration

8. Final DataIntegration

9. CustomerTracing for unverified

9. CustomerTracing for unverified

Output enhanced data

Customer Data

Then you can start to use your dataThen you can start to use your dataBy now you’re holding a more accurate and verified view of your customer with By now you’re holding a more accurate and verified view of your customer with

rules around a single view of that customer you can start to understand your rules around a single view of that customer you can start to understand your customer better :- customer better :-

Understand their potential value to you, the share of their wallet you hold and Understand their potential value to you, the share of their wallet you hold and how you can develop the relationship with them.how you can develop the relationship with them.

Enhance records with geodemographics overlays e.g. Mosaic, FSS, Acorn or Enhance records with geodemographics overlays e.g. Mosaic, FSS, Acorn or Cameo to profile them and help discriminationCameo to profile them and help discrimination

Consider holding your own suppression licenses (e.g. TBR, Mortascreen) in-Consider holding your own suppression licenses (e.g. TBR, Mortascreen) in-househouse

Lifestyle Data to understand customer lifestages and driversLifestyle Data to understand customer lifestages and drivers

Develop a strategy to apply to each customer whereby you know if you want to Develop a strategy to apply to each customer whereby you know if you want to cross-sell, upsell, retain or develop the relationship with them. cross-sell, upsell, retain or develop the relationship with them.

Understand which channel or media your customer responds best to.Understand which channel or media your customer responds best to.

The world is your lobster……The world is your lobster……

Manage ExpectationsManage Expectations All database projects take time – the bigger the database the All database projects take time – the bigger the database the

longer the timelonger the time

Define what your objectives are at the outset for any database Define what your objectives are at the outset for any database exercise – to avoid scope creepexercise – to avoid scope creep

Benchmark the data as part of the initial audit and again at the endBenchmark the data as part of the initial audit and again at the end

Establish regular MI reporting on the quality of the data compared Establish regular MI reporting on the quality of the data compared to benchmark – e.g. level of ‘returned-undelivered’to benchmark – e.g. level of ‘returned-undelivered’

Involve ‘user areas’ e.g. Marcomms in the project, even if only in Involve ‘user areas’ e.g. Marcomms in the project, even if only in an ‘informed’ capacity. Ensure they understand the ‘critical path’ of an ‘informed’ capacity. Ensure they understand the ‘critical path’ of what will be delivered and when.what will be delivered and when.

Understand the difference in matching criteria between the Understand the difference in matching criteria between the marketing database and an operational system. You may have marketing database and an operational system. You may have different levels of your SCV.different levels of your SCV.

Starting To BuildStarting To Build

SALES

TARGETING

MODEL BUILDING SEGMENTATIONATTRIBUTE

PROPAGATION

DATA QUALITY

Phil Corcoran NS&I