Improving your CATI Fieldwork

1Brochurename or the title

Improving your CATI Fieldwork

Mobile RDD Sample

Pulsed Mobile RDD

Consumer Sample

Global Coverage with more than 250 key variables

RDD onDemand

RDD onDemand provides direct access to

our global RDD data base

Geocoding Services

Enrich Data with NUTS regions or other socio-demographic data

Sample SolutionsP R E M I E R . S A M P L E . P R O V I D E R

PREMIER.SAMPLE.PROVIDER

SAMPLE SOLUTIONS

Landline RDD Sample

Pulsed landline RDD for more thann 140 countries

www.sample.solutions



Introduction

Various methods exist to improve CATI Fieldwork with regard to Interviewer training, Dialer, incentives etc. Often overlooked is the actual sampling: whether it is business sample, consumer sample, client sample or acquired sample. In this article we will look at various options that exist to improve fieldwork efficiency by developing a sampling strategy. This is a 3 part series which explores B2B Sample, RDD sample and Lifestyle sample.The article uses real cases as an example to derive lessons learned and best practices.

CONTENT

4 Brochurename or the title

3

911

Introduction

Consumer Sample

Blending CATI with SMS data

Contents

5 B2B Sample

13 Conclusion

NAME OF THE CHAPTER


B2B Sample

In many cases clients approach us with a specific SIC code range, employee ranges and a sample is purchased and loaded into the dialer. In some cases, few days later, a client would come back and explain that the sample does not meet the requirements. In our case we would then usually go through the question-naire and analyze the requirements. In many cases however, time and money can be saved by looking into the questionnaire and the quotas beforehand: What are the requirements, who is the contact person we need to speak to, what are other boundary conditions? When explaining this to a sample provider, a sample strategy can be developed prior to the actual fieldwork. Whether it is a small or large scale project, a sampling strategy can be vital for the success of a project.

1. PulsingA fraction of businesses dies every year, changes their phone number, moves addresses thus it is important to utilize a regularly updated database. Even though business databases are updated frequently, we encounter 10-20% of phone numbers not working. Before starting a project it might be worth puls-ing these records to remove these non functioning numbers so that no time is lost during fieldwork. Furthermore, it allows better judgment whether a specific number of interviews can be reached.

NAME OF THE CHAPTER


There shouldn’t be any outdated contact people in the data - changes and

updates need to be made regulary.

It is essential to have multiple contacts – in most B2B research in CATI

research several attempts are required in order to complete an interview or book an appointment for an interview.

Instead of neglecting contact people because of outdated information,

the info can be used to connect to the correct follow-up person.

2. EnrichmentGoing a step further from pulsing, we could enrich the disconnected phone numbers with verified working phone numbers. Additionally, it is possible to add other valuable information like employee count size, location, revenue or contact person. By doing so, we can further narrow down the target market or specify stratifications of our sample so that during the end of fieldwork it is possible whether the target record is still required for any of the quotas.

In the current age, we have access to a plethora of sources of secondary data as well as databases from private or public sources that can be accessed

3. ContactsContact names are essential for bypassing a gatekeeper. However, many data-bases contain only the Managing Director or CEO of a company. The chances that a good gatekeeper will pass you on to the CEO are fairly small. In many cases having a few additional contacts can be beneficial.Imagine you ask the gatekeeper to connect you to accounts payable – I am sure you will be passed on. From there it is an easy step to ask to be connected to the correct contact person.Having outdated contact people in the data is a common problem. As much as databases are kept up-to-date, changes frequently occur in our fast-paced business environment.

NAME OF THE CHAPTER


4. Employee Size & Revenue

In many cases, the target audience is not just described by means of an industry but also by employee size or revenue band (minimum, range or maximum).

These extra filters can have a significant effect on the available data and make fieldwork not feasible due to the fact that the desired targets cannot be reached. In the following example we explain how employee size and turnover can be used to increase the available sample frame:

Imagine you would like to survey manufacturing compa-nies in Belgium and the Netherlands with a revenue of EUR +10M. However, it seems we cannot get sufficient records from data sources making it impossible to reach the desired targets with the available sources. However, we can use an estimate for the required employment size. From which we then calculate around EUR 100.000 in turnover per employee, thus, we can work with an estimate of 100 employees or more. Many databases will contain a lot more information with regard to employee sizes rather than revenue. Note, this method is of course not completely accurate and will have an influence on the incidence rate. We can improve it by increasing the employee size level or decrease it by lowering it (though increasing the sampling frame).

Note that the ratio between turnover and employee size will vary from country to country, for the case of Belgium and the Netherlands, these will be similar.

5. Fine-tuning your selection

By means of sampling a specific audience, we try to create a dataset which has characteristics similar to the desired audience. In this section, we will look at some cases from the past and see how sampling is approached.

Sampling of VETS: Narrowing down the target selection

Let us take the example of animal veterinarians (we do not make use of veterinarians that work with small pets but larger animals such as horses, cattle and so on). We could start off with defining a category or a SIC code and draw the sample from there. Nevertheless, to further increase efficiency we could add some extra steps:

Remove all sample that contains the name “pet” or anything that points to smaller animals. Remove all VETs from urban areas and focus on remote rural areas. Prefer records which contain names like horse or cattle and remove Veterinarians that contain dog or cat care.Even though the sample will still contain Veterinarians which take care of smaller animals, nevertheless, we have managed to increase the incidence rate significantly.


6. Sample Sources

Selecting sample sources does not only have an influ-ence on the quality of the sample but also on the response rate. The most common databases out there require a lot of work and time to keep the records updated. This is costly and can only be covered by suf-ficient sales of these profiled records. How often do you think Fortune 500 companies, companies with +500 M revenue or IT decision makers of TOP-X companies of a country are purchased? While quality might be good, these people might have been contacted way too often for the purpose of market research but also for sales.

In many cases it might, therefore, be handy to blend data: Think of a government database, blended with Google information and maybe LinkedIn contacts. While upfront it may seem to be more work than a standard database.

7. A/B Testing

Under the section of “Fine-tuning your selection” we have discussed various options for sampling Veterinar-ians – for larger studies, it is essential to identify the most effective sampling strategy (= highest incidence rate) for different methods of sampling. Therefore, it is recom-mended to set quotas on the different sample sources. The same holds true for different sample providers. Different reasoning can be used for sampling an audi-ence, nevertheless, only when you measure it, you can manage it and increase your incidence rate.

“Measure it, to manage it!”

After completing about 10% of fieldwork, take a moment to compare the different sample sources together with refusal rates, IR and other important variables. Based on this comparison, you can make a decision with which sample source to continue using for the remainder of your fieldwork.

Sample suppliers should not charge an extra setup fee or high minimum order if you explain to them your case and want advice for your sampling strategy needed for a successful fieldwork period.

“Different methods can be used to achieve higher field productivity. It is important that a sampling strategy is developed which exceeds the standard “Some SIC codes + employees ranges)”.


1. Sample TypeFor each consumer project it is essential to analyze the options for sampling. For general population studies, for example, it is feasible to use RDD sample since it is cost-effective and yields a large coverage while lifestyle sample would be more expensive.

Nevertheless, if certain quotas need to be met (quota-sampling) a mix of RDD sample and lifestyle sample may prove useful. Initial usage of RDD sample (dual-frame) can cover the majority of the population. In order to reach special groups at the end of a survey in specific areas or age groups, targeted lifestyle sample can be used.

2. Screening techniques Though a pulsed or verified sample is slightly more expensive than a raw sample, it is worth the investment as clean samples have a much higher return on investment. Not only the hit rate of the sample should be taken into account, but also the dialing time. Modern Predictive diallers can significantly reduce waiting times for agents, nevertheless, many B2C projects are still not dialed predictive but rather through manual or automatic methods. Using a pre-dialled sample reduces waiting time.

Consumer Sample

Consumer projects can run on client sample (e.g. customer satisfaction studies), RDD sample

(e.g. public opinion polling) or lifestyle sample (e.g. brand awareness of a specific age group).

All these sample types can be used individually or combined.


3.EnrichmentScreening allows us to remove disconnected records while enrichment adds more value to the sample. Through enrich-ment additional information like names, age, locations and more can be added to the sample.Enrichment reduces the need for agents to ask for these vari-ables during fieldwork and diminishes the chance for human error. This also mitigates cost and also increases strike rate per interviewer as the average questionnaire length can be reduced.

4. ReplicatingIn projects where response rate is important, it might be use-ful to replicate sample and use a maximum amount of tries within a replicate before a new replicate is opened. By doing so it is possible to achieve a higher response rate while part of the sample remains untouched. However, at the same time, it might have an influence on the efficiency of the sample.

5. A/B TestingIn many cases, it is not known beforehand which approach for sampling yields the best possible outcome. Therefore, it is recommended to set quotas on the different sample sources. The same holds true for different sample providers. Various reasoning can be used for sampling an audience, nevertheless, only when you measure it, can you manage it and increase your incidence rate. For instance, High-Net-Worth Individuals (HNWI): can be targeted based on postcode, lifestyle sample on income.

6. GeocodingIn many consumer projects aspects such as rural/urban topol-ogy or size of the community (5000 people or more than 1,000,000 people), postcode, city or state are asked. Neverthe-less, this kind of information can be precoded into the sample which saves money by reducing interviewing time – thus geocoding provides multiple advantages.Additionally, statistical data for location and also demographics (income level, ethnographic variables etc.) can be appended in many cases.

http://www.sample.solutions/sample-cati-telephone-fieldwork/lifestyle-sample-b2c/

http://www.sample.solutions/sample-cati-telephone-fieldwork/lifestyle-sample-b2c/


Blending CATI with SMS data collectionThis article is the last in the series of discussing how CATI fieldwork, in general, can be enhanced by the usage

of SMS. In the previous pages, we discussed various methods used to improve CATI data collection by means

of advanced B2B and B2C sampling.Data collection in the future will converge more towards a mixed mode of

data collection. Therefore, a blended sampling approach assists with multi-mode data collection. Thus, we refer

to blended data collection as the use of a single sampling frame with multiple sources of data. In this article, we

focus on 1-way SMS messages which consist of the respondent receiving a link to a survey.

A. Blended Data CollectionData blending begins with the gathering of data from different sources. Once these valid sources are located and loaded, the next step is to merge the useful data into a functioning data set (sampling frame) while excluding the polluted noise of unnecessary data. The sampling frame could be a client database or a national representative (NatRep) mobile phone frame in a population with high penetration of mobile phone numbers.

In this case, one can use an initial wave of survey invitations via SMS. Since the opening rate of SMS is beyond 90% – most of the people that respond to SMS surveys will complete the mobile survey within a few hours. By using the survey link it is possible to track whether respondents have opened or begun the survey. Furthermore, an extra reminder can be sent out via SMS to respondents, while the respondents which have not reacted can be contacted via CATI.

The benefits of this method are evident:

- Cost savings due to part of the interviews being completed online on a mobile device.- Higher response rate- Different characteristics of respondents (some prefer online while others telephone) therefore lower bias.

The timelines and a number of reminders can be chosen depending on the type of survey. In the initial phase of a survey, it is rec-ommended to make use of an A/B split in order to reach maximum efficiency. Respondents can be targeted with SMS messages at any specified time.

NAME OF THE CHAPTER


C. SMS Follow-up questions Some surveys require extra follow-up questions for clarifications or weighting. In many cases finding the right respondent is tedious. Therefore, it is recommended to collect the mobile phone number of a respondent during the main fieldwork. Not only is it possible to follow-up directly with the right respondent but also make use of SMS to collect extra data either through 1-way or 2-way SMS messages.This procedure reduces the probability of selecting the wrong contact person within an organization, decreases the cost due to less time spent on agents having to find the right respondent or even removing the necessity for a CATI follow-up completely, as part of the data can be collected via SMS.

B. Pre-Notification SMSAnother simple but very effective way to decrease cost while at the same time increasing response rate is the usage of pre-notification SMS. With a pre-notification, it is possible to build up trust, inform the respondent of the purpose of the study and set a calling time.

D. Pre-Screening via SMSIn low incidence rate (IR) surveys, one of the biggest cost factors is pre-screening. An agent in the CATI facility will have to work through the respondents and check for eligibility. With some surveys having an IR below 10% it is possible to imagine not just the extra cost due to the low incidence but also the decreased motivation of the actual agent. By using SMS pre-screening one can make use of a cost-effective method for finding the correct target population. Of course, pre-screening is more feasible for B2C projects. Nevertheless,when using a client set, it is possible to add a pre-screening question. Pre-screening via 1-way or 2-way SMS can be connected to CATI software via an API in order to collect the response directly from the CATI software.

E. Quality Check via SMS Many CATI fieldwork agencies currently check around 10% of their interviews either via listening-in of call recordings or alternatively via a follow-up call in which multiple questions are asked.

By using either 2-way SMS or 1-way SMS surveys some basic data can be verified automatically thus saving time and cost. During the fieldwork period mobile phone numbers should be collected (if not already present) and furthermore respondents should be informed about potential follow-up messages.

NAME OF THE CHAPTER


Conclusion

Researchers have a lot to gain from SMS integration and should move from only using email

follow-ups to integrating SMS, as this would provide for multiple modes of interaction with a

respondent and lead to a higher response rate. Email reading rate is currently below 1% and will

decrease even further while SMS open rates are beyond 90% and read within the first 3 minutes.

Feel free to get in contact with Sample Solutions ([email protected]) or your CATI software

provider to discuss how SMS can be integrated within your current CATI fieldwork processes.

www.sample.so lut ions


Improving your CATI Fieldwork

Data & Analytics

Transcript of Improving your CATI Fieldwork