Modeling Indirect Effects of Paid Search Advertising...

CELEBRATING 30 YEARS

Vol. 30, No. 4, July–August 2011, pp. 646–665issn 0732-2399 �eissn 1526-548X �11 �3004 �0646

10.1287/mksc.1110.0635© 2011 INFORMS

Modeling Indirect Effects of Paid Search Advertising:Which Keywords Lead to More Future Visits?

Oliver J. RutzSchool of Management, Yale University, New Haven, Connecticut 06511, [email protected]

Michael TrusovRobert H. Smith School of Business, University of Maryland, College Park, Maryland 20742,

[email protected]

Randolph E. BucklinAnderson School of Management, University of California, Los Angeles, Los Angeles, California 90095,

[email protected]

Many online shoppers initially acquired through paid search advertising later return to the same websitedirectly. These so-called “direct type-in” visits can be an important indirect effect of paid search. Because

visitors come to sites via different keywords and can vary in their propensity to make return visits, traffic atthe keyword level is likely to be heterogeneous with respect to how much direct type-in visitation is generated.

Estimating this indirect effect, especially at the keyword level, is difficult. First, standard paid search data areaggregated across consumers. Second, there are typically far more keywords than available observations. Third,data across keywords may be highly correlated. To address these issues, the authors propose a hierarchicalBayesian elastic net model that allows the textual attributes of keywords to be incorporated.

The authors apply the model to a keyword-level data set from a major commercial website in the automotiveindustry. The results show a significant indirect effect of paid search that clearly differs across keywords. Theestimated indirect effect is large enough that it could recover a substantial part of the cost of the paid searchadvertising. Results from textual attribute analysis suggest that branded and broader search terms are associatedwith higher levels of subsequent direct type-in visitation.

Key words : Internet; paid search; Bayesian methods; elastic netHistory : Received: November 3, 2009; accepted: December 22, 2010; Eric Bradlow served as the editor-in-chief

and Carl Mela served as associate editor for this article. Published online in Articles in Advance June 6, 2011.

1. IntroductionIn paid search advertising, marketers select specifickeywords and create text ads to appear in the spon-sored results returned by search engines such asGoogle or Yahoo!. When a user clicks on the textad and is taken to the website, the resulting visitcan be attributed directly to paid search. Existingresearch has focused on modeling traffic and salesgenerated by paid search in this direct fashion (e.g.,Ghose and Yang 2009, Goldfarb and Tucker 2011, Rutzet al. 2010).1 In these models, the effectiveness ofpaid search is evaluated by attributing any ensuingdirect revenues and pay-per-click costs to keywordsor categories of keywords. These approaches assumeimplicitly that if a consumer does not purchase inthe session following the click-through, the money

1 A notable exception is a dynamic model developed by Yao andMela (2011). By leveraging bidding history information in a uniquedata set from a specialized search engine in the software space, theauthors are able to infer present and future advertiser value arisingfrom both direct and indirect sources.

spent on that click is “lost”; i.e., it has not gener-ated any revenue. This line of research has not yetaddressed the subsequent revenues that may be gen-erated by consumers returning to the site. Revenues(e.g., sales, subscriptions, referrals, display ad impres-sions) generated by site visitors who arrive by meansnot directly traced to paid search are typically consid-ered to be separate. Visitors, however, could decidenot to purchase at the first (paid search originated)session but return to the site later either to view morecontent or perhaps to complete a transaction. Indeed,after initially coming to a site as a result of a paidsearch, consumers could return again and again, cre-ating an ongoing monetization opportunity. The prob-lem of quantifying these indirect effects of paid searchis similar to the one described by Simester et al. (2006)in the mail-order catalog industry. They point out thata naïve approach to measuring the effectiveness ofmail-order catalogs omits the advertising value of thecatalogs and that targeting decisions based on modelswithout indirect effects produce suboptimal results.

646

Rutz, Trusov, and Bucklin: Modeling Indirect Effects of Paid Search AdvertisingMarketing Science 30(4), pp. 646–665, © 2011 INFORMS 647

In the case of paid search advertising, ignoring indi-rect effects can positively bias the attribution of salesto other sources of site traffic and downwardly biaskeyword-level conversion rates.

Consumers can access a website by typing the com-pany’s URL into their browser or using a previouslyvisited URL (stored either in the browser’s history orpreviously saved “bookmarks”). Because these visitsoccur without a referral from another source, they fallinto the class of traffic known as nonreferral, and welabel them direct type-in visits.2 Referral visits can bedue to paid search (the consumer accesses the site viathe link provided in the text ad at the search engine),organic search (the consumer clicks on a non-paid,organic link at the search engine), or other Internetlinks, such as clicking on a banner advertisement. Inthis paper, we focus on referrals from a search engine(both organic and paid) and their impact on futurenonreferral visits (the so-called direct type-in). Specif-ically, we are interested in how past paid search visitsaffect the current number of direct type-in visits toa site.

Shoppers formulate their search queries in dif-ferent ways depending on their experience, shop-ping objectives, readiness to buy, and other factors.Indeed, managers at the collaborating firm that pro-vided our data—an automotive website—told us thatwhen they switched off some keywords, a drop indirect type-in visitors occurred “a couple of dayslater,” whereas switching off others did not affectfuture direct type-in traffic. We posit that self-selectioninto the use of different keywords may reflect thepropensity of a paid search visitor to return to thesite later via direct type-in. This proposition is con-sistent with previous research in paid search (e.g.,Ghose and Yang 2009, Goldfarb and Tucker 2011, Rutzet al. 2010, Yao and Mela 2011) that showed hetero-geneity of direct effects across keywords. We seek tocontribute to this stream of research by expandingthis notion of keyword-level heterogeneity into thedomain of indirect effects. For example, imagine twoconsumers wanting to buy a DVD player. One of themsearches for “weekly specials Best Buy DVD player”and the other for “Sony DVD player deal.” An adfor http://www.bestbuy.com is displayed after bothsearches, and both consumers click on the ad. FromBest Buy’s perspective, it seems more likely that theconsumer who searched for Best Buy’s weekly spe-cials will revisit the site. This leads to the notion thatkeywords may be proxies for different types of con-sumers and/or search stages. Thus, they may be moreor less effective for the advertiser based on potentialdifferences in their indirect effects on future traffic,

2 Note that the term “direct type-in” should not be confused withthe direct effect of paid search.

and one might posit that each keyword has its ownspecific indirect effect. From the firm’s perspective,the indirect effect is important to quantify to effec-tively manage paid search advertising.

There are several challenges in doing this. First,there is typically no readily available consumer-leveldata that firms can use to accurately identify a“return” visit. Hence, attribution of future direct type-in traffic to keyword-level paid search traffic needs tobe done based on aggregate data. Second, the num-ber of keywords managed by the firm is often muchlarger than the number of observations, e.g., days,available. Finally, paid search traffic generated by sub-sets of keywords is frequently highly correlated. Tohandle these problems, we introduce a new methodto the marketing literature: a flexible regularizationand variable selection approach called the elastic net(Zou and Hastie 2005). We adopt a Bayesian ver-sion of the elastic net (Li and Lin 2010, Kyung et al.2010) that allows us to estimate keyword-level indi-rect effects across thousands of keywords even witha small number of observations. We also extend theliterature on Bayesian elastic nets by incorporating ahierarchy on the model parameters that inform vari-able selection. This allows us to leverage informationthe keyword provides by introducing semantic key-word characteristics. To extract these semantic char-acteristics from keywords, we introduce an originalstructured text-classification procedure that is basedon managerial knowledge of the business domain anddesign elements of the website. The proposed proce-dure is extendable to other data sets and is easy toimplement.

We test our model on data from an e-commercewebsite in the automotive industry. From our per-spective, the indirect effects should be especially pro-nounced in categories such as big-ticket consumerdurable goods, where the consumer search process islong (e.g., Hempel 1969, Newman and Staelin 1972).Given the nature of the product and evidence for aprolonged search process (e.g., Klein 1998, Ratchfordet al. 2003), we believe that this category is well suitedto study indirect effects in paid search. For example,Ratchford et al. (2003) find that consumers, on aver-age, spend between 15 and 18 hours searching beforepurchasing a new car.

Our intended contribution is both methodologicaland substantive. Methodologically, given its robustperformance in handling very challenging data sets(i.e., many predictors, few observations, correlation),we believe that the (Bayesian) elastic net will findmany useful applications among marketing practi-tioners and academics. To the best of our knowledge,we are the first to introduce this model to the mar-keting field. Moreover, we offer an important exten-sion to this model by allowing the variable selection

Rutz, Trusov, and Bucklin: Modeling Indirect Effects of Paid Search Advertising648 Marketing Science 30(4), pp. 646–665, © 2011 INFORMS

parameters to depend on a set of predictor-specificinformation in a hierarchy. Substantively, we expandthe paid search literature by introducing anotherdimension of keyword performance—propensity togenerate return traffic. We find significant heterogene-ity in indirect effects across keywords and demon-strate that propensity to return is, in part, revealedthrough the textual attributes of the keywords used.Our empirical findings offer new insights from textanalysis and should help firms to more accuratelyselect and evaluate keywords.

The rest of this paper is structured as follows. First,we give a brief introduction to paid search data anddiscuss the motivation for our modeling approach.We then present our model, data set, and results.Next, we discuss the implications of our findings forpaid search management. We finish with a conclusion,the limitations of our approach, and a discussion offuture research in search engine marketing.

2. Modeling Approach andSpecification

We propose to examine indirect effects of paid searchby modeling the effect of previous paid search visitson current direct type-in visits. Our goal is to do thisbased on the type of readily available, aggregate dataprovided to advertisers by search engines, along witheasily tracked visit logs from company servers.

2.1. Aggregate vs. Tracking DataIdeally, firms would have consumer-level data atthe keyword level tracked over time. Although the“ideal” scenario is attractive, our view is that it iseffectively infeasible. There are two places wherethese data can be potentially collected—firms’ web-sites or consumers’ computers. All other intermedi-ate sources (such as search engines and third-partyreferral sites) cannot provide a complete picture of theinteraction between consumers and firms. For exam-ple, Google may be able to track all paid and organicvisits to firms’ websites originated from its engine,but it will not have much information about directvisits or traffic generated through referrals. Needlessto say, because of consumer privacy concerns, searchengines are highly unlikely to share consumer-leveldata with advertisers (e.g., Steel and Vascellaro 2010).Alternatively, tracking on the consumers’ end couldbe implemented by either recruiting a panel or usingdata from a syndicated supplier who maintains apanel. Unfortunately, the size of such a panel neededto ensure adequate data on the keywords used by agiven firm would be impractical—especially for lesspopular keywords in the so-called “long tail.” Cross-session consumer tracking on firms’ websites is alsoproblematic for several reasons—practical, technical,

and corporate policy. First, firms may require eachentering visitor to provide log-in identification infor-mation that can help with tracking across multiplevisits. Although this approach is indeed used forexisting customers of some sites—often subscription-type services (e.g., most of the features of Netflix orFacebook require users to be logged in)—it is unlikelyto be effective with new customers. Because newcustomer acquisition is one of the primary goals ofpaid search advertising, asking for registration at anearly stage is likely to turn many first-time visitorsaway. Second, although tracking is typically imple-mented by use of persistent cookies,3 this approachhas become less reliable in recent years. Currentreleases of major Internet browsers (e.g., InternetExplorer, Mozilla Firefox, Google Chrome) support afeature that automatically deletes cookies at the endof each session. In addition, many antivirus and anti-adware programs—now in use on more than 75% ofInternet-enabled devices (PC Pitstop TechTalk 2010)—do the same. Although there seems to be no con-sensus on the percentage of consumers who deletecookies regularly, reported numbers range up to 40%(Quinton 2005, McGann 2005). A cookie-based track-ing approach would misclassify any repeat visitor as anew visitor if the cookie was deleted, biasing any esti-mate that is based on the ensuing faulty repeat versusnew visitor split. Finally, as a corporate policy, manycompanies in the public sector (39% of state and 56%of federal government websites) and some companiesin the private sector (approximately 10%) simply pro-hibit tracking across sessions (West and Lu 2009).

In the absence of tracking data, aggregate data areavailable that capture how visitors have reached thesite. Visitors to any given website are either refer-ral or nonreferral visitors. A nonreferral visit occursif a consumer accesses the website by typing in thecompany’s URL, uses a URL saved in the browser’shistory, or uses a previously saved (“bookmarked”)URL. We call these visits direct type-in. A referral visitis one that comes from another online source. Forexample, consumers can use a search engine to findthe site and access it by clicking on an organic or apaid search result.4 In this paper, we focus on visitorsreferred by search engines and call these organic andpaid search visits. We note that the proposed modelcan be extended to accommodate other referral trafficsuch as banner ads or links provided by listing ser-vices such as Yellow Pages.

3 IP address matching is another alterative, but as Internet serviceproviders commonly use dynamic addressing and corporate net-works “hide” the machine IP address from the outside world, it ismostly infeasible.4 Companies can also try to “funnel” consumers to their site byusing online advertising. A company can place a banner ad on theInternet, and consumers can access the company’s site by clickingon that banner ad.


2.2. Method MotivationFrom a methodological perspective, the objective ofour study is to demonstrate how a modified versionof the Bayesian elastic net (BEN; e.g., Li and Lin 2010,Kyung et al. 2010) can be used in marketing appli-cations where (1) the number of predictor variablessignificantly exceeds the number of available obser-vations, (2) predictors are strongly correlated, and(3) model structure (but not just a forecasting perfor-mance) is of focal interest to the researcher or prac-titioner. Zou and Hastie (2005, p. 302), who coinedthe term “elastic net,” describe their method using theanalogy of “a stretchable fishing net that retains allthe big fish.” Indeed, the method is meant to retain“important” variables (big fish) in the model whileshrinking all predictors toward zero (therefore let-ting small fish escape). We offer an extension to theBayesian elastic net model proposed by Kyung et al.(2010) by incorporating available predictor-specificinformation into the model in a hierarchical fashion.Before we present the details of the proposed model,we offer a brief motivation for our selected statisticalapproach.

Normally, when the number of observations sig-nificantly exceeds the number of predictors, estima-tion/evaluation of the effects of individual predictorsis straightforward. However, when the number ofpredictors is substantially larger than the numberof observations, estimation of individual parametersbecomes problematic. The model becomes saturatedwhen the number of predictors reaches the numberof observations. Using terminology proposed by Naiket al. (2008), the paid search data we use fall into acategory of so-called VAST (Variables × Alternatives× Subjects × Time) matrix arrays where the V dimen-sion is very large compared with all other dimensions.The problem of having a large number of predictorscoupled with a relatively low number of observationsis referred to as the “large p, small n” problem (West2003). Naik et al. (2008) review approaches to the“large p, small n” problem and identify the follow-ing three broad classes of methods—regularization,inverse regression, and factor analysis. The followingconditions, found in our empirical application, makethese approaches unsatisfactory: (1) the focus of theanalysis is on identifying all relevant predictors thathelp to explain the dependent variable, rather thanjust on forecasting; (2) more than n predictors should beallowed to be selected; and (3) subsets of predictorsmay be strongly correlated. The first condition rules outmethods that reduce dimensionality by combiningpredictors in a manner that does not allow recoveryof predictor-level effects. For example, methods suchas partial inverse regression (Li et al. 2007), principalcomponents (e.g., Stock and Watson 2006), or factor

regressions (e.g., Bai and Ng 2002) make such infer-ence difficult. Some regularization techniques, suchas Ridge regression (e.g., Hoerl and Kennard 1970),do not allow subset selection but instead produce asolution in which all of the predictors are includedin the model. The second condition eliminates tech-niques that become “saturated” when the number ofpredictors reaches the number of observations. Forthis reason models such as LASSO regression (e.g.,Tibshirani 1996, Genkin et al. 2007) are not applica-ble in our context. Although LASSO allows exploringthe entire space of models, the “best” model cannothave more than n predictors selected. Finally, meth-ods that address dimensionality reduction performpoorly when high correlations exist among subsets ofpredictors. The multicollinearity problem is particu-larly important in high dimensions when the pres-ence of related variables is more likely (Daye and Jeng2009). For example, regularization through a stan-dard latent class approach (i.e., across predictors) orwith LASSO may fail to pick more than one predictorfrom a group of highly correlated variables (Zou andHastie 2005, Srivastava and Chen 2010).

Another class of methods that can potentiallyaddress the above conditions is based on a two-groupdiscrete mixture prior with one mass concentratedat zero and another somewhere else—also known asthe so-called “spike-and-slab” prior (e.g., Mitchell andBeauchamp 1988). This class of models is generallyestimated by associating each predictor with a latentindicator that determines group assignment (e.g.,George and McCulloch 1993, 1997; Brown et al. 2002).Bayesian methods are commonly used to explore thespace of possible models and to determine the bestmodel as the highest frequency model (stochasticsearch variable selection, or SSVS). Although this classof methods has been shown to perform well in a“typical” (n > p5 setting (e.g., Gilbride et al. 2006),the application to a “large p, small n” context is notwithout limitations, and computational issues arisefor large p (Griffin and Brown 2007).5

The Bayesian elastic net belongs to the class of reg-ularization approaches and is designed to handle theabove-mentioned three conditions. Like Ridge regres-sion, it allows more than n predictors to be selectedin the “best” model. Like LASSO, it offers a sparsesolution (not all available predictors are selected) and

5 As p becomes large, it becomes hard to identify high-frequencymodels—the key outcome of the SSVS approach (Barbieri andBerger 2004). Consequently, for high-dimensional problems theposterior mean or BMA (Bayesian model averaging) estimator istypically used. Because a BMA estimator naturally contains manysmall coefficient values, it does not offer a sparse solution (Ish-waran and Rao 2010), which is of interest in our case. Though BMAimproves on variable selection in terms of prediction, its drawbackis that it does not lead to a reduced set of variables (George 2000).


allows for grouping across predictors; strongly corre-lated predictors are added to or excluded from themodel together. In the modeling situation describedby the above conditions and faced by firms in thepaid search domain, the Bayesian elastic net offers avery good match as a state-of-the-art approach to the“large p, small n” problem.

2.3. Bayesian Elastic NetWe specify as our dependent variable the numberof direct type-in visitors to the website at a giventime t (DirV). In our data this activity is tracked atthe daily level. We investigate the effect, if any, ofprior paid search visits (PsV) on the number of directtype-in visitors. Our model allows for keyword-leveleffects, which we see as an empirical question.6 Wealso account for seasonal effects, i.e., using daily andmonthly dummies, and for variations in overall web-site traffic patterns by using lagged organic visitors7

(OrgV) as a control. Finally, potential feedback effectsare incorporated by including lagged direct type-invisitors. We use an exponentially smoothed lag struc-ture (ESmo), allowing us to estimate only one param-eter per keyword.8 Our model is given by

DirVt = �+6∑

i=1�

dayi I

dayt +

4∑

i=1�monthi Imonth

t +�DirV ESmot

+ �OrgV ESmot +

K∑

k=1�kPsV

ESmo1kt + �t1

(1)

DirV ESmot =

LDirV∑

l=1

wDirVl DirVt−l1

LDirV∑

l=1

wDirVl = 11

wDirV=(

wDirV1 · · ·wDirV

LDirV

)′1

OrgV ESmot =

LOrgV∑

l=1

wOrgV

l OrgVt−l1

LOrgV∑

l=1

wOrgV

l = 11

wOrgV=

(

wOrgV1 · · ·w

OrgVLOrgV

)′

1

PsV ESmo1kt =

LPsV∑

l=1

wPsVl PsV k

t−l1LPsV∑

l=1

wPsVl = 11

wPsV=(

wPsV1 · · ·wPsV

LPsV

)′1

6 A consistent finding of paid search research (e.g., Ghose andYang 2009) is heterogeneity in response across keywords. One pos-sible reason could be consumer “self-selection” into keywords—consumers reveal information on preferences through the use ofkeywords.7 Organic visits can have a similar effect as paid visits. A consumeraccesses the site through an organic link and might be coming backas a direct visitor going forward.8 Alternatively, we could specific the lag structure explicitly. How-ever, this would require us to estimate multiple (i.e., the number oflags) parameters per keyword—a steep increase for cases in whichthe firm bids on thousands of keywords. To be consistent, we alsouse exponential smoothing on DirV and OrgV.

w000l =

exp4−l× �5∑L000

k=1 exp4−k× �51

where DirVt is the number of direct type-in visitorsat time t, 8Iday

t 1 Imontht 9 is a set of daily and monthly

indicator variables, OrgVt is the number of organicvisitors at time t, PsV k

t is the number of paid searchvisitors from keyword k at time t, K is the number ofkeywords the firm is using, �t ∼ N401�25 and �1� =

4�day1 · · ·�

day6 �month

1 · · ·�month4 5′, �, �, �= 4�1 · · ·�K5

′, andwDirV , wOrgV , wPsV , LDirV , LOrg , LPsV , � , and � areparameters (vectors) to be estimated.

A challenge in estimating Equation (1) is that mostfirms bid on hundreds or even thousands of key-words. Based on the available daily data provided bysearch engines, firms would need to run search cam-paigns for multiple years before one would be able toestimate the model given in Equation (1). For exam-ple, our focal company bid on over 15,000 keywords.One way to address this problem is to estimate themodel specified in Equation (1) for each individualkeyword at a time (controlling for all other variables).A problem with this approach is that the daily traf-fic generated by different keywords is correlated. Thebias from omitting the effects of other keywords inEquation (1) is likely to produce inflated estimates forthe role of any given individual keyword in generat-ing returning direct type-in visitors.

To address this challenge, we propose a Bayesianelastic net (Kyung et al. 2010) that discriminatesamong keywords based on their effectiveness usinga Laplace–Gaussian prior to address the data defi-ciency. The prior ensures that “weaker” predictorsget shrunk toward zero faster than “stronger” predic-tors (see also Park and Casella 2008). A BEN allowsus to estimate keyword-level parameters even whenthe number of keywords is significantly larger thanthe number of observations (n � p5. The Laplace–Gaussian prior encourages a grouping effect, wherestrongly correlated predictors are selected or droppedout of the model together. Finally, it allows for thebest model to include more than n predictors. Ourmodeling approach also has been designed to directlyaddress concerns expressed by practitioners. We havebeen told that “when we turn off sets of keywords,we see a drop in the number of visitors that cometo the site directly. That drop has a time delay of acouple of days.” The goal of our model is to iden-tify which keywords drive this effect, allowing us tomeasure the additional indirect value these particu-lar keywords have above and beyond generating sitevisits via paid search click-through.

Along with data on traffic, firms have detailed key-word lists. In general, keywords can be decomposedinto certain common semantic characteristics, e.g., akeyword is related to car financing. We propose a


novel text mining approach to extract semantic char-acteristics from keywords (for details, see the sec-tion on empirical application), allowing us to leveragesemantics to shed light on why some keywords areeffective and others are not. To this end, we proposea hierarchical model that can incorporate predictor-specific information such as semantics. Note that anelastic net shrinks all predictors to zero—a typicalfeature of regularization approaches. Thus, contraryto the way in which hierarchies on parameters havebeen implemented in the marketing literature, wecannot impose a hierarchical mean on the keyword-specific parameter �k in the standard fashion. In aBEN, parameters � determine the amount of shrink-age toward zero and thus have a significant influenceon the parameter estimate. Our hierarchy is basedon these parameters and we define the BEN prior asfollows:

�∝ exp

[

−1

√�2

K∑

k=1

�1k ��k� −1

2�2

K∑

k=1

�2k�2k

]

0 (2)

We define the hierarchy on the keyword-specificparameters �1k and �2k as follows:

�1k ∼ gamma4c1k1d5 and �2k ∼ gamma4c2k1d51 (3)

where c1k and c2k are parameters to be estimated forall keywords.9 We use a log-normal model to repre-sent the effect of the semantic keyword characteristicson c1k and c2k:

log4ci5∼ N(

U�i1�2i

)

1 i = 1121 (4)

where U is a 4K × S5 matrix of semantic keywordcharacteristics, S is the number of semantic keywordcharacteristics, ci is a 4K × 15 parameter vector con-taining cik, and �i4S × 15 and �i41 × 15 are parame-ters (vectors) to be estimated. (For details, see WebAppendix A in the electronic companion to thispaper, available as part of the online version that canbe found at http://mktsci.pubs.informs.org/.) Ourintegrated Bayesian approach addresses the dimen-sionality issue with the Laplace–Gaussian prior andprovides a parsimonious solution to an otherwiseintractable problem. Our approach provides two keybenefits. First, it enables us to estimate the indirecteffect on a keyword level. Second, our hierarchicalBEN allows us to investigate whether differences inkeyword effectiveness can be traced back to semantickeyword characteristics.

We have also addressed two common model-ing concerns, endogeneity and serial correlation. Inour modeling situation, endogeneity may arise from

9 Note that a gamma has a location (c > 0) and a scale (d) parameter.We set the scale parameter d = 1 for identification.

two potential sources: managerial reaction (feedbackmechanism) to changes in direct traffic and omit-ted variable bias. We probed extensively for endo-geneity, and none of the tests we conducted indicatethat endogeneity is of concern (for details, please seeWeb Appendix B in the electronic companion). Toaddress concerns with regard to serial correlation, weinclude an AR415 error process in our model (e.g.,Koop 2003) implemented in a fully Bayesian frame-work (for details, please see Web Appendix A in theelectronic companion).

2.4. Model ComparisonsWe compare our proposed model against two bench-marks. First, we model direct type-in visits with-out including paid search as an explanatory variable(NO KEYWORDS). This is implemented by estimat-ing Equation (1) without PsV. Second, we modeldirect type-in visits assuming all keywords have thesame (homogeneous) effect, i.e., setting �k = �1∀k(ALL KEYWORDS). Our proposed model followsthe specification described in Equations (1)–(4) aboveand is labeled BEN. Moving from NO KEYWORDSto ALL KEYWORDS enables us to assess the pres-ence and magnitude of the indirect effect but atan aggregate (across all keywords) level. Movingfrom ALL KEYWORDS to BEN enables us to deter-mine the keyword-level effect on direct type-in vis-its and the value of modeling the indirect effect on akeyword level.

3. Data3.1. Data Set DescriptionOur empirical application is based on a data setfrom a commercial website in the automotive indus-try.10 Almost all commercially available cars andlight trucks are offered, prices are fixed (no hagglingpolicy), and consumers can either order their newcars online or obtain referrals to a dealership. Thecompany also brokers after-market services such asfinancing and roadside assistance. The data set spans117 days, from May to September 2006. The com-pany tracks how visitors come to its website: typ-ing in the URL or using a bookmark (direct type-invisits per day, DirV), clicking on an organic searchresult (organic visits per day, OrgV), or clicking on apaid search result (paid search visits per day, PsV).The majority of the visits to the site (see Figure 1)are direct type-in visits (63%, daily average of 30,669visits). Organic visits account for 27% of the traf-fic (daily average of 13,248 visits), and paid searchvisits account for 10% (daily average of 4,791 vis-its). Note that the company focused its advertising

10 The company wishes to stay anonymous.


Figure 1 Number of Site Visitors by Source

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

8/7/

06

8/10

/06

8/13

/06

8/16

/06

8/19

/06

8/22

/06

8/25

/06

8/28

/06

8/31

/06

9/3/

06

9/6/

06

9/9/

06

9/12

/06

9/15

/06

9/18

/06

9/21

/06

9/24

/06

9/27

/06

9/30

/06

Direct visitors

Organic visitors

Paid search visitors

strategy on paid search alone—it did not use ban-ner ads nor links on other websites nor advertise off-line. The data set also includes information on dailypaid search traffic obtained through Google AdWords:the number of daily impressions and clicks, and theaverage cost per click. Because of the wide range ofproducts and services the company offers, the cam-paign consisted of 15,749 keywords. Paid search led,on average, to 230,392 impressions and 4,791 clicks (orvisits) per day (see Table 1). The average cost per click(CPC) is $0.33, resulting in an average daily spend-ing of $1,568 on paid search for Google. The averageclick-through rate (CTR) is 2.1%. Over the 117-dayperiod, 560,558 paid search visits (i.e., clicks) to thesite generated a cost of $183,559. Revenue stemmedfrom selling cars and services as well as from adver-tisements on the site. Nearly all car makers advertiseon the site and are charged based on the number ofpage views. Thus, even if a consumer does not buyfrom the company, he still generates advertising rev-enue by browsing the site.

As noted previously, a key challenge in estimatingindividual keyword effects is that the number of key-words is much larger than the number of observa-tions. In our case, we need to estimate the effects of15,749 keywords from 117 data points (see Table 2).From a practical perspective, it seems unlikely thatall keywords would have a strong effect on returnvisits. For example, keywords that do not lead toclicks are unlikely candidates for generating directvisitors after the search. A paid search ad consists

Table 1 Paid Search Campaign Summary Statistics

Paid search Impressions Clicks Cost ($) CPC ($) CTR (%)

Total 26,955,850 560,558 183,559 0.33 2.1Daily average 230,392 4,791 1,568

of two lines of text and competes for attention withmany other paid ads as well as organic results. Thus,we feel that it is very unlikely that a searcher whodid not click on a paid search ad memorizes theURL displayed in the ad and types it in at a laterpoint to visit the site. In our 117-day sample period,almost half of the keywords (7,312) did not generateany clicks at all, and we exclude these from furtherconsideration. We also note that our data exhibit thelong-tail phenomenon generally found in paid searchcampaigns. In our case, 96% of the cost is spent on the38% (3,186) of keywords with 10 clicks or more overthe observation period. From a managerial perspec-tive, focusing on these keywords seems valid. Theremaining keywords (those not in the top 3,186) havevery few observations over the period of the data.Indeed, one might expect any algorithm to strugglewith those keywords. We tested our model using sim-ulated data and find that very sparse predictors arenot recovered with any reasonable accuracy. We there-fore propose that excluding all keywords with fewerthan 10 clicks11 might be the best approach to theproblem. Thus, we focus our investigation on the top3,186 keywords.12

11 We also tested other cutoff points in the same range and find thatour semantic inference is not affected by the cutoff point.12 Dropping a subset of predictors (keywords) could bias oursemantic analysis of keyword effectiveness (for an example, seeZanutto and Bradlow 2006). Given that we focus on evaluatingthe effects of semantic characteristics on the keyword’s propensityto generate return visits (but not paid search traffic), the subsam-pling used in our case is not conditioned on the dependent variable(�i5, but rather it is based on an associated summary measure (i.e.,paid search visits). If �i and paid search visits are independent,then there is no need for correction, because the dropped �i canbe viewed as missing at random. The analysis of our empiricalresults reveals that there is a trivial correlation between �i and paidsearch visits. However, if this correlation is found to be nontrivial,


Table 2 Summary Statistics for All Keywords vs. Focal Keywords

Impressions Clicks Cost CPC ($) CTR (%)

All keywords (15,749)Total 26,955,850 560,558 $183,559 0.33 2.1Daily average 230,392 4,791 $1,568

Focal keywords (3,186)Total 24,177,614 521,016 $171,420 0.33 2.2(Percentage of all (86.0) (92.9) (95.9)

keywords)Daily average 206,646 4,453 $1,465

3.2. Extracting Semantic InformationOne of the questions we address in this study iswhether useful semantic information can be extractedfrom keywords to understand differences in indi-rect effects. Current academic research on this topicis in its early stages, and very few papers on paidsearch consider semantic properties of key phrases(e.g., Ghose and Yang 2009, Yang and Ghose 2010,Rutz et al. 2010). In these papers, only a few attributescharacterizing a keyword are handpicked in an adhoc manner—e.g., retailer name, brand name, or geo-graphic location. A consistent finding is that thesekeyword characteristics have statistical power in pre-dicting click-through rates and purchases. Based onthese findings, we posit that a semantic analysiscould also prove effective in understanding differ-ences across keywords when it comes to predict-ing return visitation propensity. However, we believethat existing ad hoc procedures of dummy cod-ing can be improved by introducing a more struc-tured approach. In this paper, we propose a noveltwo-stage approach to extract semantic keywordattributes. In the first step, we identify potentiallyrelevant attributes allowing us to “decompose” key-words. In the second step, each keyword is codedusing the identified attributes. In the extreme, eachword or combination of words can be treated as aunique attribute. The downside of this approach isthe dimensionality of the estimation problem—theattribute space formed by unique words can easilybecome unmanageable. Dimensionality reduction canbe achieved by creating a set of attributes where eachattribute is associated with multiple words. For exam-ple, words such as “Ford,” “Toyota,” and “Honda”can be described by the attribute “Make.” Hence,the keyword “Toyota Corolla lease” contains theattributes “Make,” “Model,” and “Financial.” If the

we suggest that the proposed model can be expanded to addressthe subsampling problem. Specifically, we may assess the correla-tion between �i and the corresponding paid search visits based onthe retained keywords only, and we adjust for selection bias whenestimating the effects of semantic attributes �, assuming that thiscorrelation holds true for low-volume keywords as well (e.g., in thespirit of a Tobit model).

set of attributes (e.g., “Make”) and associated words(e.g., “Ford,” “Toyota,” “Honda”) are known, key-word coding can easily be automated. The key chal-lenges, however, lie in attribute definition. First, weneed to determine what attributes should be usedto characterize the underlying keywords; e.g., in theautomotive domain, ”Make,” “Model,” “Year,” and“Financial” might be among potential candidates. Sec-ond, we need to determine which words are associ-ated with each attribute; e.g., we need to know that“Leasing” and “Payments” should be associated withthe attribute “Financial.” One way to approach thisproblem is to use cluster analysis techniques designedfor qualitative data (e.g., Griffin and Hauser 1993).The basic idea behind this class of methods is to con-sider the entire set of words that need to be assignedinto an unspecified number of clusters. Clusteringand classification are done simultaneously by humancoders. The downside of this approach is that imple-mentation becomes quite costly when the number ofwords to be classified becomes large.

The approach proposed in this paper also requireshuman input with regard to the cluster (attribute)definition; however, overall, it is less labor inten-sive than existing approaches. Specifically, we startby collecting unstructured data that describe the busi-ness domain of the collaborating firm. Two sourcesof information are considered in this respect: inter-actions with firm’s management and content of thefirm’s website. We then manually analyze the col-lected data to identify the set of attributes capturingdifferent aspects of the firm’s business, which mightbe of interest to the target market and, hence, mightbe searched for online. Finally, we use these attributesto characterize (code) available keywords. In contrastto the cluster analysis techniques discussed above,our proposed procedure can be seen as a top-downapproach to classification, where we define clustersfirst and then assign keywords to them. From a prac-tical standpoint, the downside of this approach is thatit is possible to miss some semantic dimensions thatmight differentiate keywords. On the other hand, ourapproach may help to discover dimensions of thebusiness environment not yet covered by the set ofkeywords used by the firm. As another benefit, oncethe set of attributes has been defined, the remainingclassification task is fully automated.13

We now outline the key steps taken to extractsemantic attributes from keywords.Step 1. Based on a set of interviews, questionnaires,

and/or other communications with the firm’s man-agement, identify the key areas of business in which

13 For the expanded version of this procedure and detailed exam-ples, please refer to Web Appendix C in the electronic companion.


the firm is offering services to consumers.14 For exam-ple, “Car sales,” “Vehicle pricing,” “Auto insurance,”and “Financial service” were among the identifiedareas.Step 2. For each of the identified business areas,

define a set of words that would (preferably unam-biguously) describe the corresponding area. Forexample, “Financial services” is associated with“Leasing,” “Financing,” “Credit,” and “Loan.”Step 3. Extract cognitive synonyms (synsets) and

hypernyms for each word defined in Step 2 using theapplication program interface of the public domaintool WordNet 2.115 (Miller 1995). For example, for“Financing,” the term “Funding” is a synonym and“Finance” is a hypernym.Step 4. Perform stemming (word reduction to its

base form; see Lovins 1968) for both the list of wordsgenerated in Step 3 and all the words found in key-words used by the firm in a paid search. For example,for “Funding,” the base form is “Fund.”Step 5. For each keyword used by the firm, find the

match in the list of synonyms or hypernyms gener-ated in Step 3. If a match is found, code the corre-sponding keyword on the matched attribute.Step 6. Review the list of classified keywords; if

needed, expand the list of synonyms and repeat fromStep 4.Step 7. For the remaining nonclassified keywords

(if any), introduce additional descriptive categories(e.g., code all conjunctions as category “Grammar”).

In application to our data set, the proposedapproach yields a list of 30 characteristics, of which29 were coded as indicator variables in the empir-ical model. Note that more than one attribute canbe associated with each keyword because most key-words in our empirical data set contain two words ormore and/or because some of the individual wordsare classified into more than one category. Specifi-cally, only 2% of keywords were associated with asingle indicator, whereas 23%, 54%, 19%, and 3% ofkeywords were described by 2, 3, 4, and 5 or moreindicators, respectively. Although the proposed pro-cedure is not fully automated, it significantly reduces

14 A similar procedure was followed to identify key functional areasof the website design. Although many of the functional areas over-lap with the business areas identified by the firm’s management,there was a significant number of unique functional attributes. Forexample, photo gallery, newsletter, and specific make and modelinformation all are functional attributes.15 WordNet® is a large lexical database of English, developed underthe direction of George A. Miller. Nouns, verbs, adjectives, andadverbs are grouped into sets of cognitive synonyms (synsets), eachexpressing a distinct concept. Synsets are interlinked by means ofconceptual-semantic and lexical relations. The resulting network ofmeaningfully related words and concepts can be navigated withthe browser (source: http://wordnet.princeton.edu/).

the effort needed to classify keywords based onsemantic characteristics. Another promising directionfor using keyword information is to leverage devel-opments in text mining and natural language process-ing (e.g., from applications based on user-generatedcontent). Several recent studies have argued that user-generated media, i.e., “the language of the consumer,”may serve as an indispensable resource to under-stand consumers’ preferences, interests, and percep-tions (Archak et al. 2007, Bradlow 2010, Feldman et al.2010). Although the application of text mining to paidsearch has not yet appeared in marketing, we believeit has significant potential.

4. Estimation Results4.1. Model SelectionWe use a predictive approach to assess model fit andto determine model selection. Specifically, we baseour analysis on the posterior predictive distributionusing so-called posterior predictive checks (PPCs).16

Calculating PPCs requires generating predicted (orreplicated) data sets based on the proposed modeland the parameter estimates. Comparing these predic-tions to the observed data is generally termed a pos-terior predictive check (e.g., Gelman et al. 1996, Luoet al. 2008, Braun and Bonfrer 2010, Gilbride and Lenk2010). PPCs offer certain advantages over standardfit statistics. First, a wide range of diagnostics canbe defined based on the distribution of predictionsunder a model. Second, posterior predictive simula-tion explicitly accounts for the parametric uncertaintythat is usually ignored by alternative approaches.Using PPCs we can not only evaluate whether a pro-posed model provides the best fit compared withother models but we can also investigate whetherthe proposed model adequately represents the data.However, model checking by means of PPCs is open-ended, and gold standards have yet to be established(Gilbride and Lenk 2010). We use PPCs to investigatemodel fit in-sample as well as model predictive powerout-of-sample.

As discussed above, in a posterior predictive checkwe compare predicted data sets with the actualdata set at hand. These comparisons are based onso-called discrepancy statistics T . A large number ofpotential discrepancy statistics is available, and theresearcher is faced with an inherent trade-off. Somestatistics, e.g., root mean square discrepancy (RMSD)as used by Luo et al. (2008), utilize the whole datato describe fit, whereas others, e.g., minimum andmaximum as suggested Gelman et al. (1996), allowinvestigation of model fit on certain dimensions of

16 We thank an anonymous reviewer for this suggestion.


Table 3 Posterior Predictive Checks

(a) In-sample

Model RMSD MAD MAPD

NO KEYWORDS 3,291 2,606 0.1194ALL KEYWORDS 1,838 1,451 0.0721BEN 804 621 0.0205

(b) Out-of-sample

Model RMSD MAD MAPD

NO KEYWORDS 3,386 2,738 0.1479ALL KEYWORDS 1,875 1,485 0.0877BEN 1,087 583 0.0263

interest, e.g., the tails. We acknowledge this trade-off and investigate how the model fits the data ona global level as well as on specific dimensions ofinterest to the researcher.17 To test for fit on a globallevel, we use statistics such as RMSD, mean abso-lute discrepancy (MAD), and mean absolute percent-age discrepancy (MAPD). In our case, a regressionmodel with a specific type of shrinkage structure, thedata are relatively basic compared with many mar-keting applications, e.g., no individual-level data asin Gilbride and Lenk (2010). As such, we focus ourinvestigation on specific dimensions of fit using dif-ferent descriptives: fit in the tails by investigatingminimum, maximum, and the 25%, 50%, and 75%quantiles. In addition, we also investigate fit of themodel with regard to an important moment, the vari-ance. We do this because with models of our type,overfitting in-sample and poor fit out-of-sample areconcerns. Finally, we provide box-and-whiskers plotsfollowing Braun and Bonfrer (2010) to illuminate howwell the different models describe the data based onthe predicted data sets.

4.1.1. In-Sample. Following Gelman et al. (1996),we simulate sets of predicted DirVpred at each sweepof the sampler and calculate the proposed discrep-ancy statistics to investigate how close the predictedvalues (DirVpred5 resemble observed values (DirV).Based on the set of discrepancy statistics that cap-ture the whole data (RMSD, MAD, and MAPD), wefind that the proposed BEN model outperforms theNO KEYWORDS and the ALL KEYWORDS models(see Table 3, panel a). Inspecting the figures withrespect to the posterior predictive densities, we seethat the models replicate the data well (see Fig-ures 2(a) and 2(b)). The BEN, in general, providesa better fit in-sample as shown by the smaller vari-ances for the posterior predictive densities. This isalso supported by the p-values reported (see Fig-ures 2(a) and 2(b)). Moreover, close inspection of the

17 We thank the editor for this suggestion.

box-and-whiskers plots (see Figure 3) indicates thatthe BEN is fitting the data better. We expect a modellike ours to outperform a simpler model in-sample.The more critical question is whether a model with alarge number of parameters—which can lead to over-fitting in-sample—will also outperform a much sim-pler model in terms of out-of-sample.

4.1.2. Out-of-Sample. We hold out the last weekof the data and reestimate our models using theremaining 110 data points. Based on our estimates ateach step of the sampler, we predict a set of DirVpred_F

for the holdout sample. We use the same set of dis-crepancy statistics as in sample. We find that our pro-posed BEN outperforms the NO KEYWORDS and theALL KEYWORDS models (see Table 3, panel b). Thisalleviates the concern that in-sample overfitting mightbe driving model selection. Comparing the densities,we find that the BEN replicates the data well. Com-pared with the ALL KEYWORDS and the NO KEY-WORDS models, the BEN provides a better forecast(as shown by the significantly smaller variance of allthe posterior predictive densities; see Figures 4(a) and4(b)). This is again supported by the p-values (seeFigures 4(a) and 4(b)). As was true in-sample, thebox-and-whiskers plots show the BEN to be better atcapturing the data points in the holdout sample (seeFigure 5)—i.e., it is closer to the true value and pro-vides a forecast with a smaller variance.

4.2. Coefficient EstimatesIn Table 4, we present the estimated model coeffi-cients for our proposed approach (BEN) along withthose for the two benchmark models (NO KEY-WORDS and ALL KEYWORDS). We will focus ourdiscussion on the estimates for the BEN model.18 Themodel effectively captures the seasonality present inonline activity and the auto industry. First, onlineactivity is much higher during the week and fallsoff during the weekend (Pauwels and Dans 2001).The highest direct type-in traffic occurs on Mondays(included in the intercept) with increasingly negativeestimates for the daily dummy variables as the weekprogresses. Second, accounting for monthly effectsreveals the typical summer seasonality in the autoindustry. Turning to the effects of previous directtype-in, organic, and paid search visits on futuredirect type-in visits, we calculate both short- andlong-run effects.19 We find that the mean immediate(i.e., next period) effect of DirV is 0.53 and the meanlong-run effect is 1.15. The effect for an organic visit(OrgV) is similar: the mean immediate effect is 0.57

18 In our empirical investigation we set �2p = �2 for all p. We alsoinvestigated a full model and found the results to be very similar.19 We do this at each sweep of the sampler based on effect_LR =

effect_SR/41 −�5 and report the mean of the effects.


Figure 2(a) PPCs—In-Sample

NO KEYWORDS

PPC

var

ianc

e:7.

54 ×

107 (

in-s

ampl

e)

p-value:0.13

p-value:0.15

p-value:0.09

p-value:0.82

p-value:0.30

p-value:0.62

p-value:0.32

p-value:0.30

6 8 10

× 107

6 8 10

× 107

6 8 10

× 107

4 5 6

× 104

4 5 6

× 104

4 5 6

× 104

0.8 1.81.61.41.21.0

× 104

0.8 1.81.61.41.21.0

× 104

0.8 1.81.61.41.21.0

× 104

p-value:0.54

PPC

min

imum

:1.

36 ×

104 (

in-s

ampl

e)PP

C m

axim

um:

5.11

× 1

04 (in

-sam

ple)

ALL KEYWORDS BEN

Figure 2(b) PPCs by Quantile—In-Sample

NO KEYWORDS

PPC

25%

qua

ntile

2.14

× 1

04 (in

-sam

ple)

p-value:0.32

p-value:0.26

p-value:0.35

p-value:0.04

p-value:0.14

p-value:0.19

p-value:0.17

p-value:0.26

2.0 2.2 2.4 2.0 2.2 2.4 2.0 2.2 2.4

× 104 × 104 × 104

3.0 3.5 4.0 3.0 3.5 4.0 3.0 3.5 4.0

× 104 × 104 × 104

2.5 3.0 2.5 3.0 2.5 3.0

× 104 × 104× 104

p-value:0.10

PPC

50%

qua

ntile

2.56

× 1

04 (in

-sam

ple)

PPC

75%

qua

ntile

3.47

× 1

04 (in

-sam

ple)

ALL KEYWORDS BEN


Figure 3 Box-and-Whiskers Plot—In-Sample

Days

1

2

3

4

5

6

2

3

4

5

1

2

3

4

5

6

Days

Days

Dir

ect v

isits

× 1

04D

irec

t vis

its ×

104

Dir

ect v

isits

× 1

04

(1) NO KEYWORDS

(2) ALL KEYWORDS

(3) BEN

and the mean long-run effect is 1.23. The posteriormean of the exponential smoothing parameter � is2.97 (�� = 0048), indicating that any lags beyond t − 2have virtually no effect.20 We find evidence for seri-ally correlated errors; the posterior mean of � basedon an AR415 process is 0.24 (�� = 0009).

We now discuss whether keywords differ withrespect to the effects that they have on future directtype-in visits. Based on our BEN model, we find thateffects differ significantly across keywords. First, ofthe 3,186 keywords, 599 have a significant indirecteffect; i.e., the 95% highest posterior density (HPD)coverage interval does not contain zero. Figure 6presents a histogram of the estimated posterior meankeyword-level parameters. The large spike at zerorepresents the 2,587 keywords that have no significantindirect effect. We also analyzed some of the proper-ties of the 599 keywords identified as having signif-icant effects. For example, these keywords represent66% of the paid search clicks (341,360) but only 54%(13,586,685) of the impressions (see Table 5). Note thatthe click-through rate on the significant keywords ishigher: 2.5% versus 1.7%. This potentially indicatesthat the keywords that better match user needs alsoattract visitors with a higher propensity to return tothe site. Of the 599 significant keywords, 57 generate

20 For example, based on using up to three lags, the weight of timeperiod t − 1 is 94.9%, t − 2 is 4.8%, and t − 3 is 0.3%.

more than 1,000 clicks over the observation period,whereas 132 have between 999 and 100 clicks and410 have fewer than 100 clicks. For the insignificantkeywords, only 15 keywords have more than 1,000clicks and the large majority (2,306) have fewer than100 clicks (see Table 6).

In our setting, the majority of consumers shouldhave a relatively lengthy search process because of thenature of the automotive category (e.g., Klein 1998,Ratchford et al. 2003). We believe that our resultsusing daily search data provide, in the best case,a good estimate of the indirect effect and in theworst case (i.e., many intraday and fewer across-dayssearchers), a lower bound for the indirect effect. Tobe sure, more return visits could be occurring withina day, which is hard to identify given a daily aggre-gated data set. The fact that we find a significant indi-rect effect across days highlights the importance ofthis phenomenon. Whereas some advertisers are nowable to examine intraday data, we note that it is likelyto bring other modeling challenges along with it, e.g.,sparse data in the overnight hours.

In sum, the extent to which paid search bringsconsumers to the site also explains variation in sub-sequent direct type-in visits. This is an importantempirical finding because it indicates that invest-ments in paid search advertising may pay dividendsto firms above and beyond their immediate effects.Second, we find that not all paid search visits are “the


Figure 4(a) PPCs—Out-of-Sample

PPC

var

ianc

e:8.

11 ×

106 (

out-

of-s

ampl

e)

p-value:0.19

p-value:0.09

p-value:0.22

p-value:0.51

p-value:0.20

p-value:0.35

p-value:0.14

p-value:0.18

2 4 2 4 2 4

× 107 × 107 × 107

× 104 × 104 × 104

2.01.51.0 2.01.51.0 2.01.51.0

3.02.52.0 3.02.52.0 3.02.52.0

× 104 × 104× 104

p-value:0.11

PPC

min

imum

:1.

81 ×

104 (

out-

of-s

ampl

e)PP

C m

axim

um:

2.59

× 1

04 (ou

t-of

-sam

ple)

ALL KEYWORDS BENNO KEYWORDS

Figure 4(b) PPCs by Quantile—Out-of-Sample

NO KEYWORDS

PPC

25%

qua

ntile

2.27

× 1

04 (ou

t-of

-sam

ple)

p-value:0.83

p-value:0.33

p-value:0.59

p-value:0.63

p-value:0.23

p-value:0.36

p-value:0.15

p-value:0.47

2.0 2.5 3.0 2.0 2.5 3.0 2.0 2.5 3.0

× 104 × 104 × 104

2.51.5 2.0 2.51.5 2.0 2.51.5 2.0

2.51.5 2.0 2.51.5 2.0 2.51.5 2.0

× 104 × 104× 104

p-value:0.31

PPC

50%

qua

ntile

2.40

× 1

04 (ou

t-of

-sam

ple)

PPC

75%

qua

ntile

2.56

× 1

04 (ou

t-of

-sam

ple)

ALL KEYWORDS BEN

× 104 × 104 × 104


Figure 5 Box-and-Whiskers Plot—Out-of-Sample

Days

1.0

1.5

3.0

2.5

2.0

1.5

3.0

2.5

2.0

1.0

1.5

3.0

2.5

2.0

Days

Days

Dir

ect v

isits

× 1

04D

irec

t vis

its ×

104

Dir

ect v

isits

× 1

04

(1) NO KEYWORDS

(2) ALL KEYWORDS

(3) BEN

same.” In particular, keyword-level data are usefulin investigating how an advertiser can drive “good”consumers to a website. In our case, a subset of 599keywords can be associated with bringing consumersback to the site via direct type-in at significantlyhigher rates than other keywords.

5. Managerial Implications5.1. General FindingsWe find two classes of keywords with respect to gen-erating future repeat visits: 2,587 keywords are noteffective, whereas 599 keywords can be linked to fluc-tuations in direct type-in traffic. We now ask how tovalue this indirect effect. Advertising (by car makers)on the site accounts for a significant part of com-pany revenue. Every viewed page that displays ban-ner ads generates incremental advertising revenue,because banner ads are billed on an exposure basis.Using our results, we estimate the ad revenue gener-ated by the additional direct type-in visits attributableto paid search and then contrast it with the actualcosts of the paid search campaign. Our evaluation is

conservative in that some repeat visitors may subse-quently go on to purchase from the company or bereferred to a dealer, both of which generate substan-tial income. Because of data limitations, our analysisdoes not incorporate these additional benefits fromrepeat visitation. On average, a direct type-in visitorviews 2.7 pages with advertising.21 Based on averageadvertising revenue of $31.59 per 1,000 page views,the value of a direct type-in visitor in terms of adver-tising revenue is roughly $0.08. Our model estimatesimply that, in the long run, significant keywords gen-erate, on average, about 3.3 return visitors per click(or $0.26 in advertising revenue). The average CPCfor significant keywords is $0.22. Thus, our analysisindicates that an advertising profit of about $0.04 perclick is generated by the significant keywords. Indeed,the total value of the estimated additional advertisingrevenue is $90,385.22 This recoups approximately 49%

21 The average number of page views for a direct visitor is10.9. Note that not all pages have revenue-generating advertisingon them.22 Calculated as

∑

class revenue ×LR_effect × no0 of clicks.


Table 4 Coefficient Estimates—Mean and Coverage Intervals

Coefficient estimatesa

Covariates NO KEYWORDS ALL KEYWORDS BEN

Intercept � 4,863.9 4,375.5 4,380.3(1,269.4, 8,503.2) (2,219.1, 6,524.9) (2,806.5, 6,034.4)

Tuesday �day1 −2,781.8 −1,951.4 −2,110.9

(−4,566.3, −896.9) (−2,997.1, −945.4) (−2,911.9, −1,245.1)

Wednesday �day2 −5,587.7 −3,681.4 −4,054.4

(−7,342.7, −3,777.0) (−4,789.9, −2,568.1) (−4,884.5, −3,183.3)

Thursday �day3 −6,937.0 −5,289.1 −5,400.6

(−8,708.6, −5,245.9) (−6,354.7, −4,269.7) (−6,228.2, −4,585.4)

Friday �day4 −7,112.7 −6,076.3 −6,244.5

(−8,773.1, −5,496.4) (−7,140.5, −5,114.4) (−7,020.9, −5,433.2)

Saturday �day5 −8,191.1 −7,387.8 −7,720.7

(−9,939.7, −6,482.6) (−8,444.8, −6,387.3) (−8,502.0, −6,958.4)

Sunday �day6 −5,963.7 −5,404.5 −5,644.0

(−7,643.0, −4,343.3) (−6,322.2, −4,419.0) (−6,368.0, −4,928.0)June �month

1 5,442.9 7,445.8 5,431.0(3,633.2, 7,011.5) (6,360.5, 8,523.8) (4,651.5, 6,241.5)

July �month2 6,937.8 7,021.1 7,267.0

(4,127.2, 9,493.1) (5,473.8, 8,567.3) (6,008.6, 8,570.3)August �month

3 1,171.6 1,491.7 523.2(−731.8, 3,056.9) (432.4, 2,643.6) (−343.3, 1,436.4)

September �month4 1,928.2 −675.4 1,173.9

(−35.0, 3,798.2) (−1,847.9, 618.8) (264.9, 2,048.7)Direct � 0.691 0.434 0.529

(0.595, 0.783) (0.368, 0.499) (0.488, 0.572)Organic � 0.531 0.474 0.566

(0.349, 0.721) (0.373, 0.576) (0.485, 0.651)All keywords � — 1.865 —

(1.628, 2.107)

aWe report posterior means as well as 95% HPD intervals.

of the total spending on paid search of $183,559 (seeTable 1).

When simply inspecting the top significant key-words, we see that terms with the firm’s brand name

Figure 6 Histogram of the Posterior Means of Individual KeywordEffects

2,600

80

00 10–10 20

and very general terms such as “Buy car,” “Cars,” and“Car sales” have the highest number of clicks. The topinsignificant keywords are very narrow searches suchas “Toyota Avalon specs” and “Used Pontiac Vibe,” aswell as terms that relate to selling a car such as “Usedcar selling.”23 Although we cannot show a completelist of the keywords here, we note that significantterms often appear broader than insignificant terms.Empirically, we have observed in this data set andin other paid search data sets that broader keywords,e.g., “Cars online,” are more expensive (we believemostly because of heightened competition). In ourdata this remains true even when holding positionconstant. Whereas the broader keywords are moreexpensive, it is not clear which type of keyword(based on CPC) would be more prone to attract visi-tors who are more likely to return via direct type-in.Indeed, when computed for the significant 599 key-words, the correlation between keyword effectiveness

23 Note that the company had only recently started to sell andoffer used cars and was not yet an established venue for sellingused cars.


Table 5 Significant vs. Insignificant Keywords—Summary Statistics

Paid search Impressions Clicks Cost ($) CPC ($) CTR (%)

Significant effect 13,586,685 341,360 110,926 0.32 2.5Insignificant effect 10,590,929 179,656 60,493 0.34 1.7

Table 6 Significant vs. Insignificant Keywords—Click Perspective

Paid search Clicks >1,000 999–500 499–100 <100

Significant effect 57 23 109 410Insignificant effect 15 27 239 2,306

in generating return visits and CPC (CTR) is −0.04(−0.15). This suggests that campaign statistics, e.g.,CPC or CTR, may not be good proxies for indirecteffects. We investigate in the next section whethersemantic keyword characteristics help to better under-stand observed differences in indirect effects acrosskeywords.

We end this section by contrasting the implicationsof the BEN model with the ALL KEYWORDS bench-mark model. If one were to rely on the aggregateeffect estimated from the ALL KEYWORDS model,this could significantly bias the indirect value of paidsearch. In this model the mean long-run effect of apaid search visit is 3.9 direct type-in visits, imply-ing $0.32 in revenue. The additional advertising rev-enue would be $650,017—a severe overestimation ofthe additional revenue of $90,385 indicated by thesuperior BEN model.

5.2. Investigating Semantic KeywordCharacteristics

We investigate whether semantic keyword charac-teristics can be used to understand keyword effec-tiveness in driving direct type-in traffic. In our BENmodel, semantic keyword characteristics enter in ahierarchical fashion via the mean c1 of the shrink-age parameters � based on a log-normal model(i.e., we model log4c15). In our setup, a higher meanwill lead to a higher �, which in turn will introducemore shrinkage of the parameter toward zero andincrease the probability that the corresponding key-word has no significant indirect effect. We find thatsemantic keyword characteristics explain variation inthe mean of �1; the mean R2 for this regression is0.23.24 We group the semantic keyword characteristicsinto three classes for ease of exposition: one with neg-ative effects, one with positive effects, and one withno significant effects (see Table 7 for results). Begin-ning with the semantic characteristics that have a neg-ative effect (less shrinkage), we find that keywords

24 The R2 is calculated for each sweep of the sampler.

Table 7 Results for Semantic Keyword Characteristics

Semantic Coverage Change incharacteristic Estimatea intervalb shrinkagec (%)

Intercept 0048 (0.46, 0.51) N/AAuto −0018 (−0.20, −0.16) −5001Buying −0003 (−0.04, −0.02) −707Car −0017 (−0.20, −0.15) −4607Comparison −0002 (−0.04, −0.01) −601Image −0027 (−0.30, −0.23) −7206Grammar −0010 (−0.11, −0.09) −2704Information −0010 (−0.12, −0.09) −2802Make −0002 (−0.02, −0.01) −402Model −0005 (−0.06, −0.04) −1302Sale −0004 (−0.05, −0.03) −1106Search −0026 (−0.29, −0.23) −6907Company −0039 (−0.49, −0.28) −10703Web −0007 (−0.09, −0.05) −1801

Truck 0003 (0.01, 0.05) 904Category 0007 (0.05, 0.09) 1906Channel 0002 (0.01, 0.03) 503Condition 0033 (0.28, 0.37) 8906Inventory 0008 (0.05, 0.10) 2105Feature 0006 (0.04, 0.08) 1602Mileage 0028 (0.25, 0.32) 7702Price 0014 (0.12, 0.16) 3809

Financial 0001 (−0.01, 0.02) —Selling 0002 (−0.01, 0.04) —Vehicle −0002 (−0.04, 0.01) —Year −0010 (−0.14, 0.05) —New −0003 (−0.07, 0.03) —Used −0004 (−0.09, 0.03) —Word Count 0011 (−0.10, 0.29) —NewByYear −0002 (−0.07, 0.02) —OldByYear −0003 (−0.09, 0.02) —

aPosterior means.b95% HPD interval.cCalculated as change relative to mean shrinkage.

including the company’s name (the website opera-tor) have the strongest negative effect. Indeed, key-words that include the company’s name are foundto create return visits by our model. One interpreta-tion is that consumers who are already aware of thecompany (and search for it) have a higher propen-sity to return. Thus, “branded”25 or trademarked key-words are more valuable for the firm in this respect.Next, inclusion of the term “Search” has the second-biggest effect. We can speculate that a consumer whouses the phrase “Search” is actively searching fora car and thus likely to come back after his ini-tial visit and to spend more time engaged in search.Consumers who are searching for photos of a car(as captured by the “Image” attribute) also tend toreturn to the site. The effects can be understood inthe following way: semantic characteristics help toexplain the amount of shrinkage in the corresponding

25 “Branded” here means that the brand name of the company isincluded, not the brand name of the product.


parameters. For example, keywords associated withthe attribute “Auto” (“Image”) are getting approxi-mately 50% (38%) less shrinkage compared with key-words without this attribute (refer to the right columnof Table 7). Combined, the amount of this inducedshrinkage and the keyword’s intrinsic effectiveness ingenerating return traffic translates to the probabilityof the keyword being included in the model. In thecase of the attribute “Auto” (“Image”), this probabil-ity increases by roughly 35% (50%).26

Several interesting patterns emerge from exam-ining the estimates in Table 7. First, phrases suchas “Search,” “Information,” and “Comparison” areindicative of a consumer actively searching, want-ing to gather information, and wanting to comparecars, e.g., “Car shopping guide” or “Compare midsizecars.” Most likely, this type of consumer has not yetdecided which car to purchase but is trying to nar-row his options and move down the funnel. As such,he is more likely to continue searching and returningto our focal site. The use of “Web” might indicate aconsumer who is comfortable searching (and poten-tially shopping) for a car online and therefore morelikely to visit again, e.g., “Buy car online.” Second,keywords including very broad terms such as “Auto,”“Car,” “Buying,” and “Sale” are also more likely to beeffective. Third, keywords including product brands(“Make” and “Model”) are also more effective, e.g.,“Audi cars” or “300SL.” This is an interesting exten-sion to previous findings on the effects of brandedterms on click-through and conversion rate: Ghoseand Yang (2009) show this for product brands, andRutz and Bucklin (2011) show this for a firm’s brand.Our findings indicate that branded keywords thatalready outperform their nonbranded counterparts ontraditional paid search metrics also perform better inattracting return visitors to our focal site.

Next, we turn our attention to semantic character-istics that have a positive impact (those related tolower effectiveness in generating return visits). Com-pared with the characteristics that have a negativeimpact, these are narrower. These characteristics arepart of a more detailed search—for example, includ-ing price-related information (“Price”) or specific cardetails (“Features”), e.g., “BMW 325i sports packagecheap.” Specific details for used cars (“Mileage” and“Condition”) also have a positive effect, e.g., “HondaCivic 20k miles like new.” In both cases, a consumeris looking for something much more specific and ismost likely further down the search/purchase fun-nel. After inspecting our website and finding what hesearched for (or not), the propensity to return is lower

26 Results for the other keywords are excluded for brevity and areavailable from the authors upon request.

compared with a specific detailed search. We specu-late that two different mechanisms may be at work:consumers who search using a narrow detailed searchare more likely to buy directly but are less likely torevisit the site in the future. This could be due to laterstages in the buying process because these consumersalready know what they want to buy and are search-ing for a good deal. If they find an offer they like, theyare apt to buy. If not, they look for other vendors anddo not revisit an already-searched vendor. In our case“Channel” refers to searches with the goal of findinga dealership, e.g., “Ford dealership CA.” Assumingthis information is found and the consumer connectsto a dealership, there is not much incentive to comeback to our focal site. Finally, “Category” and “Truck”can also be seen as a narrower search.

Some semantic characteristics have no effect. Asmentioned before, our focal firm had recently startedselling used cars and was not yet offering a com-petitive set compared with other sites. Thus, modelyear and information on whether a used or new caris searched for both do not have an effect. Finally,word count is not a significant explanatory semanticfactor. Whereas Ghose and Yang (2009) report that alower word count increases click-through and conver-sion rate, we find that the length of the keyword isnot significantly related to its effect on future directtype-in visits.

In sum, we found evidence for a difference in effec-tiveness between broad and narrow keywords. Basedon our findings, broad searches lead to more returnvisits whereas narrow searches do not. Thus, we con-clude that, for our firm, a narrow search seems toindicate a consumer who knows what he is lookingfor. This consumer has, most likely, made up his mindand checks whether our firm has an offer that fits hisneeds or not, without the need to return. A broadsearch seems indicative of an early stage in the funnelthat leads to more information gathering and returnvisits. This interesting pattern might be used by firmsto cast a new light on CPC. Firms often scrutinizetheir most expensive keywords closely and also seek“diamonds in the rough,” i.e., small keywords that arecheap but effective. Our results suggest that the broadand expensive keywords might offer more value thanpreviously thought if long-term effects such as returnvisits are taken into account. On the other hand,our semantic findings are specific to our firm. Paidsearch keywords can represent a variety of very dif-ferent searches and differ across categories and indus-tries. Nonetheless, the BEN model together with ourapproach to create the semantic keyword characteris-tics can be applied by other firms to data from GoogleAnalytics, thus largely avoiding the need to collectadditional information.


6. ConclusionPaid search can be conceptualized as a hybrid betweendirect response (impression → click → sale) and indirecteffects; i.e., the consumer might return/buy at a latertime, but the visit/purchase is, in part, attributable tothe original paid search visit. Because of the fairlyrecent advent of paid search advertising, researchershave so far focused mainly on the direct responseaspect. In this research, we attempt to look beyonddirect response and examine the effects of paid searchfrom an indirect viewpoint. The value of a customerinitially attracted to the firm’s website through paidsearch does not end with the first visit. If the samecustomer comes back to the site in the future (andsuch visits present a monetization opportunity to thefirm), one should seek to attribute the repeat visitsback to the original source—paid search. The objec-tive of this paper is to develop a modeling approachto enable researchers and managers to do this. Ourstudy is based on data from an e-commerce companyin the automotive industry, where direct type-in visitsare of key interest. A major part of the company’s rev-enue comes from banner advertising on the site thatthe company sells to car makers on a page-view basis,so the company seeks traffic, even if the visitors donot purchase. Thus, for collaborating the firm’s busi-ness, the ability to quantify the indirect effects of apaid search is quite important.

The notion that some of the first-time visitorsacquired through paid search advertising later returnto the site as direct type-in visitors is intuitivelyappealing. It can be easily tested using aggregatepaid search data provided by popular search engines,together with internal Web analytic reports. Theharder question is whether the propensity to generatethese return visits varies across keywords and, if so,how. Assessing such keyword-level effects presentssignificant modeling challenges because many firmsuse thousands of keywords in their paid search adcampaigns. Whereas effects need to be evaluatedacross keywords, firms do not have the luxury ofwaiting an extended period of time while a data sam-ple of sufficient duration is collected. Thus, traditionalstatistical methods cannot be applied to gauge theindirect effect at the keyword level.

The modeling approach proposed in this paperallows paid search practitioners to make inferencesbased on limited data, reducing the time needed togenerate important marketing metrics. We propose aflexible regularization and variable selection methodcalled (Bayesian) elastic net, which facilitates suchinference. Our model allows us to estimate keyword-level parameters for thousands of keywords based ona small number of observations that can be collectedover a couple of months. We extend the current liter-ature on Bayesian elastic nets by using semantic key-word characteristics in a hierarchical setup to inform

the model parameters that drive variable selection.Using posterior predictive checks in-sample and out-of-sample, we find that our proposed model outper-forms traditional regression approaches that eitherignore paid search in generating return traffic or treatindividual keyword effects as homogeneous.

Substantively, we find that paid search visitsexplain significant variation in subsequent directtype-in visits to the site. The positive effect of clicks(visits) is consistent with a common search hypothe-sis: paid search is used to initially locate the site, andconsumers later return as direct type-in visitors. Wefind that this effect of paid search should be mod-eled on a keyword-level. Our analysis reveals thatout of the 3,186 keywords that account for 96% ofthe firm’s total spending on paid search, 599 key-words have a significant effect in terms of generat-ing return traffic. We find that the aggregate modelgrossly overestimates the effect of paid search ondirect visits and could lead managers to incorrectlyconclude that the indirect effect of paid search is big-ger than what a more careful keyword-level analysisreveals. Our findings are in line with other studiesthat focus on the direct channel: paid search shouldbest be managed on a keyword level. Our proposedmodel enables us to quantify the indirect effect on akeyword level, allowing managers to adjust existingkeyword-level management strategies to reflect theindirect effect of paid search.

We close our investigation with an analysis of thefactors associated with keyword effectiveness. Wepropose an original approach to generate semanticcharacteristics based on a given set of keywords.We use a novel hierarchical setup to include seman-tic keyword characteristics into our keyword-levelmodel. First, we find evidence that the previouslydocumented value of brands in keywords extendsfrom direct to indirect effects. Second, for our focalfirm, broader keywords, which are generally moreexpensive because of higher levels of competition,are better at producing return visitors compared withnarrow keywords. Our findings have the potential tochange campaign strategy, which at many firms is tar-geted toward the long tail and finding “diamonds inthe rough.” Our analysis shows that broad keywordshave additional indirect effects that narrow keywordsseem to lack. Thus, our model allows managers todetermine which keywords are likely candidates toproduce strong indirect effects and to consider thiswhen determining the firm’s paid search strategy.

In this study we focused on just one type of indirecteffect of paid search—the ability to generate subse-quent direct type-in visits. Additional indirect effectscould also exist, perhaps in less tangible domainssuch as attitude and awareness. A limitation of ourstudy is that we are not able to examine whether


these effects are present. One could imagine thatkeywords are not simply more or less effective buthave different levels of effectiveness on different con-structs relevant to managers. However, if this typeof data set becomes available, our model could beextended to a multivariate dependent variable frame-work. Although our study is based on data availableto managers, a second limitation is that the analysisis based on behavior aggregated across consumers.A large-scale panel-based clickstream data set wouldenable us to directly investigate the premise of ourstudy. In the meantime, our model provides a usefulway to leverage the data that each company adver-tising on search engines such as Google have avail-able on a daily basis. We see our approach as a steptoward understanding the online consumer searchprocess and the role of search engine use. Last, wedo not observe the design of the ad and the landingpage. This means that differences attributed to key-words could be driven by ad copy and landing-pagedesign. We note, however, that most firms we haveworked with manage their paid search campaigns onkeywords and keep ad copy and landing-page designrelatively static, even over extended periods of time.This was certainly the case with the collaboratingfirm in this study. Of course, determining whether adcopy and landing-page design also influence keywordeffectiveness is an interesting and important topic forfuture research.

7. Electronic CompanionAn electronic companion to this paper is available aspart of the online version that can be found at http://mktsci.pubs.informs.org/.

AcknowledgmentO. J. Rutz’s current affiliation is the Foster School of Busi-ness, University of Washington, Seattle, WA 98195.

ReferencesArchak, N., A. Ghose, P. G. Ipeirotis. 2007. Show me the money!

Deriving the pricing power of product features by mining con-sumer reviews. Proc. 13th ACM SIGKDD Internat. Conf. Knowl-edge Discovery Data Mining, San Jose, CA, 56–65.

Bai, J., S. Ng. 2002. Determining the number of factors in approxi-mate factor models. Econometrica 70(1) 191–221.

Barbieri, M. M., J. O. Berger. 2004. Optimal predictive model selec-tion. Ann. Statist. 32(3) 870–897.

Bradlow, E. T. 2010. Automated marketing research using onlinecustomer reviews. Working paper, University of Pennsylvania,Philadelphia.

Braun, M., A. Bonfrer. 2010. Scalable inferences of customer similar-ities from interactions data using Dirichlet processes. Workingpaper, Massachusetts Institute of Technology, Cambridge.

Brown, P. J., M. Vannucci, T. Fearn. 2002. Bayes model averag-ing with selection of regressors. J. Roy. Statist. Soc. Ser. B 64(3)519–536.

Daye, Z. J., X. J. Jeng. 2009. Shrinkage and model selection withcorrelated variables via weighted fusion. Comput. Statist. DataAnal. 53(4) 1284–1298.

Feldman, R., J. Goldenberg, O. Netzer. 2010. Mine your own busi-ness: Market structure surveillance through text mining. Work-ing paper, Columbia University, New York.

Gelman, A., X.-L. Meng, H. Stern. 1996. Posterior predictive assess-ment of model fitness via realized discrepancies. StatisticaSinica 6(4) 733–807.

Genkin, A., D. D. Lewis, D. Madigan. 2007. Large-scale Bayesianlogistic regression for text categorization. Technometrics 49(3)291–304.

George, E. I. 2000. The variable selection problem. J. Amer. Statist.Assoc. 95(452) 1304–1308.

George, E. I., R. E. McCulloch. 1993. Variable selection via Gibbssampling. J. Amer. Statist. Assoc. 88(423) 881–889.

George, E. I., R. E. McCulloch. 1997. Approaches for Bayesian vari-able selection. Statistica Sinica 7(2) 339–373.

Ghose, A., S. Yang. 2009. An empirical analysis of search engineadvertising: Sponsored search in electronic markets. Manage-ment Sci. 55(10) 1605–1622.

Gilbride, T. J., P. J. Lenk. 2010. Posterior predictive model checking:An application to multivariate normal heterogeneity. J. Market-ing Res. 47(5) 896–909.

Gilbride, T. J., G. M. Allenby, J. D. Brazell. 2006. Models of hetero-geneous variable selection. J. Marketing Res. 43(3) 420–430.

Goldfarb, A., C. Tucker. 2011. Search engine advertising: Channelsubstitution when ricing ads to context. Management Sci. 57(3)458–470.

Griffin, J. E., P. J. Brown. 2007. Bayesian adaptive lassos with non-convex penalization. Technical report, IMSAS, University ofKent, Kent, UK.

Griffin, A., J. R. Hauser. 1993. The voice of the customer. MarketingSci. 12(1) 1–27.

Hempel, D. J. 1969. Search behavior and information utilizationin the home buying process. P. R. McDonald, ed. MarketingInvolvement in Society and the Economy. American MarketingAssociation, Chicago, 241–249.

Hoerl, A. E., R. W. Kennard. 1970. Ridge regression: Applicationsto nonorthogonal problems. Technometrics 12(1) 69–82.

Ishwaran, H., J. S. Rao. 2010. Generalized ridge regression: Geome-try and computational solutions when p is larger than n. Work-ing paper, Cleveland Clinic, Cleveland.

Klein, L. R. 1998. Evaluating the potential of interactive mediathrough a new lens: Search versus experience goods. J. Bus.Res. 41(3) 195–203.

Koop, G. 2003. Bayesian Econometrics, 2nd ed. John Wiley & Sons,Hoboken, NJ.

Kyung, M., J. Gil, M. Ghosh, G. Casella. 2010. Penalized regres-sion, standard errors, and Bayesian lassos. Bayesian Anal. 5(2)369–412.

Li, Q., N. Lin. 2010. The Bayesian elastic net. Bayesian Anal. 5(1)151–170.

Li, L., R. D. Cook, C.-L. Tsai. 2007. Partial inverse regression.Biometrika 94(3) 615–625.

Lovins, J. B. 1968. Development of a stemming algorithm. Mech.Translation Comput. Linguistics 11(1–2) 22–31.

Luo, L., P. K. Kannan, B. T. Ratchford. 2008. Incorporating subjec-tive characteristics in product design and evaluations. J. Mar-keting Res. 45(2) 182–194.

McGann, R. 2005. Study: Consumers delete cookies at surprisingrates. ClickZ (March 14), http://www.clickz.com/3489636.

Miller, G. A. 1995. WordNet: A lexical database for English. Comm.ACM 38(11) 39–41.


Mitchell, T. J., J. J. Beauchamp. 1988. Bayesian variable selectionin linear regression [with discussion]. J. Amer. Statist. Assoc.83(404) 1023–1036.

Naik, P., M. Wedel, L. Bacon, A. Bodapati, E. Bradlow, W. Kamakura,J. Kreulen, P. Lenk, D. Madigan, A. Montgomery. 2008. Chal-lenges and opportunities in high-dimensional choice data anal-yses. Marketing Lett. 19(3) 201–213.

Newman, J. W., R. Staelin. 1972. Prepurchase information seekingfor new cars and major household appliances. J. Marketing Res.9(3) 249–257.

Park, T., G. Casella. 2008. The Bayesian lasso. J. Amer. Statist. Assoc.103(482) 681–686.

Pauwels, K. E. Dans. 2001. Internet marketing the news: Lever-aging brand equity from marketplace to marketspace. BrandManagement 8(4) 303–314.

PC Pitstop TechTalk. 2010. The state of PC security. (May 13), http://techtalk.pcpitstop.com/2010/05/13/the-state-of-pc-security.

Quinton, B. 2005. Cookies and the deletion dilemma. Chief Mar-keter (April 9), http://chiefmarketer.com/online_marketing/search/cookies_ deletion_0409.

Ratchford, B. T., M.-S. Lee, D. Talukdar. 2003. The impact of theInternet on information search for automobiles. J. MarketingRes. 40(2) 193–209.

Rutz, O. J., R. E. Bucklin. 2011. From generic to branded: A modelof spillover in paid search advertising. J. Marketing Res. 48(1)Forthcoming.

Rutz, O. J., R. E. Bucklin, G. Sonnier. 2010. A latent instru-mental variables approach to modeling keyword conversionin paid search advertising. Working paper, Yale University,New Haven, CT.

Simester, D. I., P. Sun, J. N. Tsitsiklis. 2006. Dynamic catalog mailingpolicies. Management Sci. 52(5) 683–696.

Srivastava, S., L. Chen. 2010. Comparison between the SSVS andthe LASSO for genome-wide association studies. Comm. Inform.Systems 10(1) 39–52.

Steel, E., J. E. Vascellaro. 2010. Facebook, MySpace confront privacyloophole. Wall Street Journal (May 21) B1.

Stock, J. H., M. W. Watson. 2006. Forecasting with many predictors.G. Elliott, C. W. J. Granger, A. Timmermann, eds. Handbook ofEconomic Forecasting. North-Holland, Amsterdam, 515–554.

Tibshirani, R. 1996. Regression shrinkage and selection via thelasso. J. Roy. Statist. Soc. Ser. B 58(1) 267–288.

West, M. 2003. Bayesian factor regression models in the “largep, small n” paradigm. Bayesian Statistics: Proc. Seventh Valen-cia Internat. Meeting. Oxford University Press, New York,723–732.

West, D. M., J. Lu. 2009. Comparing technology innovation inthe private and public sectors. Brookings Institution. http://www.brookings.edu/papers/2009/06_technology_west.aspx.

Yang, S., A. Ghose. 2010. Analyzing the relationship betweenorganic and sponsored search advertising: Positive, negative,or zero interdependence? Marketing Sci. 29(4) 602–623.

Yao, S., C. F. Mela. 2011. A dynamic model of sponsored searchadvertising. Marketing Sci. 30(3) 447–468.

Zanutto, E., E. Bradlow. 2006. Data pruning in consumer choicemodels. Quant. Marketing Econom. 4(3) 267–287.

Zou, H., T. Hastie. 2005. Regularization and variable selection viathe elastic net. J. Roy. Statist. Soc. Ser. B 67(2) 301–320.

Modeling Indirect Effects of Paid Search Advertising...

Documents

Transcript of Modeling Indirect Effects of Paid Search Advertising...