FRUSTRATION AND CONSUMER EVALUATION OF SEARCH …utilitarian products provide consumers tangible...
Transcript of FRUSTRATION AND CONSUMER EVALUATION OF SEARCH …utilitarian products provide consumers tangible...
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 122
FRUSTRATION AND CONSUMER EVALUATION OF SEARCH ADVERTISING AND
SEARCH ENGINE EFFECTIVENESS:
THE CASE OF HEDONIC VERSUS UTILITARIAN PRODUCT
Qin Sun
College of Business Administration
Trident University International
5757 Plaza Dr., Suite 100
Cypress, CA 90630, USA
Nancy Spears
Department of Marketing and Logistics
College of Business Administration
University of North Texas
1307 West Highland Street
Denton, TX, 76201,USA
ABSTRACT
The purpose of this paper is to investigate the role of frustration in online search endeavors. The research
applies precepts from frustration theory to investigate the nature of search goals for hedonic vs. utilitarian product
and the moderating influence of encountered frustration during online searches on attitude toward keyword search
advertising (AAd), attitude toward searched brand (Abrand), purchase intention (PI), and attitude toward search engine
effectiveness. An experiment with a pretest was conducted. The results of the pretest suggest that hedonic product
searches associate with the goal of finding relevant search results, while utilitarian product searches relate to time
saving. The findings of the experiment indicate that the influence of hedonic vs. utilitarian search on AAd and search
engine effectiveness is moderated by the level of frustration. The present research contributes to the substantive and
conceptual domains of frustration theory, Internet search advertisements, search engine effectiveness, and electronic
commerce.
Keywords: frustration, flow theory, keyword search advertising, attitude toward search ad, search engine
effectiveness
1. Introduction
Due to the increasing popularity of search engine and its special appeal for small to medium firms with niche
markets, businesses and advertisers start to use search advertising to promote their products or services. According
to Search Engine Marketing Professional Organization (SEMPO), advertisers in North America spent about $13.5
billion dollars on search engine marketing (SEM) in 2008 and the Northern American SEM expenditure will reach
$26 billion dollars by 2013. The two top objectives of paid placement programs among advertisers are to increase
brand awareness and to sell products, services, and contents directly online [SEMPO 2008]. In addition, over 50% of
Internet browsers use a search engine every few days and search engines generate more traffic to commercial sites
with product and price information than other online search methods [Sen et al. 2006, Feng et al. 2007]. Because of
search advertising’s potential for greater return on investment relative to other forms of advertising [Burns 2005] as
well as its increasing market share in online advertising, there is a great need to have a clearer understanding of
search engine effectiveness [ Marketing Week 2008, Chen et al. 2009].
Consumers expect to find relevant information when they use search engines, and they hardly go beyond the top
30 listings for a single keyword search [Sen 2005, Sen et al. 2011]. Although search engines intend to match the
consumers’ search with the online sellers’ offerings, there are concerns about the negative impact of a poor match
between expectations and the output list of the search engine [Su 2008]. It is even frustrating for many Internet
surfers to search the web due to the potential confusion or inaccuracy caused by the heterogeneous commerce
related terms [Spink & Jansen 2008]. Therefore, it is critical to the online sellers to evaluate the effectiveness of
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 123
search engines and develop search engine marketing strategies to improve the possibility of match between their
offerings and the expectations of consumers, as well as to promote targeted access to their commercial websites [Sen
2005, Spink & Jansen 2008, Su 2008]. In addition, search engines as a transformation agent influence the business
environment in a tremendous way, and it is important to better understand their effectiveness in the cause of changes
[Rangaswamy et al. 2009].
Furthermore, flow theory has been extensively utilized to examine the positive online search experience and
relevant attitudinal and behavioral constructs [Korzaan 2003, Mathwick & Rigdon 2004, Hoffman & Novak 2009].
Although negative emotions incurred during the online keyword search could have a critical influence on attitude
toward the product and website, as well as attitude toward sponsored links, little research to date has been conducted
to investigate how web searchers handle encountered frustrations [Campbell & Wright 2008, Lin & Hung 2009].
Therefore, this research provides an alternative view on flow theory by investigating negative subjective experience
when the consumers’ web exploration is not in a flow state in some way [Hoffman & Novak 1996, Hoffman &
Novak 2009].
Although research has been done to understand keyword advertising campaigns using the paradigm of buying
funnel [Jansen & Schuster 2011], as well as the pricing strategies employed by major search engines [Xiao et al.
2009], there is still a need to investigate the influence of search engines and keyword advertisements on consumer
online information search and purchase behavior. To contribute to our understanding of Internet search advertising
and search engine effectiveness, a qualitative study has been conducted to reveal the search goals of web surfers
(Sun & Spears 2011). Extending the qualitative research that we have done, the present study attempts to apply
precepts from frustration theory to examine the relation between online keyword search goals, the product category
and key online search response variables. To accomplish this central research objective, an experiment was designed
to investigate: (1) online search goals for hedonic vs. utilitarian products; and (2) the moderating role of frustration
on the relationships between product category (search for hedonic vs. utilitarian product) and key online search
response variables such as attitude toward search ad, attitude toward searched brand, purchase intention toward
searched brand, and search engine effectiveness. In the next section, we discuss frustration theory and develop the
hypotheses. Then we report the experiment, its pretest and the results. Finally, the theoretical and managerial
implications along with a future research agenda conclude the remark.
2. Frustration Theory and Hypotheses Development
Different from dissatisfaction, frustration is a highly negative emotion that people experience when their goals
are not achieved or the expected rewards are missed [Rosenzweig 1944, Shorkey & Crocker 1981, Berkowitz 1989,
Colman 2001]. This unpleasant inner condition results in a variety of outcomes that may include cognitive
restructuring of the task at hand, goal modification, aggressive acts, resignation, glossing over the situation,
behaviors such as product avoidance, or consumer protest. In the online keyword search context, precepts from
frustration theory suggest that consumers approach keyword searches with specified goals and become frustrated
when those goals are thwarted. From the perspective of frustration theory, a two-stage process is possible in the
course of an online keyword search. First, a particular search goal is specified by the consumer, such as finding
relevant results. Second, the identified search goal may be blocked, hindering goal attainment. Despite the insights
from previous research about the role of frustration in online keyword search [Sun & Spears 2011], unanswered
questions remain. For example, do primarily hedonic vs. utilitarian product searches have different keyword search
goals? If so, does the frustration of a thwarted search attempt produce differing consumer responses with different
searches? If this is the case, how are they different? In the following section, we discuss the legitimacy of these
issues based on extant literature and develop hypotheses.
Prior research indicates that emotions impact consumer decision making [Ding et al. 2005, Darke et al. 2006].
In particular, the ineffectiveness of search engines could leave consumers frustrated and impact attitude toward the
searched product [Campbell & Wright 2008, Su 2008]. In addition, frustration resulting from an unsuccessful search
experience has the potential to influence evaluation of keyword search advertising, the searched product or brand,
and purchase intention [MacKenzie et al. 1986, Burke & Edell 1989, MacKenzie & Lutz 1989, Brown & Stayman
1992, Brown et al. 1998, Mathwick & Rigdon 2004, Lin & Hung 2009 ]. Furthermore, affective responses such as
mood, can impact consumers’ product evaluations and purchase decisions [Darke et al. 2006]. Likewise, affective
responses were found to serve as a cue when the message information is not easily accessible [Bakamitsos 2006],
influence the food consumption [Garg et al. 2007], and impact post-consumption behavior [Soscia 2007]. Thus, the
findings of previous literature suggest that affective responses, such as the experience of feeling frustrated, have the
potential to influence attitude toward the search ad, attitude toward the brand, and purchase intention. Guided by
these ideas, it is likely that, when the search result is successful in meeting search goals, consumers will experience
less frustration, have a more positive evaluation of the search ad and the brand, and present higher intention to make
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 124
a purchase. By comparison, consumers experiencing high levels of frustration in keyword search should have more
negative evaluations.
Depending upon product category, affective responses differentially serve as information cues. For example,
utilitarian products provide consumers tangible functional features while hedonic products fulfill sensory or
experiential needs [Hirschman & Holbrook 1982, Dhar & Wertenbroch 2000]. Therefore, consumers evaluate
utilitarian products cognitively and rationally, and focus mainly on informational cues relating the objective and
practical benefits of the products [Shiv & Fedorikhin 1999]. By comparison, the evaluation process for hedonic
products tends to be affect-driven, subjective and emotional, and reliant upon affective cues, such as frustration [Ryu
et al. 2006]. Consumers searching for hedonic products also consider less diverse and smaller number of brands than
those for utilitarian products since the latter group is more likely to avoid risk and uncertainty [Chernev 2004, Suh
2009].
Based on extant literature, we expect that the effect of search for hedonic vs. utilitarian products on key search
response variables will be moderated by frustration with the search effort. In particular, we argue that affective
feelings, such as frustration, during information search play a more critical role for consumers who shop for hedonic
products than for utilitarian products, which leads to differential search response attitudes. Therefore, if the search
experience is frustrating, the consumers for hedonic products will more negatively evaluate important search
response variables. On the other hand, when the search experience is less frustrating, the consumers for hedonic
products are more likely to have a more positive attitude toward the ad message and the searched brands, and show
higher buying intention than for utilitarian products. As a result, we hypothesize:
H1: (a) In high levels of frustration, search for a utilitarian product will produce more positive attitudes toward
search ads than search for a hedonic product. (b) In low levels of frustration, search for a hedonic product will
produce more positive attitudes toward search ads than search for a utilitarian product.
H2: (a) In high levels of frustration, search for a utilitarian product will produce more positive attitudes toward
the searched product than search for a hedonic product. (b) In low levels of frustration, search for a hedonic
product will produce more positive attitudes toward the searched product than search for a utilitarian product.
H3: (a) In high levels of frustration, search for a utilitarian product will produce higher purchase intention
than search for a hedonic product. (b) In low levels of frustration, search for a hedonic product will produce
higher purchase intention than search for a utilitarian product.
In addition, personal relevance of the online advertising is found to influence attitude toward the website that
carries the advertisements [Campbell & Wright 2008]. It is reasonable to argue that the failure of search advertising
to be relevant to consumers’ expectation or search goals will lead to a negative opinion of the search engine.
However, searches for utilitarian vs. hedonic products are quite different. For example, consumers searching for
utilitarian products are more likely to search different information sources and more likely to extensively use search
engines to find the information. By comparison, consumers searching for hedonic products are prone to search
products less intensively through different channels [Shiv & Fedorikhin 1999]. It is possible that consumers consider
the effectiveness of the search engine as a tool in online search. If this is the case, then a search for a utilitarian
(hedonic) product would likely rely more (less) extensively on the effectiveness of the search tool. It makes sense
that, in high levels of frustration, search for a utilitarian product would result in more negative evaluations of search
engine effectiveness, compared to search for a hedonic product. However, in the less frustrated state, a consumer in
search of a utilitarian product is likely to view the search engine more effective since it is an important tool in their
search efforts. Following this logic, we hypothesize that:
H4: (a) In high levels of frustration, search for a hedonic product will produce more positive attitudes toward
search engine effectiveness than search for a utilitarian product. (b) In low levels of frustration, search for a
utilitarian product will produce more positive attitudes toward search engine effectiveness than search for a
hedonic product.
3. The Research Design
3.1 The Pretest
The objective of pretest is to provide substantiation on the connection between keyword search goals and
product categories (e.g. hedonic vs. utilitarian products), which should provide foundation for the experiment.
According to a qualitative study on the online keyword search goals, consumers tend to have two major search
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 125
goals: one is to save time and the other is to find relevant results [Sun & Spears 2011]. So it is reasonable to
speculate if the online keyword search for different products associates with these two search goals.
Based on extant literature on consumer search behavior, consumers search for and purchase goods and services
for two basic purposes, hedonic gratification and utilitarian reasons based on more functional rationale [Hirschman
& Holbrook 1982, Batra & Ahtola 1990]. Arguably, these purchase searches are quite different in terms of goals and
potential frustrations. For example, Okada [2005] recently found that consumers are willing to expend more time in
search for hedonic goods and services, compared to utilitarian ones. Indeed, Okada [2005] concluded that the
convenience goal of time savings is likely to be a more effective option for utilitarian purchases than for hedonic
ones.
Building on the conclusions of research above [Okada 2005, Sun & Spears 2011], it is likely that the central
search goal for utilitarian purchase is time savings. However, search for hedonic goods and services tend to deliver
experiential enjoyment [Okada 2005]. Thus, it is likely that discovering and sifting through relevant, associated, and
interrelated keyword search results may yield a certain degree of experiential enjoyment. Based on these ideas, we
can reason that an online search for a utilitarian product will result in a higher proportion of time savings search
goals than that for a hedonic product, while an online search for a hedonic product will result in a higher proportion
of relevant search goals than that for a utilitarian product. However, since there is no empirical study to support this
argument, a pretest is needed to provide support for this stipulated relationship.
3.1.1 Sampling
A convenience sample of fifty-five undergraduates from an American university participated in the study. Since
people use keywords in search engines to find relevant information on the searched product and/ or save time [Sun
& Spears 2011], two scenarios were developed. Since fictitious scenario could provide similar results as the real
case, we used fictitious scenarios in this study (Theofilou 2011). The only difference between two scenarios is the
focal product. Since utilitarian products tend to provide mainly functional features such as personal computers and
the hedonic products offer primarily experiential fun and pleasure such as vacation packages [Hirschman &
Holbrook 1982, Dhar & Wertenbroth 2000], two products were chosen to represent utilitarian vs. hedonic products:
the GPS brand Star and a movie. The scenario one is for GPS Star and reads as follow:
Imagine that you are using a search engine to look for information about a handheld Global Positioning
System (GPS) brand Star. After you typed in the keyword “GPS”, the results pull up pages of information on GPS.
Then you click through the search results to look for the information on the brand Star.
The scenario two on movie reads as follow:
Imagine that you are using a search engine to look for a movie. After you typed in the movie name in the
search engine, the results pull up pages of information. Then you click through the search results to look for the
information on the movie.
After reading the scenario, the respondents were asked to “check which ONE of the following goals best
represents your online search goal for GPS brand star” in scenario one with “Relevant search results” and “Time
savings” as the options. Then, a semantic differential scale with “1= weakly represents my online search goal” and
“7= strongly represents my online search goal” was used to evaluate “how strongly this goal represents your online
search goal for the GPS brand star”. An open-ended question was followed to gain insights on the respondents’
acceptance range to check the keyword search results by asking the question “Please indicate how many pages you
usually click through after the search engine shows the results for your typed keyword or phrase. Please explain
why.” The demographic information such as gender, age, educational background and ethnicity was asked at the end
of the questionnaire.
3.1.2 Results
A chi-square analysis was employed to test for interdependence between search goal and product category. The
minimum expected cell frequency for each cell in a contingency table should exceed five (Conover 1971). Each cell
in our test exceeded this minimum (see Table 1). Thus, a sample size of fifty-five was adequate for the test of
association. Based on the results of chi-square analysis, the search goal for GPS brand Star is mainly time saving
(59%), while that for movie is mostly relevant search results (74%). This difference is statistically significant,
demonstrating that respondents thought the search goal for GPS brand Star is to save time and that for movie is to
find relevant results (ᵡ 2(1, 56) = 6.10, p = .01; Table 1).
Therefore, the pretest provides the support for the speculated relationship between online keyword search goals
and the product category. Further analyses were run to deepen our understanding of search goals for each product
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 126
category. For example, ANOVA with representativeness of online search goal as the dependent variable and the
product category (GPS brand Star and Movie) as the independent variable indicated that there is a high agreement
on the search goals and no significant difference between two products on their representativeness of search goals
was found (MGPS = 5.38 and Mmovie = 5.59; p = 0.53), adding confidence to our conclusion that the selected search
goals represents their intended goals.
Additionally, a qualitative content analysis on the open question reveals that 54% (28 out of 52) of participants
preferred to find the searched product on the first page of the search results. Only four respondents were willing to
check 5 or more pages of the search results. Further chi-square analysis indicates that there is no significant
difference between the pages of the search results for GPS and movie (ᵡ 2(2,56)= 0.89, p = .88). That is, 15
respondents preferred to find the searched GPS on the first page of search results while 13 participants only looked
at the first page of search results for movie. 10 respondents looked at the first 2-3 pages of search results for GPS
and 11 for movie. At the same time, 3 respondents would check 5 or more pages for GPS while only one was willing
to do so for movie. In addition, 7 respondents explained to rephrase the keyword to find the searched product.
Table 1: Chi-Square Analysis for Pretest
Search Goal
Relevant Search Results Time Saving
Product Type
GPS Brand Star
Count 12 17
Expected Count 17 12
% within Product Type 41 59
Count 20 7
Movie Expected Count 15 12
% within Product Type 74 26
Pearson Chi-square 6.10
Sig. 0.01
3.1.3 Discussion of the Pretest
The results of pretest support the idea that the nature of the good or service is associated with the type of online
search goal. While the results indicate that there is no difference between the acceptable number of pages to click
through for the hedonic and the utilitarian product search, previous research nevertheless indicates that different
frustration responses associate with different search goals [Sun & Spears 2011]. Based on the findings of this
pretest, which suggest the presence of distinct search goals for hedonic vs. utilitarian products, four scenarios were
designed for the experiment to investigate the impact of frustration on key consumer response variables in online
search for hedonic vs. utilitarian products. That is, the purpose of the experiment discussed in the next section is to
examine the moderating role of frustration on the relationships between product category (hedonic vs. utilitarian
product) and key online search response variables such as attitude toward search ad, attitude toward searched brand,
purchase intention toward searched brand, and search engine effectiveness.
3.2 The Experiment
3.2.1 Method
To test the hypotheses, a 2×2 (high frustration vs. low frustration and GPS vs. Movie) experiment was designed.
Four scenarios were developed based on the pretest results, which included searching a utilitarian product with high
frustration (scenario 1), searching a utilitarian product with low frustration (scenario 2), searching a hedonic product
with high frustration (scenario 3) and search a hedonic product with low frustration (scenario 4). The scenarios 1 and
3 were designed to evoke a high level of frustration and the only difference between two scenarios is the searched
product/brand (GPS Star for scenario 1 and movie for scenario 3). Scenario 1 reads as follow:
Imagine that you are using a search engine to look for information about a handheld Global Positioning
System (GPS) brand Star. After you typed in the keyword “GPS”, the results pull up pages of information on GPS.
You clicked through the first five pages of the result, but nothing was what you expected. Then you typed in “Star”
to search, but you still did not find it in the first 5 pages of the results. You also tried different combinations of the
keyword, and you could not find it.
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 127
The scenario 2 and scenario 4 were designed to evoke a low level of frustration with different searched
product/brand (GPS Star for scenario 2 and movie for scenario 4). The scenario 2 reads as follow:
Imagine that you are using a search engine to look for information about a handheld Global Positioning
System (GPS) brand Star. After you typed in the keyword “GPS”, the results pull up pages of information on GPS.
You clicked though the different links and found the handheld GPS brand Star information you expected on the first
page of the results.
Manipulation checks were designed to determine the level of frustration in each scenario and the search
purpose. Based on the results of previous qualitative study on online keyword search goals [Sun & Spears 2011],
four Likert items were used for manipulation checks on level of frustration: “I was distressful when I was trying to
find what I was searching for”, “I was prevented from getting my results quickly”, “I was irritated during my
keyword search”, and “I was sad that I could not find the relevant information”. Two items were used for
manipulation checks on search purpose: “The purpose of the search was to find something very practical” and “The
purpose of the search was for pleasure”. To account for important control variables, two questions were asked after
each scenario. In scenarios 1 and 2, the items are as follows: “I have experience with GPS systems”, and “I am a
frequent user of keyword searches”. In scenarios 3 and 4, the first control question was changed to “I frequently use
keyword searches to find movies” and the second one is the same as that in scenario 1 and 2. The Likert items used
7-point scale anchored with 1 = strongly disagree and 7 = strongly agree.
According to Yahoo! Advertising Solutions, search ad is “a way to advertise your business in the search results
that are returned by search engines in response to a consumer’s search query. Ads typically consist of a clickable
title, a description underneath it, and the URL of the advertising site.” [www.yahoo.com]. Although the keyword
search results include both paid search advertisement and unpaid search results, most search results are refined
through search engine optimization (SEO) and could be regarded as a new form of search ad to promote a product,
service, company or an idea [SEL 2008]. Therefore, to measure the respondents’ attitude toward search advertising,
they were asked to indicate their feeling about the keyword search results. Six semantic differential scales were used
here with 1= unpleasant/ unlikable/ boring/ tasteless/ artless/ bad and 7= pleasant/ likable/ interesting/ tasteful/
artful/ good [Madden et al. 1988, Spears & Singh 2004]. In order to measure the respondents’ attitude toward the
searched product/brand, five semantic differential scales were used with 1= unappealing/ bad/ unpleasant/
unfavorable/ unlikable and 7 = appealing/ good/ pleasant/ favorable/ likable [Spears & Singh 2004]. Five other
semantic differential scales were used to measure the participants’ intention to purchase the searched product/brand:
1 = definitely not buy it/ probably not buy it/ low purchase interest/ never/ definitely not intent to buy and 7 =
definitely buy it/ probably buy it/ high purchase interest/ definitely/ definitely intent to buy [Spears & Singh 2004].
Then, the respondents were asked to indicate their agreement or disagreement with the following statements using 7-
point scale (1= strongly disagree and 7 = strongly agree): “This search engine is not effective”, “I would not use the
same search engine again”, “I would use other search engines instead of this one”, “I would not use search engines
any more”, and “I would use other information sources.” Finally, participants were asked to provide demographic
information (i.e., gender, age, highest level of education and ethnicity). After completing the questionnaire,
participants were briefed and thanked.
Effective responses were collected at a Southeastern University in U.S.A. In total, 130 students participated in
this study. Class credit was offered as an incentive to the participants. Each scenario was assigned randomly to the
participants. As a result, 32 responses were collected for scenario 1, 32 for scenario 2, 32 for scenario 3 and 34 for
scenario 4. There were 46% males and 54% of females. 82% of the participants were between 21- 25 and 95% were
juniors and seniors. Of ethnicity, 74% were Caucasian, 5% were African American, 9% Hispanic, 7% Asian and 4%
were “others”.
3.2.2 Results
Exploratory factor analysis (EFA) was used to classify the items used for frustration manipulation since these
items were pooled together from the content analysis results of the qualitative study [Sun & Spears 2011]. The result
of EFA indicated that only one factor “frustration” was extracted. Confirmatory factor analysis (CFA) was
conducted for items used for attitude toward search ad, attitude toward searched product/brand, and purchase
intention toward searched product/brand since we used existing scales. To check scale reliabilities, Cronbach’s α
was calculated using the entire data set. Adequate reliabilities were found for frustration (.85), attitude toward search
advertising (.86), attitude toward searched product/brand (.95), and purchase intention toward searched
product/brand (.93) [Nunally 1978].
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 128
3.2.3 Manipulation Check
An ANOVA was performed to assess the study’s manipulations. Participants in high frustration scenarios
showed much higher level of frustration (MGPS = 4.58 and Mmovie = 4.85) than those in low frustration scenarios
(MGPS = 3.13 and Mmovie = 2.86) for both products. Differences between these means were all statistically significant
(omnibus F = 19.85; p < .000). Therefore, our manipulation of frustration was reasonable.
Another ANOVA was conducted to test the manipulations for the search purpose. The results indicated that
participants agreed that the GPS scenarios described the practical search purpose (MGPS = 5.52) instead of hedonic
search purpose (Mmovie = 3.88; F = 95.23; p < .000). At the same time, the participants agreed that the movie
scenarios described hedonic search purpose (Mmovie = 5.38) rather than practical search purpose (MGPS = 3.48; F =
80.46; p < .000). Therefore, our manipulations of search purpose were reasonable.
3.2.4 Hypotheses Test
A series of ANOVAs were performed to examine hypotheses. To test hypothesis 1, we conducted an ANOVA
with attitude toward search advertising (Aad) as the dependent variable and frustration level (high vs. low) and
product category (hedonic movie and functional GPS Star) as the independent variables. Consistent with prediction,
a significant two-way interaction between frustration level and product category was observed for Aad (F(1,62) =
4.42, p = 0.04), confirming H1. Further analysis indicated that in a low level of frustration, participants showed
significantly more favorable attitude toward the search advertising for movie (Mmovie = 4.99) than that toward the
search ad for GPS brand Star (MGPS = 4.31). On the other hand, in a high level of frustration, participants had more
favorable attitude toward the search advertising for GPS brand Star (MGPS = 3.95) than that toward the search ad for
movie (Mmovie = 3.77).
Figure 1: Hypothesis One Test Result for Dependent Variable: Attitude toward Search Ad (Aad)
Note: The semantic differential scale used: 1= Most negative attitude and 7 = Most positive attitude
Another ANOVA was performed with attitude toward brand (Abrand) as the dependent variable and the same
independent variables as H1. The results indicated that the main effect of product category is significant (Fproduct =
11.48, p = 0.00), but the two-way interaction between frustration level and product category was not significant for
Abrand (F(1,62) = 1.66, p = 0.20), rejecting H2. Similar results were found for the dependent variable of purchase
intention (PI). The ANOVA results showed significant main effect of frustration level (Ffrustration = 6.38, p = 0.01)
and insignificant interaction between frustration level and product category for PI (F(1,62) = 0.00, p = 0.99).
Therefore, H3 is not supported.
In addition, ANOVAs were utilized to evaluate the effectiveness of the search engine. The results indicated a
significant two-way interaction between frustration level and product category for the effectiveness of focal search
engine (F(1,62) = 8.32, p = 0.01), supporting H4. The results reveal that, in high levels of frustration, the
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 129
effectiveness of the search engine is more positive (Mmovie = 5.44) for the movie than for the utilitarian product (MGPS
= 4.61). However, in low levels of frustration, the effectiveness of the search engine is more positive (MGPS = 3.34)
for the utilitarian product than for the hedonic one (Mmovie =2.47).
Figure 2: Hypothesis Two Test Result for Dependent Variable: Attitude toward Brand (Abrand)
Note: The semantic differential scale used: 1= Most negative attitude and 7 = Most positive attitude
Figure 3: Hypothesis Three Test Result for Dependent Variable: Purchase Intention (PI)Note: The semantic
differential scale used: 1= Most negative purchase intention
and 7 = Most positive purchase intention
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 130
Figure 4: Hypothesis Four Test Result for Dependent Variable: Search Engine Effectiveness
Note: The scale used: 1= strongly disagree and 7 = strongly agree
3.2.5 Ancillary Tests
The results of ANOVA showed that the participants in four scenarios were not significantly different from each
others on the following control variables: experience with GPS systems/ using keyword searches to find movies
(MScenario1 = 5.16 vs. MScenario2 = 4.69 and MScenario3 = 4.53 vs. MScenario4 = 5.03; F = .86; p = .466) and frequent use of
keyword searches (MScenario1 = 5.78 vs. MScenario2 = 5.69 and MScenario3 = 5.28 vs. MScenario4 = 6.18; F = 1.99; p = .119).
Post hoc, multiple comparisons were used to test if there were significant differences in the participants’
attitudes toward search advertising (1) in GPS high frustration situation and those in movie high frustration situation.
No significant difference was found between two groups (MGPS = 3.93 vs. Mmovie = 3.76; p = .954); and (2) in GPS
low frustration situation and those in movie low frustration situation. There was also no significant difference found
between two groups (MGPS = 4.31 vs. Mmovie = 4.99; p = .139).
Also Post hoc, multiple comparisons were performed to test if there were significant differences in the
participants’ attitudes toward searched product/brand (1) in GPS high frustration situation and those in movie high
frustration situation. No significant difference was found between two groups (MGPS = 4.50 vs. Mmovie = 5.05; p =
.415); and (2) in GPS low frustration situation and those in movie low frustration situation. However, significant
difference was found between two groups (MGPS = 4.59 vs. Mmovie = 5.65; p = .014).
Post hoc, multiple comparisons were further utilized to test if there were significant differences in the
participants’ purchase intention to the searched product/brand (1) in GPS high frustration situation and those in
movie high frustration situation. No significant difference was found between two groups (MGPS = 3.83 vs. Mmovie =
3.75; p = .996); and (2) in GPS low frustration situation and those in movie low frustration situation. There was also
no significant difference found between two groups (MGPS = 4.42 vs. Mmovie = 4.31; p = .990).
3.2.6 Discussion of the Experiment
Several outcomes of the experiment are noteworthy. First, the results indicate that the influence of searched
product type (hedonic vs. utilitarian) on key consumer response variables is moderated by frustration level. Second,
the findings suggest that the impact of frustration appears to be related to the types of cues that are considered when
searching for utilitarian vs. hedonic products. Specifically, the results suggest that, highly frustrated consumers are
more positive about the search ad when searching for a utilitarian product but more positive about the effectiveness
of the search tool when looking for a hedonic product. On the other hand, the less frustrated consumers appear to
have a more positive opinion of the search ad when searching for a hedonic product but a more positive opinion of
the search tool when looking for a utilitarian product. One logical implication is that, for a hedonic product search,
consumers seem to respond to high frustration with more negative evaluations of the search ad. However, they
appear to be more tolerant of and positively predisposed toward the search tool or search engine effectiveness when
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 131
frustrated. This makes sense when one considers that the search tool is a more functional device, compared to the
search ad. Another logical conclusion is that, for a utilitarian product search, consumers seem to be less tolerant of
and more negatively predisposed toward the functional search tool when highly frustrated. This makes sense
considering the important role of functional features in a utilitarian product search, compared to a hedonic product
search.
Taken together with the findings of pretest, the findings of the experiment seem to suggest that when frustrated,
consumers ‘take it out’ on the effectiveness of the search tool when time savings is an important goal as is the case
for a utilitarian product search. However, consumers appear to ‘take it out’ on the keyword search ad when relevant
search results is important as is the case for a hedonic product search. Finally, the findings support the idea that the
nature of the frustration resolution process may underlie such responses. In particular, when the primary goal is
relevancy, the blame for block to goal pursuit is placed on oneself, while the blame is placed on the search engine
when the primary goal is time savings [Sun & Spears 2011]. It is not surprising then that frustrations in hedonic
product searches appear to emanate from affective cues, such as irrelevant search ads, while frustrations in utilitarian
searches seem to stem from search tools that hinder a timely result.
4. General Discussion
The central purpose of this study is to contribute to the advertising and electronic commerce literature by
investigating the differing goals associated with different product category, as well as the moderating role of
frustration on the relationships between the search for hedonic vs. utilitarian products and the key search response
variables. The study also enhances our present understanding of online search goals and frustration responses when
goals are thwarted. Furthermore, the present research contributes to our understanding of the relationships between
important ad response variables and online search advertising, as well as how the search for hedonic vs. utilitarian
products intertwine with search frustrations to impact ad responses and evaluations of search engine effectiveness.
The results of pretest indicate that the search goal for GPS brand Star is to save time and that for movie is to
find relevant results. In addition, the findings of the experiment show that frustration and product category have a
significant influence not only on attitude toward search ads but also on evaluation of search engine effectiveness.
Highly frustrated consumers were less positive in their evaluations of search engine effectiveness when they were
searching for a utilitarian product than when they were searching for a hedonic one. However, consumers highly
frustrated with online search had more favorable attitudes toward search ads when they were searching for a
utilitarian product than those searching for a hedonic product. In concert, these findings suggest that when searching
for a utilitarian product, highly frustrated consumers are less tolerant of search engine ineffectiveness but more
positive about search ads. It is quite possible that these consumers view the search engine as a tool that should be
effective in helping them in search but do not factor in search frustration when evaluating search ads. This assertion
is consistent with the findings of previous scholars that consumers evaluate utilitarian products rationally and focus
mainly on informational cues relating practical benefits of the products instead of incoming emotional cues, such as
frustration [Shiv & Fedorikhin 1999]. Highly frustrated consumers who were searching for a hedonic product were
more tolerant of the ineffectiveness of the search engine but less so of search ads. Taken together, these findings
suggest that when searching for a hedonic product, highly frustrated consumers are not so reliant upon the search
engine as a tool that should be effective in helping them search but are more reliant upon affective cues, such as
frustration, in evaluating search ads [Ryu et al.2006].
4.1 Theoretical and Managerial Implications
The present research contributes to the substantive and conceptual domains of Internet search advertisements,
search engine effectiveness, and frustration theory. The results of this experiment further our understanding of the
influences of the emotional reactions (e.g. frustration) on consumers’ evaluation of the search advertisement and
searched product as well as their purchase intention. The significant results confirm and extend the previous findings
in advertising and consumer behavior literature that affective feeling has a significant influence on consumers’
attitude toward search ads [Darke et al.2006]. Specifically, in low level of frustration situations, participants are
more favorable to the search advertising for movie than that to the search ad for GPS brand Star. On the other hand,
in a high level of frustration scenario, participants hold more favorable attitude toward the search advertising for
GPS brand Star than that toward the search ad for movie. The results for evaluations of the effectiveness of the
search engine were in an opposite direction. Thus, highly frustrated consumers looking for a hedonic product were
more tolerant of ineffectiveness of search engines than highly frustrated consumers searching for a utilitarian
product. The findings of the present studies highlight the potential of frustration theory in investigating the role of
thwarted online search goals in influencing search attitudes. Likewise, the results of this investigation suggest that
frustration theory has the potential to guide our understanding of how negative emotional responses, such as
frustration, are resolved in an online context.
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 132
This study also expands the framework of flow theory by investigating the negative consequences resulted from
the interruption of flow experience in online keyword search behavior setting. Extant research on flow argues that
consumers’ pleasant online experience could result in higher purchase intention toward online retailers, as well as
better attitude toward the website and the brand [Korzaan 2003, Mathwick & Rogdon 2004, Hoffman & Novak
2009]. This study fills a literature blank by focusing on the less pleasant experience the consumers may have during
an online search, as well as the relevant consumer attitudinal and behavioral consequences. Therefore, this research
complements flow theory to give a fuller picture of online consumer search and purchase behavior. In particular,
flow theory investigates the pleasant online experience while this research examines the not-so-pleasant situations
experienced by online consumers.
The findings of the present study should be very useful for marketing managers. Given current interests of the
search engine providers, search advertising agencies and their clients, as well as consumers, this study is both
appropriate and timely. The findings suggest that managers should be mindful of varying search goals of their target
audience toward different products. The findings of pretest reveal that different products represent different search
goals in the consumers’ mind. For example, the consumers search a GPS with the major objective to find the
relevant information while they search a movie with the aim to save time and find what they want as quickly as
possible. In other words, consumers are more willing to spend more time on durable or functional products (e.g.
GPS) than on non-durable or hedonic product (e.g. movie), which confirming the previous findings of the
differential effects of product category on the consumers’ information search and purchase behavior [Schaninger &
Sciglimpaglia 1981, Nysveen & Pederson 2005].
In addition, the marketing managers or advertisers should notice the importance of search engine effectiveness
on their advertising results. Since the search engine’s failure to satisfy the consumers’ search goals could lead to
unfavorable feeling toward the search engine, which ultimately decrease the attractiveness of their advertisements
and drive away their potential customers, it is critical to choose a right search engine provider to deliver the relevant
message to the target community and create a pleasant experience for the online consumers. Although various
analytical tools have been created to optimize the use of online search engines, there is a need to establishing
feasible measures to evaluate the search engine’s effectiveness with respect to consumer search goals.
4.2 Limitations and Future Research Directions
The student sample and fictitious scenarios are two major limitations of this study. Additional research is
needed to expand the student sample to a more representative sample of general population. More realistic
experimental manipulation involving search and creating some real frustration such as field experiments is also
needed in future research endeavor.
Although there are no significant relationships between frustration, product category, and attitude toward
searched product and purchase intention, additional analysis suggests the potential influence of product type. That is,
people in the similar emotional state will experience different evaluation of the searched product depending on what
kind of products they are looking for. Therefore, there is a need to examine the differential impacts of other products
such as cars, houses, or financial services on the consumers online search behavior as well as their affective and
behavioral reactions.
Since online consumers who are in a flow state intend to explore the websites longer, spread positive word-of-
mouth, and show better purchase intention than those who are not, it is necessary to compare these two groups of
people in a more integrative way. For example, how do the consumers in a flow state differ in their attitudes toward
search ads, searched brands and purchase intention compared with those in a frustration state? What are the
theoretical implications for flow theory and frustration theory? What are the practical implications to the online
marketers or retailers? In addition, actual purchase behaviors should be investigated and compared for the
consumers in different emotional states.
Future research can also proceed in a number of directions. For example, the results show that consumers have
heterogeneous search goals for different products, and it is necessary to conduct a survey with larger sample size to
verify this finding. In addition, elaboration likelihood model of persuasion could be used to test the differential
effects of the central message and the position of the ad in the search results on the evaluation of the search ad,
searched product and purchase intention [Petty & Cacioppo 1986]. Other factors such as level of consumers’
keyword search experience and the task involvement could be examined in the future study to provide in-depth
insights on the effectiveness of search advertisements and search engine.
REFERENCES Bakamitsos, G.A., “A Cue Alone or A Probe to Think? The Dual Role of Affect in Product Evaluations,” Journal of
Consumer Research, 33: 403-412, December 2006.
Journal of Electronic Commerce Research, VOL 13, NO 2, 2012
Page 133
Batra, R., and O.T. Ahtola, “Measuring the Hedonic and Utilitarian Sources of Consumer Attitudes,” Marketing
Letters, Vol. 2, No. 2:159-170, 1990.
Berkowitz, L., “Frustration-aggression Hypothesis: Examination and Reformulation,” Psychology Bulletin, Vol.
106, No. 1: 59-73, 1989.
Brown, S. P., P. M Homer, and J. Inman, “A Meta-analysis of Relationships Between Ad-evoked Feelings and
Advertising Responses,” Journal of Marketing Research, Vol. 35:114-126, February 1998.
Brown, S. P., and D. M. Stayman, “Antecedents and Consequences of Attitude toward The Ad: A Meta- analysis,”
Journal of Consumer Research, Vol. 19: 35-51, June 1992.
Burke, M. C., and J. A Edell, “The Impact of Feelings on Ad-based Affect and Cognition,” Journal of Marketing
Research, Vol. 26: 69-83, February 1989.
Campbell, D. E., and R. T. Wright, “Shut-up I Don’t Care: Understanding the Role of Relevance and Interactivity on
Customer Attitudes toward Repetitive Online Advertising,” Journal of Electronic Commerce Research, Vol. 9,
No. 1:62-76, 2008.
Chen, J., D. Liu, and A.B. Whinston, “Auctioning Keywords in Online Search,” Journal of Marketing, Vol. 73: 125-
141, July 2009.
Chernev, A., “Goal-attribute Compatibility in Consumer Choice,” Journal of Consumer Psychology, Vol. 14, No.
1&2: 141-150, 2004.
Colman, A. M., A Dictionary of Psycholgoy. New York, NY: Oxford University Press, 2001.
Conover, W. J., Practical Nonparametric Statistics, New York, NY: Wiley, 1971.
Darke, P. R., A. Chattopadhyay, and L. Ashworth, “The Importance and Functional Significance of Affective Cues
in Consumer Choice,” Journal of Consumer Research, Vol. 33, No. 3: 322-328, 2006.
Dhar, R., and K.Wertenbroch, “Consumer Choice between Hedonic and Utilitarian Goods,” Journal of Marketing
Research, Vol. 37, No. 1: 60-71, 2000.
Ding, M., J. Eliashberg, J. Huber, and R. Saini, “Emotional Bidders—An Analytical Experimental Examination of
Consumers’ Behavior in a Priceline-like Reverse Auction,” Management Science, Vol. 51, No. 3: 352-364,
2005.
Eagly, A. H., and S. Chaiken, The Psychology of Attitudes. Orlando, FL: Harcourt Brace, 1993.
Feng, J., H. K. Bhargava, and D. M. Pennock, “Implementing Sponsored Search in Web Search Engines:
Computational Evaluation of Alternative Mechanism,” INFORMS Journal on Computing, Vol. 19, No. 1: 137-
148, 2007.
Garg, N., B. Wansink, and J. Inman, “The Influence of Incidental Affect on Consumers’ Food Intake,” Journal of
Marketing, Vol. 71:194-206, January 2007.
Hirschman, E. C., and M. B. Holbrook, “Hedonic Consumption: Emerging Concepts, Methods, and Propositions,”
Journal of Marketing, Vol. 46: 92-101, Summer 1982.
Hoffman, D. L., and T. P. Novak, “Marketing in Hypermedia Computer-Mediated Environments: Conceptual
Foundations,” Journal of Marketing, Vol. 60:50-68, July1996.
Hoffman, D. L., and T. P. Novak, “Flow Online: Lessons Learned and Future Prospects,” Journal of Interactive
Marketing, Vol. 23, No. 1: 23-34, 2009.
Jansen, B.J., and S. Schuster, “Bidding on the Buying Funnel for Sponsored Search and Keyword Advertising,”
Journal of Electronic Commerce Research, Vol. 12, No. 1: 1-18, 2011.
Korzaan, M. L., “Going With the Flow: Predicting Online Purchase Intentions,” Journal of Computer Information
Systems, Vol. 43: 25-31, Summer 2003.
Lin, F-H, and Y-F. Hung, “The Value of and Attitude toward Sponsored Links for Internet Information Searchers,”
Journal of Electronic Commerce Research, Vol. 10, No. 4: 235-251, 2009.
Madden, T.J., C.T. Allen, and J. L. Twible, “Attitude toward The Ad: An Assessment of Diverse Measurement
Indices under Different Processing ‘Sets’,” Journal of Marketing Research, Vol. 25:242-252, August1988.
MacKenzie, S. B., and R. J. Lutz, “An Empirical Examination of the Structural Antecedents of Attitude toward the
Ad in An Advertising Pretesting Context,” Journal of Marketing, Vol. 53: 48-65, April1989.
MacKenzie, S. B., R. J. Lutz, and G. E. Belch, “The Role of Ad as a Mediator of Advertising Effectiveness: A Test
of Competing Explanations,” Journal of Marketing Research, Vol. 23:130-143, May1986.
Mathwick, C., and E. Rigdon, “Play, Flow, and the Online Search Experience,” Journal of Consumer Research, Vol.
31: 324-332, September 2004.
Marketing Week, “Search Marketing: Why Quality Content is King,” pp. 31, Jan. 24, 2008.
Nunnally, J. C., Psychometric Theory. , New York: McGraw-Hill, 1978.
Nysveen, H., and P. E. Pederson, “Search Mode and Purchase Intention in Online Shopping Behavior,”
International Journal of Internet Marketing and Advertising, Vol. 2, No. 4: 288-306, 2005.
Sun & Spears: Frustration and Consumer Evaluation of Search Advertising and Search Engine Effectiveness
Page 134
Okada, E. M., “Justification Effects on Consumer Choice of Hedonic and Utilitarian Goods,” Journal of Marketing
Research, Vol. 42: 43-53, February 2005.
Petty, R. E., and J. T. Cacioppo, Communication and persuasion: central and peripheral routes to attitude change.
New York : Springer Verlag, 1986.
Rangaswamy, A., C .Giles, and S. Seres, “A Strategic Perspective on Search Engines: Thought Candies for
Practitioners and Researchers,” Journal of Interactive Marketing, Vol. 23, No. 1: 49-60, 2009.
Rosenzweig, S., An Outline of Frustration Theory. In Hunt, J. M. (Ed.), Personality and Behavior Disorders, Vol. 1:
380-383, New York: Ronald Press Co., 1944.
Ryu, G., J. Park, and L. Feick, “The Role of Product Type and Country-of-Origin in Decisions about Choice of
Endorser Ethnicity in Advertising,” Psychology & Marketing, Vol. 23, No. 6: 487-513, 2006.
Schaninger, C.M., and D. Sciglimpaglia, “The Influence of Cognitive Personality Traits and Demographics on
Consumer Information Acquisition,” Journal of Consumer Research, Vol. 8, No. 2: 208-216, 1981.
SEL, Total Search Marketing, Part 1, retrieved on July 15, 2008 from http://searchengineland.com/total-search-
marketing-part-1-14310.
SEMPO, The State of Search Engine Marketing 2008, retrieved on June 21, 2008 from
http://www.sempo.org/learning_center/research/2008_exec_summary.pdf.
Sen, R., “Optimal Search Engine Marketing Strategy,” International Journal of Electronic Commerce, Vol. 10, No.
1: 9-25, 2005.
Shiv, B., and A. Fedorikhin, “Heart and Mind in Conflict: Interplay of Affect and Cognition in Consumer Decision
Making,” Journal of Consumer Research, Vol. 26, No. 3: 78-90, 1999.
Shorkey, C. T., and S.B. Crocker, “Frustration Theory: A Source of Unifying Concepts for Generalist Practice,”
Social Work, Vol. 26, No. 5: 374-379, 1981.
Soscia, I., “Gratitude, Delight, or Guilt: The Role of Consumers’ Emotions in Predicting Postconsumption
Behaviors,” Psychology & Marketing, Vol. 24, No. 10: 871-894, 2007.
Spink, A., and B.J. Jansen, “Trends in Searching for Commerce Related Information on Web Search Engines,”
Journal of Electronic Commerce Research, Vol. 9, No. 2: 154-161, 2008.
Spears, N., and S. N. Singh, “Measuring Attitude toward the Brand and Purchase Intentions,” Journal of Current
Issues and Research in Advertising, Vol. 26, No. 2: 53-66, 2004.
Sen, R., R. King, and M. Shaw, “Buyer’s Chice of Online Search Strategy and Its Managerial Implications,” Journal
of Management Information Systems, Vol. 23, No. 1: 211-238, 2006.
Sen, R., S. Bandyopadhyay, J.D. Hess, and J. Jaisingh, “Pricing Paid Placement on Search Engines,” Journal of
Electronic Commerce Research, Vol. 9, No. 1: 33-50, 2008.
Su, B-C, “Characteristics of Consumer Search Online: How Much Do We Search?,” International Journal of
Electronic Commerce, Vol. 13, No.1: 109-129, 2008.
Suh, J-C, “The Role of Consideration Sets in Brand Choice: The Moderating Role of Product Characteristics,”
Psychology & Marketing, Vol. 26, No. 6: 534-550, 2009.
Sun, Q., and N. Spears, “Frustration Theory: Toward an Understanding of Keyword Search Effectiveness and
Consumer Responses,” Journal of Customer Behaviour, Vol. 10, No. 1: 35-48, 2011.
Theofilou, A.E, “Methodological Consideration in Crisis Management Research: Fictitious Scenarios vs. Real
Crises,” 2011 Cambridge Business & Economics Conference, ISBN : 9780974211428.
Xiao, B., W. Yang, and J. Li, “Optimal Reserve Price for the Generalized Second-price Auction in Sponsored
Search Advertising,” Journal of Electronic Commerce Research, Vol. 10, No. 3: 114-129, 2009.
Yahoo.com, What is ‘Search Advertising’?, retrieved on Oct. 6, 2011 from
http://help.yahoo.com/l/us/yahoo/advertising/faqs/what_search.html.