Proceedings of the ASME 2018 International Design...
Transcript of Proceedings of the ASME 2018 International Design...
1 Copyright © 2018 by ASME
Proceedings of the ASME 2018 International Design Engineering Technical Conference & Computers and Information in Engineering Conference
IDETC/CIE 2018 August 26-29, 2018, Quebec City, Quebec, Canada
DETC2018-86291
EFFECTS OF COLLAGE PRIMING ON SUSTAINABLE DESIGN IDEA CREATION AND ASSESSMENT
Ting Liao Graduate Student
Mechanical Engineering Stanford University Stanford, CA, USA
Erin F. MacDonald Assistant Professor
Mechanical Engineering Stanford University Stanford, CA, USA
ABSTRACT Priming is a psychological technique that can alter
designers’ mindsets prior to conceptual design exercises [1]. For
example, priming the five senses enhanced designers’ abilities to
communicate sustainability through the product features they
designed [2,3]. Although the three pillars of sustainable design—
social desirability, economic competitiveness, and
environmental friendliness—are all important, they are not
necessarily equally accessible or salient during the design
process. This paper applies the collage priming method of [2] to
(1) increase/improve ideas related to the sustainability pillars, in
the eyes of users, and (2) reduce ownership bias and cause a more
favorable judgment of others’ ideas, when compared to one’s
own ideas. An experiment tests (1) and (2) for the collage
priming method versus a reading preparation activity and no
prime/activity for effectiveness in these two applications. The
participants included graduate design student attendees at the
2016 IDETC conference and graduate engineering students at
Stanford. For (1), collage priming is proven to be successful in
helping designers to generate ideas that are more
environmentally friendly but less successful in helping designers
generate ideas related to social desirability and economic
competitiveness, as judged by potential users; no more
successful than a reading exercise. For (2), we find evidence that
the collage priming reduces ownership bias in designers, as
measured in their judgment of other (simulated) designers’ ideas,
and in this case the reading exercise does not have the same
effect.
1 INTRODUCTION Sustainability has three important pillars: social/user
desirability, economic competitiveness, and environmental
friendliness [4]. Social/user desirability refers to the well-being
of a community and its members, as a sustainable system is
expected to have a positive influence on human equality, social
justice, members’ happiness, etc. [5-6]. In this study, as it is hard
to fully investigate the well-being of a community, we focus on
the local level of the social pillar, namely user desirability.
Economic competitiveness means that a sustainable business or
product must allocate its resources efficiently to be consistently
profitable. Again, to focus on a local level, we interpret
economic competitiveness as the manufacturing cost and use
cost of a product. Of the three pillars, environmental friendliness
relates most closely to the training of engineers, such as in
material selection, resource consumption, design of assembly
and disassembly, etc. Thus, it makes sense that when engineers
approach sustainable product design, they may tend to gravitate
toward environmental friendliness and pay less attention to the
aspects of social impact/user desirability and economic
competitiveness.
A designer’s mindset has a strong impact on the outcome of
the design process, especially their mindset in the early
conceptual stages of the design process, such as ideation. This
mindset is a combination of longstanding knowledge as well as
recently acquired stimuli and new information. One method to
alter a designer’s mindset is through priming—a psychological
technique that aims to affect performance on a task via exposure
to a stimulus that activates a particular idea, contextualization, or
feeling [7]. Priming alters a designer’s mindset by not only
bringing background knowledge into the foreground, but also by
introducing new information that subsequently influences, for
example, ideas generated. The psychological mechanism of
priming is rooted in the concept of “perceptual readiness”
proposed by Bruner [8], which claims that the information and
feelings that are currently cognitively accessible lead to
corresponding thoughts and behavior. Priming, which activates
a specific set of information and/or feelings in the brain,
increases the accessibility of thoughts, memories, and feelings
2 Copyright © 2018 by ASME
associated with this activation, and thus motivates related
perspectives, decisions, and behavior [1,9]. For numerous
examples, refer to [1]. Priming, which is subconscious, is
different from directly, consciously engaging a topic, such as a
reading an article about environmental design.
She and MacDonald found that their collage priming
method—where participants physically handled images of
products to create a paper picture collage—enhanced student
engineers’ abilities to generate features that communicate
environmental friendliness to the user [2]. This collage priming
method was developed based on the work of Guyton, who
demonstrated collage activity an effective approach to establish
product semantics for sustainable products and to inspire
sustainability concerns [2, 10]; for more information on the
motivation and origin of the method, refer to [2]. Subsequent
testing of prototypes built to include the best of the features
thought-up with this method demonstrated that potential
customers increased their consideration of sustainability during
hypothetical purchase decisions [3]. Thus, the collage priming
method, with respect to designing improved communication of
sustainability, seems effective.
Here, we tested to see if the collage priming method,
adapted from [2], can effectively increase/improve the ideas
related to all three pillars of sustainable design during ideation,
especially for social and economic factors. We also wondered if
performing the priming collage task could better prepare
designers for the next step of the design process in a
collaborative environment, namely idea evaluation, as priming
has been known to affect decision-making [1]. While we were
not able to fully-investigate this question, we investigated how
designers, with and without collage priming, judged their own
ideas during a mock-up of an idea selection process. Onarheim
and Christensen [11] found that every participant proved to be
more likely to select his or her own ideas over other ideas when
individually screening ideas in engineering design. Designers
tend to unconsciously favor their own ideas, known as ownership
bias, which can affect their objectivity during idea selection and
ultimately the final product [11].
We ask: (1) Does the collage prime increase/improve ideas
related to the sustainability pillars, in the eyes of users? and (2)
Does the collage prime reduce ownership bias and cause a more
favorable judgment of others’ ideas, when compared to one’s
own ideas?
In this study, we divided designers into five treatment
conditions. There were three treatment categories: (A) a priming
stimulus of a collage activity relating to either social, economic,
or environmental aspects of sustainability; (B) a reading activity
that directly and consciously engaged the reader on the
environmental aspect of sustainability; or (C) no stimuli. The
designers, working alone, generated ideas for washing machines,
and then made judgments about their ideas and the ideas of other
designers (which were simulated). Then, we had potential users
on Amazon Mechanical Turk assess all ideas generated. Next,
we analyzed all results to test the ability of the primes to increase
the number of effective design ideas generated by designers, to
improve the quality (defined by originality and feasibility) of
ideas, and to reduce ownership bias during idea evaluation.
This paper is organized as follows: Section 2 provides a
brief review of sustainable design and of priming techniques in
psychology, marketing, and engineering design. Section 3 lists
our propositions and hypotheses. Section 4 gives an overview of
the experiment, while Section 5 is an explanation of how we
prepared the experiment. Section 6 describes the experiment
itself in detail. Data and statistical analysis is shown in Section
7, and Section 8 provides our discussion. Finally, Section 9 is the
conclusion and a description of future work.
2 BACKGROUND
2.1 Sustainable Design and Framing of Sustainability Product design has a huge effect on global sustainability
because many design decisions, such as choice of materials or
manufacturing plan, extensively impact a product’s entire life
cycle [12]. To create more sustainable products and more
sustainable processes, engineers have investigated novel design
tools, methods, and models to guide industry and business
[2,12,13]. Objective tools, such as life cycle assessment [12,14],
the House of Ecology [14], and quality function deployment [14],
are widely included in design processes; so too are strategic
methods, like sustainable product and service development [13].
Designers sometimes equate “environmentally friendly”
products with a fully-sustainable product. This is likely due, in
part, to their training in engineering, in subjects such as material
selection and energy usage. They may also be influenced by the
users’ familiarity with environmental friendliness [13,15], but
unfamiliarity with the broader definition of sustainability. In
addition, environmental friendliness is easily quantified with
existing metrics [2], while manufacturing processes and
economic issues are usually considered beyond the scope of
product design.
The social pillar of sustainable design is somewhat
malleable in its meaning. For example, Cuthill [16] writes that it
involves social policy and community development, while Littig
and Griessler [17] describe it as “a quality of societies that
satisfies an extended set of human needs.” In this study, we focus
on a consumer product for the United States. Therefore, the
social pillar definition(s) that relate more to the developing world
do not apply. Although a product such as a washing machine
does have a societal impact, the U.S. is equipped to handle many
problems with a well-developed community system, such as a
sewer system to process dirty water and electrical grid to allow
for variable electricity usage. In this study, we interpret the social
pillar of sustainability at an extreme local level, namely the user
desirability of the product. An environmentally friendly product
does not benefit the environment if no sales are made, and in
order to sell, user desires must be balanced with costs and
environmental concerns.
The economic pillar of sustainability has been defined as
“cost of supply chain” [18] and “total net profits for a product”
[19]. The full concerns of economic sustainability are too broad
for a designer to consider in a single ideation setting. However,
3 Copyright © 2018 by ASME
manufacturing and use costs are a large component of economic
sustainability and are addressable during ideation; thus, we chose
them to represent the economic pillar for this experiment.
2.2 Priming Techniques Psychological priming is a robust method used extensively
in psychology, behavioral economics, and organizational
behavior to activate specific mindsets, thereby making relevant
knowledge more accessible to participants [2,20,21]. Priming
has been shown to affect purchase decisions [20], increase
awareness of a social problem, or change behavior [21]. Priming
has inspired sustainability concerns [2,10], pro-social behavior
[22-24], rudeness toward the experimenter [9], mimicking of
advanced age [25], competitiveness and selfishness [26], and
overcoming of racial stereotypes [27]. Various mental and
physical activities are used as a priming method, including
unscrambling words [28], placing objects [29], creating sensory
collages [2,10], simulating a user scenario [9], writing stories
[29-31], answering questions [9], and playing games [24]. The
priming method functions without the awareness or intent of the
individual, and its effect can last up to 24 hours or more [32].
Priming also works in real-world situations. For example, voters
whose polling places are located in schools are more likely to
support educational propositions on the ballot; this is known as
contextual priming [33].
Unscrambling sentences containing harsh words lead to
participants drawing more hostile features, such as spikes and
claws, in their sketches of hypothetical aliens [28]; answering
questions about nutrition habits improved participants’
productivity and the quality of their ideas on how to improve
health [31]; positive affective priming—participants were shown
a picture of a laughing baby—increased the quality of the ideas
generated in an alternative uses task [34]; mimicking limited
mobility, by wearing gloves, increased participants’ empathy for
elderly users, improved the originality of their concepts, and
alleviated their design fixation [9]; and counterfactual priming—
participants consider events that happened as well as events that
almost happened—increased participants’ performance on the
Drucker candle problem by triggering more consideration of
alternative uses of the given supplies [35].
2.3 Ownership Bias Ownership bias refers to preferring one’s own ideas over the
ideas of others during concept selection in engineering design
[36]. This unintentional tendency potentially introduces
partiality to the decision-making process and challenges team
collaboration. Since idea selection is considered one of the most
crucial steps in engineering design in a collaborative
environment, this bias can have a large impact on the success of
the design process [11].
To understand the causes of bias, behavioral economists
investigated the phenomenon directly and found that the feeling
of ownership originated from the appreciation of solutions
developed by individuals [37] and perception of themselves [38-
39]. Thus, the bias can be affected by personality traits and
gender [39]. To facilitate the idea selection process and eliminate
ownership bias, many systematic selection methods have been
developed, such as the Analytic Hierarchy Process (AHP) [40]
and Pugh method [41]. However, many designers still prefer
informal selection methods to screen a myriad of early-stage
ideas [42]. Based on the explanations of the reasons for
ownership bias, we suspect that changing the mindset of
designers via priming my change the level of ownership bias they
experience when evaluating ideas.
3 RESEARCH PROPOSITIONS AND HYPOTHESES Proposition 1: The collage priming activity can increase the
number of effective ideas per designer from design ideation in
terms of as-rated user desirability, manufacturing/use costs, and
environmental friendliness. To determine if priming successfully
activates one of these mindsets/perspectives, we compare the
number of effective ideas per designer from the primed
conditions with the number of effective ideas per designer of a
control group, and additionally with the number of effective
ideas per designer of a group who engaged in a reading activity
(a conscious engagement with the topic of environmental
sustainability). We hypothesize:
H1(a,b,c,d): Priming for (User Desirability, Cost, Cost,
Environmental priming) results in a larger number of effective
ideas generated by each designer, when judged on (User
desirability, Manufacturing Cost, Use Cost, Environmental
Impact) by novice users, than those generated by the control
condition.
The collage priming activity can also improve the quality of
outcomes of individual ideation by priming. We hypothesize that
priming that focuses on one of three aspects of sustainability
helps designers in general by increasing the originality and
feasibility (two characteristics of quality) of their design ideas
compared to ideas generated by the control group. Section 7.1.2
gives detailed information on the rationale behind the metric
selection.
H1(e,f): Priming of any aspect of sustainability results in a
larger number of effective ideas, when judged on (originality,
feasibility) by novice users than ideas generated by the control
group.
Proposition 2: Priming can reduce ownership bias in
designers for other designers’ ideas. Priming potentially makes
latent information related to stimuli more accessible, not only
improving the designers’ design ability in an active way, but also
potentially changing the designers’ subconscious attitude toward
the targeted perspective and goals of the design process. We
investigated if designers who were primed were more likely to
embrace ideas generated by hypothetical others when evaluating
a mixed set of ideas that included their own ideas.
H2: Collage priming for any aspect of sustainability results
in a higher instance of a designer changing their mind about the
importance of their own ideas when exposed to ideas generated
by hypothetical others, than the control group.
4 Copyright © 2018 by ASME
4 EXPERINMENT OVERVIEW In this section we give highlights of the experiment, with
more detail in the following sections. We designed a test-versus-
control experiment to test the hypotheses of Section 3. Figure 1
shows the five conditions: user desirability prime (A1), cost
prime (A2), and environmental prime (A3), reading education
activity (B); and control (C) of no activity (neither priming nor
reading).
FIGURE 2 TESTING CONDITIONS
Figure 2 gives an overview of the experiment’s six major
steps. All activities were performed alone, not in condition
groups. The collage activities assessed coffee cups. The main
design task of the experiment was to design a washing machine.
Explanations for these choices are included in Sections 5.1.1 and
5.3.
I. Pre-ideation Activity: Participants in conditions A did the
collage prime and B did the reading activity. Those in the
control conditions began in Step II. The collage and
reading activities are described in Sections 5.1 and 5.2.
II. Ideation 1: Participants generated design solutions for a
washing machine. Selection of this product is described in
Section 5.3.
III. Video Watching: Participants watched a montage of
videos that highlighted sustainability aspects (user
desirability, manufacturing, and environmental
friendliness) of washing machines. This step was included
to further stimulate design ideas, as described in Section
5.4.
IV. Ideation 2: Participants generated design solutions again
for the same product.
V. Idea Evaluation: Participants first evaluated a set of their
own design ideas. Then they evaluated a mixed set of ideas,
which included the same set of their ideas and a number
of predetermined, sustainability-related ideas, presented
as coming from other designers. The selection of those
ideas and more details are described in Section 5.5.
VI. Questionnaire: All participants filled out a questionnaire
about their demographics, knowledge of sustainable
design, and awareness of characteristics in sustainable
design.
5 ACTIVITY AND STIMULI PREPARATION AND DESCRIPTION
5.1 Collage priming activity In this activity, participants rated their preference for
various types of coffee cups by creating a collage. First, they
physically placed product images on a piece of butcher paper
which already had a set of labeled axes drawn on it, as illustrated
in Figure 3. Second, participants described their selections by
choosing words from a list we provided and writing these words
FIGURE 1 ILLUSTRATION OF EXPERIMENT PROCEDURE
5 Copyright © 2018 by ASME
next to the product images. Each word had to be used at least
once, and each cup had to be described by using at least one word.
Two aspects of the activity created the priming condition: 1)
The Y-axis directly related to the aspect of sustainability that was
being primed; for User Desirability we used “boring” to
“delightful,” for Cost we used “inexpensive to make” to
“expensive to make,” and for Environmental Friendliness we
used “low environmental impact” to “high environmental
impact.” 2) Our list of descriptive words were chosen based on
the strength of each word’s association with the concepts “social,”
“economic”, and “environmental” (described further in Section
5.1.3).
FIGURE 3 ILLUSTRATION OF THE COLLAGE PRIMING ACTIVITY
FIGURE 4 EXAMPLE OF COLLAGE FROM THE EXPERIMENT
5.1.1 Selecting a Product for the Collage Activity It was important that the collage activity of experiment Step
I did not include the same product as in experiment Steps II–V.
In other words, participants must be primed using a different
product than the one they will design. This was to ensure that it
was the priming that caused the effects we saw in the ideations,
and not, for example, the effects of assessing/comparing a
variety of washing machines (the focal product of the design
exercises).
After pilot-testing various options, we chose coffee cups as
the product category for collage priming because: coffee cups
are a common product that most participants are familiar with;
they represent a large variety of appearances, prices, and
materials, which demonstrate different levels of user desirability,
cost, and environmental impact; and the coffee cup incorporates
social elements because coffee is a part of many social
experiences. They are also very different from the product in the
main design task, washing machines. We also considered other
products, for example transportation modes (planes, cars), but
post-pilot-test interviews showed that environmental impact and
cost, but not user desirability, were driving assessments even in
the user desirability prime.
5.1.2 Choosing Axes Labels All three collage activities used the same x-axis labels
(“like” / “dislike”) so that participants could cognitively load as
much as they want (e.g., opinion, features, aesthetics) on it. The
y-axis labels varied according to the priming conditions (see
Table 1).
5.1.3 Descriptive Words To determine our list of descriptive words, we conducted a
pilot study in which graduate students at Stanford University first
generated a list of words by consulting design documents and
thesauri. Then they sorted the words into 4 categories (Social,
Economic, Environmental, and Other), and then rated the
strength of a word’s association with that concept. We used the
12 highest-rated words in each category for our lists (Table 1).
In pilot testing, students that were given the task to generate
Economic words produced fewer ideas than other students. The
words in the cost prime could have stifled ideation if they
focused too much on the negative consequences of spending
money; therefore, we re-evaluated this list to make sure none of
the words were too negative.
TABLE 1 AXES LABELS AND DESCRIPTIVE WORDS FOR THE PRIMING COLLAGE ACTIVITY
User
Desirability Cost
Environmental
Friendliness
Horizontal
Axis
Labels
Dislike/Like Dislike/like Dislike/Like
Vertical
Axis
Labels
Boring/
Delightful
Inexpensive/
Expensive to
make
Low/High
environmental
impact
Collage
Words
Sociable
Friendly
Warm
Helpful
Ergonomic
Modern
Unfriendly
Useless
Useful
Solitary
Playful
Stylish
Affordable
Budget
Cheap
Economical
Bargain
Valuable
Inexpensive
Indulgent
Overpriced
Expensive
Rich
Costly
Recyclable
Natural
Efficient
Organic
Conserving
Polluting
Disposable
Synthetic
Inefficient
Wasteful
Reusable
Consumable
6 Copyright © 2018 by ASME
5.2 Educational Reading Activity Participants assigned to this condition did not engage in the
collage priming activity but instead read a document about eco-
design drawn from the educational materials of the Sustainable
Minds website [43], with edits. The material focused mostly on
the environmental aspect of sustainability. Our goal was to
compare the effectiveness of the reading activity (B) with the
environmental collage priming (A3) to see if directly engaging
with a conscious, educational activity (which is not considered
“priming”) was more effective than subconsciously activating a
certain mindset (priming).
5.3 Ideation Activity In this activity, all participants were prompted to generate
ideas for a next-generation sustainable washing machine within
a given time. We chose washing machines because they are
common consumer products that are familiar to most designers
and students in the U.S. Washing machines have much room
for improvement in user desirability, cost, and environmental
impact.
The prompt given to designers was “Imagine that you are
working on a product design team at Smithfield Appliances, and
you have been asked to design the next generation of sustainable
washing machine. Product ideas may be related to reducing
water and energy usage, improving manufacturing, and
increasing the product’s appeal to users. Generate ideas that
address this design challenge. You may use scratch paper if
necessary but please type all individual ideas in the boxes below.
You do not have to fill in all of the boxes. You will have 8 minutes
to complete this task.”
We included explicit instructions, e.g. reducing water, to
help trigger participants’ thoughts of sustainability, because She
and MacDonald [2] found that participants perform much better
if more explicit design directions are provided.
5.4 Video Montage
In between Ideation 1 and 2, all participants watched a
montage of videos (available at https://youtu.be/7RKSZxOv2g4)
for approximately 10 minutes. The videos included balanced
mentions of user desirability (a video describing a sleek, modern
washing machine by Samsung with extensive user-centered
features, approximately 4 minutes), environmental impact (two
news segments on highly environmentally-friendly washing
machines, approximately 2.5 minutes), and manufacturing (a
segment from the television show “How It’s Made” on front-
loading washing machines, approximately 4 minutes). This
video montage was added in response to pilot testing, which
showed that participants needed more inspiration to generate an
analyzable number of ideas. Pilot testing honed the video for
balance and demonstrated that watching the video increased the
number of ideas generated in Ideation 2. All participants saw the
same video. It is possible that the collage priming activity
enhanced the effect of the video, allowing primed participants to
obtain more information from the video than non-primed
participants, but this is difficult to specifically test.
5.5 Idea Evaluation This activity occurred after Ideation 2, and its purpose is to
see if priming reduced ownership bias, as articulated in Research
Proposition 2 in Section 3. To design the evaluation activity, we
consulted with an expert on team behavior. First, the activity
creates an anchor by asking participants to rate 5 of their own
ideas (randomly drawn from their previous answers using a
JavaScript in Qualtrics). They classify them into 3 categories:
“definitely want to explore,” “possibly want to explore,” and “do
not want to explore,” as shown in Figure 5. On the next screen,
the participants see these five ideas again, now randomly
interspersed with 12 preselected ideas from hypothetical others.
All participants see the same 12 other ideas, which represent
varying sustainability aspects (user desirability, cost,
environmental impact) and quality characteristics (originality
and feasibility). The participants classified all 17 ideas using the
same categories as before. The analysis below in Section 7.2
checks to see if participants demoted the classification of any of
their five ideas when exposed to the ideas of others.
6 EXPERIMENT PROCEDURE Seventy-three participants were recruited at the
International Design and Technical Conference (IDETC) 2016 in
Charlotte, North Carolina, and on Stanford University’s campus.
To control the experimental environment, we used similar desk
set-ups in both locations. Most of the participants were graduate
students in engineering. Participants were contacted by e-mail
and reminded by a text message. They received $50.00 for
participation. The experiment took on average 82.4 minutes to
complete. Participants were encouraged to sketch or draw on
scratch paper, but only typed responses were processed in this
study; because the quality of drawings might affect the judges’
perception of the quality of the idea [44], we wanted to avoid
such influence.
All activities except the collage priming activity took place
on an iPad, and all data was collected using Qualtrics.
Participants in conditions A1–A3 and B each spent 12 minutes
interacting with the collage or reading material before designing,
while participants in the control condition started with Ideation
I, which lasted for 8 minutes for all participants. Then, all
participants watched the video for 10.5 minutes. Next,
participants had another design session to generate additional
ideas (Step IV, Ideation 2). If a participant generated fewer than
5 ideas total from both sessions, the participant would see a
design prompt for additional encouragement, but in practice,
only one participants required this prompt. Lastly, in Step VI
Questionnaire, participants answered a survey about their
7 Copyright © 2018 by ASME
demographics, their past experience designing sustainable
products, and their appreciation of various qualities of the
washing machine.
7 DATA AND ANALYSIS We collected responses from 73 participants in three
separate sections: Ideation, Idea Evaluation, and Questionnaire.
In Ideation (Ideation 1 and 2), participants generated 914 design
ideas in total and 670 unique ideas. Because ratings of the
number of effective ideas in Ideation 1 and 2 are not significantly
different for all conditions, all ideas were treated as one data set
for analysis in this paper.
For the Questionnaire (Step VI), participants reported their
previous experience of sustainable products and awareness of
different characteristics of sustainable designs, from extremely
important to not at all important (Figure 6). 49.3% of participants
reported having no experience working on sustainability-related
projects, and 86% of participants reported not being familiar
with the three pillars of sustainability. Figure 6 shows that
environmental impact, cost, and user desirability all were ranked
FIGURE 5 ILLUSTRATION OF THE INTERFACE IN STEP V, IDEA EVALUTION
FIGURE 6 SELF-REPORTED AWARENESS OF CHARACTERISTICS OF SUSTAINABLE DESIGNS, AT END OF EXPERIMENT
8 Copyright © 2018 by ASME
among the most important aspects of design. Note that this is
after priming or educational reading, and video exposure.
7.1 Judging Ideas
7.1.1 Preparation and Procedure We first reviewed all ideas generated in Ideation 1 and 2 to
correct typos and grammatical errors, so that the type of
expression and the clarity of the language would not affect the
assessment of ideas. Next, very similar ideas were identified and
grouped to a single idea for rating by the judges. This (1) ensured
that two very similar ideas would not get different ratings; (2)
prevented confusion, should judges be presented with two
similar ideas within their randomly-drawn set; and (3) reduced
the number of ideas from 927 to 670, requiring less judging
work. The cleaned ideas were entered into a Qualtrics survey
designed to present a judge with 15 randomly-drawn ideas for
judging, such that each idea was rated on each metric at least 5
times by 5 separate judges.
Judges were recruited on Amazon Mechanical Turk
(AMT)—a platform that efficiently crowdsources users [45]—
and, in this paper, are referred to as “AMT judges” or simply
“judges.” They had minimum design experience but were
familiar with washing machines. Typical screening procedures
were applied, using a test to qualify judges, and dropping the
responses with response times shorter than 1 standard deviation
of the mean response time (35 respondents). Judges were paid $4
for completing the survey, which took on-average 18 minutes to
complete.
She and MacDonald found that the ratings per designer by
experts and novices aligned with a Pearson correlation
coefficient of 0.7 when evaluating the quality of the design ideas
[2]. In addition, novice judges who passed the screening were
expected to understand rating criteria and to be qualified to give
rating scores. Additionally, using professional judges on 670
features brings its own issues related to internal consistency and
judgment fatigue. Therefore, only AMT judges were recruited in
this study.
The rating survey consisted of two sections: one for judging
ideas related to aspects of sustainability (user desirability,
manufacturing cost, use cost, and environmental impact) and
another section for ideas related to characteristics of intrinsic
quality (originality and feasibility). In each section, AMT judges
received a task overview and then training about the rating
categories, including score descriptions, example ideas, and
corresponding rating scores (see Table 2). Judges first took
quizzes to test if they paid attention to the training, and those
who passed rated 15 randomly ordered ideas in each section, for
a total of 30 ideas.
7.1.2 Idea Metrics The rating scales used to judge the effectiveness of User
Desirability, Manufacturing Cost, Use Cost, and Environmental
Impact are shown in Table 2. Cost was broken into
“manufacturing cost” and “use cost” based on pilot judging,
which indicated a confusion in rating cost as a whole. Separately,
we measured originality and feasibility to assess the intrinsic
quality of a design. A global definition of design quality does not
exist, and research yielded different definitions and metrics
under different circumstances [46]: Shal et al. [47-48] proposed
four effectiveness measures: quality, quantity, novelty, and
variety; Kudrowitz et al. suggested three attributes to measure
early-stage product ideas: novelty, usefulness, and feasibility
[36]. In this study, it is appropriate to adopt a subset of the
existing metrics based on the hypotheses at hand, and originality
and feasibility were chosen to represent how well a design
expands the design space. These metrics are discussed further in
[36,47-48], on which we based our rating scale. All ratings were
on a scale of 1 to 5, with 5 being the most positive rating. This
5-point scale is adopted from the previous study by She and
MacDonald [2] for the comparability of the results.
7.1.3 Judging Results for Ideation 73 participants generated 914 design features, including
non-unique ones. Table 3 shows the total number of ideas
generated per experimental condition and the average number of
ideas generated per participant. Ratings of ideas were averaged
across judges; Table 4 summarizes the average rating scores
across each condition for all rating categories, and Table 5
summarizes the median rating scores for all rating categories.
To test H1(a,b,c,d), we counted the number of above-
median-rating ideas generated by each designer for each
sustainability-related rating category. For brevity we call these
above-median ideas “effective ideas.” We compared the average
count of effective ideas per designer for each experimental
condition against the average count in the control.
Figure 7 plots the average count of effective ideas generated
per designer across all conditions and rating categories, and
Table 6 gives the statistical details.
To determine statistical significance, we used the Kruskal-
Wallis test [49] for comparison of two non-normally distributed
samples from the sample distribution. This test is similar to one-
way ANOVA but is used when the measurement data does not
meet the normality assumption or has a small size. Results:
• Manufacturing Cost (Fig. 7b): The mean value of the
number of effective ideas generated by designers in the
reading activity condition is significantly higher than that of
the control condition (8.29 vs. 4.80, p = 0.05*).
• Environmental Impact (Fig. 7d): The mean values of the
number of effective ideas generated by designers in the cost
and in the environmental collage priming conditions are
significantly higher than the control (cost: 7.07 vs. 4.27, p =
0.007*; environmental: 6.13 vs. 4.27, p = 0.02**). The mean
value of numbers of effective ideas generated by designers
in the reading activity condition (B) is significantly higher
than that of the control condition (7.57 vs 4.27, p = 0.05*).
• In general, the average count of effective “user desirability”
and “use cost” ideas generated per designer in three collage
priming conditions and the reading activity condition are
higher than the average count of effective ideas in the
control condition, but the differences are not significant.
To test H1(e,f), we counted the number of effective (above-
median-rating) ideas generated by each designer for each
9 Copyright © 2018 by ASME
quality-related rating category. We compared the average count
of effective ideas per designer for each experimental condition
against the average count in the control, again using Kruskal-
Wallis. No significant effects were found, although “feasibility”
average count was higher for all conditions versus the control.
TABLE 2 RATING CATEGORIES AND DESCRIPTIONS OF SCORES 5, 3, AND 1
Rating
Category
Description of
Score 5
Description of
Score 3
Description of
Score 1
Use
r
Des
irab
ilit
y Desired by a
reasonable
number of
customers;
pleasing;
attractive
Okay to have;
somewhat
desired by
customers
Unpleasant or
should be
avoided
Man
ufa
ctu
rin
g
Cost
Manufactured
with feasible
process and
cost saving
Neither
decreasing nor
increasing
process time or
manufacturing
cost compared
to the current
design
Manufactured
with high cost
or tremendous
amount of
effort
Use
Cost
Operated at a
lower cost or
helpful at
reducing the
cost of using
(electricity, gas,
water etc.)
compared to
the
conventional
design
Having no
significant
effect on the
cost of using,
compared to
the
conventional
design
Operated at a
higher cost or
increasing the
cost of using,
compared to
the
conventional
design
En
vir
on
men
tal
Imp
act
Good for the
environment
Neutral for the
environment
Bad for the
environment
Orig
inali
ty
Not expressed
before and
ingenious
Not typical,
and show some
imagination
Common and
boring
Feasi
bil
ity
Easy to
implement
without major
changes or
violation of
known
constraints
(financially and
physically)
Could be
implemented
with minor
changes to
existing
conditions
Very hard to
implement
given the
existing
conditions
and/or
requiring
significant
research and
development
TABLE 3 NUMBER OF IDEAS GENERATED PER CONDITION AND AVERAGE PER DESIGNER
Condition
Number of
participants
Total number
of ideas
generated
(including non-
unique ones)
Average number
of ideas
generated per
participant
User
desirability
prime 13 159 12.23
Cost prime 15 195 13.00
Environmental
prime 16 191 11.93
Reading 14 204 14.57
Control 15 165 11.00
TOTAL 73 914 12.50
TABLE 4 MEAN SCORE (AND STANDARD DEVIATION) OF RATING
CATEGORIES PER CONDITION
Con
dit
ion
Rating Category
User
desira
bility
Manuf
acturin
g cost
Use
cost
Enviro
nment
al
impact
Origin
ality
Feasibi
lity
User
desirabi
lity
prime
3.64 2.50 3.42 3.82 2.95 3.47
(0.74) (0.57) (0.52) (0.66) (0.88) (0.79)
Cost
prime
3.57 2.54 3.44 3.90 3.03 3.34
(0.75) (0.65) (0.55) (0.71) (0.88) (0.77)
Environ
mental
prime
3.55 2.58 3.46 3.86 2.86 3.470
(0.73) (0.62) 0.561 0.72 0.86 0.782
Reading 3.38 2.67 3.42 3.85 2.95 3.33
(0.80) (0.69) (0.53) (0.68) (0.84) (0.81)
Control 3.59 2.54 3.40 3.69 3.08 3.25
(0.78) (0.59) (0.53) (0.71) (0.80) (0.81)
Grand 3.54 2.57 3.43 3.83 2.97 3.37
(0.76) (0.63) (0.54) (0.70) (0.86) (0.80)
10 Copyright © 2018 by ASME
TABLE 5 MEDIAN RATING SCORES FOR EACH CATEGORY
User
desirability
Manufacturing
cost Use cost
Environmental
impact Originality Feasibility
3.66 2.52 3.32 3.89 3.03 3.40
TABLE 6 AVERAGE NUMBER OF EFFECTIVE IDEAS GENERATED BY EACH DESIGNER (An “effective idea” is one
that was rated above the median score of that rating category; **p<0.05 and *p<0.1.)
Rating Category
User
desirability
Manufacturing
Cost Use Cost
Environmental
Impact Originality Feasibility
Con
dit
ion
s
User
Desirability
Prime
6.92 5.77 5.77 6.00 6.31 6.69
Cost Prime 6.80 6.27 6.73 7.07 *
(p=0.07) 6.80
5.93
Environmental
Prime
6.13
6.13 6.31
6.31 **
(p=0.02) 5.38 6.31
Reading
Activity 6.00
8.29 **
(p=0.04)
7.21
7.57 **
(p=0.04)
6.71
7.00
Control 5.53 4.80 5.27 4.27 6.20 4.93
FIGURE 7 COUNT OF EFFECTIVE IDEAS GENERATED BY EACH DESIGNER ACROSS ALL CONDITIONS (An “effective idea” is one that was rated above the median score of that rating category; error bars show the standard deviations.)
11 Copyright © 2018 by ASME
7.2 Idea Evaluation
This analysis tests Proposition 2: It is possible to reduce
ownership bias of designers by using the collage prime.
Specifically, it investigates if designers re-classify the
importance of further investigating their own ideas when also
asked about the importance of other designers’ ideas, see Section
5.5.
The analysis focuses on the classification exercise of the five
randomly-drawn ideas from the design participants responses in
the ideation phase of the experiment. This mimics idea selection
in a collaborative environment, which is one of the crucial
components in the design process. The classification categories
were “Definitely,” “Possibly,” and “Do Not” want to explore
(this idea further), and a fourth classification, “Untouched,” of
not dragging the idea into any of the three boxes. The participants
were asked to evaluate these ideas twice, first alone, which we
call evaluation section ES1, and then in the presence of 12 ideas
from hypothetical other designers, which we call ES2.
7.2.1 Binomial Proportion Test The proportion of changing the classifications of
participants’ own ideas from ES1 to ES2 for each testing
condition is quantified as Eq. 1.
𝑝𝑐ℎ𝑎𝑛𝑔𝑒,𝑖 = ∑ 𝐶𝑜𝑢𝑛𝑡 𝑜𝑓 𝑅𝑒𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝐼𝑑𝑒𝑎𝑠𝑖 𝑖𝑛 𝐸𝑆2
∑ 𝐶𝑜𝑢𝑛𝑡 𝑜𝑓 𝐼𝑑𝑒𝑎𝑠 𝑃𝑙𝑎𝑐𝑒𝑑 𝑖𝑛 𝐵𝑜𝑥𝑒𝑠 𝑖𝑛 𝐸𝑆1𝑖 (1)
Table 7 shows the proportions of changing for all conditions
and the p-values when comparing each value of the experimental
groups (A1–A3, B) to the control group by binomial proportion
test. The cost collage prime (A2) has a dramatic and statistically
significant effect, with 40% of classified ideas changing
positions. The other conditions have no significance, although all
are greater than the control.
TABLE 7 PROPORTIONS OF RECLASSICATIONS
Condition
ES2: Count of
reclassified ideas
ES1: Count of Ideas
Placed in Boxes
% Reclassified
User
Desirability 17 63 27% (p=0.28)
Cost 30 75 40% (p =2e-4*)
Environment 20 71 28% (p=0.15)
Reading 20 68 29% (p=0.10)
Control 15 71 21% -
7.2.2 Wilcoxon Signed Rank Test To explore the change of classifications at the individual
level from ES1 to ES2, for example, an idea was classified as
“possibly want to explore” in ES1 and “definitely want to
explore” in ES2, the non-parametric Wilcoxon signed rank test
was used to compare the paired ES1-to-ES2 classification results
without and with the predetermined ideas for each condition.
Specifically, higher rank indicates the direction of changes is
towards higher importance.
TABLE 8 SIGNED-RANKS BETWEEN ES1 AND ES2
Condition Test statistics
User Desirability 36 (p = 0.03*)
Cost 30 (p = 0.09*)
Environment 140 (p = 0.20)
Reading 100 (p = 0.9)
Control 70 (p = 0.5)
Table 8 shows the test statistics along with the p-values for
five conditions by the Wilcoxon signed rank test. The results
show that both user desirability collage priming and cost collage
priming have a significant effect on encouraging designers to
classify their own ideas as less important and potentially to
explore others’ ideas more in a collaborative environment.
8 DISCUSSION The experiment demonstrates that exposing designers to a
collage priming activity associated with sustainable design (user
desirability, economic competitiveness, environmental
friendliness) is partially effective in generating effective ideas
and in reducing ownership bias. A comparison reading activity is
just as or more effective than the priming activities in generating
effective environmentally friendly ideas and manufacturing cost
ideas, but not effective in reducing ownership bias. This suggests
that priming is an effective way to produce environmentally
friendly design ideas while keeping an open mind to ideas of
others. Unfortunately, priming did not provide a method for
activating the less-readily-accessible pillars of sustainability—
social/user desirability and economic competitiveness—during
ideation. Reading material on sustainable design was effective in
producing a greater number of effective manufacturing cost
ideas, but not in reducing ownership bias in decision evaluation,
compared to control.
There were 670 unique ideas generated—a huge amount—
so we used novice AMT judges with limited in-survey training,
and averaged multiple evaluations from separate judges for the
statistical testing of effectiveness of ideas. As the AMT judges
were imperfect and inexperienced raters, we defined an effective
idea as one that was ranked higher than the median rating for
each rating category, instead of relying on a pure rating of greater
than 3 to determine effectiveness, as we may have with expert
judges. While it can be argued that using novice judges
diminishes the findings of the study, as compared to using expert
judges, the argument can also be made that as a product’s
sustainability and quality will be judged in the end by novice
users, it makes sense to use them for research judgments as well.
In previous research, we found good alignment, with a Pearson
correlation of 0.7 [2], between expert judges and AMT judges on
a similar rating task.
The effectiveness of the reading education activity is
notable. In terms of pure numbers, not with significance, this
condition had the most ideas generated per designer, and had the
highest average number of effective ideas per design for
Manufacturing Cost, Use Cost, Environmental Impact, and
12 Copyright © 2018 by ASME
Feasibility—four out of six rated categories. This suggests that
the student design participants, on the whole, did not have much
latent content in their brains that could be activated by priming
to address sustainable design. The fact that 63 out of 73 designers
had no concept of the three pillars of sustainability supports this
explanation. Instead, providing basic sustainable design
education was more effective. We did not combine the reading
activity with a collage priming activity, but this could have been
even more effective. Possibly, priming more-experienced
designers, who are accustomed to addressing cost and user
desirability concerns, would have been more effective than basic
education (which they would already have). Or a better approach
might be to combine a priming activity and an education activity
for student designers.
Our specific hypotheses produced mixed results. The
collage priming activity that was associated with the
environmental pillar significantly increased the number of
above-median-rating (effective) ideas generated per designer
when judged on “environmental impact,” which supports
hypothesis H1d. This also supports the thoughts of the paragraph
above, as it is likely that of the three pillars, student designers
have the most experience with environmental impact design,
such as reducing energy or resources used during manufacture
and/or use of the product. The results also show that the collage
priming activity associated with cost and the reading education
activity also resulted in a significant increase in the number of
effective “environmental impact” ideas. Reducing cost is closely
linked with reducing resource use and consumption, so this is
logical.
Hypotheses H1a, b, and c are rejected; that is, the collage
priming activity associated with user desirability had no
significant effect on increasing the average number of effective
“user desirability” ideas, and the collage priming activity
associated with cost had no significant effect on increasing the
average number of effective “manufacturing cost” and “use cost”
ideas.
Hypotheses H1e and f are also rejected. There were no
significant effects on improving quality, as measured by
feasibility and originality. While other priming activities have
been demonstrated to increase novelty, the activities in this
experiment were ineffective.
Hypotheses H2 tested if priming activities could lead to
more receptivity and less ownership bias when evaluating one’s
own ideas in the context of others’ design ideas. It is supported
for the Cost priming activity by a binomial proportion test and
Wilcoxon Signed Rank test, and for the User Desirability
priming activity by only the Wilcoxon Signed Rank test. The
Cost priming resulted in a substantial (40%) reclassification of
the importance of further investigating one’s own ideas when
exposed to the ideas of others. H2 is not supported for the
Environmental collage priming activity. It may be that this
prime, instead of underscoring ownership bias, triggers social
desirability bias, or doing what is morally right, and creates an
overemphasis on the importance of environmental impact as
compared to the other two pillars of sustainability. Thus, when a
variety of ideas are presented, designers stick to their own
environmentally-oriented ideas in favor of ideas that address the
other pillars of sustainability. The reading activity also did not
reduce ownership bias.
This study has several limitations. The first is that the study
used student designers instead of practicing designers. Students
have less experience and education with the broader concerns of
engineering, and thus less latent mental content can be triggered
by sustainability primes. The success of the education activity
supports this limitation. Ideation outcomes were assessed by
multiple online judges, who are potential users of the focal
product, but who may have very different purchase habits and
life goals from each other. This setting reflects the situation of
sustainable products in the real market; yet the individual
differences between judges is not taken into account in this study.
Expert judgments may yield different results, although a past
study indicated a good correlation between expert and novice
judges. In addition, participants in the priming conditions
performed activities, either physically placing pictures or
reading a document, while participants in the control condition
directly started to ideate. The acceptance of others’ ideas was
only performed with simulated other designers. There were too
many possible factors, such as personality fit, familiarity,
language differences, variability in creativity and
communication, etc., that were not controlled for, to get an
accurate read on change in ownership bias in a real team setting.
Also, the participants in our study were unable to do design
research; perhaps in a more involved exercise, with a washing
machine on-hand, users to interview, and Life Cycle Assessment
analysis available, the priming activities could have triggered
design actions, which may have led to even better ideation
sessions.
There are also a number of potential sources of technical
error in the study. For example, the data is not normally
distributed, and although the Kruskal-Wallis test was used to
compensate for this, it can still result in incorrect predictions.
The sample size was small, at approximately 15 designers per
condition. Doubling this number could improve the distribution
of results and also the robustness of the tests. Another potential
source of error is that only one design task was performed on one
target product. Testing multiple products would increase the
assuredness of the findings. While the reliability of AMT judges
has been proven trustworthy, if automatic assessments where
recorded instead of those of actual novice users, this could also
impact the results.
9 CONCLUSION AND FUTURE WORK The experiment further illustrates the difficulty designers
have when tackling social and economic aspects in ideation
processes. As stated in the introduction, designers naturally
gravitate toward the environmental aspect in the early stage of
sustainable design, even though social and economic aspects are
also essential for sustainable products to succeed in the market.
This issue is well demonstrated in this experiment: designers
who were exposed to the user desirability prime did not improve
significantly on creating a greater number of effective user-
desirable ideas, and designers who were exposed to the cost
13 Copyright © 2018 by ASME
prime did not generate a greater number of effective designs that
improved manufacturing cost or use cost of the product, as
judged by novice users. As mentioned above, this is likely due to
a lack of education on these participants—without embedded
knowledge in the brain to “activate,” priming cannot work.
A good follow-up study would be to test the effectiveness of
priming activities in combination with education activities.
Questions of the order of the two activities, and the best pairings
of types of activities should be explored. Effort to expose student
designers to broader design concerns, such as manufacturing,
marketing, and human-centered design, must continue in order
to create available mental content that can be accessed during
ideation. Life Cycle Assessment and other environmentally
motivated engineering training is important, but without design
attention to the other two pillars of sustainability, the product will
not be profitable and popular with users and may ultimately fail
in the market and in use.
Overall, the study contributes to growing evidence that
priming is an effective tool to improve ideation, with mixed
results for the experiment at-hand. We demonstrate that collage
priming activities can help designers generate a greater number
of effective ideas that address the environmental pillar of
sustainability, but not that address cost (representing the
economic pillar) or user desirability (representing the social
pillar). We believe lack of education and experience is to blame
for the ineffectiveness of the priming activities for these pillars.
A reading education activity in this experiment generated a larger
number of effective sustainable design ideas but did not reduce
ownership bias. The Economic and Social pillar collage priming
activities are effective in reducing ownership bias to others’ ideas
when evaluating design ideas to pursue further, but the
Environmental collage priming activity may trigger a social bias
for environmentally driven ideas and enhance ownership bias.
Combining these two findings, the Economic priming activity
was the most effect prime overall, although not in the manner
that we predicted: it was not effective toward the economic pillar
of sustainability, but rather the environmental one, and also
showed evidence of reducing ownership bias over the ideas of
other designers.
ACKNOWLEDGMENTS This work was funded by the Hasso Plattner Design
Thinking Research Program. We thank Hasso Plattner Insitut for
supporting this project. We thank Ufuoma Ovienmhada for her
assistance in data collection. The premise of this study was
conceived with Carolyn Conner Seepersad at UT Austin. We
thank Dylan Moore and Professor Pamela Hinds for their
thoughtful contributions to the design of the experiment.
REFERENCES [1] She, J., Seepersad, C. C., Holtta-Otto, K., & MacDonald, E.
F., 2018, “Priming Designers Leads to Prime Designs,”
Design Thinking Research, pp. 251-273, Springer, Cham.
[2] She, J., & MacDonald, E., 2014, “Priming Designers To
Communicate Sustainability,” ASME J. Mech. Des., 136(1),
p. 011001.
[3] She, J., & MacDonald, E. F., 2018, “Exploring the Effects of a
Product’s Sustainability Triggers on Pro-environmental
Decision-making,” ASME J. Mech. Des., 140(1), p. 011102.
[4] Keeble, B. R., 1988, “The Brundtland Report: ‘Our Common
Future’,” Medicine and War, 4(1), pp. 17–25.
[5] Cuthill, M., 2009, “Strengthening the ‘Social’ in Sustainable
Development: Developing A Conceptual Framework for Social
Sustainability and a Rapid Urban Growth Region an
Australia,” Sustain. Dev., 18(6), pp. 362–373.
[6] Littig, B., & Griessler, E., 2005, “Social Sustainability: A
Catchword Between Political Pragmatism and Social Theory,”
Inter. J. Sustain. Dev., 8(1-2), pp. 65–79.
[7] Friedman, R. S., Fishbach, A., Forster, J., & Werth, L., 2003,
“Attentional Priming Effects On Creativity,” Creat. Res. J.,
15(2–3), pp. 277–286.
[8] Bruner, J. S., 1957, “On Perceptual Readiness,” Psychol.
Rev., 64(2), p. 123.
[9] Bargh, J. A., Chen, M., and Burrows, L., 1996, “Automaticity
Of Social Behavior: Direct Effects Of Trait Construct And
Stereotype Activation On Action,” J. Pers. Soc. Psychol.,
71(2), pp. 230–244.
[10] Guyton, A. A., 2006, “Developing Sustainable Product
Semantics for Consumer Products: A Sustainable Designer’s
Guide,” Doctoral dissertation, Georgia Institute of
Technology.
[11] Onarheim, B., and Christensen, B. T., 2012, "Distributed
idea screening in stage-gate development processes," J. Eng.
Des., 23(9), pp. 660-673.
[12] Ramani, K., Ramanujan, D., Bernstein, W. Z., Zhao, F.,
Sutherland, J., Handwerker, C., Choi, J. K., Kim, H., and
Thurston, D., 2010, “Integrated Sustainable Life Cycle
Design: A Review,” ASME J. Mech. Des., 132(9), p. 091004.
[13] Maxwell, D., & Van der Vorst, R., 2003, “Developing
Sustainable Products And Services,” J. Cleaner Prod., 11(8),
pp. 883-895.
[14] Halog, A., 2001, “Using Quality Function Deployment for
Technique Selection for Optimum Environmental
Performance Improvement,” J. Cleaner Prod., 9, pp. 387–394.
[15] Grunert, K. G., Hieke, S., & Wills, J., 2014, “Sustainability
Labels On Food Products: Consumer Motivation,
Understanding And Use,” Food Policy, 44, pp. 177–189.
[16] Cuthill, M., 2009, “Strengthening the ‘Social’ in Sustainable
Development: Developing A Conceptual Framework for Social
Sustainability and a Rapid Urban Growth Region an
Australia,” Sustain. Dev., 18(6), pp. 362–373.
[17] Littig, B., & Griessler, E., 2005, “Social Sustainability: A
Catchword Between Political Pragmatism and Social Theory,”
Inter. J. Sustain. Dev., 8(1-2), pp. 65–79.
[18] Mota, B., Gomes, M. I., Carvalho, A., & Barbosa-Povoa, A. P.,
2015, “Towards Supply Chain Sustainability: Economic,
Environmental and Social Design and Planning,” J. Clean.
Prod., 105, pp. 14-27.
[19] United Nations, Department of Economic., 2007, “Indicators
of Sustainable Development: Guidelines and Methodologies,”
United Nations Publications.
[20] Mandel, N., and Johnson, E. J., 2002, “When Web Pages
Influence Choice: Effects of Visual Primes on Experts and
Novices,” J. Consum. Res., 29(2), pp. 235–245.
[21] Stapel, D. A., 2011, “Priming as Proxy: Understanding the
Subjectivity of Social Life,” Cognitive Methods in Social
Psychology, pp. 148–183.
[22] Shariff, A. F., and Norenzayan, A., 2007, “God Is Watching
14 Copyright © 2018 by ASME
You: Priming God Concepts Increases Prosocial Behavior In
An Anonymous Economic Game,” Psychol. Sci., 18(9), pp.
803–809.
[23] Pfeffer, J., and DeVoe, S. E., 2009, “Economic Evaluation:
The Effect Of Money And Economics On Attitudes About
Volunteering,” J. Econ. Psychol., 30(3), pp. 500–508.
[24] Martin, L. J., Hathaway, G., Isbester, K., Mirali, S., Acland,
E. L., Niederstrasser, N., Slepian, P. M., Trost, Z., Bartz, J. A.,
Sapolsky, R. M., Sternberg, W. F., Levitin, D. J., and Mogil, J.
S., 2015, “Reducing Social Stress Elicits Emotional
Contagion Of Pain In Mouse And Human Strangers,” Curr.
Biol., 25(3), pp. 326–332.
[25] Bargh, J. A., Chen, M., and Burrows, L., 1996, “Automaticity
Of Social Behavior: Direct Effects Of Trait Construct And
Stereotype Activation On Action,” J. Pers. Soc. Psychol.,
71(2), pp. 230–244.
[26] Kay, A. C., Wheeler, S. C., Bargh, J. A., and Ross, L., 2004,
“Material Priming: The Influence Of Mundane Physical
Objects on Situational Construal and Competitive Behavioral
Choice,” Organ. Behav. Hum. Decis. Process., 95(1), pp. 83–
96.
[27] Sassenberg, K., and Moskowitz, G. B., 2005, “Don’t
Stereotype, Think Different! Overcoming Automatic
Stereotype Activation By Mindset Priming,” J. Exp. Soc.
Psychol., 41(5), pp. 506–514.
[28] Marsh, R. L., Bink, M. L., and Hicks, J. L., 1999,
“Conceptual Priming In A Generative Problem-Solving Task,”
Mem. Cognit., 27(2), pp. 355–363.
[29] Reed, A., Aquino, K., Levy, E., and Ii, R., 2007, “Moral
Identity and Judgments of Charitable Behaviors,” J. Mark.,
71(1), pp. 178–193.
[30] Molinsky, A. L., Grant, A. M., and Margolis, J. D., 2012,
“The Bedside Manner Of Homo Economicus: How And Why
Priming An Economic Schema Reduces Compassion,” Organ.
Behav. Hum. Decis. Process., 119(1), pp. 27–37.
[31] Rietzschel, E. F., Nijstad, B. A., and Stroebe, W., 2007,
“Relative Accessibility Of Domain Knowledge and
Creativity: The Effects Of Knowledge Activation On The
Quantity And Originality Of Generated Ideas,” J. Exp. Soc.
Psychol., 43(6), pp. 933–946.
[32] Srull, T. K., and Wyer, R. S., 1979, “The Role Of Category
Accessibility In The Interpretation Of Information About
Persons: Some Determinants And Implications,” J. Pers. Soc.
Psychol., 37(10), pp. 1660–1672.
[33] Berger, J., Meredith, M., and Wheeler, S. C., 2008,
“Contextual Priming: Where People Vote Affects How They
Vote,” Proc. Natl. Acad. Sci. U. S. A., 105(26), pp. 8846–
8849.
[34] Lewis, S., Dontcheva, M., and Gerber, E., 201, May,
“Affective Computational Priming And Creativity,” In
Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (pp. 735-744). ACM.
[35] Galinsky, A. D., and Moskowitz, G. B., 2000,
“Counterfactuals as Behavioral Primes: Priming the
Simulation Heuristic and Consideration of Alternatives,” J.
Exp. Soc. Psychol., 36(4), pp. 384–409.
[36] Kudrowitz, B. M., and Wallace, D., 2013, “Assessing the
quality of ideas from prolific, early-stage product ideation,” J.
Eng. Design, 24(2), pp. 120-139.
[37] Kahneman, D., Knetsch, J. L., and Thaler, R. H., 1990,
"Experimental Tests of the Endowment Effect and the
Coase Theorem," J. Political Econ., 98(6), pp.
1325-1348.
[38] Alicke, M. D., 1985, "Global Self-Evaluation as
Determined by the Desirability and Controllability of Trait
Adjectives," J. Pers. Soc. Psychol., 49(6), pp. 1621-1630.
[39] Toh, C. A., Patel, A. H., Strohmetz, A. A., and Miller, S. R.,
2015, “My Idea Is Best! Ownership Bias and its Influence on
Engineering Concept Selection,” ASME 2015 IDETC/CIE,
Aug. 2-5, Boston, MA.
[40] Marsh, E. R., Slocum, A. H., and Otto, K. N., 1993,
"Hierarchical decision making in machine design," MIT
Precision Engineering Research Center.
[41] Pugh, S., 1991, Total design: integrated methods for
successful product engineering, Addison-Wesley,
Wokingham.
[42] López-Mesa, B., and Bylund, N., 2011, "A study of the use
of concept selection methods from inside a company," Res.
Eng. Des., 22(1), pp. 7-27.
[43] “Ecodesign overview” [Online]. Available:
https://app.sustainableminds.com/learning-center/ecodesign-
and-lca/ecodesign-overview desirability.
[44] Yang, M. C., 2009, “Observations on concept generation and
sketching in engineering design,” Res. Eng. Des., 20(1), pp.
1-11.
[45] Kittur, A., Chi, E. H., and Suh, B., 2008, “Crowdsourcing
User Studies with Mechanical Turk,” CHI 2008, Apr. 5–10,
Florence, Italy.
[46] Reeves, C. A., and Bednar, D. A., 1994, “Defining quality:
alternatives and implications,” Acad. Manag. Rev., 19(3), pp.
419-445.
[47] Shah, J. J., Smith, S. M., and Vargas-Hernandez, N., 2003,
“Metrics for Measuring Ideation Effectiveness,” Des. Stud.,
24(2), pp. 111-134.
[48] Shah, J. J., Kulkarni, S. V., and Vargas-Hernandez, N., 2000,
“Evaluation of Idea Generation Methods for Conceptual
Design: Effectiveness Metrics and Design of Experiments,”
ASME J. Mech. Des., 122(4), pp. 377-384.
[49] Kruskal, W. H., & Wallis, W. A., 1952, “Use of Ranks in One-
Criterion Variance Analysis,” J. Am. Stat. Assoc., 47(260), pp.
583-621.
[50] Howell, D. C., 2012, “Statistical methods for psychology,” pp.
127-131, Cengage Learning.
[51] Sidney, S., 1957, “Nonparametric Statistics for The Behavioral
Sciences,” J. Nerv. Ment. Dis., 125(3), p. 497.