2014-Ideation-Methods-A-First-Study-with-Measured-Outcomes-with-Personality-Type-DETC2014-34954...

8/10/2019 2014-Ideation-Methods-A-First-Study-with-Measured-Outcomes-with-Personality-Type-DETC2014-34954 (1).pdf

http:///reader/full/2014-ideation-methods-a-first-study-with-measured-outcomes-with-personality-type-detc2014-3 1/12

1

Proceedings of the ASME 2014 International Design Engineering Technical Conferences &Computers and Information in Engineering Conference

IDETC/CIE 2014August 17-20, 2014, Buf falo, New York, USA

DRAFT:DETC2014-34954

IDEATION METHODS: A FIRST STUDY ON MEASURED OUTCOMES WITH PERSONALITY TYPE

Pui Kun ChooSingapore University ofTechnology & Design

Zhi Ning LouSingapore University ofTechnology & Design

Bradley A. CamburnSingapore University ofTechnology & Design

Kristin L. WoodSingapore University ofTechnology & Design

Ben KooTsinghua University

Beijing, China

Francois GreyCentre for Nano and Micro

Mechanics, TsinghuaUniversity, Beijing

ABSTRACTThe research reported here considers an experiment and

subsequent data coding and analysis to extract correlations

between personality type and ideation outcome from several

methods. This article presents the background theory,research methodology, and empirical results associated with

the experiment. The experiment is based on observations of

designers developing a real product, and associated

assessment tools, where the goal is to correlate the quality,

quantity, and variety of design outcomes with respect to

personality type. This approach lays the foundation for a

tailored ideation method or a suite of ideation methods that

takes advantage of the preferences and strengths of

individuals. We find that there are significant correlations

between type and ideation metrics and that these correlations

are supported by related theory from psychology and business

management.

Keywords: ideation methods, brainstorming, mind-

mapping, Method 6-3-5, C Sketch, personality type, Myers-

Briggs Type Indicator, Six Thinking Hats, empirical study

1. INTRODUCTIONStructured ideation methods are critical for the progress of

many projects in engineering design. Technically accurateinformation on the effectiveness of these methods is equally

critical [1]. It is important to assess the methods through

formal research methodologies and obtain insights on how andwhy ideation methods can be made effective.

Design managers have been grouping or seeking to

understand teams based on personality type for a long time[2,3]. One possible avenue for research is to elicit the

relationships between personality types and ideation method.

Since different personality types have been shown to

communicate differently [4,5]; and different ideation methodsare based on different types of communication [6,7], it is only

intuitive to infer that there may be differences in the

effectiveness of an ideation method for different personalitytypes. This information could potentially be used to develop a

structured ideation method or a suite of ideation methods that

is tailored for individuals or different compositions of teams.This study follows the practice of designers engaged in a

sponsored project. Procedures are employed to extract

concrete data about their performance on a design task. These

data are then post processed to make inferences about the

correlation of personality type and ideation methods. The typeof research employed in this study is often referred to as an

empirical study [1]. Other studies have in fact contributed to

examining personality types during design. Individuals fromdifferent fields tend to span a range of particular personality

types, and different personality types prefer particular stagesof the design process [8,9]. There has also been comparativeresearch on the various ideation methods used in this paper.

Design outcome metrics have been used to compare the

quantitative results of different ideation methods [6,7,9,10].

However, the authors believe this may be the first time that

different ideation methods and personality type are directly

compared for the ideation phase of design.The objective of empirical studies is to verify and record

observable phenomena. This allows for the quantificaion of

what may previosuly have been intuitive knowledge. Theexperimental cycle allows this at a high level of granularity.

Along this line, the following research questions were used to

guide and develop this study:1. What are the statistically significant trends (if any) inquality, quantity, novelty, and variety of design solutions



2

produced by different personality types across different

ideation methods, and between personality types withineach method?

2. Are there statistically significant differences in how

participants of different personality types self-perceive theoutcome of different ideation methods?

We address these questions by creating a structuredideation environment as part of an active design group, and

measure the outcome quantitatively using established metrics.

These metrics include quality, quantity, novelty, and variety of

the design outcomes as well as self-efficacy of the participant

designers. The following sections describe the ideation andpersonality type assessments employed, the experimental

setup, the design problem and its context, the participants, data

encoding and analysis procedures, as well as results andconclusions of the study.

2. BACKGROUND THEORY2.1 Brainstorming

As an intuitive method [11] of idea generation, thebrainstorming method by Osborn [12] encourages divergent

thinking [11] in design problem solving. It can be employed

individually or in a group typically including five to fifteen

people. Individually, one may start by identifying a fewconcepts to build on and generate as many solutions as

possible. In a group, members verbally communicate ideas to

one another for thirty to forty-five minutes. Ideally, themembers should not be inhibited in expressing any ideas to

achieve a comprehensive range of solutions. Development of

individual ideas result as members respond by making

connections to others ideas. These connections will vary aseach member differs in skill sets, experience and personality,

creating diversity in solutions. However, individualbrainstorming has been shown to be more productive than

group brainstorming. In a group, domination by a single or afew group members may occur [13]. Inhibition could also

happen in the presence of an expert [13], or when groups are

unreceptive to new ideas, and may result in discussing only

existing solutions. A facilitator could be appointed to ensureparticipation by all, while restricting negative criticism. A

complementary mind-mapping or 6-3-5 / C-Sketch session

combined with or held after initial brainstorming may lead to

greater effectiveness. An example brainstorming sheet fromthe study is shown in Figure 1.

Figure 1 An individual Brainstorming sheet

2.2 Mind-mapping

To effectively maximize the results from brainstorming, one

can use mind-mapping. Mind-mapping is an intuitive semanticand categorization technique that emulates a process similar to

how we organizes ideas in long-term memory. First, a key

idea is placed at the center of a piece of paper. Next, possiblesolution categories are added, branching off the key idea.

Finally, specific solutions to the problem are added to thesekey categories. Thus, each solution generated is related to theoriginal problem statement. Research has demonstrated that

mind-mapping may significantly increase the number of ideas

generated compared to the classic brainstorming approach.

This result is attributed to the categorizing of ideas, which

arranges concepts hierarchically, hence suggesting thedifference between the design avenue or category of ideas and

specific solutions. Moreover, mind mapping facilitates piggy-

backing and leap-frogging of ideas due to its a two-dimensional graphical map structure. This structure opens up

the opportunity to identify and fill in gaps in the possible

design space; for example, upon creating the mind-map, one

may notice a certain branch of solutions is less complete than

another [14]. An example mind -map from the study is shownin Figure 2.

Figure 2 An individual Mind-map

2.3 C-SketchAs an extension of the 6-3-5 method, which uses writtendescription for idea generation, the C-Sketch method is an

intuitive method that uses graphical descriptions instead of

written descriptions. It is usually employed after the problemdefinition and clarification stage of design [15]. In a team,

each member is given a sheet of blank paper on which they are

to sketch three solutions with respect to the design problemstatement. After tminutes, the papers are to be passed on to

the next person on the right. Another t minutes will be given

to add modifications or additional ideas to each idea. Theprocess of passing repeats until all members have contributedto every individual paper. The number of people in each team

is typically six, although a range of three to eight members

may work well. Likewise, the time duration is variable; it can

be fifteen minutes initially followed by ten-minute alterationsessions. Each individual is usually given a uniquely colored

pen or marker to encourage no elimination of ideas, and allow

members to easily identify their contributions for later

discussion. There is no verbal communication allowed toprevent domination of the session by a single or small group

of members, while encouraging participants to make

individual inferences of the sketches that may result in

unanticipated ideas. Labeling should also be kept to aminimum, but instead focus only on main keywords. It is also

important to refrain from negative criticism, but instead to



3

focus on further developing the ideas. Brainstorming may be

combined with a C-Sketch session, either before or after, fordevelopment of ideas through verbal communication. C-

Sketch facilitates leap frogging of ideas [7], and achieves

diversity in design [15]. Provocative stimuli [15] fromsketches of other members reduces design fixation, and helps

to develop new ideas. Research has also shown that C-Sketchinduces a forty percent increase in quantity of ideas producedover a variety of comparable methods [11]. An example of a

solution produced in C-Sketch can be seen in Figure 3.

Figure 3 A concept developed during C-Sketch

Each of the above ideation methods was deployed in the

experiment. A summary of the methods is provided in Table 1.

There are many useful methods, these three were chosen for

several reasons. First because they require relatively littleintroductory training and allowed for the full deployment of

several methods in limited time. Secondly these methods

represent a mixture of structured versus open-ended

approaches to ideation, and individual versus groupformatting. Finally, there was a rich compliment of empirical

literature to evaluate the properties of these methods readily

available for review. The remainder of Section 2 details theMyers Briggs Personality Type Indicators and Six Thinking

Hats.

TABLE 1: SUMMARY & COMPARISON OF METHODS

Method Communication Style

Individual

Brain-stormingWritten word only

Provocative stimuli, use of

analogiesGroupBrain-storming

More verbal thanwritten word

Individual

Mind-mappingWritten word only

Categories, structured andorganizedGroup

Mind-mapping

Verbal and writtenword

C Sketch Sketching primarilyImagery, graphical,

provocative stimuli

2.4 MBTIThe Myers-Briggs Type Indicator (MBTI) [4] assesses an

individuals level of preference in four categories that indicate

aspects or approaches to problem solving, decision-makingand communication of information or ideas. The categories are

based on C.G. Jung's theory of psychological types (Table 2).A total of sixteen types result from the permutation of the

categories. Research has shown that engineers are more likely

to be ISTJ, followed closely by ESTJ, then ENTP [9]. Jungs

theory of eight cognitive modes, representing problem solvingapproaches, is related to the two dominant modes or sub-types

as shown in Table 3 [3].

MBTI can be used in team formation strategies[2,3,16,17] to achieve diverse teams [2,3,16]. It ensures a

mixture of members with a variety of cognitive styles,providing groups with a spectrum of viewpoints and problemsolving methods. Through identification of each individuals

type, it informs and encourages understanding amongst

members [3].

TABLE 2: OVERVIEW OF MBTI

Orientation

Extraversion (E)

OR

Introversion (I)

Prefer working in groups and

through external interaction,

often taking a breadth-of-knowledge approach.

More comfortable working alone

reflectively, taking a depth-of-

knowledge approach into ideasand concepts.

Perception

Sensing (S)

OR

Intuition (N)

Gather information through

practical experience, focusingon observable phenomena,

facts and details.

Perceive through imagination

and internal sensing, focusing onthe big picture, theories, and new

possibilities.

Judgment Thinking (T)

OR

Feeling (F)

Analytical and logical, judging

objectively through impersonal

evaluations.

Subjective and weigh human

factors, often making decisions

based on personal values.

Style

Judging (J)

OR

Perceiving (P)

Decisive and planners,

preferring structure and order.

Keep options open, are flexible,

spontaneous and exploratory.

TABLE 3: JUNGS COGNITIVE MODES [3]

Information Collection Decision-Making

ES EN ET EFExperiment Ideation Organization Community

IS IN IT IFKnowledge Imagination Analysis Evaluation

2.5 Six Thinking HatsThe Six Thinking Hats model by Edward de Bonodistinguishes six modes of thinking, represented by six colored

hats [5] (Table 4).

TABLE 4: OVERVIEW OF SIX HATS

White Hat Red Hat

Concerned with facts, and objectiveinformation.

Utilizes emotions and intuition.

Black Hat Yellow Hat

An analyst, the devils advocatewho gives negative but logical

criticism, identifying why something

might not work.

An optimist, giving logical positivecriticism on why something might

work.

Green Hat Blue Hat

Creative, generates new possibilities

and solves problems through lateral

thinking.

Often the leader facilitating,

overseeing and organizing thinking

processes to achieve the agenda.

Role-playing hats in a group facilitates group-thinkingprocesses, as multiple perspectives can be covered [18].

Otherwise, if each individuals hat is known, those of different

hats can be grouped together to achieve a balance of thinking

types within a group [2,3,16]. By identifying onescorrelations, association, or preferences for particular hats, itfocuses and amplifies the particular preferred mode of



4

thinking. This improves communication as thinking modes are

used deliberately, giving greater freedom to the thinker toexpress thoughts through the mode chosen [18].

Research has shown that groups that are formed with just

either MBTI or six hats are more effective than groups that donot meet the criteria of the respective team formation

strategies [2]. Groups that are formed based on both MBTIand six hats results and team formation strategies aresignificantly more effective than groups formed with just

either MBTI or six hats, under certain conditions [2].

3. EXPERIMENTPrevious research shows that there are distinct differences in

the results of various ideation methods [6,7], and that adesigners personality type is correlated with different

behavioral preferences in problem solving and communication

[17,20]. Cross-correlating these two variables could help todevelop a tailored ideation method or suite of ideation

methods that reinforces the proclivity of each individual on a

team, or the team dynamics as a whole. However, to achieve

such a method, an investigation and development of quantified

model of correlations are needed.

An experiment is thus pursued to measure outcomes at theintersection of personality types and ideation methods.

Personality tests are given before participants, composed ofdesigners working on a coordinated project, arrive on site.

These tests result in both MBTI and Six Hats type indicators.

During the design challenge, participants are provided theopportunity to employ design ideation methods in a controlled

environment. Ideation outcomes are recorded for post

analysis. Several methods of ideation are employed:

brainstorming, mind-mapping and the C-Sketch method.

Brainstorming and mind-mapping are employed both

individually and in groups. The following section describesthe experimental procedures and subsequent analysis in detail.

3.1 ContextThis experiment involved deploying design ideation methods

and tracking results during a product design challenge. The

design challenge, known as Lego2Nano, was the third in a

series of China-UK Summer Schools between Tsinghua

University, Peking University and the University College

London, held on the campus of Tsinghua University. Anumber of individuals from diverse educational backgrounds

were selected to work together for five days to design and

build a low-cost Atomic Force Microscope1(AFM) suitable

for use in Chinese high schools.An aspirational theme of the challenge was to determine

how a low-cost AFM might transform science teaching inschools. To make this part of the challenge much more

1Traditionalopticalmicroscopesareunabletoresolvefeaturessmallerthanaboutonemicrometerathousandthofamillimeter thewavelengthof

visiblelight.TheAFMusesdirectphysicalcontactbetweenasharptipanda

surface todetectfeaturesonasurfacethataremuchsmallerthana

micrometer.Suchmicroscopescanevensensesingleatoms.TheAFMwas

inventedinthe1980s.Ithasprovedveryusefulformanyfieldsofresearch,

includingstudyingnewmaterialsforenergystorage,measuringimportant

biologicalaspectsofDNAmolecules,andfabricatingnoveltypesofelectronic

devices.Butthistypeofmicroscopetypicallycosts$100,000ormorefora

professionalquality

version,

and

even

so

called

"educational

models"

are

at

least$20,000.

concrete, students from local Chinese high schools were

invited to participate in the event, and were interviewed by theLego2Nano teams on the first day of the challenge, to

understand context of the high school students needs and

constraints.The Lego2Nano challenge focused on teamwork. Teams

were selected based on carefully balancing age, gender,nationality and their different technical backgrounds, as wellas personality traits such as whether individuals are intuitive

or critical, extrovert or introvert. An important focus in the

first couple of days was on activities that helped the teammembers learn about each other and appreciate their

complementary skills.Professional scientists and engineers have spent many

years trying to improve the AFM, so theres no reason to

expect that young scientists many of whom didnt evenknow what an AFM was at the outset of the event - could

complete this challenge in just one week. Still, a significant

step forward towards a low-cost AFM was made. And, at thesame time, the event represented a radically new approach to

teaching science and technology, promoting teamwork andencouraging internationalization and interdisciplinary as partof a Chinese system.

This new approach is also called XLP, short for eXtreme

Learning Process. XLP is a learning activity design

methodology intended to explore the boundaries of cognitive

capabilities of groups of people with diverse talents. XLP

activities divide participants into two groups, namelyChallenge Designers (developers and organizers) and

Missionaries (participants).

In general, Challengers first play out learning related taskson themselves, months ahead of time, so that they can assess

how much time and resources are needed to accomplish

certain tasks. After trying out the tasks and identifying at leastone feasible solution, then, Challenge Designers will prepare

the event according to the necessary success factors for an

intended audience, called the Missionaries. Challenge

Designers will stand by Missionaries during the intensive

workshop, usually four to five days. The purpose is to guideMissionaries when necessary, but not to do the design tasks

for them. Some times, Challenge Designers will serve as

technicians to help Missionaries perform certainimplementation tasks, but for a price, usually measured in

virtual currencies. A comprehensive XLP would consist of

simulated Banks, Courts, and Patent Offices. More detailedexplanation of XLP can be found in [21].

3.2 Participants



5

The challenge participants were composed of PhD and post-

doctoral students in engineering and physics. Some of theparticiapnts specialize in the development and use of

nanotechnology equipment, while others focus on graphic and

industrial design. Additionally, some of the participants haveconsiderable experience building and using nanotechnology

tools. These nanotechnologists also had minimal experience

with design methodologies which is positive, as they remainunbiased towards to any particular method due to previous or

personal experience. There were thirty-one (31) participants,

coming from top universities around the world. The designers

and nanotechnologists were distributed evenly between teams.

We were able to collect a complete set of data from twenty-five of the participants, as the process was voluntary, and

participants completed only those methods they chose toengage their efforts in. Figure 4 depicts the breakdown of

personality types.

3.3 Design ProblemThe unique design problem for this challenge was developed

as part of a goal to create a novel research collaboration

between China and the UK. Participants are aware of the fact

that their results will become very real and are thus highlymotivated to produce a novel design. For the purposes of a

challenge event, the technical challenge was divided into fiveaspects:

1.

Resolution of the force scanning device2.

Creative engineering and technical design aspects of the

device

3. Scientifically meaningful applications as supported by thedevice

4. Suitability of the device for use by high school students

5. Ingenuity in sharing and crowdsourcing the deviceproduction and its applications

The participants were given one week in which toconceive a design and produce a working prototype. They

were provided with a few basic prototyping components

including LEGO construction sets, MindStorms, a few piezocrystals, and an AFM probe or cantilever tip. For context,

although prototyping will not be discussed in this paper, thefinal prototype is shown seen under construction in Figure 5.

3.4 Description of the Workshop TutorialsThe format in which the researchers engaged in this activity

was to provide the participants with a series of instructional

videos and brief information sessions on applying ideationmethods, and then walk participants through completion of

each method. There were videos for brainstorming, mind-

mapping and C-Sketch methods. Each instructional period wastimed at ten minutes. Participants were then given fifteen

minutes to work through each of the methods.

Figure 5 Construction of a final proto type

3.5 Data RecordingThe methods were deployed in sequence during the challenge,

and participant communication was controlled, as was theamount of time allotted to each method. The sequence and

communication levels of each method are as shown in Table 1.During individual methods, participants were not allowed

to talk. During team methods, teams were not allowed to

discuss technical issues with other teams but individuals were

allowed to discuss freely within their own team. The exceptionto this approach is C-Sketch, in which communication only

occurs by passing the sheets of paper even though it is a team

exercise.

The participants were instructed to produce as many ideasas they could during the fifteen minutes allotted to each

method. Data collection occurred at the beginning of the

session and after each method, consisting of a self-efficacysurvey and collection of all concept sketches. Each individualwas provided with a uniquely colored and coded pen so that

Figure 4: Breakdown of percentage of participants by their identified dominant type (rounded to the nearest integer). The

three type sets above are indepedent, each ind ividual is represented once in each chart.



6

his or her solutions could be tracked anonymously. It was still

possible to correlate personality type to ideation solution asindividuals provided this same code on their personality

surveys.

The methods of self-efficacy assessment via surveys andraw ideation solution collection provide for an analysis of the

effectiveness of each method not only in terms of designcontent but also self-perceived ability or satisfaction. Previousstudies have shown that self-efficacy is positively related to

actual performance both in the past and future [19, 20]. Taking

a picture with a high resolution DSLR camera allowed forlater reference and analysis of the drawings.

3.6 Solution EncodingThe researchers worked to encode the solutions into a

common format to remove any biases due to handwriting

quality, or irrelevant aspects of the original drawing. Thesolutions were described using a common format and re-listed

as entries in an excel table. Images were also translated into

descriptions of solutions. A link was maintained to the original

image file for reference. For those solutions that consist of

multiple aspects, each aspect is listed as a unique entry. This

approach provides an equal comparison between a solutionthat only covered a single aspect of the problem and those that

combined multiple aspects in a single solution. For example, itwould not be equitable to rate a solution that covers probe

design and stage motion control in a single drawing

equivalently to a solution that only includes a probe design.

3.7 MetricsThe analysis of results utilizes a standard set of metrics

employed in design science literature for the purpose of

evaluating ideation outcome. These are: quality, novelty,

variety, and quantity as first introduced by Shah and adaptedby Chan et al. [1,10]. Self-efficacy was another metric used to

assess the design outcomes. In parallel both the Myers Briggs

Personality Type Indicator (MBTI) and Six Hats methodswere used to record personality type. Three raters with

background in the challenge and solutions encoded the data.

Pearsons correlation coefficient for inter-rater agreement forsolution binning by function was calculated as a 0.73 raw

score, and 1.0 after discussion and resolution of each

mismatched specific solution.

The chosen metrics were first introduced as a generic tool

to provide quantitative evaluation of creative results producedin ideation sessions and for design research [1,6,7,10]. The

metrics provide information about the performance of theindividuals during an ideation session, and overall fromcertain methods in a numerical form so that statistical analysis

can be employed to test for significance of these findings.

3.7.1 Quality. Quality is a measure of the feasibility of adeveloped design or system in question to satisfy design

requirements. For example, the challenge assessment ofquality might be a normalized measure of the resolution of a

microscope, where a microscope with a higher relativeresolution is rated with a higher quality. Since the designs in

question were at a conceptual level, experienced

nanotechnology researchers provided input on the potential

quality of each solution according to the scale in Table 5.

3.7.2 Quantity .Quantity is a direct and basic measure of thenumber of ideas produced (either in total for a single method

or by an individual). Quantity can be measured as either

unique ideas, that is, ideas that a rater determines to be uniquefunctionally with respect to other ideas; and raw quantity of

ideas, which is the total number of ideas listed during an

exercise, even including repeats. Repetition is identified

through Novelty and Variety.

3.7.3 Novelty and Variety. Variety is a measure of theexplored solution space during the idea generation process [1].

The generation of similar ideas indicates a lower variety, andtherefore corresponds to a lower probability of finding better

ideas in the possible solution space. We calculate variety, inthis study using the equation adapted from Shah by Chan et al.

for evaluation of design ideation methods [1,10]. The equation

for applying this approach to a set of ideas is stated as

(1)

where is the novelty of specific solution i; is the

total number of times a specific solution was generated forthat sub-function of the problem in the given ideation method;

and is the total number of times the specific solution to beevaluated was generated in the given ideation method.

(2)

where n, is the total number of solutions generated by an

individual with a particular ideation method.

Novelty is a measure of uniqueness of a solution [10];

and, in a complementary way, variety is a measure of the

uniqueness of a set of design solutions. Mathematically, it issimply the average novelty for a set of solutions. In our case,

solutions were collected in concept variant bins. The solutions

in a bin all perform the same function with the same basicprinciple. These are considered a specific solution. For

TABLE 5: QUALITY SCORING RUBRIC EMPLOYED DURING EVALUATION

Score Level Sublevel Examples (Control of Probe Approach)

0 Not a valid concern or idea Use magic

1 Valid idea but not implementable Low Accuracy is challenging

2 Medium Probe angle is hard to control

3 High Probe angle is hard to control due to atomic reaction forces

4 Valid idea that is implementable Low Probe angle must be controlled

5 Medium Use the same controller for probe approach angle and tapping mode scanning

6 High Use a low power high voltage controller for probe

7 Specific implementable solution Low Use Arduino to control probe

8 Medium Use an Arduino with PID function to control probe

9 High Arduino controller, linked with USB microscope and z-piezo as sensors for PID control



7

instance for the subfunction of stage scanning scheme, two

drawings from different participants depicting a probe that isfree to move in x, y, and z alongside a stage which is fixed

would be listed as the same specific solution. However, a

solution depicting a probe that moves only in z and a stagethat scans in x, and y would be a different unique solution

under this subfunction.

3.7.4 Self Efficacy Surveys. A method is also required todetermine the participants own perception of the results ofeach method. This is important not only to determine the

participants psychological reaction to the method but also as a

parallel test of the metrics. This is possible, as it has been

established that self-efficacy is correlated with actual

performance [19,20]. Paper surveys are used to establish self-efficacy. Each individual was given a single page multiple-

choice survey across ideation methods, and five minutes to

complete the survey. We asked participants to rate theeffectiveness of each ideation session in terms of quantity,

quality, novelty, and variety of the ideas they generated, as

well as the over-all usefulness of the method from theirperspective. The questions are structured as a five-point Likert

scale that ranges from Strongly Agree to Strongly

Disagree.

4. RESULTS AND DISCUSSIONOver all we find the session to have been largely productive.

We found that the 25 participants who opted in to analysis for

the study produced 1095 individual solutions and 321 uniquesolution bins. That means the average individual produced

more than 43 individual solutions and more than 12

completely unique ideas in the total 75 minutes of

brainstorming. The detailed results section consists of three

segments detailing overall trends of the ideation session,MBTI results, and finally Six hat results. We applied paired t-

test analyses for mean shift in the data results,

correspondingly, all pvalues reported in the results sectionare the significance estimates reported from this test.

4.1 GENERAL IDEATION RESULTSBefore examining comparisons between types it is important

to review results of the ideation methods as a whole. The

average results across all participants can be seen in Figure 6.

As would be expected from the literature, mind-mapping andC-Sketch were effective methods. Individual mind-mapping

scores were higher in quantity than individual brainstorming

(p = 0.008), and group brainstorming (p = 0.015). C-Sketchalso significantly outperformed individual brainstorming (p=

0.058), and group brainstorming (p = 0.064) in quantity. C-

sketch is more importantly known for permitting the

refinement and advancement of ideas, accordingly C-Sketch

saw a significant increase of the quality of ideas produced,with ideas produced in C-Sketch having a higher mean than all

other methods in quality (p



8

Figure 6 depicts the average number of new solutions

introduced per person. Comparisons must carefully account

for this. For lateral comparisons, (between types within one

method), the comparison can be direct. For longitudinal

comparisons, (between different methods within one type) thecomparison is relative to the average of each method.

4.2 MBTI COMPARISONSFor MBTI analysis, Jungs cognitive modes (Table 3) are

cross-compared as in Wildes Teamology. Studies have found

them to be the dominant indicators of individual performancein team dynamics. There are eight possible modes, however

we had a small sample size of Introverted Feeler types

(Evaluation) and Introverted Sensing types (Knowledge);

therefore we do not report results on these types. This deficit

occurred as participants elected to join this project of theirown accord and we did not have the opportunity to screen for

an even number of each type.

The results of this comparison can be seen in Figure 7.Each bar in Figure 7 is the average score for all individuals in

a given cognitive mode. Additionally, each plot represents anindependent data set. The ET-EF-IT results are separated fromthe EN-ES-IN graphs. For instance, there are no Introverted

Thinkers that are also Extroverted Feelers, but there may be

some Extroverted Feelers that are also Extroverted Intuitors;

thus, those two sets are not directly comparable.

By examining the results seen in Figure 7, a number of

insights can be found that relate the cognitive modes toideation results and the related theory. Results of the decision

making types (ET-EF-IT) will be discussed first. It would be

expected that the ET or Organizer types would score highly inmind-mapping. They do outperform IT or Analysts. IT or

Analyst types would be expected to perform most highly in C-

Sketch as it is the most analytical method. Indeed theysignificantly outperformed EF or Community types (p =0.05).

This supports the theory that Analysts excel in the solution

refinement process of C-Sketch. Lastly, what would be

expected from theory on the decision making types is for EF

or Community types to perform well in group methods. In factthey are the only type that actually did better in group

brainstorming than individual brainstorming, but not quite

significantly so (p = 0.15). With regards to quality, EF orCommunity types produced the best ideas in group mind-

mapping. This would be expected as group mind-mapping

requires a lot of group integration, their scores weresignificantly higher than for IT or Analyst types (p = 0.01).

Finally, for variety, IT or Analyst types had greater varietythan ET or Organizer types in GBS (p = 0.01), it may be that

Analyst types were more comfortable without an

organizational structure.

Quantity

Quality

Variety

NewSolutionBins

Figure 6 Total averages for all participants across theideation session. Error bars are 1 standard error . Thevertical axis is performance in each metric (see section

3.7) The horizontal axis i s the method code: I =indivudual, G = group; BS = brainstorming, MM =

mindmapping, and 635 = C-Sketch



9

Similarly, comparisons of interest can be made to the

theory for information collection types (EN-ES-IN). The ENor Ideation types would generally be expected to perform well

across the board in group or extroverted ideation methods.

There were no particular methods in which EN types did

better than other types, this may be because group methods

were placed after individual methods. IN or Imagination typeswould be expected to perform well at individual methods.

They indeed have the highest performance in C-Sketch among

the information collection group comparisons, withsignificance (p = 0.01) that they produce more ideas. This

indicates that the segmentation approach of C-Sketch allows

for individual introspective or imagination type ideators toflourish. With regards to quality, some comparisons were

significant also. ES or Experimenter types outperformed ENIdeation types in C-Sketch average quality (p = 0.02). This

could indicate that the C-Sketch process of iteratively

evaluating an idea has an aspect comparable toexperimentation. ES or Experimenter types similarly

outperformed EN or Ideation types in Variety of C-Sketch

ideas also.Table 6: Self efficacy for MBTI. Range is from 1 to 5, where

5 is strong agreement.

Information Collection Decision Making

BS MM 635 BS MM 635

p

EN/ES 0.171 0.130 0.281 EF/ET 0.482 0.400 0.088

ES/IN 0.058 0.313 0.500 ET/IT 0.001 0.144 0.049

EN/IN 0.001 0.031 0.166 EF/IT 0.012 0.090 0.417

Type

EN 3.8 3.6 3.6 EF 3.7 3.6 3.04

ES 3.5 3.3 3.1 ET 3.7 3.5 3.9

IN 2.8 3.1 3.1 IT 2.9 3.2 3.2

Self-efficacy results for the cognitive modes appear to

align with the quantitative results for EF and ET, Introverts

tended to self-assess results lower than extroverts, which may

be why IT reports a lower effectiveness of their ideation, thesame can be seen with IN types. Table 6 summarizes the

average score for self-assessed high quality, novelty and

variety of ideas in the method.

4.3 Six Hat ComparisonsSimilarly for the Six Hats type

indicators, significant differences in performance results werefound between different types. To ensure that the type sets

were independent, a similar process of separating groups wasemployed before comparison. An individual can be evaluated

as having a high score for multiple hats. Each individual must

be assessed according to their strongest hat preference to

ensure that comparisons are independent for inter-typecomparisons. However, some individuals who participated

scored a strong but equal indication for several hats. These

individuals were removed from analysis in the six hatscomparison. Once those individuals were removed from the

set, there were only enough individuals remaining to make

statistically significant comparisons between strong Yellow

Hat thinkers and strong Blue Hat thinkers. The results of thiscomparison are shown in Figure 8.

Decision Making Personality Type Set

Quantity

Quality Variety

Information Gathering Personality Type Set

Quantity Quality Variety

Figure 7 Comparison between type averages for Jungs Cognitive modes. Error bars are 1 standard error . The vertical axisis performance in each metric (see section 3.7) The horizontal axis is the method code: I = indivudual, G = group; BS =

brainstorming, MM = mindmapping, and 635 = C-Sketch


http:///reader/full/2014-ideation-methods-a-first-study-with-measured-outcomes-with-personality-type-detc2014- 10/12

10

It is clear that yellow hats are generally productive in the

brainstorming and mind-mapping techniques. As would beexpected from the theory, indicating that the characteristic of

yellow hats is to expand on existing ideas act as optimisticideators. It is also interesting to note that yellow hats produce

this high quantity in both individual and group methods. The

statistical significance of Yellow Hats produced a higherquantity of ideas than Blue hats in group brainstorming was

only p =0.069. This falls below the .05 threshold, but given

the small sample size, it is a suggestion that with a largersample size the difference may prove significant. Furthermore,

it was significant that Yellow hats produced a higher thanaverage quality in group brainstorming (p=0.047). This is also

in accordance with extant Six Hat theory on Yellow hats as

idea supporters, since group brainstorming permits piggybacking and leap-frogging.

Blue hats are characterized by preference for process

driven problem solving. It would be expected that Blue hatsperform well in 6-3-5 or C-Sketch, because it is a very

systematic method. Indeed, the mean performance of Blue

hats is higher for quantity (p=0.49), and variety (p=0.25) in635 than that of Yellow hats, but not significantly. This may

be due to the fact that Yellow hats are generally strongideators and thus comparison to the other hats is required inthe future when data is available. However, there was

significance (p= 0.036) to the difference of mean scores for

self-efficacy with Blue hats reporting an average indicationthat that they ideated more effectively than Yellow hats in C-

Sketch. On a five point Likert scale, the Blue hats listed an

average 4.0 equivalent to Agree that their ideas had high

quality, quantity, and novelty during C-Sketch. Yellow hats

only listed 3 or Neutral for C-Sketch performance. Theremainder of self-efficacy results can be seen in Table 7. Other

comparisons in Table 7 also agree with the quantitative

results.

Table 7: Self efficacy for Six Hats. Range is f rom 1 to 5,where 5 is strong agreement.

Self-Efficacy for High Quality, Novelty and

Variety of Ideas in the Method

P value Yellow Blue

Brainstorming

Methods0.360 3.1 3

Mind-Mapping

methods0.220 3.2 3

C-Sketch 0.035 3 4

5. CONCLUDING REMARKSThe results of this paper provide a clear indication that there

are significant differences in the ideation results of differentpersonality types across a set of ideation methods. It is shownthat these significant comparisons match what would be

expected from the theory of types. For example, Jungs

cognitive modes for decision-making and informationgathering can both be used to interpret the characteristics of

ideation results. This type of comparison allows a deeper

understanding of ideation suites. On one hand, an emphasis

on one ideation technique will not fully explore the potential

of a group of individuals with differing communication anddecision skills, and their ideation preferences. On the other

hand, the results open consideration of new ideation methods

or sequencing of methods that would leverage thecharacteristics of each communication style simultaneously.

6. LIMITATIONS AND FUTURE WORKThe personality type indicators have been applied in a varietyof team formation, management, and psychology contexts;

however, this study was only able to evaluate the

characteristics of MBTI and Six hats indicators as compared to

ideation method in a single design problem. In support of our

approach is the consideration that this design problem was ahighly multidisciplinary one, touching on nano-technology,

controls, kinematics, interaction design, and programming.

Additionally, the type indicators may have some imprecisionin evaluating personality type. The objective of our study was

to explore trends across a set of individuals and therefore

attempt to reduce any effects that might be a reflection of theindividual. In regards to participants, there were not enough

Quantity

Quality

Variety

Figure 8 Total averages for all participants across theideation session. Error bars are 1 standard error . Thevertical axis is performance in each metric (see section

3.7) The horizontal axis i s the method code: I =individual, G = group; BS = brainstorming, MM =

mindmapping, and 635 = C-Sketch



11

individuals of certain types and a number of unanswered

questions remain regarding the properties of those types forwhich we did not have sufficient data. Finally, there are

potentially a number of other analyses and cross-comparisons

which could be developed using data from this workshop suchas technical skill sets and other personality type assessments,

but this would exceed the space allotted to this paper toproperly examine.As with any psychological study a primary objective of

future work is to increase sample size. Additionally we hope

to explore the inter-relations with other aspects of the designprocess (such as prototyping) and personality type. It could

also be useful to allow participants significantly more time to

ideate. It has been shown that some ideation methods permit

continual production of ideas if given a longer span of time. In

general, we find that this study was a fruitful and intriguinglook into the comparison of personality and ideation and find

this research area to be open for continued efforts.

ACKNOWLEDGEMENTS

This work is supported by the Singapore University ofTechnology and Design (SUTD) and the SUTD-MITInternational Design Center (IDC, idc.sutd.edu.sg). This

project is partially supported by Chinas Natural Science

Foundation, project number: 70971073. Additionally, this

work is made possible by collaboration with Tsinghua

University in Beijing, and Center for Nano and Micro

Mechanics. The authors would also like to thank PekingUniversity and University College London participants for

their design efforts and patience in this study.

REFERENCES[1] Shah, J. J., Smith, S. M., and Vargas-Hernandez, N., 2003,

Metrics For Measuring Ideation Effectiveness, Design Studies,24(2), pp. 111-134.

[2] Jensen, D., Feland, J., Bowe, M., and Self, B., 2000, A 6-Hats

Based Team Formation Strategy: Development And ComparisonWith An MBTI Based Approach, In Proceedings of the ASEE

Annual Conference, Session 2425.[3] Wilde, D. J., 2008, Teamology: The Construction And

Organization Of Effective Teams, Springer.[4] Myers, I. B., McCaulley, M. H., and Most, R., 1985, Manual: A

Guide To The Development And Use Of The Myers-Briggs Type

Indicator, Consulting Psychologists Press, Palo Alto, CA.

[5] DeBono, E., 1985, Six Hats Thinking, Little, Brown and Co,Boston, MA.

[6] Linsey, J., Green, M. G., Murphy, J. T., Wood, K. L., andMarkman, A. B., 2005, Collaborating To Success: An

Experimental Study Of Group Idea Generation Techniques,DETC2005-85351, In ASME Design Theory and Methodology

Conference, pp. 24-28.[7] Linsey, J. S., Clauss, E. F., Kurtoglu, T., Murphy, J. T., Wood, K.

L., and Markman, A. B., 2011, An Experimental Study OfGroup Idea Generation Techniques: Understanding The Roles OfIdea Representation And Viewing Methods, Journal ofMechanical Design, 133(3), 031008.

[8] Puccio, G., and Grivas, C., 2009, Examining The RelationshipBetween Personality Traits And Creativity Styles, Creativity andInnovation Management, 18(4), pp. 247-255.

[9] Durling, D., Cross, N., and Johnson, J., 1996, Personality And

Learning Preferences Of Students In Design And Design-RelatedDisciplines, In IDATER 1996 Conference, LoughboroughUniversity, Loughborough.

[10] Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., andKotovsky, K., 2011, On The Benefits And Pitfalls Of Analogies

For Innovative Design: Ideation Performance Based OnAnalogical Distance, Commonness, And Modality Of

Examples, Journal Of Mechanical Design, 133, 081004.[11] White, C., Wood, K., and Jensen, D., 2012, From

Brainstorming to C-Sketch to Principles of Historical Innovators:Ideation Techniques to Enhance Student Creativity, Journal of

STEM Education, 13(5), pp. 12-25.

[12] Osborn, A. F., 1957,Applied Imagination, Scribner, New York.[13] Diehl, M., and Stroebe W., 1987. Productivity Loss in

Brainstorming Groups: Toward the Solution of a Riddle, Journalof Personality and Social Psychology, 53(3), pp. 497-509.

[14] Otto, K., and Wood, K., 2001, Product Design: Techniques InReverse Engineering And New Product Development, PrenticeHall, Upper Saddle River, NJ.

[15] Shah, J. J., Vargas-Hernandez, N. O. E., Summers, J. D., andKulkarni, S., 2001, Collaborative Sketching (C-Sketch) An

Idea Generation Technique For Engineering Design, TheJournal of Creative Behavior, 35(3), pp. 168-198.

[16] Jensen, D., Wood, J., and Wood, K., 2003, Hands-OnActivities, Interactive Multimedia And Improved Team

Dynamics For Enhancing Mechanical Engineering Curricula,International Journal of Engineering Education, 19(6), pp. 874-

884.[17] Shen, S. T., Prior, S. D., White, A. S., and Karamanoglu, M.,

2007, Using Personality Type Differences To Form EngineeringDesign Teams, Engineering Education, 2(2), pp. 54-66.

[18] Carl III, W. J., 1996, Six Thinking Hats: Argumentativeness

and Response to Thinking Model.[19] Bandura, A., 1971, Social Learning Theory, General Learning

Corporation, New York.[20] Locke, E. A., Frederick, E., Lee, C., and Bobko, P., 1984,

Effect Of Self-Efficacy, Goals, And Task Strategies On TaskPerformance, Journal Of Applied Psychology, 69(2), pp. 241-251.

[21] Gu, X. Y., Wang, D, Y., Zhou, S. Y., Yang, F. F, Lu, D. R.,2013, Distributed Learning Workflow: An Operating Systemthat Integrates Information Technologies and the Real Campus

Research in Higher Education of Engineering, 2013(2), pp. 72-89.

Additional Resources:

self efficacy

Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-

efficacy, and intrinsic interest through proximal self-motivation.Journal of personality and social psychology, 41(3), 586.

Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation.

Journal of personality and social psychology, 41(3), 586.Bandura, A. (1982). Self-efficacy mechanism in human agency.

American psychologist, 37(2), 122.Bandura, A. (1994). Self-efficacy. John Wiley & Sons, Inc..

Carberry, A. R., Lee, H. S., & Ohland, M. W. (2010). MeasuringEngineering Design Self-Efficacy. Journal of Engineering

Education, 99(1), 71-79.Schunk, D. H. (1995). Self-efficacy, motivation, and performance.

Journal of Applied Sport Psychology, 7(2), 112-137.Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn.

Contemporary educational psychology, 25(1), 82-91.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic

performance.Journal of educational psychology, 82(1), 33.Tsenn, J., McAdams, D., Linsey, J., A Comparison of Design Self-

Efficacy of Mechanical Engineering Freshmen, Sophomores, andJuniors, In Proceedings of the 2013 American Society for

Engineering Education Annual Conference and Exposition,Atlanta GA

ideationBlair, B. M., & Hltt-Otto, K. (2012, August). Comparing theContribution of the Group to the Initial Idea in Progressive Idea



12

Generation. In ASME 2012 International Design EngineeringTechnical Conferences and Computers and Information in

Engineering Conference (pp. 425-436). American Society ofMechanical Engineers.

Fu, Katherine, et al. "The Meaning of Near and Far: The Impactof Structuring Design Databases and the Effect of Distance of

Analogy on Design Output."Journal of Mechanical Design135

(2013): 021007.Linsey, J. S., Markman, A. B., & Wood, K. L. (2012). Design by

analogy: A study of the wordtree method for problem re-representation.Journal of Mechanical Design, 134, 041009

creativity metrics

Oman, S. K., Tumer, I. Y., Wood, K., & Seepersad, C. (2013). Acomparison of creativity and innovation metrics and samplevalidation through in-class design projects. Research in

Engineering Design, 24(1), 65-92.

Shah, J. J., Kulkarni, S. V., & Vargas-Hernandez, N. (2000).Evaluation of idea generation methods for conceptual design:effectiveness metrics and design of experiments. Journal of

Mechanical Design, 122, 377.

interaction and personality traits

Carr, P. G., De La Garza, J. M., & Vorster, M. C. (2002).

Relationship between personality traits and performance forengineering and architectural professionals providing design

services.Journal of Management in Engineering, 18(4), 158-166.Feijs, L., Kyffin, S., & Young, B. (2005, November). Design and

semantics of form and movement. InDeSForM Workshop.De Bono, E. (1995). Serious creativity. Harper Business.

De Bonos, E. (2012). Six Thinking Hats.Meneely, J., & Portillo, M. (2005). The adaptable mind in design:

Relating personality, cognitive style, and creative performance.

Creativity Research Journal, 17(2-3), 155-166.Richardson, A. L. (2008). Tinkering interactions on freshman

engineering design teams. In Proceedings of the 2008 American

Society for Engineering Education Annual Conference and

Exposition.Ullman, D. G. (1992). The mechanical design process(Vol. 2). New

York: McGraw-Hill.

2014-Ideation-Methods-A-First-Study-with-Measured-Outcomes-with-Personality-Type-DETC2014-34954...

Documents

Transcript of 2014-Ideation-Methods-A-First-Study-with-Measured-Outcomes-with-Personality-Type-DETC2014-34954...