Forecasting Technological Change (3)

04/10/23 1

Forecasting Technological ChangeForecasting Technological Change

Session 3. Expert Opinion Session 3. Expert Opinion TechniquesTechniques

Paul A. Schumann, Jr.Paul A. Schumann, Jr.

Glocal Vantage, Inc.Glocal Vantage, Inc.

04/10/23 2

SessionsSessions

• IntroductionIntroduction

• Trend Analysis TechniquesTrend Analysis Techniques

• Expert Opinion TechniquesExpert Opinion Techniques

• Integrative TechniquesIntegrative Techniques

• ClosingClosing

04/10/23 3

SessionsSessions

• IntroductionIntroduction

• Trend Analysis TechniquesTrend Analysis Techniques

• Expert Opinion TechniquesExpert Opinion Techniques

• Integrative TechniquesIntegrative Techniques

• ClosingClosing

04/10/23 4

3. Expert Opinion Techniques3. Expert Opinion Techniques

• InterviewsInterviews

• SurveysSurveys

• GroupsGroups

04/10/23 5

RationaleRationale

• Some individuals know more about a Some individuals know more about a technology or industry than otherstechnology or industry than others

• Therefore their knowledge will be betterTherefore their knowledge will be better

• However, individual experts are subject However, individual experts are subject to bias and scotomasto bias and scotomas

• Therefore multiple experts will provide Therefore multiple experts will provide better knowledge of potential futures better knowledge of potential futures

04/10/23 6

Expert Opinion Not Good for TFExpert Opinion Not Good for TF

• Single expertSingle expert

• CommitteesCommittees

• Task forcesTask forces

• Team/group meetingsTeam/group meetings

• Business meetingsBusiness meetings

04/10/23 7



• SurveysSurveys

• GroupsGroups

04/10/23 8

InterviewsInterviews

• Not market research/opinion pollNot market research/opinion poll

• A conversation with a respected expertA conversation with a respected expert

• Must approach on a professional levelMust approach on a professional level

• Use diversityUse diversity

• Can be exploratory or reactionaryCan be exploratory or reactionary

• In person better than phoneIn person better than phone

• Statistical significance not a goalStatistical significance not a goal

04/10/23 9

ConversationConversation

Values

Info

rma

tio

n

Same Different

Sam

eD

iffe

ren

t

ConverseConverse

ConverseConverse

ArgueArgue

Tem

pera

men

t

BuildBuildIntimacyIntimacy

CreativityCreativity

04/10/23 10

Johari WindowJohari Window

Self

Oth

ers

Know Don’t Know

Kn

ow

Do

n’t

Kn

ow

04/10/23 11

ConversationConversation

Interviewer

Expert

Result

04/10/23 12

TemperamentsTemperamentsObjective/External

Subjective/Internal

Pas

t

Fu

ture

SJTraditionalist

StablizerConsolidator

NFCatalyst

SpokespersonMotivator

SPTroubleshooter

NegotiatorFirefighter

NTVisionary

System ArchitectIntegrator

Pre

sen

t

Source: Grant (www.glocalvantage.com/StrategicPartners/Grant.html

04/10/23 13

InterviewsInterviews

Highly

Structured

Unstructured

Us

efu

lne

ss f

or

TF

04/10/23 14

Interview PlanningInterview Planning

• Become knowledgeableBecome knowledgeable

• Define objectivesDefine objectives

• Select expertsSelect experts– SnowballSnowball– DiversityDiversity

• Structure interviewStructure interview

• Arrange interviewsArrange interviews

04/10/23 15

Interview ImplementationInterview Implementation

• Establish professional rapportEstablish professional rapport• Explain processExplain process• Allow expert to vocalizeAllow expert to vocalize• Get into conversationGet into conversation• Keep referring back to structure & objectivesKeep referring back to structure & objectives• Take notesTake notes• Close with open ended questionClose with open ended question• Useful to have two people interviewingUseful to have two people interviewing

04/10/23 16

Interview ReportInterview Report

• Transcribe notesTranscribe notes

• Adjust interviews as necessaryAdjust interviews as necessary

• Complete all interviewsComplete all interviews

• Analyze notesAnalyze notes

• Synthesize resultsSynthesize results

04/10/23 17

Interviews SummaryInterviews Summary

• Can provide insight into change processCan provide insight into change process

• Critically dependent upon:Critically dependent upon:– Selection of expertsSelection of experts– Interview questionsInterview questions– Interview methodologyInterview methodology– Analysis & synthesis of resultsAnalysis & synthesis of results

• ExpensiveExpensive

04/10/23 18



• SurveysSurveys

• GroupsGroups

04/10/23 19

Types of SurveysTypes of Surveys

• NormalNormal• DelphiDelphi

04/10/23 20


• NormalNormal

• DelphiDelphi

04/10/23 21

Normal SurveyNormal Survey

• Qualitative or quantitativeQualitative or quantitative• For TF, quantitative is betterFor TF, quantitative is better• Typical questionsTypical questions

– In what year will ____ be commercially available in the US?In what year will ____ be commercially available in the US?– Looking at the historical trend line (shown below), in what Looking at the historical trend line (shown below), in what

year will the next advance be made and what will be the year will the next advance be made and what will be the amount of improvement?amount of improvement?

– Estimate probability and impact of an event, trend or Estimate probability and impact of an event, trend or scenario?scenario?

• Lacks the dialog of an interviewLacks the dialog of an interview

04/10/23 22

WFS SurveyWFS Survey

• 2004: Confessions to artificial intelligence priests2004: Confessions to artificial intelligence priests• 2005: Designer babies2005: Designer babies• 2010: Video tattoos2010: Video tattoos• 2012: Insect like robots used for crop pollination2012: Insect like robots used for crop pollination• 2015: ID cards replaced by biometric scanning2015: ID cards replaced by biometric scanning• 2020: Nanorobots in toothpaste attack plaque2020: Nanorobots in toothpaste attack plaque• 2025: Thought recognition becomes everyday input means2025: Thought recognition becomes everyday input means• 2030: First bionics Olympics2030: First bionics Olympics• 2030: Emotion control chips used to control criminals2030: Emotion control chips used to control criminals• 2040: Moon base size of a small village is built2040: Moon base size of a small village is built

Source: Editors, The Futurist

04/10/23 23

Global Futures ForumGlobal Futures Forum

• Expert panel survey Expert panel survey (160)(160)

• Which scenarios Which scenarios (key factors/driving (key factors/driving forces) are most forces) are most likely & impact they likely & impact they will havewill have

• Focus on business Focus on business strategystrategy

Panelists Type

Futurists50%

Industry Analysts/Consultants25%

Practitioners25%

04/10/23 24

Sample PageSample Page

Source: Global Futures Forum

04/10/23 25


• NormalNormal• DelphiDelphi

• Developed in the early Developed in the early 1950’s by Rand 1950’s by Rand Corporation for military Corporation for military applicationsapplications

• FeaturesFeatures– AnonymityAnonymity– Controlled feedbackControlled feedback– IterationIteration– Statistic group Statistic group

responseresponse

04/10/23 26

Delphi Survey ProcessDelphi Survey Process

First First RoundRound

Second Second RoundRound

ThirdThirdRoundRound

FourthFourthRoundRound

Expert Panelists

UnstructuredForecast

EventsTrends

CombinePrioritizeSummarize

Forecast time of event or trend

Statisticalsummary of the forecasts

Remain the same or change If outlier, must give reason

Statistical summarywith reasons

04/10/23 27

Conducting a DelphiConducting a Delphi

• Select experts with regard to the topic to be Select experts with regard to the topic to be forecast (15)forecast (15)– Inside vs. outsideInside vs. outside– PeersPeers– Publications, awards or patentsPublications, awards or patents

• Obtain agreement to serve on panelObtain agreement to serve on panel• Explain Delphi process completelyExplain Delphi process completely• Make the questionnaire easyMake the questionnaire easy

04/10/23 28

Conducting a Delphi (cont.)Conducting a Delphi (cont.)

• Limit number of questions (<25)Limit number of questions (<25)• Payment to panelistsPayment to panelists• Carefully word questionsCarefully word questions

– Avoid compound eventsAvoid compound events– Avoid ambiguous statement of eventsAvoid ambiguous statement of events– Avoid too little or too much information in Avoid too little or too much information in

event statementsevent statements

04/10/23 29

Conducting a Delphi (cont.)Conducting a Delphi (cont.)

• ExamplesExamples– A commercial nuclear fusion plant for generating A commercial nuclear fusion plant for generating

electricity using deuterium from sea water will electricity using deuterium from sea water will begin operation in the year ___.begin operation in the year ___.

– By the year ___, remote access computer By the year ___, remote access computer terminals will be common in private homes.terminals will be common in private homes.

– By ___, the per capita electric power consumption By ___, the per capita electric power consumption of Africa will be 25% of the US per capita of Africa will be 25% of the US per capita consumption.consumption.

04/10/23 30

GWU Forecast of Emerging TechnologiesGWU Forecast of Emerging Technologies

• Scanning & trend analysisScanning & trend analysis

• 85 prominent emerging technologies 85 prominent emerging technologies have been identifiedhave been identified

• Grouped into 12 fieldsGrouped into 12 fields

• Longitudinal Delphi type surveyLongitudinal Delphi type survey

• 50 panelists50 panelists

• Three, decade scenariosThree, decade scenarios Source: www.gwforecast.gwu.edu

04/10/23 31

GW Forecast of Emerging Technologies GW Forecast of Emerging Technologies (cont.)(cont.)

Source: www.gwforecast.gwu.edu

04/10/23 32

Japanese & German DelphiJapanese & German Delphi

• Japan has been conducting a Delphi Japan has been conducting a Delphi every five years since 1971every five years since 1971

• Joint survey in 1993Joint survey in 1993

• 405 Japanese and 459 German experts405 Japanese and 459 German experts

• Looking for cultural differencesLooking for cultural differences– Little difference in realizationLittle difference in realization– Some difference in importanceSome difference in importance

Source: www.nistep.go.jp

04/10/23 33

Importance to S&TImportance to S&T

04/10/23 34

Importance to EconomyImportance to Economy

04/10/23 35

Korean DelphiKorean Delphi

• Round 1Round 1– 25,000 experts25,000 experts– Forecast over next 20 yearsForecast over next 20 years– 5,000 experts responded5,000 experts responded– >5 suggestions per expert>5 suggestions per expert– 9,000 topics selected and arranged into 15 9,000 topics selected and arranged into 15

areasareasNote: about 75% of the ideas were different than the Japanese Note: about 75% of the ideas were different than the Japanese

surveysurveySource: Shin

04/10/23 36

Korean Delphi (cont.)Korean Delphi (cont.)

• Round 2Round 2– Selection committees formedSelection committees formed– 1,127 topics selected for second round1,127 topics selected for second round– 5,000 experts5,000 experts– 1,600 returned surveys1,600 returned surveys– Each expert responded to less than 50 Each expert responded to less than 50

questions of two areas on averagequestions of two areas on average

04/10/23 37

Korean Delphi (cont.)Korean Delphi (cont.)

• Round 3Round 3– 1,200 returned surveys1,200 returned surveys

• Purpose: Input into a national S&T planPurpose: Input into a national S&T plan• Implemented in 1994 & 1999Implemented in 1994 & 1999

04/10/23 38



• SurveysSurveys

• GroupsGroups

04/10/23 39

Group TechniquesGroup Techniques

• Nominal group conferencingNominal group conferencing

• Impact analysisImpact analysis

• Morphological analysisMorphological analysis

• Relevance treesRelevance trees

04/10/23 40






04/10/23 41

Nominal Group ConferenceNominal Group Conference• Designed to overcome unproductive aspects of Designed to overcome unproductive aspects of

face-to-face meetings while stimulating face-to-face meetings while stimulating creativitycreativity

• Ideal for 7 to 9 peopleIdeal for 7 to 9 people• Relatively quick and simpleRelatively quick and simple• Dependent on source and range of expertiseDependent on source and range of expertise• Generates more ideas than face-to-face and Generates more ideas than face-to-face and

DelphiDelphi• Requires 3 to 4 hoursRequires 3 to 4 hours

04/10/23 42

Nominal Group ProcessNominal Group Process

Pose question or Pose question or problemproblem

Select expertsSelect expertsArrange meetingArrange meeting

Explain purposeExplain purposeExplain processExplain processPose problem or Pose problem or

questionquestion

Moderator leads Moderator leads discussiondiscussion

Each idea in turnEach idea in turnNon-judgmentalNon-judgmental

Each expert given Each expert given chance to explainchance to explain

their ideatheir ideaNot justifyNot justifyIn silenceIn silence

In rotationIn rotationEach expert records Each expert records

an idea on a an idea on a chart or wall board chart or wall board

for all to seefor all to seeAll ideas recordedAll ideas recorded

Additional ideation Additional ideation encouragedencouraged

In silence (20 minutes)In silence (20 minutes)Experts record all Experts record all

the ideas or solutions the ideas or solutions they can come up withthey can come up with

BreakBreak

Moderator explains Moderator explains rating systemrating system

In silenceIn silenceEach expert rates Each expert rates each of the ideaseach of the ideas

Moderator Moderator summarizes resultssummarizes results

04/10/23 43

TF ApplicationsTF Applications

• Identify driving forcesIdentify driving forces

• Identify emerging technologiesIdentify emerging technologies

• Identify competing technologiesIdentify competing technologies

• Identify potential futuresIdentify potential futures

04/10/23 44






04/10/23 45

Impact AnalysisImpact Analysis

• Method to use experts to forecast future Method to use experts to forecast future impacts of an existing or potential event impacts of an existing or potential event or trendor trend

• 7 to 9 experts optimal7 to 9 experts optimal

• Brainstorming with structureBrainstorming with structure

• Quick and inexpensiveQuick and inexpensive

• Requires 1 to 4 hoursRequires 1 to 4 hours

04/10/23 46

Impact Analysis ProcessImpact Analysis Process

• PlanningPlanning• Structured Structured

brainstormingbrainstorming• QuantificationQuantification• ReportingReporting

04/10/23 47




• Identify event or Identify event or trendtrend

• Describe event or Describe event or trend in a short trend in a short paragraphparagraph

• Use data if existingUse data if existing• Identify and invite Identify and invite

expertsexperts• Arrange meetingArrange meeting

04/10/23 48




• Explain processExplain process• Describe event or trendDescribe event or trend• Initiate brainstormingInitiate brainstorming• Solicit and record all Solicit and record all

first order impacts first order impacts suggestedsuggested

• Use “hub and spoke” Use “hub and spoke” way of recordingway of recording

04/10/23 49




• If higher order impacts If higher order impacts are suggested, note but are suggested, note but don’t record don’t record

• Get at least 5 - 7 Get at least 5 - 7 impactsimpacts

• If biased positive or If biased positive or negative, solicit impacts negative, solicit impacts to balanceto balance

04/10/23 50




• When idea flow slows, When idea flow slows, take each of the first take each of the first order impacts and order impacts and solicit second order solicit second order impactsimpacts

• Continue process until Continue process until third (fourth) order third (fourth) order impacts are identifiedimpacts are identified

04/10/23 51

Hub & Spoke ModelHub & Spoke Model

First orderImpacts

Second OrderImpacts

Third OrderImpacts

Event orTrend

04/10/23 52

Example of Impact Analysis (partial)Example of Impact Analysis (partial)

Source: Vanston, TFI

04/10/23 53




• Although the technique Although the technique is primarily qualitative, is primarily qualitative, quantified results can quantified results can be obtainedbe obtained

• Convert “hub & spoke” Convert “hub & spoke” into a table or into a table or spreadsheetspreadsheet

• Have each expert Have each expert individually score each individually score each impact (probability, impact (probability, significance, rank, time)significance, rank, time)

04/10/23 54

Hub & Spoke ModelHub & Spoke Model

First orderImpacts

Second OrderImpacts

Third OrderImpacts

Event orTrend

0.5 10.3

0.15

04/10/23 55




• Always convert “hub Always convert “hub & spoke” into table & spoke” into table or linear flow chartor linear flow chart

• Summarize the Summarize the quantification if you quantification if you have obtained ithave obtained it

04/10/23 56






04/10/23 57

Morphological AnalysisMorphological Analysis

• A way to discover alternative paths to a A way to discover alternative paths to a potential future state or an existing statepotential future state or an existing state– TrendsTrends– EventsEvents– FutureFuture– ProductProduct– TechnologyTechnology

04/10/23 58

Morphological Analysis (cont.)Morphological Analysis (cont.)

• 7 to 9 experts7 to 9 experts

• Structured brainstormingStructured brainstorming

• Quick and easyQuick and easy

• 1 to 4 hours1 to 4 hours

04/10/23 59

Morphological Analysis ProcessMorphological Analysis Process

• PlanningPlanning• ImplementationImplementation• QuantificationQuantification• ReportingReporting

04/10/23 60




• Identify future or Identify future or present statepresent state

• Describe future or Describe future or present state in a present state in a short paragraphshort paragraph

• Use data if existingUse data if existing• Identify and invite Identify and invite

expertsexperts• Arrange meetingArrange meeting

04/10/23 61



• Describe the Describe the purpose and purpose and processprocess

• Explain the state to Explain the state to consideredconsidered

• As a group, break As a group, break the state into as the state into as many sub-states as many sub-states as necessarynecessary

04/10/23 62



• For each sub-state, For each sub-state, brainstorm all the brainstorm all the ways that sub-state ways that sub-state could become a could become a realityreality

• As a group, explore As a group, explore possible possible combinations of combinations of sub-statessub-states

04/10/23 63

Morphological Analysis ProcessMorphological Analysis ProcessPotential Future Stateor Existing State

Sub-states

Potential Ways to AchieveSub-state

04/10/23 64

ExampleExample

Source: Bright

04/10/23 65

ExampleExample

Source: Bright

04/10/23 66

ExampleExample

Source: Bright

04/10/23 67




• Although the technique Although the technique is primarily qualitative, is primarily qualitative, quantified results can quantified results can be obtainedbe obtained

• Have each expert Have each expert individually score each individually score each sub-state (probability, sub-state (probability, significance, rank, time)significance, rank, time)

04/10/23 68



• Don’t focus on Don’t focus on process, but on the process, but on the resultsresults

• Communicate as Communicate as scenariosscenarios

• Utilize probabilities Utilize probabilities (if you have them)(if you have them)

04/10/23 69

Morphological Analysis SummaryMorphological Analysis Summary

• Best useBest use– In conjunction with normative forecast to In conjunction with normative forecast to

discover road map to get to the futurediscover road map to get to the future– In conjunction with trend analysis In conjunction with trend analysis

techniques to show how progress could be techniques to show how progress could be mademade

– To find competing technologiesTo find competing technologies

04/10/23 70






04/10/23 71

Relevance TreesRelevance Trees

• A way of logically deriving the relevance A way of logically deriving the relevance of alternative paths to a futureof alternative paths to a future

• 7 to 9 people7 to 9 people

• Logical processLogical process

• Quick & easy (2 to 4 hours)Quick & easy (2 to 4 hours)

• Way to quantify the output of other TF Way to quantify the output of other TF processesprocesses

04/10/23 72

Relevance Tree - Morphological AnalysisRelevance Tree - Morphological Analysis

Criteria Weight Sub 1a Sub 1b Sub 1c

Cost 0.6 0.5 0.1 0.4

Reliability 0.3 0.1 0.6 0.3

Weight 0.1 0.2 0.3 0.5

Score 0.35 0.27 0.38

04/10/23 73

Expert OpinionExpert Opinion

• InterviewsInterviews• SurveysSurveys• GroupsGroups

Data, InsightData, InsightSurveillance Material

ChangeChangeTrend Analysis

Formal

ApplicationApplication ProductiveExpertOpinion

Glocal Vantage, Inc. 74

Glocal Vantage, Inc.Glocal Vantage, Inc.

• PO Box 161475PO Box 161475

• Austin, TX 78716Austin, TX 78716

• (512) 632-6586(512) 632-6586

• [email protected]@glocalvantage.com

• www.glocalvantage.com

• http://incollaboration.com

• Twitter: innovant2003Twitter: innovant2003

Paul SchumannPaul Schumann

• Futurist and innovation consultantFuturist and innovation consultant• Application of web 2.0 to market & strategic Application of web 2.0 to market & strategic

intelligence systemsintelligence systems• Web 2.0 tools & technologiesWeb 2.0 tools & technologies• Application of web 2.0 to democratic processesApplication of web 2.0 to democratic processes• Broad perspectives on the futureBroad perspectives on the future• ServicesServices

– Strategic market research & technology forecastingStrategic market research & technology forecasting– Intelligence systems consultingIntelligence systems consulting– Seminars, webinars & presentationsSeminars, webinars & presentations

Glocal Vantage, Inc. 75

This work is licensed under the Creative Commons Attribution

license. You may distribute, remix, tweak, and build upon this work,

even commercially, as long as you credit me for the original creation as Paul Schumann, Glocal Vantage

Inc, www.glocalvantage.com.

Forecasting Technological Change (3)

Technology

Transcript of Forecasting Technological Change (3)