Web-based knowledge elicitation and application to planned experiments for

product development

Sue Lewis

University of Southampton, UK

David Dupplaw, David Brunson,

Anna-Jane Vine, Colin Please, Angela Dean, Andy Keane, Marcus Tindall

EPSRC, Jaguar Cars, Hosiden Besson, Goodrich Engine Control Systems

S.M.Lewis@maths.soton.ac.uk

Outline

1. Background- conventional versus web-based approach

2. Design of experiments- screening many factors

3. Methodology

4. Software implementation

5. Case study- Jaguar Cars- optimisation of cold start engine

performance

Knowledge elicitation

• List of factors to be varied/ held constant

• Available knowledge on how factors affect performance

Knowledge elicitation

Two-stage group screening

• aims to find factors whose effects are sufficiently large to produce a substantive improvement in performance

• each factor is investigated at two levels

- “high” and “low”

• factors are of two kinds: control (design) or noise

• factors divided into groups

• factors within each group are varied together

- new grouped factor

Two-stage group screening

Stage 1 experiment– on small number of grouped factors– decide which groups are important

Stage 2 experiment– on individual factors from important

groups– estimate main effects and interactions

Strategies: at stage 1– factor main effects only

• classical group screening– both main effects and interactions

• interaction group screening

Criteria for choosing a grouping and strategy

1. Total number of runs (Stage 1 + Stage 2)– total no. S of main effects and interactions

that have to be examined by experiment

Want E(S) as small as possibleAlso P(S>target) as small as possible

2. Risk of failing to detect important main effects and interactions– as small as possible

• tension between these aims• software to guide our choice

Software implementation • central server - Linux and Windows XP• all interactions through a web browser

- Internet Explorer, Netscape, Opera• different levels of user

- administrators- users- guests

• uses open source software• requires installation of

Apache web server, MySQL, PHP tools, grouping and simulation code

Methodology

Setup (administrator)Eg define performance measure and initial factors Acquisition of information (users and guests)Eg elicit factor importance, experts’ confidence, new

factors

Summary of importance (admin. and users)Eg guide factor groupings

Choice of groupings and strategy (admin. and users)Eg consider probability distribution of total size of

experiment

Simulation (admin. and users)Eg examine how often important effects may be missed

Questionnaire

Questionnaire

Questionnaire

Information on 4 of the 5 factors complete – Portability not yet considered

Questionnaire

Questionnaire summary

Input of group sizes and groupings for assessment by software

Comparison of 4 strategies under the criterion minimize P(S>target)

Simulation software then run to calculate the proportions of times important effects

are missed

Case Study

12 control or design factors– AFR– spark time– calibration– engine off timing– idle speed– plug type– injection timing– spark advance– transient fuel with

calibration– plug gap– variable valve timing– injector spray angle and

direction

2 noise factors- injector tip variation- humidity/temperature

Performance measure: given no. of engine cycles with a low spark resistance

Planning the experiment: size of S

Grouping

0.010.1557.25102.3322,2,2,12,3

P(S>120)P(S>110)Var(S)E(S)

2 indiv noise

7 indiv des

5 v. likely ind design

Probability of 5 very likely design main effects being active = 1.0

Distribution of S

Risk of exceeding a target

Planning the experiment: risks

2 – 15 52 – 56 design x noise interactions

3 – 24 2 – 18 design x design interactions

00 design main effects

separategrouped

Noise factors

Use simulation to calculate percentages of active effects missed

Conclude: keep noise factors separate –not grouped together

Choosing a strategy: risks

2 – 15 71 – 73 design x noise interactions

3 – 24 69 – 70 design x design interactions

00 – 1individual design main effects

IGSCGS

Strategy

Percentages of active effects missed

CGS misses more active individual effects than IGS

Conclusions

• Group screening methodology defined• Software implementation validated in

industry• Software at beta testing stage

- enquiries and feedback welcome

- S.M.Lewis@maths.soton.ac.uk