10/16/02copyright Brian Williams, 20021 courtesy of JPL Diagnosing Multiple Faults Brian C. Williams...

1copyright Brian Williams, 200210/16/02courtesy of JPL

Diagnosing Multiple Faults

Brian C. Williams16.412J/6.834JOctober 16th, 2002

Brian C. Williams, copyright 2000

2copyright Brian Williams, 200210/16/02

Outline

Properties of Diagnosis Diagnostic Agents Diagnosis using Conflicts Estimating Modes Finding the Most Likely

Conflict-directed A*

10/16/02 copyright Brian Williams, 2002 3

Failures may be hidden: STS-93Symptoms:

• Engine temp sensor high• LOX level low• GN&C detects low thrust• H2 level low (???)

Problem: Liquid hydrogen leakEffect:

• LH2 used to cool engine• Engine runs hot• Consumes more LOX

Failures may be detected by qualitative arguments


• Mars Observer

courtesy of JPL

Failures are Often Novel


Multiple Faults Occur

courtesy of NASA

• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off).

• Power limitations too severe to perform new mission.

• Novel reconfiguration identified, exploiting LEM batteries for power.

APOLLO 13


• Mars Observer• . . . .• Mars Climate Orbiter• Mars Polar Lander• Deep Space 2

courtesy of JPL

Failures Are Subtle,Frequent and Remote


Mars Polar Lander Diagnosis:

• Legs deployed during descent.

• Noise spike on leg sensors latched by software monitors.

• Laser altimeter registers 50ft.

• Begins polling leg monitors to determine touch down.

• Latched noise spike read as touchdown.

• Engine shutdown at ~50ft.

Failures Involve Complex Software/Hardware Interactions


Candidate: Assignment to all component modes.

3

2

2

3

3

M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

10Adder(i): G(i):

Out(i) = In1(i)+In2(i) U(i): 12

Diagnosis identifies consistent modes

Candidate = {G(A1) & G(A2) & G(M1) & G(M2) & G(M3)}

Given Model and Observations:


Diagnosis identifies consistent modes

Adder(i): G(i):

Out(i) = In1(i)+In2(i) U(i):

Candidate: Assignment to all component modes. Diagnosis D: Candidate consistent with model Phi and

observables OBS.

3

2

2

3

3

M1

M3

A1

A

B

C

D

E

F

G

X

Y

Z

10

12

Diagnosis = {G(A1) & U(A2) & G(M1) & G(M2) & G(M3)}

Given Model and Observations:


Outline



senseP(s)

WORLD

observations actions

AGENT Diagnostic Agent:

• Monitors & Diagnoses

• Repairs & Avoids

• Probes and Tests

Plant

act

Consistency-based From Symptoms


Goal: Support programmers with Embedded Languages thatmonitor, diagnose and repair

(Model-based Programming)

Programming Diagnostic Agents

System Model

Control Program

RMPL Model-based Program

Control Sequencer

Deductive Controller

CommandsObservations

Plant

Titan Model-based Executive

State goalsState estimates

ModeEstimation

ModeReconfiguration

Trackslikely

plant states

Tracks least cost goal states

Executes concurrently Preempts Queries (hidden) states Asserts (hidden) state

ClosedClosed

ValveValveOpenOpen StuckStuck

openopen

StuckStuckclosedclosed

OpenOpen CloseClose

0. 010. 01

0. 010. 01

0.010.01

0.010.01

inflow = outflow = 0

Generates target goal statesconditioned on state estimates

Example: The model-based program sets engine = thrusting, and the deductive controller . . .

Determines that valveson the backup engine

will achieve thrust, andplans needed actions.

Deduces that a valve failed - stuck closed

Plans actionsto open

six valves

Fuel tankFuel tankOxidizer tankOxidizer tank

Deduces thatthrust is off, and

the engine is healthy


Outline



Representing Inconsistent Candidates Compactly

M1

M2

A1

A

B

C

D

E

3

2

2

3

F

G

X

Y

Z

10

Conflict

• A set of component modes M that are inconsistent with the model and observations.

• Every subset of a conflict is a conflict

• Only need conflicts that are minimal under subset

• Logically, not M is an implicate of model & Obs

6

6

12

M3

A2


??3

223

3

M1

M3

A1

A

BCD

E

F

G

X

Y

Z

10

12?

Kernel Diagnosis

= {U(A2) & U(M2)}

Representing Diagnoses Compactly

Partial Diagnosis: A set of component modes M all of whose extensions are diagnoses.

• M removes all symptoms

• M entails Model & Obs (implicant)

Kernel Diagnosis: A minimal partial diagnosis K

• M is a prime implicant of model & obs


Using Conflicts to Divide and Conquer

Given model Phi and observations OBS1. Find all symptoms and

diagnose each symptom separately

2. Merge diagnoses

General Diagnostic Engine[de Kleer & Williams, 87]


Using Conflicts to Divide and Conquer

Given model Phi and observations OBS1. Find all symptoms and

diagnose each symptom separately find minimal conflicts and constituent diagnoses

2. Merge diagnoses find kernel diagnoses

General Diagnostic Engine[de Kleer & Williams, 87]


1. Find Symptom

M1

M2

A1

A

B

C

D

E

3

2

2

3

F

G

X

Y

Z

10

Symptom: F is observed 10, but should be 12 if M1 & M2 & A1 are okay.

6

6

12

M3

A2


1. … and its Conflict

M1

M2

A1

A

B

C

D

E

3

2

2

3

F

G

X

Y

Z

Symptom: F is observed 10, but should be 12 if A1, M1& M2 are okay.

Conflict: G(A1) & G(M1) & G(M2) is inconsistent

F 10

126

6


M1

M2

A1

A

B

C

D

E

3

2

2

3

F

G

X

Y

Z

Symptom: F is observed 10, but should be 12

Conflict: G(A1) & G(M1) & G(M2) is inconsistent

U(A1) or U(M1) or U(M2) removes conflict.

10

126

6



1. Find Another Symptom

3

2

3

M1

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

10

12

4

6

10

Symptom: G is observed 12, but should be 10 ...


3

2

3

M1

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

10

12

4

6

10

Symptom: G is observed 12, but should be 10

Conflict: G(A1) & G(M2) & G(M1) & G(M3) is inconsistent

U(A1) or U(A2) or U(M1) or U(M3) removes conflict

Conflict not just upstream from symptom



Kernel Diagnoses =

2. Merge Diagnoses

U(A1) or U(M1) or U(M2) remove conflict 1.

U(A1) or U(A2) or U(M1) or U(M3) remove conflict 2

“Smallest” sets of modes that remove all conflicts

Constituent Kernel Diagnoses


Kernel Diagnoses = {U(A1)}

3. Merge Diagnoses






Kernel Diagnoses = {U(M1)}{U(A1)}

3. Merge Diagnoses






3. Merge Diagnoses



Kernel Diagnoses = {U(A2) & U(M2)} {U(M1)}{U(A1)}




Kernel Diagnoses = {U(M2) & U(M3)}{U(A2) & U(M2)}{U(M1)}{U(A1)}

3. Merge Diagnoses






Worst Case Complexity

Given model Phi and observations OBS Generate all minimal conflicts

An instance of Implicate generation - NP Hard

Generate kernel diagnosis from conflicts An instance of Implicant generation - NP Hard

Performance is a pragmatic question: How large is the set of conflicts and kernels? Is it faster to go through conflicts,

than to go directly to the kernels?


Outline



Diagnosis With Only the Unknown

Inverter(i): G(i): Out(i) = not(In(i)) U(i):

X YA B C0 00 0

Nominal and Unknown Modes

• Isolates surprises• Doesn’t explain


Diagnosis With Only the Known

Inverter(i): G(i): Out(i) = not(In(i)) S1(i): Out(i) = 1 S0(i): Out(i) = 0

X YA B C0 00 0

Exhaustive Fault Modes

• No surprises• Explains


Solution: Diagnosis as Estimating Behavior Modes

Inverter(i): G(i): Out(i) = not(In(i)) S1(i): Out(i) = 1 S0(i): Out(i) = 0 U(i):

X YA B C0 00 0

Nominal, Fault and Unknown Modes

• Isolates surprises• Explains


Example Diagnoses

X YA B C0 01

Diagnosis: [S1(A),G(B),U(C)]

0 0

Sherlock[de Kleer & Williams, 89]


Example Diagnoses

X YA B C0 01

Diagnosis: [S1(A),G(B),U(C)]

Kernel Diagnosis: [U(C)]

X YA B C0 0??

0 0

0 0

Sherlock[de Kleer & Williams, 89]


1. Find Symptoms & Conflicts

Conflict:

not G(A), G(B) and G(C)

X YA B C0 0

1 0G G

G0

01

0

0


More Symptoms & Conflicts

Not S1(A), G(B), and G(C)

X YA B C0 0

1 0S1 G

G0

01

0

0


not S0(B) and G(C)

X YA B C0 0

0S0

G0

01


0

0


not S1(C)

X YA B C0 0

1S10

0



All Conflicts

< S1(C) >

< S0(B), G(C) >

< S1(A), G(B), G(C) >

< G(A), G(B), G(C) >


2. Constituent Diagnoses from Conflicts

< S1(C) >=> G(C),S0(C) or U(C)

< S0(B), G(C) >

=> G(B),S1(B),U(B),S1(C),S0(C) or U(C)

< S1(A), G(B), G(C) >

=> G(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C) or U(C)

< G(A), G(B), G(C) >

=> S1(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C) or U(C)


3. Generate Kernel Diagnoses

[U(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

[G(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C),U(C)]

[S1(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C),U(C)]


[U(C)] [S0(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]



3. Generating Kernel Diagnoses


[U(C)] [S0(C)]

[U(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]





[U(C)] [S0(C)]

[U(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]



[S1(B),G(C)]



[U(C)] [S0(C)]

[U(B),G(C]

[S1(B),G(C)]

[U(A),G(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]





[U(C)] [S0(C)]

[U(B),G(C]

[S1(B),G(C)]

[U(A),G(B),G(C)]

[S0(A),G(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]



3. Generate Kernel Diagnoses


Outline


Computing Likelihood Conflict-directed A*


Diagnoses: (42 of 64 candidates)

Fault Models [G(A),G(B),S0(C)] [G(A),S1(B),S0(C)] [S0(A),G(B),G(C)]

. . . Fault Free [G(A),G(B),U(C)] [G(A),U(B),G(C)] [U(A),G(B),G(C)]

Partial Diagnoses [G(A),U(B),S0(C)] [U(A),S1(B),G(C)] [S0(A),U(B),G(C)]

. . .

X YA B C0 00 0


Changes to Event Space

U

Candidates withUNKNOWN failure modes

Candidates withKNOWN failure modes

Good Good

G

F1Fn

G

U


Candidate Initial (prior) Probabilities

p(c) p(m)mc

A B C

p(G) .99 .99 .99

p(S1) .008 .008 .001

p(S0) .001 .001 .008

p(U) .001 .001 .001

p([G(A),G(B),G(C)]) = .97

p([S1(A),G(B),G(C)]) = .008

p([S1(A),G(B),S0(C)]) = .00006

p([S1(A),S1(B),S0(C)]) = .0000005

Assume FailureIndependence


00.20.40.60.8

11.2

OK

S0(A)

S1(B)

S0(C)

U(A

)

U(B

)

U(C

)

S1(A),

S0(C)

X YA B C0 00


Posterior Probability, after Observation x = v

P(x=v|c) estimated using Model:

If previous obs, c and Phi entails x = vThen p(x = v | c) = 1

If previous obs, c and Phi entails x <> vThen p(x = v | c) = 0

If Phi consistent with all values for x Then p(x = v | c) is based on priors E.g., uniform prior = 1/m for m possible values of x

p(c | xv) p(x v | c)p(c)

p(x v)Bayes’Rule

Normalization Term


Observe out = 1: C = [G(A),G(B),G(C)] P(C) = .97 P(out = 1 | C) = ? = 1 P(C | out = 0 ) = ? = .97/p(x=v)


p(x v)

X YA B C 10


Observe out = 0: C = [G(A),G(B),G(C)] P(C) = .97 P(out = 0 | C) = ? = 0 P(C | out = 0 ) = ? = 0 x .97/p(x=v) = 0


p(x v)

X YA B C 00


X YA B C0

A B C

p(S1) .008 .008 .001

p(S0) .001 .001 .008

p(U) .001 .001 .001

0

How do the single faults change?• which are eliminated?• which predict observations?• Which are agnostic?

Priors for Single Fault Diagnoses:


00.20.40.60.8

11.2

OK

S0(A)

S1(B)

S0(C)

U(A

)

U(B

)

U(C

)

S1(A),

S0(C)

X YA B C0 00


0

0.1

0.2

0.3

0.4

0.5

OK

S0(A)

S1(B)

S0(C)

U(A

)

U(B

)

U(C

)

S1(A)S

0(C)

X YA B C0 00 0

Top 6 of 64 = 98.6% of P


Outline




When you have eliminated the impossible, whatever remains, however improbable, must be the truth.

- Sherlock Holmes.

The Sign of the Four.

Exploring the Improbable


Focusing on Probable Candidates: Conflict-directed A*

GenerateCandidate

TestCandidate

Consistent?Keep

ComputePosterior p

BelowThreshold?

RecordConflict

DoneYes No

Yes No

Leading CandidateBased on Priors


IncreasingCost

Feasible

Infeasible

A*


IncreasingCost

Feasible

Infeasible



IncreasingCost

Feasible

InfeasibleConflict 1



IncreasingCost

Feasible




IncreasingCost

Feasible


Conflict 1



IncreasingCost

Feasible

Infeasible

Conflict 3

Conflict 2

Conflict 1



IncreasingCost

Feasible

Infeasible

Conflict 3

Conflict 2

Conflict 1


• Feasible regions described by the implicants of known conflicts (Kernel Assignments)

•Want kernel assignment containing the best cost candidate



Given Current Conflicts: Generate best-cost kernel of conflicts Extend best kernel to best candidate Test consistency

Failure: new-conflict.


3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

Assume Independent failures:

PG(mi) >> PU(mi)

Psingle >> Pdouble

PU(M2) > PU(M1) > PU(M3) > PU(A1) > PU(A2)

Example: Diagnosis


3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

Conflicts / Constituent Diagnoses none

Best Kernel: {}

Best Candidate: ?

First Iteration


{ }

?(M1) & ?(M2) & ?(M3) & ?(A1) & ?(A2)

G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Select most likely value for unassigned modes


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12

Extract Conflict and Constituent Diagnoses:


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12



Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12


Not [G(M2) & G(M1) & G(A1)]


Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)


Not [G(M2) & G(M1) & G(A1)]

U(M2) or U(M1) or U(A1)

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12


Conflicts / Constituent Diagnoses U(M2) or U(M1) or U(A1)

Best Kernel: U(M2)

Best Candidate: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Second Iteration

3

2

2

3

3

10M1

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

6

12


Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

A1


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

6

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

610

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

610

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1

Extract Conflict:


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

610

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1

Extract Conflict:

Not [G(M1) & G(M3) & G(A1) & G(A2)]


3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

610

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

A1

Extract Conflict:

Not G(M1) & G(M3) & G(A1) & G(A2)

U(M1) or U(M3) or U(A1) or U(A2)


Conflicts / Constituent Diagnoses U(M2) or U(M1) or U(A1)

U(M2) or U(M3) or U(A1) or U(A2)

Best Kernel: U(M1)

Best Candidate: U(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Second Iteration3

2

2

3

3

10M1

M3

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

4

610

A1


Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12


3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)


3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)


3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)


Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

4


Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)

3

2

2

3

3

10

M2

M3

A1

A2

A

B

C

D

E

F

G

X

Y

Z

12

6

6

4

12

Consistent!


U(A2) U(M1)

U(M3)U(A1)U(A1), U(A2), U(M1), U(M3)

U(A1) U(M1) U(M2)

U(A1) U(M1) U(M1) & U(A2) U(M2) & U(M3)

Conflict-Directed A*: Generating Best Kernels

U(A1), U(M1) , U(M2)

Constituent Diagnoses

• Kernel assignments found by minimal set covering

• To find the best kernel, expand tree in best first order


Outline




Research in Model-based Diagnosis

Methods exist for:

• Focusing on likely diagnoses

• Active probing

• Diagnosing dynamic systems and monitoring behavior

• Repairing and compensating for failures

• Diagnosing hybrid discrete/continuous systems

• Performing distributed diagnosis

10/16/02copyright Brian Williams, 20021 courtesy of JPL Diagnosing Multiple Faults Brian C. Williams...

Documents

Transcript of 10/16/02copyright Brian Williams, 20021 courtesy of JPL Diagnosing Multiple Faults Brian C. Williams...