Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier...

19
Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky , Jonathan Sprinkle Electrical and Computer Engineering University of Arizona, Tucson, AZ [email protected] This work was supported by the AFOSR DDDAS program under grant #FA9550-15-1-0143.

Transcript of Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier...

Page 1: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

Model-based Fuzzy Logic Classifier Synthesis for

Optimization of Data-Adaptable Embedded Systems

Adrian Lizarraga, Roman Lysecky, Jonathan SprinkleElectrical and Computer Engineering

University of Arizona, Tucson, [email protected]

This work was supported by the AFOSR DDDAS program under grant #FA9550-15-1-0143.

Page 2: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Data-adaptable System Model (DASM)

• Objective: • Develop a data-adaptable modeling framework for DDDAS applications to

enable an efficient runtime framework that continually adapts, re-composes, and re-optimizes the system implementation

• Approach:• Model-based design capturing functional and non-functional application

requirements, as well as application structure• Understands how a system can be optimally composed as the availability of

sensing, computing, communication, and applications requirements change• Quantify the optimality of system compositions under various dynamic

execution scenarios

VideoCapture

Background Subtraction

Downsampling Morphological Filter

Feature Detection&Tracking

Inverse Perspective

Mapping

Position Estimation

Min. Braking Distance Calc.

Downsampling

Velocity Estimation

Acceleration Estimation

Physical Application

Attribute Constraint Calculator

Optimization Evaluator

Runtime DSE and Optimizer

VehicleControl

StateOptimize? Constraints

System Configuration

2

Page 3: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

A B D

C Latency: 100msAccuracy: 75%

Increased data size leadsto congestion in Task B

Video

GPS

A B’ D

C Latency: 35msAccuracy: 70%

Adapt implementation of TaskB to improve latency

GPS

Video

A B D

C Latency: 30msAccuracy: 75%

µP

µP

µP

µP

Video

GPS

Constraint: Latency < 38ms

Change in Data

• Task implementations should automatically adapt to changes in data quality, data availability, application requirements, and availability of computing resources

• Complexities of optimization necessitate new modeling approaches and supporting tools to facilitate design and enable runtime task adaptability

3

Data-adaptable System Modeling and Runtime Adaptation

Page 4: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

µP

µP

µP

µP

A B D

C Latency: 100msAccuracy: 75%

Increased data size leadsto congestion in Task B

Video

GPS

A B’ D

C Latency: 35msAccuracy: 70%

Adapt implementation of TaskB to improve latency

GPS

Video

A B D

C Latency: 30msAccuracy: 75%

µP

µP

µP

µP

Video

GPS

Constraint: Latency < 38ms

A B’ D’

C Latency: 37msAccuracy: 65%

Adapt implementation of Tasks Band D to improve latency

GPS

Video

Change in Data

Change in Resources

Processor core becomes unavailable

• Task implementations should automatically adapt to changes in data quality, data availability, application requirements, and availability of computing resources

• Complexities of optimization necessitate new modeling approaches and supporting tools to facilitate design and enable runtime task adaptability

4

Data-adaptable System Modeling and Runtime Adaptation

Page 5: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

µP

µP

µP

µP

A B D

C Latency: 100msAccuracy: 75%

Increased data size leadsto congestion in Task B

Video

GPS

A B’ D

C Latency: 35msAccuracy: 70%

Adapt implementation of TaskB to improve latency

GPS

Video

A B D

C Latency: 30msAccuracy: 75%

µP

µP

µP

µP

Video

GPS

Constraint: Latency < 38ms

A B’ D’

C Latency: 37msAccuracy: 65%

Adapt implementation of Tasks Band D to improve latency

GPS

Video

A B’’ D

C’’ Latency: 65msAccuracy: 80%

Adapt implementation of Task B and C to improve accuracy at the expense of latency

GPS

VideoConstraint: Latency < 70ms

Change in Data

Change in Resources

Change in Requirements

Processor core becomes unavailable

• Task implementations should automatically adapt to changes in data quality, data availability, application requirements, and availability of computing resources

• Complexities of optimization necessitate new modeling approaches and supporting tools to facilitate design and enable runtime task adaptability

5

Data-adaptable System Modeling and Runtime Adaptation

Page 6: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Video-based Vehicle Tracking and Collision Avoidance (VBVTCA)

6

• Analyze video to detect vehicles in real time

• Calculate necessary minimal travel distance (MTD) to slow down and match lead vehicle’s speed

• Implementation must adapt to:• Data: Video resolution, SNR, • Environmental conditions• Dynamic execution

requirements (e.g., residential or highway)

𝑫𝑫𝟎𝟎

Vehicle Detected

Speed Matched

Page 7: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

DASM: Modeling Application Task Flow

7

Video

Done

BSDS1 MF FDT

DownsampledVideo

Frame width Frame height

SNRLatency

DA

EADA

EA

Foreground Mask

Frame width Frame height

SNRLatency

Accuracy

DA

EADA

EAEA

DS2 IPM

PVMTDC A

• Task: Any executable computation (e.g., algorithm, filter, simulation) that consumes data and produces output. Ex: Background Subtraction (BS)

• Data types: Atomic units, or tokens, transferred between tasks. • Data attributes: Properties that describe a data type. Data attributes model the

assumptions made on the data inputs and outputs of a task. • Evaluation attributes: Models formalizing or estimating metrics to evaluate

composability of tasks. Ex: Accuracy, Latency, Uncertainty

Page 8: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

DASM: Modeling Task Options

8

• Task option: Specific implementation of a task. • For a computational task, task options represent different implementations for the

same high-level task.• For a simulation task, task options may represent different models

Video DoneBS

Gaussian Mix

Adaptive

1 Gaussian

DS1 MF FDTDS2 IPM P V MTDCA

1x

2x

4x

1x

2x

4x

720x480 1280x720

1280x960 1280x1024

1600x1200 1920x1080

704x480 None 2 pixel radius

4 pixel radius

6 pixel radius

7 pixel radius

18 pixel radius

Surf/Surf

Surf/ORB

Surf/Sift

Page 9: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

DASM: Modeling Attribute Transforms

9

Task Option 1

Frame Width DAT

SNR EAT

. . .

Input Data

Frame width Frame height

SNRLatency

DA

EADA

EA

Output Data

Frame widthFrame heightSNRLatency

DA

EADA

EA

• Attribute Transforms: Model or estimate of evaluation attributes for outputs based on input data/evaluation attributes.

• Distinguish between data attribute transforms (DAT) and evaluation attribute transforms (EAT)

• Enable end-to-end evaluation of attribute values• Used during design space exploration to evaluate system compositions

𝑆𝑆𝑆𝑆𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜 = 𝑆𝑆𝑆𝑆𝑆𝑆𝐼𝐼𝐼𝐼 + 20 log 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑

Page 10: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

DASM: Modeling Attribute Guards

10

Foreground Mask

Frame width Frame height

SNRLatency

DA

EADA

EA

Task Option 1

Gaussian Mixtures Background Subtraction

Adaptive Background Subtraction

Task Option N

Downsampled Video

Frame width Frame height

SNRLatency

DA

EADA

EA

Frame width: [320, 1080]Frame height: [640, 1920]SNR: [5 dB, 22dB]

Attribute Guard 1DADAEA

• Attribute Guards: Define composability of each Task Option

• Semantic Composability: Captures the composability of specific task options• Prevent use of a task option that is invalid/incompatible for some data attribute

values

• Programmatic Composability: Capture designer knowledge of suitability of task compositions

• Can formalize (or estimate) the propagation of programmatic composability

Page 11: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Model-guided Genetic Optimization Algorithm

11

• Model-guided (MG) Genetic Optimization:• Uses model information (attribute guards) to determine compatibility between task

implementations, based on the task-flow model composed by the designer• MG Population Generation: generates semantically valid yet diverse

configurations• MG Crossover: ensures semantically valid offspring configurations• MG Mutation: generates mutated offspring maintaining validity

VideoCapture

Background Subtraction

Downsampling Morphological Filter

Feature Detection&Tracking

Inverse Perspective

MappingPosition

EstimationMin. Braking

Distance Calc.

Downsampling

Velocity Estimation

Acceleration Estimation

Physical Application

Attribute Constraint Calculator

Optimization Evaluator

Runtime DSE and Optimizer

VehicleControl

StateOptimize? Constraints

System Configuration

Page 12: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Initial Model-guided Genetic Optimization Algorithm

60

70

80

90

100

0 20 40 60 80 100 120 140 160 180 200

Acc

urac

y (%

)

Generation

Optimize Accuracy [Latency < 847 ms]

model-guided standard

0

50

100

150

0

2

4

6

8

1% 2% 3% 4% 5%

Gen

erat

ions

Spee

dup

Percent of Optimal

Optimize Accuracy [Latency < 847 ms]MG gen. STD gen. Speedup

• Model-guided genetic algorithm finds optimal solution faster• Initial fitness improvements of 18.5% (up to 23.3% in different scenario)• 6.5X speedup, up to 26X in different scenario, and up to 544X compared to exhaustive

search• Standard genetic algorithm may fail to find optimal with quality constraints

• Challenges• Dynamically changing requirements necessitate adapting optimization goals• Multi-objective optimization will yield Pareto optimal compositions

12

Page 13: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

• Fuzzy Design Metric Classifications:• Define the acceptable or unacceptable values for each high-level metric

• Fuzzy Design Fitness Rules:• Define the relative importance of each metric within a particular

application• IF L IS F OR G OR S, AND A IS S, THE DESIGN IS S• IF L IS F OR G OR S, AND A IS G, THE DESIGN IS G• IF L IS F OR G OR S, AND A IS F, THE DESIGN IS F• IF L IS U, AND A IS F OR G OR S, THE DESIGN IS U

Specifying Dynamic Fitness for Optimization

• Weighted Functions:

• Difficult to determine appropriate weights.

100%80%60%40%20%

0%

Latency (sec)847 200 32 0

100%80%60%40%20%

0%

Accuracy (%)0 60 70 90 100

GoodFair

Superior

Legend

Unacceptable

13

Page 14: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

DASM: Modeling Fuzzy Logic Classification Synthesizer

• Fuzzy classification functions should change dynamically• Meaning of a “Good” latency changes as vehicle speeds change• Ex: A latency of 750ms may be Fair in a residential area, but may be Unacceptable

in a highway scenario (750ms equates to more than 20 meters of uninformed, i.e., “blind”, driving)

• Fuzzy Logic Classification Synthesizer model • Classification Attribute Transform: Designer-specified transform that can

estimate/produce a classification function based on EAs.• Ex: Latency CAT determines latency classification boundaries by calculating the

maximum tolerable latency, LatencyMAX

• Then, Unacceptable classification is any latency > LatencyMAX14

Attributes

PositionP

VelocityV

AccelA

EAEA

EA

Fuzzy Logic Classification Synthesizer

Latency CAT

Accuracy CAT

. . .

Latency Fuzzy Classification

Accuracy Fuzzy Classification

U/F F/G G/S S

U/F F/G G/S S

Page 15: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Runtime Optimization Framework

Using designer-specified models, runtime framework generates fuzzy classification functions at runtime

Fuzzy classification functions make “constraints” redundant Ex: A latency constraint can be converted to a range of “unacceptable”

values.

15

Video BSDS1 MF

FDTIPMPMTDC

DS2

VA

Application Model

Fuzzy Logic Classification

Synthesis

StateFuzzy Metric

Classification Functions

System Configuration

Runtime DSE and Optimizer

Fuzzy Fitness EvaluationDSE

• IF L IS F OR G OR S, AND A IS S, THE DESIGN IS S• IF L IS F OR G OR S, AND A IS G, THE DESIGN IS G• IF L IS F OR G OR S, AND A IS F, THE DESIGN IS F• IF L IS U, AND A IS U OR F OR G OR S, THE DESIGN IS U

100%

80%

60%

40%

20%

0%

Latency (sec)

847 200

32 0

100%

80%

60%

40%

20%

0%

Accuracy (%)

0 60 70 90 100

Page 16: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Experimental Results: Fuzzy vs Linear Weighted Function

Scenario ΔSpeed (mph)

Latency Fuzzy Classification Boundaries Accuracy Fuzzy Classification Boundaries

U/F F/G G/S S U/F F/G G/S S1 4 847 200 32 0 60% 70% 90% 100%2 8 749 200 32 0 50% 60% 85% 100%3 14 619 200 32 0 50% 60% 80% 100%4 16 508 200 32 0 50% 60% 75% 100%5 23 235 200 32 0 50% 60% 70% 100%

16

-50%

-25%

0%

25%

50%

75%

100%

Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5

Piecewise Linear VS Fuzzy Logic Based Optimization

Latency Accuracy Fitness

• Fuzzy based approach produced configurations that sacrificed latency by 8.4%,

• BUT, accuracy and total system fitness are both improved by 25.7% and 67.3%, respectively

Page 17: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

-80.0%

-40.0%

0.0%

40.0%

80.0%

Config 1 Config 2 Config 4 Config 5

Scenario 3 Latency Accuracy Fitness

-100%-80%-60%-40%-20%

0%20%40%

Config 2 Config 3 Config 4 Config 5

Scenario 1 Latency Accuracy Fitness

Experimental Results: Dynamic Composition and Optimization

• Determined the optimal configuration for each execution scenario (e.g., configuration 1 is optimal in scenario 1)

• Evaluated the performance of each configuration in the other scenarios

Improvement in latency come at the expense of lower accuracy and overall system fitness

Configurations 4 and 5 use an Unacceptable (unsafe) resolution to meet the stricter latency constraints in the respective scenarios.

17

X X

X X

Configurations 1 and 2 have Unacceptable latency

Page 18: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT

Conclusions and Future Work

Conclusions Presented modeling and optimization extensions enabling intuitive tradeoff

specifications for competing design requirements and optimization goals Fuzzy Logic Classification Synthesizer model enables runtime synthesis of

fuzzy metric classification functions DASM modeling environment supports the specification of fuzzy design

fitness rules to define the relative importance of competing metrics when determining overall system fitness

Fuzzy logic based optimization achieves compositions that tradeoff unneededlatency reduction for improvements in accuracy and overall system fitness

Future Work Formalize opportunity cost of system reconfigurations

Investigate methods to understand when dynamic adaption is advantageous/disadvantageous subject to system performance constraints

• Runtime adaptive instrumentation and performance models• Online, low overhead profiling to measure actual data/evaluation attributes

and refine/train evaluation models• Validation and Verification with Physical Hardware

• Sensing and video in-the-loop with Robotic Ford Escape

18

Page 19: Model-based Fuzzy Logic Classifier Synthesis for ... · Model-based Fuzzy Logic Classifier Synthesis for Optimization of Data-Adaptable Embedded Systems Adrian Lizarraga, Roman Lysecky,

R. Lysecky, August 11, 2016 InfoSymbiotics/DDDAS 2016, Hartford, CT19

Thank You

This work was supported by the AFOSR DDDAS program under grant #FA9550-15-1-0143.