Oded Maler Barbaric Science from a Captive...

Oded MalerBarbaric Science

from a Captive Poet

Our Work with Oded Maler and his Impact

Manfred Morari, UPenn and ETH

HSCC Montreal April 18, 2019

Research at Caltech and ETH around 1997

Automatica 35 (1999) 407—427

Control of systems integrating logic, dynamics, and constraints1

Alberto Bemporad, Manfred Morari*Institut fu( r Automatik, ETH - Swiss Federal Institute of Technology, ETHZ - ETL, CH 8092 Zu( rich, Switzerland

Received 17 March 1998; received in final form 16 September 1998

Systems described by interdependent physical laws, logic rules, and operating constraintsare described by linear equations and inequalities involving continuous and integer variables. Modelpredictive control based on mixed-integer quadratic programming provides a systematic controllersynthesis procedure.

Abstract

This paper proposes a framework for modeling and controlling systems described by interdependent physical laws, logic rules, andoperating constraints, denoted as mixed logical dynamical (MLD) systems. These are described by linear dynamic equations subject tolinear inequalities involving real and integer variables. MLD systems include linear hybrid systems, finite state machines, some classesof discrete event systems, constrained linear systems, and nonlinear systems which can be approximated by piecewise linear functions.A predictive control scheme is proposed which is able to stabilize MLD systems on desired reference trajectories while fulfillingoperating constraints, and possibly take into account previous qualitative knowledge in the form of heuristic rules. Due to thepresence of integer variables, the resulting on-line optimization procedures are solved through mixed integer quadratic programming(MIQP), for which efficient solvers have been recently developed. Some examples and a simulation case study on a complex gas supplysystem are reported. ! 1999 Elsevier Science Ltd. All rights reserved.

Keywords: Hybrid systems; Predictive control; Dynamic models; Binary logic systems; Boolean logic; Mixed-integer programming;Optimization problems

1. Introduction

The concept of model of a system is traditionally asso-ciated with differential or difference equations, typicallyderived by physical laws governing the dynamics of thesystem under consideration. Consequently, most of thecontrol theory and tools have been developed for suchsystems, in particular for systems whose evolution isdescribed by smooth linear or nonlinear state transitionfunctions. On the other hand, in many applications thesystem to be controlled is also constituted by parts de-

*Corresponding author. Tel.:#41 1 632 7626; fax:#41 1 632 1211;e-mail: [email protected].

!This paper was not presented at any IFAC meeting. This paperwas recommended for publication in revised form by Associate EditorA. L. Tits under the direction of Editor T. Bas,ar, and accepted by theGuest Editors J. M. Schumacher, A. S. Morse, C. C. Pantelides, andS. Sastry.

scribed by logic, such as for instance on/off switches orvalves, gears or speed selectors, evolutions dependent onif-then-else rules. Often, the control of these systems is leftto schemes based on heuristic rules inferred by practicalplant operation.

Recently, in the literature researchers started dealingwith hybrid systems, namely hierarchical systems con-stituted by dynamical components at the lower level,governed by upper level logical/discrete components(Grossmann et al., 1993; Branicky et al., 1998). Hybridsystems arise in a large number of application areas, andare attracting increasing attention in both academic the-ory-oriented circles as well as in industry. Our interest ismotivated by several clearly discernible trends in theprocess industries which point toward an extended needfor new tools to design control and supervisory schemesfor hybrid systems and to analyze their performance.

For this class of systems, design procedures have beenproposed which naturally lead to hierarchical, hybrid

0005-1098/99/$—see front matter ! 1999 Elsevier Science Ltd. All rights reservedPII: S 0 0 0 5 - 1 0 9 8 ( 9 8 ) 0 0 1 7 8 - 2

Automatica 35 (1999) 407—427

Control of systems integrating logic, dynamics, and constraints1

Alberto Bemporad, Manfred Morari*Institut fu( r Automatik, ETH - Swiss Federal Institute of Technology, ETHZ - ETL, CH 8092 Zu( rich, Switzerland

Received 17 March 1998; received in final form 16 September 1998

Systems described by interdependent physical laws, logic rules, and operating constraintsare described by linear equations and inequalities involving continuous and integer variables. Modelpredictive control based on mixed-integer quadratic programming provides a systematic controllersynthesis procedure.

Abstract

This paper proposes a framework for modeling and controlling systems described by interdependent physical laws, logic rules, andoperating constraints, denoted as mixed logical dynamical (MLD) systems. These are described by linear dynamic equations subject tolinear inequalities involving real and integer variables. MLD systems include linear hybrid systems, finite state machines, some classesof discrete event systems, constrained linear systems, and nonlinear systems which can be approximated by piecewise linear functions.A predictive control scheme is proposed which is able to stabilize MLD systems on desired reference trajectories while fulfillingoperating constraints, and possibly take into account previous qualitative knowledge in the form of heuristic rules. Due to thepresence of integer variables, the resulting on-line optimization procedures are solved through mixed integer quadratic programming(MIQP), for which efficient solvers have been recently developed. Some examples and a simulation case study on a complex gas supplysystem are reported. ! 1999 Elsevier Science Ltd. All rights reserved.

Keywords: Hybrid systems; Predictive control; Dynamic models; Binary logic systems; Boolean logic; Mixed-integer programming;Optimization problems

1. Introduction

The concept of model of a system is traditionally asso-ciated with differential or difference equations, typicallyderived by physical laws governing the dynamics of thesystem under consideration. Consequently, most of thecontrol theory and tools have been developed for suchsystems, in particular for systems whose evolution isdescribed by smooth linear or nonlinear state transitionfunctions. On the other hand, in many applications thesystem to be controlled is also constituted by parts de-

*Corresponding author. Tel.:#41 1 632 7626; fax:#41 1 632 1211;e-mail: [email protected].

!This paper was not presented at any IFAC meeting. This paperwas recommended for publication in revised form by Associate EditorA. L. Tits under the direction of Editor T. Bas,ar, and accepted by theGuest Editors J. M. Schumacher, A. S. Morse, C. C. Pantelides, andS. Sastry.

scribed by logic, such as for instance on/off switches orvalves, gears or speed selectors, evolutions dependent onif-then-else rules. Often, the control of these systems is leftto schemes based on heuristic rules inferred by practicalplant operation.

Recently, in the literature researchers started dealingwith hybrid systems, namely hierarchical systems con-stituted by dynamical components at the lower level,governed by upper level logical/discrete components(Grossmann et al., 1993; Branicky et al., 1998). Hybridsystems arise in a large number of application areas, andare attracting increasing attention in both academic the-ory-oriented circles as well as in industry. Our interest ismotivated by several clearly discernible trends in theprocess industries which point toward an extended needfor new tools to design control and supervisory schemesfor hybrid systems and to analyze their performance.

For this class of systems, design procedures have beenproposed which naturally lead to hierarchical, hybrid

0005-1098/99/$—see front matter ! 1999 Elsevier Science Ltd. All rights reservedPII: S 0 0 0 5 - 1 0 9 8 ( 9 8 ) 0 0 1 7 8 - 2

Research at Caltech and ETH around 1997Precursors to

• LTL specifications for controller synthesis • Tabuada, Pappas; Linear Time Logic Control of Discrete-Time Linear Systems, IEEE TAC

2006

• MPC controller design with stability and performance guarantees for linear switched (hybrid) systems• Borrelli, Constrained Optimal Control of Linear and Hybrid Systems, Springer, 2003• Borrelli, Bemporad, MM; Predictive Control for Linear and Hybrid Systems, Cambridge

2017• Marcucci & Tedrake, Mixed Integer Formulations for Optimal Control of Piecewise-Affine

Systems, HSCC 2019

Then we discoveredHybrid Systems...

…and met Oded

Oded Maler, 1957-2018

• 1957 born in Haifa• 1976-79 BA Computer Science, Technion• 1980-84 MSc Management Science, Tel-Aviv University

• “Evidence Propagation and Control Strategies in Bayesian Inference Networks” w/ Moshe Ben-Bassat

• 1986-90 PhD Computer Science, Weizmann Institute• “Finite Automata: Infinite Behavior, Learnability and Decomposition” w/ Amir Pnueli

• 2000 Habilitation, Univ. J. Fourier, Grenoble• “Discrete,Timed and Hybrid Systems”

• 1992- Researcher, Research Director, CNRS, VERIMAG


• Postdocs (12)• Peter Niebert, Hou Jianmin, Goran Frehse, Antoine Girard, Gregory Batt,

Victor Schuppan, Ramzi Ben Salah, Alexandre Donze, Scott Cotton, JianshengXing, Stefano Minopoli, Marcelo Forets

• PhD Students (19)• Thao Dang, Yasmina Abdeddaïm, Moez Mahfoudh, Abdelkarin Kerbaa,

Alexandre Donze, Ramzi Ben Salah, Dejan Nickovic, Scott Cotton, Colas Le Guernic, Aldrich Degorre, Julien Legriel, Selma Saidi, Jean Francois Kempf, Pranav Tendulkar, Jan Lanic, Thomas Ferrere, Abhinav Srivastav, Inrini-Eleftheria Mens, Dogan Ulus


• Numerous courses, organization of workshops, invited presentations, research projects• 1997 – 2000: Coordinator the Esprit/LTR project 26270 VHS

(Verification of Hybrid Systems), with the participation of UJF-Verimag, Dortmund, Nijmegen, Aalborg-Brics, Kiel, INPG-LAG, Weizmann, CWI, Ghent, Uppsala, ETH-Zurich, Nylstar SA, Sidmar, Krupp-Uhde. • 2002 – 2005: Coordinator of the project IST 33520 CC (Control and

Computation) with the participation of INPG-Verimag, ETHZ, Parades, Lund, CWI, Sienna, EDF and ABB.

• Subject: More place on the Titanic, Date: Oct. 27, 1998

Dear Alberto [Bemporad] and Manfred, Thank you for showing interest in our project. On one hand I would be glad to cooperate with you. On the other I am not very enthusiastic about re-entering into the process of proposal editing, partners consultation, Brussels discussions, and all that. We have just added a 13th partner recently and I am not sure how authorities will react to further modification. If you are not convinced by these humanitarian arguments, I can try to ask the other partners concerning their attitude as well as the European project officer….

VERIMAGUNITE MIXTE DE RECHERCHE

Centre Equation2 avenue de Vignate38610 GIERESTel. +33 4 76 63 48 48Fax +33 4 76 63 48 50

Universite Joseph FourierCentre National de la Recherche Scientifique Institut National Polytechnique de Grenoble

Work Plan: Project 26270 VHS (Verification of Hybrid Systems)• Case Studies supplied by Dortmund and [chemical] industry• Comparative study of models, techniques, tools from the verification

community• Theoretical and practical conclusions from the experience leading to

improvements of models and tools• Surveying the current practice of control software development in the

industry and suggesting improvements based on formal and semiformal methods

Discrete-Time Hybrid Modeling and Verification

Fabio Danilo Torrisi1 and Alberto Bemporad2,1

Abstract

For hybrid systems described by interconnections of lin-ear dynamical systems and logic devices, we recentlyproposed Mixed Logical Dynamical (MLD) systemsand the language HYSDEL (Hybrid System Descrip-tion Language) as a modeling tool. For MLD models,we developed a reachability analysis algorithm whichcombines forward reach-set computation and feasibil-ity analysis of trajectories by linear and mixed-integerlinear programming. In this paper the versatility ofthe overall analysis tool is illustrated on the verifica-tion of an automotive cruise control system for a carwith robotized manual gear shift.

1 Introduction

Hybrid models describe processes that evolve accordingto dynamic equations and logic rules [1]. The interestin hybrid systems has grown over the last few yearsnot only because of the theoretical challenges, but alsobecause of their impact on applications, for instancein the automotive industry. A typical example of hy-brid systems are real-time systems, where physical pro-cesses are controlled by embedded controllers. For thisreason, it is important to have tools to guarantee thatthis combination behaves as desired. Formal verifica-tion is aimed at providing such a certification. Thisamounts to solving the following reachability problem:For a given set of initial conditions and disturbances,guarantee that all possible trajectories never enter aset of unsafe states, or possibly provide a counterexam-ple. Unfortunately, it is well known that formal veri-fication of hybrid systems is an undecidable problem.In spite of this complexity, several tools for formal ver-ification of hybrid systems have been proposed in theliterature [2, 8, 10,12,16].

Formal verification is strictly related to the modelingframework used to describe the process, whose safetyproperties we need to certify. Different models lead todifferent verification algorithms. Timed automata [8]and hybrid automata [12], for instance, were proved tobe successful modeling frameworks and led to very effi-cient verification tools based on symbolic computation.

Recently, we proposed an algorithm [6, 7] based onmathematical programming for solving safety analysisproblems on Piecewise Affine (PWA) and Mixed Logi-cal Dynamical (MLD) systems as an alternative to other

1Automatic Control Laboratory, ETH Zentrum - ETL,8092 Zurich, Switzerland torrisi,[email protected]://control.ethz.ch/~hybrid

2Dip. Ingegneria dell’Informazione, Universita di Siena, ViaRoma 56, 53100 Siena, Italy [email protected]

HYSDEL Model

MLD + PWA Model

Reachability Analysis

Process

Safety Requirements Safety Properties

Figure 1: Verification of hybrid systems: The process ismodeled in HYSDEL. The model is automat-ically translated it into MLD and PWA form,and used for proving safety properties

modeling frameworks. PWA and MLD systems are for-mulated in discrete time. Despite the fact that theeffects of sampling can be neglected in most applica-tions, subtle phenomena such as Zeno behaviors [14]cannot be captured in discrete time. Nevertheless, adiscrete-time formulation allows one to develop numeri-cally tractable schemes for solving complex analysis andsynthesis problems. Several questions related to MLDsystems can indeed be suitably formulated as mixed-integer linear/quadratic optimization problems, such ascontrol [5] and scheduling problems [4].

After reviewing the basics of PWA and MLD systems,we briefly recall the reachability analysis algorithm de-scribed in [6], and address the formal verification prob-lem of an automotive control system for automatizedgear-shift. We model the automotive system in our lan-guage HYSDEL to obtain an MLD system. For formalverification, we adopt the algorithm described in [6].The overall modeling and verification procedure is de-picted in Figure 1.

2 PWA Systems, MLD Systems, and HYSDEL

PWA systems [11] are defined by partitioning the statespace into polyhedral regions, and associating with eachregion a different linear state-update equation

x(t + 1) = Ai(t)x(t) + Bi(t)u(t) + fi(t)

if x(t) ∈ Ci(t) ! {x : Hi(t)x ≤ Ki(t)}(1)

where x ∈ X ⊆ Rn, u ∈ U ⊆ Rm, {Ci}s−1i=0 is a poly-

hedral partition1 of the sets of states2 X, the matri-ces Ai, Bi, fi, Hi, Ki are constant and have suitable

1The double definition of the state-update function over com-mon boundaries of sets Ci (the boundaries will also be referred toas guardlines) is a technical issue that can be resolved by allow-ing a part of the inequalities in (1) to be strict. However, from anumerical point of view, this issue is not relevant.

2More generally, Ci are subsets the of state+input spaceRn+m, for instance when the PWA system is obtained as anequivalent representation of an MLD system [11].

ThA12-6

2899



Abstract


1 Introduction






HYSDEL Model

MLD + PWA Model


Process













ThA12-6

2899



Abstract


1 Introduction






HYSDEL Model

MLD + PWA Model


Process













ThA12-6

2899

40th IEEE CDC, 2001

Also: European Journal of Control (2001)7:382-399

!"#$%"&&'% '#$'%( $)' *&*%+ +",' *#, -++',-*$'&."/'#( 0*&0' !"# $" '+/$. $*#1 2345%-#6 $)-( /)*('7 $)')'*$'% -( ($-&& -# $)' !"#$%&' ($*$'3 8)'# $)' $'+/'%*9$5%' -# $*#1 : %'*!)'( $)' *&*%+ &'0'& !" ;#(#)*<7 $)')'*$-#6 -( (=-$!)', ">> *#, $)' !"#$%"&&'% '#$'%( $)'($*$' +,,(%&'3 ?-#*&&.7 =)'# $*#1 2 -( '+/$.7 $)' !"#9$%"&&'% 6'$( -# $)' ($*$' -)#%&%&'7 =)'%' 0*&0' !"# -(!&"(', *#, 0*&0' !"$ -( "/'#',7 *#, $)' ("&5$-"# >&"=(>%"+ $)' 5//'% $*#1 $" $)' &"='% $*#13 ?%"+ -)#%&%&'7$)' !"#$%"&&'% !*# '-$)'% (=-$!) $" $)' ($*$'.,& -> $*#1: @'!"+'( '+/$.7 "% $" (,/$ -> $)' $'+/'%*$5%' -# $*#1 :@'!"+'( &"='% $)*# $)' !%-$-!*& 0*&5' !#3

A)' *-+ -( $" 0'%->. $)*$ $)' !"#$%"&&'% (*$-(>-'( $)'>"&&"=-#6 (*>'$. %'B5-%'+'#$(C ;-< -> * +*&>5#!$-"#-#6-# $)' !""&-#6 (.($'+ "> $)' !"#,'#('% "!!5%(7 $)')'*$'% +5($ @' $5%#', ">> B5-!1&. '#"56) $" /%'0'#$,*+*6'( $" $)' !"#,'#('%7 ;--< $)' ("&5$-"# -# $*#1 : -(

,%*-#', $" $*#1 2 @'>"%' -$ ("&-,->-'(7 ;---< =)'# $)'0*&0' D:E -( "/'# $*#1 2 -( '+/$.3

F'%$->.-#6 $)*$ $)' GHF !",' (*$-(>-'( $)'(' (/'!-9>-!*$-"#( *+"5#$( $" 0'%->. $)*$ >%"+ *&& $)' -#-$-*&($*$'( -# * 6-0'# ('$ ! I $)' (.($'+ #'0'% %'*!)'( $)'($*$' (,/$7 "%7 'B5-0*&'#$&.7 $)*$ $)' (.($'+ *&=*.(%'*!)'( $)' ($*$' .,&3

!" #$%&'()* +,- () #./ 0$12

J# "%,'% $" 5(' $)' 0'%->-!*$-"# $""&( ,'(!%-@', -#K'!$-"# L7 =' #'', $" "@$*-# * ).@%-, +",'& "> $)'@*$!) '0*/"%*$"% /%"!'(( -# MH4 >"%+3 8' !"#(-,'%$)' +",'& ,'(!%-@', -# NOPQ7 =)-!) "#&. $*1'( -#$"*!!"5#$ $)' ,.#*+-!( "> $)' &'0'&( $:7 $27 *#, "> $)'$'+/'%*$5%' ! "> $*#1 :3 A)' +",'& ,'0'&"/', -# NOPQ

0(*" !" ?&"=!)*%$ "> $)' @'#!)+*%1 '0*/"%*$"% (.($'+3

0(*" 3" K$*$' $%*#(-$-"# ,-*6%*+ "> $)' !"#$%"&&'%3

%&'#()*)+!&,) -./(&0 120)3&45 "40 6)(&7&#"*&24 OP:

!"#$%"&&'% '#$'%( $)' *&*%+ +",' *#, -++',-*$'&."/'#( 0*&0' !"# $" '+/$. $*#1 2345%-#6 $)-( /)*('7 $)')'*$'% -( ($-&& -# $)' !"#$%&' ($*$'3 8)'# $)' $'+/'%*9$5%' -# $*#1 : %'*!)'( $)' *&*%+ &'0'& !" ;#(#)*<7 $)')'*$-#6 -( (=-$!)', ">> *#, $)' !"#$%"&&'% '#$'%( $)'($*$' +,,(%&'3 ?-#*&&.7 =)'# $*#1 2 -( '+/$.7 $)' !"#9$%"&&'% 6'$( -# $)' ($*$' -)#%&%&'7 =)'%' 0*&0' !"# -(!&"(', *#, 0*&0' !"$ -( "/'#',7 *#, $)' ("&5$-"# >&"=(>%"+ $)' 5//'% $*#1 $" $)' &"='% $*#13 ?%"+ -)#%&%&'7$)' !"#$%"&&'% !*# '-$)'% (=-$!) $" $)' ($*$'.,& -> $*#1: @'!"+'( '+/$.7 "% $" (,/$ -> $)' $'+/'%*$5%' -# $*#1 :@'!"+'( &"='% $)*# $)' !%-$-!*& 0*&5' !#3

A)' *-+ -( $" 0'%->. $)*$ $)' !"#$%"&&'% (*$-(>-'( $)'>"&&"=-#6 (*>'$. %'B5-%'+'#$(C ;-< -> * +*&>5#!$-"#-#6-# $)' !""&-#6 (.($'+ "> $)' !"#,'#('% "!!5%(7 $)')'*$'% +5($ @' $5%#', ">> B5-!1&. '#"56) $" /%'0'#$,*+*6'( $" $)' !"#,'#('%7 ;--< $)' ("&5$-"# -# $*#1 : -(

,%*-#', $" $*#1 2 @'>"%' -$ ("&-,->-'(7 ;---< =)'# $)'0*&0' D:E -( "/'# $*#1 2 -( '+/$.3

F'%$->.-#6 $)*$ $)' GHF !",' (*$-(>-'( $)'(' (/'!-9>-!*$-"#( *+"5#$( $" 0'%->. $)*$ >%"+ *&& $)' -#-$-*&($*$'( -# * 6-0'# ('$ ! I $)' (.($'+ #'0'% %'*!)'( $)'($*$' (,/$7 "%7 'B5-0*&'#$&.7 $)*$ $)' (.($'+ *&=*.(%'*!)'( $)' ($*$' .,&3

!" #$%&'()* +,- () #./ 0$12

J# "%,'% $" 5(' $)' 0'%->-!*$-"# $""&( ,'(!%-@', -#K'!$-"# L7 =' #'', $" "@$*-# * ).@%-, +",'& "> $)'@*$!) '0*/"%*$"% /%"!'(( -# MH4 >"%+3 8' !"#(-,'%$)' +",'& ,'(!%-@', -# NOPQ7 =)-!) "#&. $*1'( -#$"*!!"5#$ $)' ,.#*+-!( "> $)' &'0'&( $:7 $27 *#, "> $)'$'+/'%*$5%' ! "> $*#1 :3 A)' +",'& ,'0'&"/', -# NOPQ

0(*" !" ?&"=!)*%$ "> $)' @'#!)+*%1 '0*/"%*$"% (.($'+3

0(*" 3" K$*$' $%*#(-$-"# ,-*6%*+ "> $)' !"#$%"&&'%3

%&'#()*)+!&,) -./(&0 120)3&45 "40 6)(&7&#"*&24 OP:

!" # $%&'()%* #+, -+./ !"0#1 23 #**&)&.34 %#,( ("5%+6+%,)#3'7% &8 598(%* .3 # 8)#,: .- 8%)8 ). !% %;57.+%*1

<%-.+% 8)#+)&3' # 3%$ +%#,( 8%) ,./59)#)&.3 -+./ #8%) !" %;)+#,)%* -+./ )(% 8)#,:4 $% ,(%,: -.+ &3,798&.3.- !" &3 .)(%+ 3.*%8 .- #1 2- )(&8 (#55%384 8#" !" $ !=

#3* !" $ !>4 )(% 3.*% #88.,&#)%* $&)( !" &8 +%/.?%*-+./ #4 #3* )(% #+,8 #+% +%*&+%,)%*1@-)%+ @7'.+&)(/ A1= )%+/&3#)%84 )(% .+&%3)%* 5#)(8

.3 # -+./ &3&)&#7 3.*% !"0# ). )%+/&3#7 3.*%8 %"4 " &=4 ! ! ! 4$ *%)%+/&3% # 895%+8%) .- -%#8&!7% 8$&),(&3'8%B9%3,%8 %"&# & ''"0#4 ! ! ! 4 '"& ( =#)1 23 -#,)4 !%,#98%.- )(% .9)%+ #55+.;&/#)&.3 "!( .- 3%$ &3)%+8%,)&.38 !(

C8)%5 ) D4 3.) #77 8$&),(&3' 8%B9%3,%8 #+% -%#8&!7%1E%#8&!&7&)" ,#3 !% 8&/57" )%8)%* ?&# FG .?%+ )(% 8%)8 .-7&3%#+ &3%B9#7&)&%8 '%3%+#)%* !" #3 %;57&,&) +%#,( 8%)+%5+%8%3)#)&.3 #8 &3 CHD1

!" #$%&'&()*&+, +' *-$ .)*(- /0)1+%)*+%2%+($33 .$,(-4)%5

23 )(&8 8%,)&.3 $% #557" )(% )..78 5+.5.8%* &3 )(%5+%?&.98 8%,)&.38 ). )(% !%3,(/#+: 5+.!7%/ 5+.5.8%*$&)(&3 )(% IJGK2L6FLK G+.M%,) >N>H0 OPJ CO%+&-&6,#)&.3 .- P"!+&* J"8)%/8DQ #8 R#8% J)9*" =1 L(% -9778"8)%/ ,.38&8)8 .- #3 %;5%+&/%3)#7 !#),( 57#3) S>TU123 )(&8 5#5%+ $% $&77 -.,98 .37" .3 )(% *+,-./,&./010&*24 #8 5+.5.8%* &3 S>VU4 $(&,( &8 8,(%/#)&,#77"*%5&,)%* &3 E&'1 N1

L(% ,.38&*%+%* 89!8"8)%/ &8 ,./5.8%* .- )(+%%5#+)8W )(% 955%+ )#3: = C7#!%7%* #8 !" &3 E&'1 ND4 )(%7.$%+ )#3: > C!#D #3* )(% ,.3*%38%+ C$%D1 L#3: =&8 %B9&55%* $&)( # (%#)&3' 8"8)%/ C&D4 #3* &8,.33%,)%* ). )#3: > !" # 5&5% #3* # ?#7?% C'%"D4 $(&7%

)(% .9)7%) .- )#3: > &8 ,.3)+.77%* !" ?#7?% '%(1 <.)()(% (%#)&3' 8"8)%/ #3* )(% ?#7?%8 ,#3 .37" (#?% )$.,.3-&'9+#)&.38W !" C!#$"D #3* !%% C&'!($)D1 L(% 7%?%78( =4 ( > .- )(% 8.79)&.3 &3 )(% )$. )#3:84 #3* )(% )%/65%+#)9+% 3 .- )#3: = #+% *%)%,)%* !" 8%38.+81 L(%8%5+.?&*% -.9+ 7.'&, 8&'3#78W *+", - $.#*/4 *+", 0$.#*/4 +'+1.4 #3* &1/(*+''23+*2!"1 @ )#3: &8 ,.368&*%+%* %/5)" $(%3 &)8 7%?%7 &8 !%7.$ = ,/1

X9+&3' 3.+/#7 .5%+#)&.3 .- )(% 57#3)4 #3 #B9%.988.79)&.3 $&)( # 7.$ ,.3,%3)+#)&.3 .- 8#7) %3)%+8 )#3: =1L(% (%#)&3' &8 )9+3%* .34 #3* $(%3 )(% 5+.*9,)&.3 .-8)%#/ !%'&384 )(% ,.3,%3)+#)&.3 .- 8#7) &3 )#3: = 8)#+)8). +&8%1 L(% .9)'.&3' 8)%#/ -7.$8 )(+.9'( )(% ,.36*%38%+ #3* &8 *+#&3%* #$#" -+./ )(% 8"8)%/1 Y(%3 )(%,.3,%3)+#)&.3 .- 8#7) (#8 +%#,(%* # ,%+)#&3 7%?%74 )(%(%#)&3' 8"8)%/ &8 8$&),(%* .--4 ?#7?% '%" &8 .5%3%*4#3* )(% 8.79)&.3 -7.$8 ). )#3: >4 -.+ 5.8) 5+.,%88&3'.5%+#)&.381 L(% 57#3) &8 *%8&'3%* &3 89,( # $#" )(#)/.+% )(#3 .3% !#),( ,#3 !% 5+.*9,%* #) )(% 8#/%)&/%4 8. )(#) )#3: = #3* )#3: > ,#3 5+.,%88 *&--%+%3)!#),(%8 &3 5#+#77%71

23 )(&8 5#5%+ $% -.,98 .3 )(% %;,%5)&.3 (#3*7&3'+%B9&+%* $(%3 )(% ,.3*%38%+ *.%8 3.) $.+: 5+.5%+7"1J955.8% )(#) -.+ 8./% +%#8.3 C%1'1 7#,: .- ,..7&3'#'%3)D )(% ,.3*%38%+ /#7-93,)&.381 23 )(&8 ,#8%4 )(%8)%#/ ,#33.) !% ,..7%* *.$3 #3* )(% 5+%889+% &3 )#3:= +&8%81 L(% (%#)&3' 8"8)%/ 8(.97* !% 8$&),(%* .-- ).5+%?%3) *#/#'% ). )(% 57#3) *9% ). .?%+65+%889+%1 Z3)(% .)(%+ (#3*4 )(% )%/5%+#)9+% &3 )#3: = 8(.97* 3.)'%) 7.$%+ )(#3 # ,+&)&,#7 )%/5%+#)9+% 344 .)(%+$&8% )(%8#7) /#" ,+"8)#77&[% #3* %;5%38&?% 5+.,%*9+%8 $.97*!% 3%%*%* ). +%8).+% )(% .+&'&3#7 -93,)&.3#7&)"1

@GFR ,.3)+.78 )(% 57#3) #,,.+*&3' ). )(% -&3&)% 8)#)%/#,(&3% *%5&,)%* &3 E&'1 H4 $(%+% )(% %?%3) +'+1..,,9+8 $(%3 3 * 3,4 #3* &1/(*+''23+*2!" $(%3 3 + 341

L(% ,.3)+.7 8)+#)%'" ,#3 !% %;57#&3%* #8 -.77.$81Y(%3 # /#7-93,)&.3 .- )(% ,.3*%38%+ &8 *%)%,)%*4 )(%

(3, (0))

i

h

(a) j

(0)

(1, (0))

(2, (0))

(b)

6&7" !" K%#,(#!&7&)" #3#7"8&81 C#D K%#,( 8%) %?.79)&.34 '9#+*7&3% ,+.88&3'4 .9)%+ #55+.;&/#)&.3 .- # 3%$ &3)%+8%,)&.34 C!D Z9)%+ +%,)#3'97#+#55+.;&/#)&.3 .- # 3%$ +%'&.3 &3)%+8%,)&.31

Q ())5W\\$$$6?%+&/#'1&/#'1-+\OPJ\

QV0 56 7*2-./,8 *& ,96

Lessons learned:

- Polytope manipulations are numerically challenging- Take advantage of commercial MILP / MIQP solvers

Some Examples of Contributions from Oded’s Group• Antoine Girard, Colas Le Guernic

• Introduction of Zonotopes to our community• Rigorous analysis of computational complexity vs error and good heuristics to

reduce it• Use of support functions

• Goran Frehse• Extensions • Tool Development: SpaceEx

CC (Control & Computation)Mtg. 2003, Siena

JULY 2006 ICR 5

COVER STORY AND CONTENTS

Cover StoryThe front cover shows how by working together, Jura Cement and ABB Switzerland Ltd achieved the first known successful application of a MLD system on a cement mill. ABB’s wide and successful experience with this technology made it keen to see what benefits it could bring to other parts of the cement process and therefore approached Jura Cement about a collaboration to apply this cutting edge technology to one of their grinding units. The outcome for Jura Cement’s Wildegg plant has been that the mill can be run for maximum production, achieving their Blaine targets, and also ensuring energy inputs and additives are used efficiently and effectively. Jura Cement has the first plant in the world the experience the benefits of ABB applying MLD to a cement mill.

For further information, contact:ABB Switzerland LtdTel.: +41 58 586 84 44Fax: +41 58 586 7333Email: [email protected]: www.abb.com/cement

8 World newsAll the essential news from the global cement industry

Lafarge announces three year targets. Buzzi Unicem buys stakes in two Algerian producers. Cemex expects higher operating income for 2Q06. HeidelbergCement backs Sierra Leone and Oyak Cement revises its forecasts.

21 Teamwork pays dividendsby ICR Research

ICR exclusively interviews Markus Akermann, CEO of Holcim and discovers what ambitions are driving the Swiss cement giant forward, its strategies, manpower developments and market assessments.

July 2006

Industrial productivitythrough ABB

front cover July 2006.indd 1 21/6/06 14:42:42

Contents

JULY 2006

1 2 3 4

5 6 7 8

9 10 11 12

4 MANAGING EDITOR David Hargreaves: [email protected]

4 DEPUTY EDITORS Peter Bell: [email protected] Jolyne Fernandes: [email protected]

4 TECHNICAL CONSULTANT Dr. Michael Clark

4 ADVERTISING & MARKETING Gary Morton: [email protected]

4 SUBSCRIPTIONS Susan Hargreaves: [email protected]

4CONTACT Tel: +44 (0) 1306 740363/740383 Fax: +44 (0) 1306 740660 Email: [email protected] Web: www.CemNet.com

4 MAIL Tradeship Publications Ltd Old Kings Head Court, 15 High Street, Dorking Surrey RH4 1AR, United Kingdom

International Cement Review© is a Tradeship Publication and is copyright protected throughout. International Cement Review© is printed monthly in the United Arab Emirates by Emirates Printing Press LLC, Dubai, UAE. ©2006

ISSN No. 0959-6038

26 Pakistan markets flying highby ICR Research

Government inspired ‘mega projects’ at home and increased demand for cement, plus exports, has signalled a flurry of expansion and increased capacity plans for Pakistan producers.

3/9/19, 6(30 PMABB introduces ACS6080 drive for high performance motor control

Page 1 of 5https://new.abb.com/news/detail/16575/abb-introduces-acs6080-drive-for-high-performance-motor-control

Press release | Zurich, Switzerland, 07 March 2019

ABB introduces ACS6080 drive for highperformance motor control

ABB’s new all-compatible ACS6080 drive brings precision, reliability andsafety to demanding industrial processes.

ABB has expanded its all-compatible drives offering to include the ACS6080 medium

voltage (MV) drive for critical applications. The new drive introduces a breakthrough control

technology, developed by ABB, that brings high performance motor control to demanding

processes used in industries such as mining, metals and marine. The drive also enables

plants to operate more efficiently and produce higher quality output.

New ACS6080 medium voltage drive for high performance applications



Press release | Zurich, Switzerland, 07 March 2019

ABB introduces ACS6080 drive for highperformance motor control

ABB’s new all-compatible ACS6080 drive brings precision, reliability andsafety to demanding industrial processes.

ABB has expanded its all-compatible drives offering to include the ACS6080 medium

voltage (MV) drive for critical applications. The new drive introduces a breakthrough control

technology, developed by ABB, that brings high performance motor control to demanding

processes used in industries such as mining, metals and marine. The drive also enables

plants to operate more efficiently and produce higher quality output.

New ACS6080 medium voltage drive for high performance applications



Traditionally, achieving high dynamic performance and excellent power quality needed

hardware with multiple devices. However, this increase in components could decrease

reliability of the equipment. The ACS6080 drive maintains the same highly reliable hardware

as its predecessor, the ACS6000, and includes the new advanced control capability, known

as Model Predictive Pulse Pattern Control (MP C).

ABB’s MP C technology combines model predictive control with optimized pulse pattern to

modulate the semiconductors. This means that at every point in time this control can

anticipate the best motor operation point by finding the perfect compromise between

dynamics, efficiency and harmonic distortion.

The new control enables system integrators and plant operators to maintain stability, while

reducing investment and operational costs by using a smaller drive or a smaller motor. The

increased power capability of the ACS6080’s single power module reduces the footprint of

the drive by up to 20 percent. The new control helps to reduce operational cost in two ways:

it is more energy efficient than other control solutions, and maintenance costs are reduced

as there is less mechanical and thermal stress on the equipment leading to longer lifetime

of the components.

The ACS6080 provides extensive input/output connectivity and supports all major fieldbus

protocols, making it easy to integrate into existing systems and processes. It is compatible

with ABB Ability™ condition monitoring services, allowing customers to obtain real-time

data about the status and performance of the monitored equipment from any location.

“The ACS6080 MV drive provides benefits which help deliver exceptional value to industrial

processes in terms of product quality, uptime, safety and energy efficiency,” says Ester

Guidi, Product Manager, ABB Drives.

The new drive delivers a high level of safety for people and equipment via an arc-resistant

design and certified functional safety features. The ACS6080 is built on ABB’s common drive

architecture, enabling a faster and harmonized startup, commissioning and monitoring.

Once a user has learned how to use one all-compatible drive, they can then operate any

other drive in ABB’s all-compatible offering.

3

3

From: Oded Maler <[email protected]>

Subject: Hi From Dagstuhl

Date: February 21, 2018 at 1:21:11 PM GMT+1

To: "Engell, Sebastian" <[email protected]>, Morari Manfred <[email protected]>, "Bruce H. Krogh" <[email protected]>

Hi Friends, I'm there in a formal methods for synthetic biologyseminar. In parallel there is a seminar on MINLP, and the only name Irecognized there is Ignacio Grossmann but he did not recognize me. It'sa pity because I wanted to offer him a heretic guest lecture on thesacrilege and inutility of treating the Booleans as integers ;-)

Best,

--Oded

System Biology

• A cynical view• An arrogant view• A humble view• A (relatively) sober view• A sober (but subjective) view

Systems Biology: a Cynical View

I Systems Biology: the current gold rush for manymathematical and technical disciplines looking for nutrition(funding, self-esteem) in the scientific food chain

I Biophysics, Biomatics, Bioinformatics, Biostatistics...

I The story goes like this:

I I do X

I I do it for my pleasure, because I studied it, and anyway, thisis the only thing I will do in my current incarnation...

I ...fortunately X is very useful for Biology

I When you have a hammer, everything looks like a nail

I Personally this is how I came to the domain(X = automata, verification and hybrid systems)

I Fortunately, my hammer is universal

Systems Biology: an Arrogant View

I Biologists are essentially very concrete beings, spending mostof their time in the kitchen doing manual work

I They were not selected (initially) based on ability tomanipulate imaginary concepts or creativity and rigor in theabstract world of ideas but rather..

I ..based on their rigor and e�ciency at the bench

I Now when they need to make a real science out of theirdetails they need noble white collar brahmins, namely..

I ... physicists, mathematicians, computer scientists, asspiritual guides

I Like monotheists converting the pagans, these merchants ofabstract methodologies try to impress the poor savage withtheir logics and miracles

Systems Biology: a Humble View

I Biologists are working with the most fascinating, complex andmysterious real-life phenomena

I Living systems are more complex than the hydrogen atom orthe electromagnetic field (and are not e↵ectively reducibleto them)

I Living systems are more sophisticated than your dumb

terminal or smart phone or mobile robot or car

I Living systems are more mysterious and primordial than theprime numbers, the algebra of Boole or the free monoid

I If some of our dry tricks can help them, even a bit, in theirgrand march toward..

I ..understanding something about Life Itself or helpingdoctors kill less patients

I We should be very happy and proud for doing, for once,something meaningful

Systems Biology: a (relatively) Sober ViewI The dynamics of a scientific discipline may have di↵erent

periods with various trends and fashions

I This dynamics is not always optimized towards truthI Many aspects (politics, social dynamics, commercial interests,

cognitive inertia, media distortion) play an important roleI Probably most of what is published today in top journals will

go to the garbage can of history

I Few centuries ago, the science of this guy (chemistry,medicine, metaphysics) was debated extensively in primetime

Systems Biology: a Sober (but subjective) View

I Today there is an over emphasis on doing something withdata provided by new experimental machinery (omics)

I The main question about “knowing” all these details iswhether this knowledge:

I Is su�cient for understanding and learning something aboutunderlying mechanisms ? (certainly not)

I Is necessary for that ? (very hopefully not)I Is helpful or counter-productive ?

I Systems Biology is about seeking some clearer (conceptualand mathematical) models of dynamical systems at variouslevels of abstraction

I These models, if thoughtfully constructed, and carefully andsystematically analyzed/simulated may help reducing thegap between cellular biochemistry and physiology

Ouri Maler is creating stories | PatreonHey there. My name is Ouri Maler. I'm known by a couple of aliases online, including "sun tzu". I enjoy writing fiction, which I've been putting online for years. If you're reading this, odds are you enjoy some of my writing. Maybe you like Saga of Soul, my deconstructive-slash-reconstructive magical girl webnovelabout a teenage girl with a fondness for the scientific method, who applies it to her newfound abilities in-between trying to save the world from a procession of monsters and villains. Maybe you like Souls in a Vacuum, my short story of religion meeting politics meeting space-opera aboard Concordia, a massive space station where delegates from the various religions of the galaxy meet. Maybe you like my fanfics, be they comedic rewrites of My Little Pony that gradually grew into a script-format equivalent of abridged series, or more serious tales of a paragon of superheroism being transported to the the grim supervillain story of Worm, or stories of a ludicrously intelligent, hypercompetent mastermind wandering into that very same setting and doing his best to fix it with little more than clever plans.

Safety Verification and Robustness Analysis ofNeural Networks via Quadratic Constraints and

Semidefinite Programming1

Mahyar Fazlyab, Manfred Morari, George J. Pappas

1https://arxiv.org/abs/1903.01287

Canonical Problem

I Given– Trained neural network described by f : Rnx ! Rny

– Bounded set X ⇢ Rnx of infinite number of inputs

– Set S ⇢ Rny on the output space

Y := f(X )<latexit sha1_base64="d2gfe6oFuWGC7w4SQ5ibb/XEpII=">AAACBnicbVDLSsNAFL3xWesr6lKEYCvUTUnqQhGFghuXFexD2lAm00k7dDIJMxOhhK7c+CtuXCji1m9w5984aYNo64GBM+fcy733eBGjUtn2l7GwuLS8sppby69vbG5tmzu7DRnGApM6DlkoWh6ShFFO6ooqRlqRICjwGGl6w6vUb94TIWnIb9UoIm6A+pz6FCOlpa55UOwESA0wYsnd+PzSL/18W+PjYtcs2GV7AmueOBkpQIZa1/zs9EIcB4QrzJCUbceOlJsgoShmZJzvxJJECA9Rn7Q15Sgg0k0mZ4ytI630LD8U+nFlTdTfHQkKpBwFnq5Ml5SzXir+57Vj5Z+5CeVRrAjH00F+zCwVWmkmVo8KghUbaYKwoHpXCw+QQFjp5PI6BGf25HnSqJSdk7JzUylUL7I4crAPh1ACB06hCtdQgzpgeIAneIFX49F4Nt6M92npgpH17MEfGB/fORGYSw==</latexit>

X<latexit sha1_base64="ZFVtFXEVoh0sae19xqdrq2baVwU=">AAAB9HicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhd4dZmDA7u87MkpAN32FjoTG2foydf+MsUCh4kklOzrk398wJEsG1cd1vZ2Nza3tnt7BX3D84PDounZy2dJwqypo0FrHqBKiZ4JI1DTeCdRLFMAoEawfju9xvT5jSPJaPZpowP8Kh5CGnaKzkV3oRmhFFkXVmlX6p7FbdOcg68ZakDEs0+qWv3iCmacSkoQK17npuYvwMleFUsFmxl2qWIB3jkHUtlRgx7Wfz0DNyaZUBCWNlnzRkrv7eyDDSehoFdjLPqFe9XPzP66YmvPEzLpPUMEkXh8JUEBOTvAEy4IpRI6aWIFXcZiV0hAqpsT0VbQne6pfXSatW9Sx/qJXrt8s6CnAOF3AFHlxDHe6hAU2g8ATP8ApvzsR5cd6dj8XohrPcOYM/cD5/AE/6kcg=</latexit><latexit sha1_base64="ZFVtFXEVoh0sae19xqdrq2baVwU=">AAAB9HicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhd4dZmDA7u87MkpAN32FjoTG2foydf+MsUCh4kklOzrk398wJEsG1cd1vZ2Nza3tnt7BX3D84PDounZy2dJwqypo0FrHqBKiZ4JI1DTeCdRLFMAoEawfju9xvT5jSPJaPZpowP8Kh5CGnaKzkV3oRmhFFkXVmlX6p7FbdOcg68ZakDEs0+qWv3iCmacSkoQK17npuYvwMleFUsFmxl2qWIB3jkHUtlRgx7Wfz0DNyaZUBCWNlnzRkrv7eyDDSehoFdjLPqFe9XPzP66YmvPEzLpPUMEkXh8JUEBOTvAEy4IpRI6aWIFXcZiV0hAqpsT0VbQne6pfXSatW9Sx/qJXrt8s6CnAOF3AFHlxDHe6hAU2g8ATP8ApvzsR5cd6dj8XohrPcOYM/cD5/AE/6kcg=</latexit><latexit sha1_base64="ZFVtFXEVoh0sae19xqdrq2baVwU=">AAAB9HicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhd4dZmDA7u87MkpAN32FjoTG2foydf+MsUCh4kklOzrk398wJEsG1cd1vZ2Nza3tnt7BX3D84PDounZy2dJwqypo0FrHqBKiZ4JI1DTeCdRLFMAoEawfju9xvT5jSPJaPZpowP8Kh5CGnaKzkV3oRmhFFkXVmlX6p7FbdOcg68ZakDEs0+qWv3iCmacSkoQK17npuYvwMleFUsFmxl2qWIB3jkHUtlRgx7Wfz0DNyaZUBCWNlnzRkrv7eyDDSehoFdjLPqFe9XPzP66YmvPEzLpPUMEkXh8JUEBOTvAEy4IpRI6aWIFXcZiV0hAqpsT0VbQne6pfXSatW9Sx/qJXrt8s6CnAOF3AFHlxDHe6hAU2g8ATP8ApvzsR5cd6dj8XohrPcOYM/cD5/AE/6kcg=</latexit><latexit sha1_base64="ZFVtFXEVoh0sae19xqdrq2baVwU=">AAAB9HicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhd4dZmDA7u87MkpAN32FjoTG2foydf+MsUCh4kklOzrk398wJEsG1cd1vZ2Nza3tnt7BX3D84PDounZy2dJwqypo0FrHqBKiZ4JI1DTeCdRLFMAoEawfju9xvT5jSPJaPZpowP8Kh5CGnaKzkV3oRmhFFkXVmlX6p7FbdOcg68ZakDEs0+qWv3iCmacSkoQK17npuYvwMleFUsFmxl2qWIB3jkHUtlRgx7Wfz0DNyaZUBCWNlnzRkrv7eyDDSehoFdjLPqFe9XPzP66YmvPEzLpPUMEkXh8JUEBOTvAEy4IpRI6aWIFXcZiV0hAqpsT0VbQne6pfXSatW9Sx/qJXrt8s6CnAOF3AFHlxDHe6hAU2g8ATP8ApvzsR5cd6dj8XohrPcOYM/cD5/AE/6kcg=</latexit>

f(.)<latexit sha1_base64="HN0r35B88M9DtYvPC/xD1XWDwlA=">AAAB7XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJdOMSE7kkMCGd0oFKp520HRMy4R3cuNAYt76PO9/GMsxCwT9p8uU/56Tn/EHMmTau++0UNja3tneKu6W9/YPDo/LxSUfLRBHaJpJL1QuwppwJ2jbMcNqLFcVRwGk3mN4u6t0nqjST4sHMYupHeCxYyAg21upUw1r9sjosV9y6mwmtg5dDBXK1huWvwUiSJKLCEI617ntubPwUK8MIp/PSINE0xmSKx7RvUeCIaj/Ntp2jC+uMUCiVfcKgzP09keJI61kU2M4Im4lerS3M/2r9xITXfspEnBgqyPKjMOHISLQ4HY2YosTwmQVMFLO7IjLBChNjAyrZELzVk9eh06h7lu8bleZNHkcRzuAcauDBFTThDlrQBgKP8Ayv8OZI58V5dz6WrQUnnzmFP3I+fwCz3o3h</latexit><latexit sha1_base64="HN0r35B88M9DtYvPC/xD1XWDwlA=">AAAB7XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJdOMSE7kkMCGd0oFKp520HRMy4R3cuNAYt76PO9/GMsxCwT9p8uU/56Tn/EHMmTau++0UNja3tneKu6W9/YPDo/LxSUfLRBHaJpJL1QuwppwJ2jbMcNqLFcVRwGk3mN4u6t0nqjST4sHMYupHeCxYyAg21upUw1r9sjosV9y6mwmtg5dDBXK1huWvwUiSJKLCEI617ntubPwUK8MIp/PSINE0xmSKx7RvUeCIaj/Ntp2jC+uMUCiVfcKgzP09keJI61kU2M4Im4lerS3M/2r9xITXfspEnBgqyPKjMOHISLQ4HY2YosTwmQVMFLO7IjLBChNjAyrZELzVk9eh06h7lu8bleZNHkcRzuAcauDBFTThDlrQBgKP8Ayv8OZI58V5dz6WrQUnnzmFP3I+fwCz3o3h</latexit><latexit sha1_base64="HN0r35B88M9DtYvPC/xD1XWDwlA=">AAAB7XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJdOMSE7kkMCGd0oFKp520HRMy4R3cuNAYt76PO9/GMsxCwT9p8uU/56Tn/EHMmTau++0UNja3tneKu6W9/YPDo/LxSUfLRBHaJpJL1QuwppwJ2jbMcNqLFcVRwGk3mN4u6t0nqjST4sHMYupHeCxYyAg21upUw1r9sjosV9y6mwmtg5dDBXK1huWvwUiSJKLCEI617ntubPwUK8MIp/PSINE0xmSKx7RvUeCIaj/Ntp2jC+uMUCiVfcKgzP09keJI61kU2M4Im4lerS3M/2r9xITXfspEnBgqyPKjMOHISLQ4HY2YosTwmQVMFLO7IjLBChNjAyrZELzVk9eh06h7lu8bleZNHkcRzuAcauDBFTThDlrQBgKP8Ayv8OZI58V5dz6WrQUnnzmFP3I+fwCz3o3h</latexit><latexit sha1_base64="HN0r35B88M9DtYvPC/xD1XWDwlA=">AAAB7XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJdOMSE7kkMCGd0oFKp520HRMy4R3cuNAYt76PO9/GMsxCwT9p8uU/56Tn/EHMmTau++0UNja3tneKu6W9/YPDo/LxSUfLRBHaJpJL1QuwppwJ2jbMcNqLFcVRwGk3mN4u6t0nqjST4sHMYupHeCxYyAg21upUw1r9sjosV9y6mwmtg5dDBXK1huWvwUiSJKLCEI617ntubPwUK8MIp/PSINE0xmSKx7RvUeCIaj/Ntp2jC+uMUCiVfcKgzP09keJI61kU2M4Im4lerS3M/2r9xITXfspEnBgqyPKjMOHISLQ4HY2YosTwmQVMFLO7IjLBChNjAyrZELzVk9eh06h7lu8bleZNHkcRzuAcauDBFTThDlrQBgKP8Ayv8OZI58V5dz6WrQUnnzmFP3I+fwCz3o3h</latexit>

S<latexit sha1_base64="BtviJ89MRF38kIE7B/44/Itbhfw=">AAAB9HicbVBNTwIxFHyLX4hfqEcvjWDiieziQQ8eSLx4xChIAhvSLV1oaLtr2yUhG36HFw8a49Uf481/Yxf2oOAkTSYz7+VNJ4g508Z1v53C2vrG5lZxu7Szu7d/UD48ausoUYS2SMQj1QmwppxJ2jLMcNqJFcUi4PQxGN9k/uOEKs0i+WCmMfUFHkoWMoKNlfxqT2AzIpin97Nqv1xxa+4caJV4OalAjma//NUbRCQRVBrCsdZdz42Nn2JlGOF0VuolmsaYjPGQdi2VWFDtp/PQM3RmlQEKI2WfNGiu/t5IsdB6KgI7mWXUy14m/ud1ExNe+SmTcWKoJItDYcKRiVDWABowRYnhU0swUcxmRWSEFSbG9lSyJXjLX14l7XrNu6h5d/VK4zqvowgncArn4MElNOAWmtACAk/wDK/w5kycF+fd+ViMFpx85xj+wPn8AUeIkcA=</latexit>

I Goal:– Check whether f(X ) ✓ S

6

Example: Local Robustness in Image Classification

I X is the set of all perturbations of an image

f( )<latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit>

!



Y = f(X )<latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit>

Locally Robust

Local robustness: Class(x) is the same for all x 2 X

Applications 10

Example: Local Robustness in Image Classification

I X is the set of all perturbations of an image

f( )<latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="gRbt0mqrUfYJaXCrBswfLFaoGOU=">AAAB6HicbZBLSwMxFIXv1FetVatbN8FWqJsy40aXghuXFewDOkPJZDJtaJIZ8xDK0L/hxoUi/iJ3/hvTx0JbDwQ+zkm4NyfOOdPG97+90tb2zu5eeb9yUD08Oq6dVLs6s4rQDsl4pvox1pQzSTuGGU77uaJYxJz24sndPO89U6VZJh/NNKeRwCPJUkawcVbYSJsofLI4QZeNYa3ut/yF0CYEK6jDSu1h7StMMmIFlYZwrPUg8HMTFVgZRjidVUKraY7JBI/owKHEguqoWOw8QxfOSVCaKXekQQv394sCC62nInY3BTZjvZ7Nzf+ygTXpTVQwmVtDJVkOSi1HJkPzAlDCFCWGTx1gopjbFZExVpgYV1PFlRCsf3kTuletwPGDD2U4g3NoQgDXcAv30IYOEMjhBd7g3bPeq/exrKvkrXo7hT/yPn8AmUOO2g==</latexit><latexit sha1_base64="uCDeVrAKMvlxE7bQAyqcEKLh4Qk=">AAAB83icbZDLTsJAFIZP8YZ4Q126mQgmuCGtG10S3bjERMCENmQ6ncKE6bTOxYQ0vIYbFxrj1pdx59s4QBcK/skkX/5zTs6ZP8w4U9p1v53S2vrG5lZ5u7Kzu7d/UD086qrUSEI7JOWpfAixopwJ2tFMc/qQSYqTkNNeOL6Z1XtPVCqWins9yWiQ4KFgMSNYW8uvxw3kPxocofP6oFpzm+5caBW8AmpQqD2ofvlRSkxChSYcK9X33EwHOZaaEU6nFd8ommEyxkPatyhwQlWQz2+eojPrRChOpX1Co7n7eyLHiVKTJLSdCdYjtVybmf/V+kbHV0HORGY0FWSxKDYc6RTNAkARk5RoPrGAiWT2VkRGWGKibUwVG4K3/OVV6F40Pct3bq11XcRRhhM4hQZ4cAktuIU2dIBABs/wCm+OcV6cd+dj0Vpyiplj+CPn8wfOQ5A0</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit><latexit sha1_base64="vb59vzFL/kzN7nfwrD3w10PLFno=">AAAB83icbZC7TgJBFIbP4g3xhlraTAQTbMgujZZEG0tM5JKwGzI7OwsTZmfXuZgQwmvYWGiMrS9j59s4wBYK/skkX/5zTs6ZP8w4U9p1v53CxubW9k5xt7S3f3B4VD4+6ajUSELbJOWp7IVYUc4EbWumOe1lkuIk5LQbjm/n9e4TlYql4kFPMhokeChYzAjW1vKrcQ35jwZH6LI6KFfcursQWgcvhwrkag3KX36UEpNQoQnHSvU9N9PBFEvNCKezkm8UzTAZ4yHtWxQ4oSqYLm6eoQvrRChOpX1Co4X7e2KKE6UmSWg7E6xHarU2N/+r9Y2Or4MpE5nRVJDlothwpFM0DwBFTFKi+cQCJpLZWxEZYYmJtjGVbAje6pfXodOoe5bvG5XmTR5HEc7gHGrgwRU04Q5a0AYCGTzDK7w5xnlx3p2PZWvByWdO4Y+czx/O45A2</latexit>

!



Y = f(X )<latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit><latexit sha1_base64="sYlzsWtRG+pPwy9ZxD3PE0woy0g=">AAACBXicbVDLSsNAFL3xWesr6lIXwVaom5J0oxuh6MZlBfuQNpTJdNIOnUzCzEQoIRs3/oobF4q49R/c+TdO2iDaemDgzDn3cu89XsSoVLb9ZSwtr6yurRc2iptb2zu75t5+S4axwKSJQxaKjockYZSTpqKKkU4kCAo8Rtre+Crz2/dESBryWzWJiBugIac+xUhpqW8elXsBUiOMWHKXXviVn18nPS33zZJdtaewFomTkxLkaPTNz94gxHFAuMIMSdl17Ei5CRKKYkbSYi+WJEJ4jIakqylHAZFuMr0itU60MrD8UOjHlTVVf3ckKJByEni6MltSznuZ+J/XjZV/7iaUR7EiHM8G+TGzVGhlkVgDKghWbKIJwoLqXS08QgJhpYMr6hCc+ZMXSatWdTS/qZXql3kcBTiEY6iAA2dQh2toQBMwPMATvMCr8Wg8G2/G+6x0ych7DuAPjI9vtGqYCg==</latexit>

Not Locally Robust

Class(x) is NOT the same for all x 2 X

I How can we certify local robustness of a trained NN around a test image?

Applications 10

Example: Safety Verification

I Learning forward kinematics of robotic arms [Xiang et al., 2018]

x<latexit sha1_base64="YQKNc1rQ2TjMTBtSAPmCGy1VH94=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVGARO4kL1lDjbs7V1294zkwk+wsdAYW3+Rnf/GBa5Q8E02efLOTHbmDRLBtXHdb6ewtr6xuVXcLu3s7u0flA+P2jpOFcMWi0WsHgKqUXCJLcONwIdEIY0CgZ1gfD2rdx5RaR7LezNJ0I/oUPKQM2qsdVd9qvbLFbfmzkVWwcuhArma/fJXbxCzNEJpmKBadz03MX5GleFM4LTUSzUmlI3pELsWJY1Q+9l81Sk5s86AhLGyTxoyd39PZDTSehIFtjOiZqSXazPzv1o3NeGln3GZpAYlW3wUpoKYmMzuJgOukBkxsUCZ4nZXwkZUUWZsOiUbgrd88iq06zXP8m290rjK4yjCCZzCOXhwAQ24gSa0gMEQnuEV3hzhvDjvzseiteDkM8fwR87nD51UjVY=</latexit><latexit sha1_base64="YQKNc1rQ2TjMTBtSAPmCGy1VH94=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVGARO4kL1lDjbs7V1294zkwk+wsdAYW3+Rnf/GBa5Q8E02efLOTHbmDRLBtXHdb6ewtr6xuVXcLu3s7u0flA+P2jpOFcMWi0WsHgKqUXCJLcONwIdEIY0CgZ1gfD2rdx5RaR7LezNJ0I/oUPKQM2qsdVd9qvbLFbfmzkVWwcuhArma/fJXbxCzNEJpmKBadz03MX5GleFM4LTUSzUmlI3pELsWJY1Q+9l81Sk5s86AhLGyTxoyd39PZDTSehIFtjOiZqSXazPzv1o3NeGln3GZpAYlW3wUpoKYmMzuJgOukBkxsUCZ4nZXwkZUUWZsOiUbgrd88iq06zXP8m290rjK4yjCCZzCOXhwAQ24gSa0gMEQnuEV3hzhvDjvzseiteDkM8fwR87nD51UjVY=</latexit><latexit sha1_base64="YQKNc1rQ2TjMTBtSAPmCGy1VH94=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVGARO4kL1lDjbs7V1294zkwk+wsdAYW3+Rnf/GBa5Q8E02efLOTHbmDRLBtXHdb6ewtr6xuVXcLu3s7u0flA+P2jpOFcMWi0WsHgKqUXCJLcONwIdEIY0CgZ1gfD2rdx5RaR7LezNJ0I/oUPKQM2qsdVd9qvbLFbfmzkVWwcuhArma/fJXbxCzNEJpmKBadz03MX5GleFM4LTUSzUmlI3pELsWJY1Q+9l81Sk5s86AhLGyTxoyd39PZDTSehIFtjOiZqSXazPzv1o3NeGln3GZpAYlW3wUpoKYmMzuJgOukBkxsUCZ4nZXwkZUUWZsOiUbgrd88iq06zXP8m290rjK4yjCCZzCOXhwAQ24gSa0gMEQnuEV3hzhvDjvzseiteDkM8fwR87nD51UjVY=</latexit><latexit sha1_base64="YQKNc1rQ2TjMTBtSAPmCGy1VH94=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVGARO4kL1lDjbs7V1294zkwk+wsdAYW3+Rnf/GBa5Q8E02efLOTHbmDRLBtXHdb6ewtr6xuVXcLu3s7u0flA+P2jpOFcMWi0WsHgKqUXCJLcONwIdEIY0CgZ1gfD2rdx5RaR7LezNJ0I/oUPKQM2qsdVd9qvbLFbfmzkVWwcuhArma/fJXbxCzNEJpmKBadz03MX5GleFM4LTUSzUmlI3pELsWJY1Q+9l81Sk5s86AhLGyTxoyd39PZDTSehIFtjOiZqSXazPzv1o3NeGln3GZpAYlW3wUpoKYmMzuJgOukBkxsUCZ4nZXwkZUUWZsOiUbgrd88iq06zXP8m290rjK4yjCCZzCOXhwAQ24gSa0gMEQnuEV3hzhvDjvzseiteDkM8fwR87nD51UjVY=</latexit>

y<latexit sha1_base64="ilaY2cEC1JZTXXgQ7A6gXvo7Tdw=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVG+UjgQvaWOdiwt3fZ3TO5EH6CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqhi2WCxi1Q2oRsEltgw3AruJQhoFAjvB5HZe7zyh0jyWjyZL0I/oSPKQM2qs9VDNqoNyxa25C5F18HKoQK7moPzVH8YsjVAaJqjWPc9NjD+lynAmcFbqpxoTyiZ0hD2Lkkao/eli1Rm5sM6QhLGyTxqycH9PTGmkdRYFtjOiZqxXa3Pzv1ovNeG1P+UySQ1KtvwoTAUxMZnfTYZcITMis0CZ4nZXwsZUUWZsOiUbgrd68jq06zXP8n290rjJ4yjCGZzDJXhwBQ24gya0gMEInuEV3hzhvDjvzseyteDkM6fwR87nD57ZjVc=</latexit><latexit sha1_base64="ilaY2cEC1JZTXXgQ7A6gXvo7Tdw=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVG+UjgQvaWOdiwt3fZ3TO5EH6CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqhi2WCxi1Q2oRsEltgw3AruJQhoFAjvB5HZe7zyh0jyWjyZL0I/oSPKQM2qs9VDNqoNyxa25C5F18HKoQK7moPzVH8YsjVAaJqjWPc9NjD+lynAmcFbqpxoTyiZ0hD2Lkkao/eli1Rm5sM6QhLGyTxqycH9PTGmkdRYFtjOiZqxXa3Pzv1ovNeG1P+UySQ1KtvwoTAUxMZnfTYZcITMis0CZ4nZXwsZUUWZsOiUbgrd68jq06zXP8n290rjJ4yjCGZzDJXhwBQ24gya0gMEInuEV3hzhvDjvzseyteDkM6fwR87nD57ZjVc=</latexit><latexit sha1_base64="ilaY2cEC1JZTXXgQ7A6gXvo7Tdw=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVG+UjgQvaWOdiwt3fZ3TO5EH6CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqhi2WCxi1Q2oRsEltgw3AruJQhoFAjvB5HZe7zyh0jyWjyZL0I/oSPKQM2qs9VDNqoNyxa25C5F18HKoQK7moPzVH8YsjVAaJqjWPc9NjD+lynAmcFbqpxoTyiZ0hD2Lkkao/eli1Rm5sM6QhLGyTxqycH9PTGmkdRYFtjOiZqxXa3Pzv1ovNeG1P+UySQ1KtvwoTAUxMZnfTYZcITMis0CZ4nZXwsZUUWZsOiUbgrd68jq06zXP8n290rjJ4yjCGZzDJXhwBQ24gya0gMEInuEV3hzhvDjvzseyteDkM6fwR87nD57ZjVc=</latexit><latexit sha1_base64="ilaY2cEC1JZTXXgQ7A6gXvo7Tdw=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSVG+UjgQvaWOdiwt3fZ3TO5EH6CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqhi2WCxi1Q2oRsEltgw3AruJQhoFAjvB5HZe7zyh0jyWjyZL0I/oSPKQM2qs9VDNqoNyxa25C5F18HKoQK7moPzVH8YsjVAaJqjWPc9NjD+lynAmcFbqpxoTyiZ0hD2Lkkao/eli1Rm5sM6QhLGyTxqycH9PTGmkdRYFtjOiZqxXa3Pzv1ovNeG1P+UySQ1KtvwoTAUxMZnfTYZcITMis0CZ4nZXwsZUUWZsOiUbgrd68jq06zXP8n290rjJ4yjCGZzDJXhwBQ24gya0gMEInuEV3hzhvDjvzseyteDkM6fwR87nD57ZjVc=</latexit>

�1<latexit sha1_base64="DVsV5CtwIE7AFppaM2cihvS7qo0=">AAAB8XicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSUm8hHhQvaWPdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJrfzeueJayNi9YDThPsRHSkRCkbRWo/VPo450oFXHZQrbs1diKyDl0MFcjUH5a/+MGZpxBUySY3peW6CfkY1Cib5rNRPDU8om9AR71lUNOLGzxYbz8iFdYYkjLV9CsnC/T2R0ciYaRTYzoji2KzW5uZ/tV6K4bWfCZWkyBVbfhSmkmBM5ueTodCcoZxaoEwLuythY6opQxtSyYbgrZ68Du16zbN8X680bvI4inAG53AJHlxBA+6gCS1goOAZXuHNMc6L8+58LFsLTj5zCn/kfP4AiCSQJg==</latexit><latexit sha1_base64="DVsV5CtwIE7AFppaM2cihvS7qo0=">AAAB8XicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSUm8hHhQvaWPdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJrfzeueJayNi9YDThPsRHSkRCkbRWo/VPo450oFXHZQrbs1diKyDl0MFcjUH5a/+MGZpxBUySY3peW6CfkY1Cib5rNRPDU8om9AR71lUNOLGzxYbz8iFdYYkjLV9CsnC/T2R0ciYaRTYzoji2KzW5uZ/tV6K4bWfCZWkyBVbfhSmkmBM5ueTodCcoZxaoEwLuythY6opQxtSyYbgrZ68Du16zbN8X680bvI4inAG53AJHlxBA+6gCS1goOAZXuHNMc6L8+58LFsLTj5zCn/kfP4AiCSQJg==</latexit><latexit sha1_base64="DVsV5CtwIE7AFppaM2cihvS7qo0=">AAAB8XicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSUm8hHhQvaWPdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJrfzeueJayNi9YDThPsRHSkRCkbRWo/VPo450oFXHZQrbs1diKyDl0MFcjUH5a/+MGZpxBUySY3peW6CfkY1Cib5rNRPDU8om9AR71lUNOLGzxYbz8iFdYYkjLV9CsnC/T2R0ciYaRTYzoji2KzW5uZ/tV6K4bWfCZWkyBVbfhSmkmBM5ueTodCcoZxaoEwLuythY6opQxtSyYbgrZ68Du16zbN8X680bvI4inAG53AJHlxBA+6gCS1goOAZXuHNMc6L8+58LFsLTj5zCn/kfP4AiCSQJg==</latexit><latexit sha1_base64="DVsV5CtwIE7AFppaM2cihvS7qo0=">AAAB8XicbZA9TwJBEIbn8AvxC7W02QgmVuSORkuijSUm8hHhQvaWPdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJrfzeueJayNi9YDThPsRHSkRCkbRWo/VPo450oFXHZQrbs1diKyDl0MFcjUH5a/+MGZpxBUySY3peW6CfkY1Cib5rNRPDU8om9AR71lUNOLGzxYbz8iFdYYkjLV9CsnC/T2R0ciYaRTYzoji2KzW5uZ/tV6K4bWfCZWkyBVbfhSmkmBM5ueTodCcoZxaoEwLuythY6opQxtSyYbgrZ68Du16zbN8X680bvI4inAG53AJHlxBA+6gCS1goOAZXuHNMc6L8+58LFsLTj5zCn/kfP4AiCSQJg==</latexit>

�2<latexit sha1_base64="L1IyghwtEjik6ncX3ExNkWEZwQM=">AAAB8XicbZA9TwJBEIb38AvxC7W02QgmVuSORkuijSUm8hHhQvaWOdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41RxaPJax7gbMgBQKWihQQjfRwKJAQieY3M7rnSfQRsTqAacJ+BEbKREKztBaj9U+jgHZoF4dlCtuzV2IroOXQ4Xkag7KX/1hzNMIFHLJjOl5boJ+xjQKLmFW6qcGEsYnbAQ9i4pFYPxssfGMXlhnSMNY26eQLtzfExmLjJlGge2MGI7Nam1u/lfrpRhe+5lQSYqg+PKjMJUUYzo/nw6FBo5yaoFxLeyulI+ZZhxtSCUbgrd68jq06zXP8n290rjJ4yiSM3JOLolHrkiD3JEmaRFOFHkmr+TNMc6L8+58LFsLTj5zSv7I+fwBiamQJw==</latexit><latexit sha1_base64="L1IyghwtEjik6ncX3ExNkWEZwQM=">AAAB8XicbZA9TwJBEIb38AvxC7W02QgmVuSORkuijSUm8hHhQvaWOdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41RxaPJax7gbMgBQKWihQQjfRwKJAQieY3M7rnSfQRsTqAacJ+BEbKREKztBaj9U+jgHZoF4dlCtuzV2IroOXQ4Xkag7KX/1hzNMIFHLJjOl5boJ+xjQKLmFW6qcGEsYnbAQ9i4pFYPxssfGMXlhnSMNY26eQLtzfExmLjJlGge2MGI7Nam1u/lfrpRhe+5lQSYqg+PKjMJUUYzo/nw6FBo5yaoFxLeyulI+ZZhxtSCUbgrd68jq06zXP8n290rjJ4yiSM3JOLolHrkiD3JEmaRFOFHkmr+TNMc6L8+58LFsLTj5zSv7I+fwBiamQJw==</latexit><latexit sha1_base64="L1IyghwtEjik6ncX3ExNkWEZwQM=">AAAB8XicbZA9TwJBEIb38AvxC7W02QgmVuSORkuijSUm8hHhQvaWOdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41RxaPJax7gbMgBQKWihQQjfRwKJAQieY3M7rnSfQRsTqAacJ+BEbKREKztBaj9U+jgHZoF4dlCtuzV2IroOXQ4Xkag7KX/1hzNMIFHLJjOl5boJ+xjQKLmFW6qcGEsYnbAQ9i4pFYPxssfGMXlhnSMNY26eQLtzfExmLjJlGge2MGI7Nam1u/lfrpRhe+5lQSYqg+PKjMJUUYzo/nw6FBo5yaoFxLeyulI+ZZhxtSCUbgrd68jq06zXP8n290rjJ4yiSM3JOLolHrkiD3JEmaRFOFHkmr+TNMc6L8+58LFsLTj5zSv7I+fwBiamQJw==</latexit><latexit sha1_base64="L1IyghwtEjik6ncX3ExNkWEZwQM=">AAAB8XicbZA9TwJBEIb38AvxC7W02QgmVuSORkuijSUm8hHhQvaWOdiwt3fZnTMhF/6FjYXG2Ppv7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41RxaPJax7gbMgBQKWihQQjfRwKJAQieY3M7rnSfQRsTqAacJ+BEbKREKztBaj9U+jgHZoF4dlCtuzV2IroOXQ4Xkag7KX/1hzNMIFHLJjOl5boJ+xjQKLmFW6qcGEsYnbAQ9i4pFYPxssfGMXlhnSMNY26eQLtzfExmLjJlGge2MGI7Nam1u/lfrpRhe+5lQSYqg+PKjMJUUYzo/nw6FBo5yaoFxLeyulI+ZZhxtSCUbgrd68jq06zXP8n290rjJ4yiSM3JOLolHrkiD3JEmaRFOFHkmr+TNMc6L8+58LFsLTj5zSv7I+fwBiamQJw==</latexit>





safe region for (x,y)









safe region for (x,y)

Learn

Verify

Applications 8

Safety Verification Problem

I Goal: Check whether Y ✓ S

Y := f(X )<latexit sha1_base64="d2gfe6oFuWGC7w4SQ5ibb/XEpII=">AAACBnicbVDLSsNAFL3xWesr6lKEYCvUTUnqQhGFghuXFexD2lAm00k7dDIJMxOhhK7c+CtuXCji1m9w5984aYNo64GBM+fcy733eBGjUtn2l7GwuLS8sppby69vbG5tmzu7DRnGApM6DlkoWh6ShFFO6ooqRlqRICjwGGl6w6vUb94TIWnIb9UoIm6A+pz6FCOlpa55UOwESA0wYsnd+PzSL/18W+PjYtcs2GV7AmueOBkpQIZa1/zs9EIcB4QrzJCUbceOlJsgoShmZJzvxJJECA9Rn7Q15Sgg0k0mZ4ytI630LD8U+nFlTdTfHQkKpBwFnq5Ml5SzXir+57Vj5Z+5CeVRrAjH00F+zCwVWmkmVo8KghUbaYKwoHpXCw+QQFjp5PI6BGf25HnSqJSdk7JzUylUL7I4crAPh1ACB06hCtdQgzpgeIAneIFX49F4Nt6M92npgpH17MEfGB/fORGYSw==</latexit>



S<latexit sha1_base64="BtviJ89MRF38kIE7B/44/Itbhfw=">AAAB9HicbVBNTwIxFHyLX4hfqEcvjWDiieziQQ8eSLx4xChIAhvSLV1oaLtr2yUhG36HFw8a49Uf481/Yxf2oOAkTSYz7+VNJ4g508Z1v53C2vrG5lZxu7Szu7d/UD48ausoUYS2SMQj1QmwppxJ2jLMcNqJFcUi4PQxGN9k/uOEKs0i+WCmMfUFHkoWMoKNlfxqT2AzIpin97Nqv1xxa+4caJV4OalAjma//NUbRCQRVBrCsdZdz42Nn2JlGOF0VuolmsaYjPGQdi2VWFDtp/PQM3RmlQEKI2WfNGiu/t5IsdB6KgI7mWXUy14m/ud1ExNe+SmTcWKoJItDYcKRiVDWABowRYnhU0swUcxmRWSEFSbG9lSyJXjLX14l7XrNu6h5d/VK4zqvowgncArn4MElNOAWmtACAk/wDK/w5kycF+fd+ViMFpx85xj+wPn8AUeIkcA=</latexit>

I Challenge: Computation of Y is NP-complete– Exhaustive evaluation of an “infinite” number of input examples

I Our goal: Convex relaxation of this problem: polynomial-time solvable

Problem Statement 13

Related Work

I Exact (complete) verifiers for ReLU networks with exponential run-time(in the worst-case)

– Satisfiability Modulo Theory (SMT) [Ehlers, 2017, Katz et al., 2017,

Lomuscio and Maganti, 2017, Carlini and Wagner, 2017]

– Mixed-Integer Linear Programming (MILP) solvers

[Fischetti and Jo, 2017, Dutta et al., 2018, Tjeng et al., 2017]

Related Work


I Inexact (incomplete) verifiers– Relaxation of activation functions [Kolter and Wong, 2017,

Raghunathan et al., 2018a, Raghunathan et al., 2018b,

Mirman et al., 2018, Weng et al., 2018, Zhang et al., 2018]

Related Work


I Inexact (incomplete) verifiers

I This work: convex relaxation by adapting tools from robust control

ReLU Function

I Precisely described by 3 constraints

y = max(0, x) �) y2 = xy y � 0 y � x<latexit sha1_base64="eR6jUe4ODrkfaiYek93ukIMUXgw=">AAACbXicbZDLbhMxFIad4VaGWwCxAqEj0opWQtVMNmVTqRIsWBaJtJEyIfJ4jhOrvkxsT8loNK/Ge7BnC7wBwpMGAS3/xv/5j4999OWlFM4nyZdedO36jZu3tm7Hd+7eu/+g//DRiTOVZThiRho7zqlDKTSOvPASx6VFqnKJp/nZm65/eo7WCaM/+LrEqaJzLbhg1Ido1h/vNBk32nOqhKybJbNtFl5D5ru0jbdrOIRM0dVu8mq1B5ngHOqPw8NVDdmyogWEc45LSH6X62q1DbP+INlP1oKrJt2YAdnoeNb/mRWGVQq1Z5I6N0mT0k8bar1gEts4qxyWlJ3ROU6C1VShmzZrAi3shKQAbix0a8M6/Xuiocq5WuXhpqJ+4S73uvB/vUnl+etpI3RZedTs4iNeSfAGOpxQCBtYyToYyqwIuwJbUEuZD9DjTpnGT36BxqJqCuRCi45827z94yEoDsDSy3iumpPhfhr8++HgaLhBt0Wekhdkl6TkgByRd+SYjAgjn8lX8o187/2InkTPoucXV6PeZuYx+UfRy19h37r3</latexit><latexit sha1_base64="eR6jUe4ODrkfaiYek93ukIMUXgw=">AAACbXicbZDLbhMxFIad4VaGWwCxAqEj0opWQtVMNmVTqRIsWBaJtJEyIfJ4jhOrvkxsT8loNK/Ge7BnC7wBwpMGAS3/xv/5j4999OWlFM4nyZdedO36jZu3tm7Hd+7eu/+g//DRiTOVZThiRho7zqlDKTSOvPASx6VFqnKJp/nZm65/eo7WCaM/+LrEqaJzLbhg1Ido1h/vNBk32nOqhKybJbNtFl5D5ru0jbdrOIRM0dVu8mq1B5ngHOqPw8NVDdmyogWEc45LSH6X62q1DbP+INlP1oKrJt2YAdnoeNb/mRWGVQq1Z5I6N0mT0k8bar1gEts4qxyWlJ3ROU6C1VShmzZrAi3shKQAbix0a8M6/Xuiocq5WuXhpqJ+4S73uvB/vUnl+etpI3RZedTs4iNeSfAGOpxQCBtYyToYyqwIuwJbUEuZD9DjTpnGT36BxqJqCuRCi45827z94yEoDsDSy3iumpPhfhr8++HgaLhBt0Wekhdkl6TkgByRd+SYjAgjn8lX8o187/2InkTPoucXV6PeZuYx+UfRy19h37r3</latexit><latexit sha1_base64="eR6jUe4ODrkfaiYek93ukIMUXgw=">AAACbXicbZDLbhMxFIad4VaGWwCxAqEj0opWQtVMNmVTqRIsWBaJtJEyIfJ4jhOrvkxsT8loNK/Ge7BnC7wBwpMGAS3/xv/5j4999OWlFM4nyZdedO36jZu3tm7Hd+7eu/+g//DRiTOVZThiRho7zqlDKTSOvPASx6VFqnKJp/nZm65/eo7WCaM/+LrEqaJzLbhg1Ido1h/vNBk32nOqhKybJbNtFl5D5ru0jbdrOIRM0dVu8mq1B5ngHOqPw8NVDdmyogWEc45LSH6X62q1DbP+INlP1oKrJt2YAdnoeNb/mRWGVQq1Z5I6N0mT0k8bar1gEts4qxyWlJ3ROU6C1VShmzZrAi3shKQAbix0a8M6/Xuiocq5WuXhpqJ+4S73uvB/vUnl+etpI3RZedTs4iNeSfAGOpxQCBtYyToYyqwIuwJbUEuZD9DjTpnGT36BxqJqCuRCi45827z94yEoDsDSy3iumpPhfhr8++HgaLhBt0Wekhdkl6TkgByRd+SYjAgjn8lX8o187/2InkTPoucXV6PeZuYx+UfRy19h37r3</latexit><latexit sha1_base64="eR6jUe4ODrkfaiYek93ukIMUXgw=">AAACbXicbZDLbhMxFIad4VaGWwCxAqEj0opWQtVMNmVTqRIsWBaJtJEyIfJ4jhOrvkxsT8loNK/Ge7BnC7wBwpMGAS3/xv/5j4999OWlFM4nyZdedO36jZu3tm7Hd+7eu/+g//DRiTOVZThiRho7zqlDKTSOvPASx6VFqnKJp/nZm65/eo7WCaM/+LrEqaJzLbhg1Ido1h/vNBk32nOqhKybJbNtFl5D5ru0jbdrOIRM0dVu8mq1B5ngHOqPw8NVDdmyogWEc45LSH6X62q1DbP+INlP1oKrJt2YAdnoeNb/mRWGVQq1Z5I6N0mT0k8bar1gEts4qxyWlJ3ROU6C1VShmzZrAi3shKQAbix0a8M6/Xuiocq5WuXhpqJ+4S73uvB/vUnl+etpI3RZedTs4iNeSfAGOpxQCBtYyToYyqwIuwJbUEuZD9DjTpnGT36BxqJqCuRCi45827z94yEoDsDSy3iumpPhfhr8++HgaLhBt0Wekhdkl6TkgByRd+SYjAgjn8lX8o187/2InkTPoucXV6PeZuYx+UfRy19h37r3</latexit>

�<latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit>

�<latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit>=<latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit>




I Relaxation: for any (�, ⌫, ⌘) 2 R ⇥ R+ ⇥ R+

�(y2 � xy) + ⌫(y � x) + ⌘y � 0

I Can be written as2

4xy1

3

5> 2

40 � �⌫� �2� ⌫ + ⌘

�⌫ ⌫ + ⌘ 0

3

5

2

4xy1

3

5 � 0.

I The multipliers (�, ⌫, ⌘) will be decision variables

Abstraction of Activation Functions 21

ReLU Function

I Precisely described by 3 constraints


�<latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit>

�<latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit><latexit sha1_base64="LzWoZ6NMEx6yVIPwHqO3ZdZO5S8=">AAACxnicbVHLbhMxFHWGVxleKSzZWKRFbIgyQQiWlWDRZZGatlImijz2ncSqH4N9pyGyRmLJgq9hCz/C3+BJR4ImnM09Ovf4+j6KSkmPo9HvXnLr9p279/bupw8ePnr8pL//9Mzb2nGYcKusuyiYByUNTFCigovKAdOFgvPi8kObP78C56U1p7iuYKbZwshScoZRmvdfHoa8tAZLpqVah8/cNXmsBhxbtUnTg5yz6mDeH4yGow3oLsk6MiAdTub7vW+5sLzWYJAr5v00G1U4C8yh5AqaNK89VIxfsgVMIzVMg5+FzUANPYyKoKV1tO2CbtR/XwSmvV/rIjo1w6XfzrXi/3LTGsv3syBNVSMYfv1RWSuKlrbboUK6OLpaR8K4k7FXypfMMY5xh2ma5gZWuATrQAcBpTSyXWMTPv7lNOKGr4tNOO0IjXWYEGgVmAUuQ26F8FKAZm4hY7HX2fBtjFsmuAKz5do1IXzBlRRx+vAmHqU1pPF22faldsnZeJhF/mk8OBp3V9wjz8kL8opk5B05IsfkhEwIJ9/JD/KT/EqOE5PUyeramvS6N8/IDSRf/wDSJuAh</latexit>=<latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit><latexit sha1_base64="D5AzdP6dMhJLSx6HjzN9czX8LWU=">AAACw3icbVHLbhMxFHWGVxleLSzZWKSV2BDNpEKwQaoESCyL1LSVMlHlse8kVvwY7DstkTV7+Bq28Cn8DZ50JGjC2dyjc4+v76OslfSYZb8Hya3bd+7e27mfPnj46PGT3b2np942jsOEW2Xdeck8KGlgghIVnNcOmC4VnJXL913+7BKcl9ac4KqGmWZzIyvJGUbpYnd4EIrKGqyYlmoVvnDXFrEacOzUNk333+1HVzbK1qDbJO/JkPQ4vtgbfCuE5Y0Gg1wx76d5VuMsMIeSK2jTovFQM75kc5hGapgGPwvraVp6EBVBK+to1wJdq/++CEx7v9JldGqGC7+Z68T/5aYNVm9nQZq6QTD8+qOqURQt7VZDhXRxbrWKhHEnY6+UL5hjHOMC0zQtDFzhAqwDHQRU0shuh2348JfTiBu+PrbhpCc01mFCoFVg5rgIhRXCSwGaubmMxV7lo9cxbpjgEsyGa9uE8BWvpIjTh8N4lM6Qxtvlm5faJqfjUR755/HwaNxfcYc8Jy/IS5KTN+SIfCLHZEI4+U5+kJ/kV/IxWSYuwWtrMujfPCM3kLR/AEIZ3rA=</latexit>




� = �10, ⌫ = 2, ⌘ = 1<latexit sha1_base64="OM9CYdy5JC0S/H7VLphn6ObJygw=">AAACEXicbZC7TsMwFIadcivhFmBksWiROkCVdIGBSpVYGItEL1ITVY7jtFYdJ7IdpCrqK7DwKiwMIMTKxsbb4LQZoOVYlj79/zmyz+8njEpl299GaW19Y3OrvG3u7O7tH1iHR10ZpwKTDo5ZLPo+koRRTjqKKkb6iSAo8hnp+ZOb3O89ECFpzO/VNCFehEachhQjpaWhVau6THcHCDbhhWOfQzc/PG02CiQKNZ2qaQ6til235wVXwSmgAopqD60vN4hxGhGuMENSDhw7UV6GhKKYkZnpppIkCE/QiAw0chQR6WXzjWbwTCsBDGOhL1dwrv6eyFAk5TTydWeE1Fgue7n4nzdIVXjlZZQnqSIcLx4KUwZVDPN4YEAFwYpNNSAsqP4rxGMkEFY6xDwEZ3nlVeg26o7mu0aldV3EUQYn4BTUgAMuQQvcgjboAAwewTN4BW/Gk/FivBsfi9aSUcwcgz9lfP4A29mYkQ==</latexit><latexit sha1_base64="OM9CYdy5JC0S/H7VLphn6ObJygw=">AAACEXicbZC7TsMwFIadcivhFmBksWiROkCVdIGBSpVYGItEL1ITVY7jtFYdJ7IdpCrqK7DwKiwMIMTKxsbb4LQZoOVYlj79/zmyz+8njEpl299GaW19Y3OrvG3u7O7tH1iHR10ZpwKTDo5ZLPo+koRRTjqKKkb6iSAo8hnp+ZOb3O89ECFpzO/VNCFehEachhQjpaWhVau6THcHCDbhhWOfQzc/PG02CiQKNZ2qaQ6til235wVXwSmgAopqD60vN4hxGhGuMENSDhw7UV6GhKKYkZnpppIkCE/QiAw0chQR6WXzjWbwTCsBDGOhL1dwrv6eyFAk5TTydWeE1Fgue7n4nzdIVXjlZZQnqSIcLx4KUwZVDPN4YEAFwYpNNSAsqP4rxGMkEFY6xDwEZ3nlVeg26o7mu0aldV3EUQYn4BTUgAMuQQvcgjboAAwewTN4BW/Gk/FivBsfi9aSUcwcgz9lfP4A29mYkQ==</latexit><latexit sha1_base64="OM9CYdy5JC0S/H7VLphn6ObJygw=">AAACEXicbZC7TsMwFIadcivhFmBksWiROkCVdIGBSpVYGItEL1ITVY7jtFYdJ7IdpCrqK7DwKiwMIMTKxsbb4LQZoOVYlj79/zmyz+8njEpl299GaW19Y3OrvG3u7O7tH1iHR10ZpwKTDo5ZLPo+koRRTjqKKkb6iSAo8hnp+ZOb3O89ECFpzO/VNCFehEachhQjpaWhVau6THcHCDbhhWOfQzc/PG02CiQKNZ2qaQ6til235wVXwSmgAopqD60vN4hxGhGuMENSDhw7UV6GhKKYkZnpppIkCE/QiAw0chQR6WXzjWbwTCsBDGOhL1dwrv6eyFAk5TTydWeE1Fgue7n4nzdIVXjlZZQnqSIcLx4KUwZVDPN4YEAFwYpNNSAsqP4rxGMkEFY6xDwEZ3nlVeg26o7mu0aldV3EUQYn4BTUgAMuQQvcgjboAAwewTN4BW/Gk/FivBsfi9aSUcwcgz9lfP4A29mYkQ==</latexit><latexit sha1_base64="OM9CYdy5JC0S/H7VLphn6ObJygw=">AAACEXicbZC7TsMwFIadcivhFmBksWiROkCVdIGBSpVYGItEL1ITVY7jtFYdJ7IdpCrqK7DwKiwMIMTKxsbb4LQZoOVYlj79/zmyz+8njEpl299GaW19Y3OrvG3u7O7tH1iHR10ZpwKTDo5ZLPo+koRRTjqKKkb6iSAo8hnp+ZOb3O89ECFpzO/VNCFehEachhQjpaWhVau6THcHCDbhhWOfQzc/PG02CiQKNZ2qaQ6til235wVXwSmgAopqD60vN4hxGhGuMENSDhw7UV6GhKKYkZnpppIkCE/QiAw0chQR6WXzjWbwTCsBDGOhL1dwrv6eyFAk5TTydWeE1Fgue7n4nzdIVXjlZZQnqSIcLx4KUwZVDPN4YEAFwYpNNSAsqP4rxGMkEFY6xDwEZ3nlVeg26o7mu0aldV3EUQYn4BTUgAMuQQvcgjboAAwewTN4BW/Gk/FivBsfi9aSUcwcgz9lfP4A29mYkQ==</latexit>

� = 2, ⌫ = 2, ⌘ = 1<latexit sha1_base64="RFprRJbA+IhUZ39YhxC/wInEbFk=">AAACD3icbZBNS8MwGMfT+TbrW9Wjl+CmeJDR7qIHBwMvHie4F1jLSNN0C0vTkqTCKPsGXvwqXjwo4tWrN7+N6VZBN58Q+PH/Pw/J8/cTRqWy7S+jtLK6tr5R3jS3tnd296z9g46MU4FJG8csFj0fScIoJ21FFSO9RBAU+Yx0/fF17nfviZA05ndqkhAvQkNOQ4qR0tLAOq26THcHCDZg/Ry6+eFp4weJQg2napoDq2LX7FnBZXAKqICiWgPr0w1inEaEK8yQlH3HTpSXIaEoZmRquqkkCcJjNCR9jRxFRHrZbJ8pPNFKAMNY6MsVnKm/JzIUSTmJfN0ZITWSi14u/uf1UxVeehnlSaoIx/OHwpRBFcM8HBhQQbBiEw0IC6r/CvEICYSVjjAPwVlceRk69Zqj+bZeaV4VcZTBETgGZ8ABF6AJbkALtAEGD+AJvIBX49F4Nt6M93lryShmDsGfMj6+AfUfmCE=</latexit><latexit sha1_base64="RFprRJbA+IhUZ39YhxC/wInEbFk=">AAACD3icbZBNS8MwGMfT+TbrW9Wjl+CmeJDR7qIHBwMvHie4F1jLSNN0C0vTkqTCKPsGXvwqXjwo4tWrN7+N6VZBN58Q+PH/Pw/J8/cTRqWy7S+jtLK6tr5R3jS3tnd296z9g46MU4FJG8csFj0fScIoJ21FFSO9RBAU+Yx0/fF17nfviZA05ndqkhAvQkNOQ4qR0tLAOq26THcHCDZg/Ry6+eFp4weJQg2napoDq2LX7FnBZXAKqICiWgPr0w1inEaEK8yQlH3HTpSXIaEoZmRquqkkCcJjNCR9jRxFRHrZbJ8pPNFKAMNY6MsVnKm/JzIUSTmJfN0ZITWSi14u/uf1UxVeehnlSaoIx/OHwpRBFcM8HBhQQbBiEw0IC6r/CvEICYSVjjAPwVlceRk69Zqj+bZeaV4VcZTBETgGZ8ABF6AJbkALtAEGD+AJvIBX49F4Nt6M93lryShmDsGfMj6+AfUfmCE=</latexit><latexit sha1_base64="RFprRJbA+IhUZ39YhxC/wInEbFk=">AAACD3icbZBNS8MwGMfT+TbrW9Wjl+CmeJDR7qIHBwMvHie4F1jLSNN0C0vTkqTCKPsGXvwqXjwo4tWrN7+N6VZBN58Q+PH/Pw/J8/cTRqWy7S+jtLK6tr5R3jS3tnd296z9g46MU4FJG8csFj0fScIoJ21FFSO9RBAU+Yx0/fF17nfviZA05ndqkhAvQkNOQ4qR0tLAOq26THcHCDZg/Ry6+eFp4weJQg2napoDq2LX7FnBZXAKqICiWgPr0w1inEaEK8yQlH3HTpSXIaEoZmRquqkkCcJjNCR9jRxFRHrZbJ8pPNFKAMNY6MsVnKm/JzIUSTmJfN0ZITWSi14u/uf1UxVeehnlSaoIx/OHwpRBFcM8HBhQQbBiEw0IC6r/CvEICYSVjjAPwVlceRk69Zqj+bZeaV4VcZTBETgGZ8ABF6AJbkALtAEGD+AJvIBX49F4Nt6M93lryShmDsGfMj6+AfUfmCE=</latexit><latexit sha1_base64="RFprRJbA+IhUZ39YhxC/wInEbFk=">AAACD3icbZBNS8MwGMfT+TbrW9Wjl+CmeJDR7qIHBwMvHie4F1jLSNN0C0vTkqTCKPsGXvwqXjwo4tWrN7+N6VZBN58Q+PH/Pw/J8/cTRqWy7S+jtLK6tr5R3jS3tnd296z9g46MU4FJG8csFj0fScIoJ21FFSO9RBAU+Yx0/fF17nfviZA05ndqkhAvQkNOQ4qR0tLAOq26THcHCDZg/Ry6+eFp4weJQg2napoDq2LX7FnBZXAKqICiWgPr0w1inEaEK8yQlH3HTpSXIaEoZmRquqkkCcJjNCR9jRxFRHrZbJ8pPNFKAMNY6MsVnKm/JzIUSTmJfN0ZITWSi14u/uf1UxVeehnlSaoIx/OHwpRBFcM8HBhQQbBiEw0IC6r/CvEICYSVjjAPwVlceRk69Zqj+bZeaV4VcZTBETgGZ8ABF6AJbkALtAEGD+AJvIBX49F4Nt6M93lryShmDsGfMj6+AfUfmCE=</latexit>

� = �2, ⌫ = 2, ⌘ = 0<latexit sha1_base64="akbqnjh8WCRbIWf8HpBVgD6vqXY=">AAACEHicbZBPS8MwGMbT+W/Wf1WPXoKb6EFHu4seHAy8eJzg3GAtI03TLSxNS5IKo+wjePGrePGgiFeP3vw2plsF3XxD4MfzvC/J+/gJo1LZ9pdRWlpeWV0rr5sbm1vbO9bu3p2MU4FJG8csFl0fScIoJ21FFSPdRBAU+Yx0/NFV7nfuiZA05rdqnBAvQgNOQ4qR0lLfOq66THcHCDbgWf0UuvnhaeMHiUINu2qafati1+xpwUVwCqiAolp969MNYpxGhCvMkJQ9x06UlyGhKGZkYrqpJAnCIzQgPY0cRUR62XShCTzSSgDDWOjLFZyqvycyFEk5jnzdGSE1lPNeLv7n9VIVXngZ5UmqCMezh8KUQRXDPB0YUEGwYmMNCAuq/wrxEAmElc4wD8GZX3kR7uo1R/NNvdK8LOIogwNwCE6AA85BE1yDFmgDDB7AE3gBr8aj8Wy8Ge+z1pJRzOyDP2V8fANlRZhX</latexit><latexit sha1_base64="akbqnjh8WCRbIWf8HpBVgD6vqXY=">AAACEHicbZBPS8MwGMbT+W/Wf1WPXoKb6EFHu4seHAy8eJzg3GAtI03TLSxNS5IKo+wjePGrePGgiFeP3vw2plsF3XxD4MfzvC/J+/gJo1LZ9pdRWlpeWV0rr5sbm1vbO9bu3p2MU4FJG8csFl0fScIoJ21FFSPdRBAU+Yx0/NFV7nfuiZA05rdqnBAvQgNOQ4qR0lLfOq66THcHCDbgWf0UuvnhaeMHiUINu2qafati1+xpwUVwCqiAolp969MNYpxGhCvMkJQ9x06UlyGhKGZkYrqpJAnCIzQgPY0cRUR62XShCTzSSgDDWOjLFZyqvycyFEk5jnzdGSE1lPNeLv7n9VIVXngZ5UmqCMezh8KUQRXDPB0YUEGwYmMNCAuq/wrxEAmElc4wD8GZX3kR7uo1R/NNvdK8LOIogwNwCE6AA85BE1yDFmgDDB7AE3gBr8aj8Wy8Ge+z1pJRzOyDP2V8fANlRZhX</latexit><latexit sha1_base64="akbqnjh8WCRbIWf8HpBVgD6vqXY=">AAACEHicbZBPS8MwGMbT+W/Wf1WPXoKb6EFHu4seHAy8eJzg3GAtI03TLSxNS5IKo+wjePGrePGgiFeP3vw2plsF3XxD4MfzvC/J+/gJo1LZ9pdRWlpeWV0rr5sbm1vbO9bu3p2MU4FJG8csFl0fScIoJ21FFSPdRBAU+Yx0/NFV7nfuiZA05rdqnBAvQgNOQ4qR0lLfOq66THcHCDbgWf0UuvnhaeMHiUINu2qafati1+xpwUVwCqiAolp969MNYpxGhCvMkJQ9x06UlyGhKGZkYrqpJAnCIzQgPY0cRUR62XShCTzSSgDDWOjLFZyqvycyFEk5jnzdGSE1lPNeLv7n9VIVXngZ5UmqCMezh8KUQRXDPB0YUEGwYmMNCAuq/wrxEAmElc4wD8GZX3kR7uo1R/NNvdK8LOIogwNwCE6AA85BE1yDFmgDDB7AE3gBr8aj8Wy8Ge+z1pJRzOyDP2V8fANlRZhX</latexit><latexit sha1_base64="akbqnjh8WCRbIWf8HpBVgD6vqXY=">AAACEHicbZBPS8MwGMbT+W/Wf1WPXoKb6EFHu4seHAy8eJzg3GAtI03TLSxNS5IKo+wjePGrePGgiFeP3vw2plsF3XxD4MfzvC/J+/gJo1LZ9pdRWlpeWV0rr5sbm1vbO9bu3p2MU4FJG8csFl0fScIoJ21FFSPdRBAU+Yx0/NFV7nfuiZA05rdqnBAvQgNOQ4qR0lLfOq66THcHCDbgWf0UuvnhaeMHiUINu2qafati1+xpwUVwCqiAolp969MNYpxGhCvMkJQ9x06UlyGhKGZkYrqpJAnCIzQgPY0cRUR62XShCTzSSgDDWOjLFZyqvycyFEk5jnzdGSE1lPNeLv7n9VIVXngZ5UmqCMezh8KUQRXDPB0YUEGwYmMNCAuq/wrxEAmElc4wD8GZX3kR7uo1R/NNvdK8LOIogwNwCE6AA85BE1yDFmgDDB7AE3gBr8aj8Wy8Ge+z1pJRzOyDP2V8fANlRZhX</latexit>

� = 2, ⌫ = 0, ⌘ = 1<latexit sha1_base64="PTVZ8ahnVvS7lNK1/25iyEAMRLc=">AAACD3icbZC7TsMwFIadcivhFmBksWhBDKhKusBApUosjEWiF6mpqhPXaa06TmQ7SFXVN2DhVVgYQIiVlY23wWkzQOFYlj79/zmyzx8knCntul9WYWV1bX2juGlvbe/s7jn7By0Vp5LQJol5LDsBKMqZoE3NNKedRFKIAk7bwfg689v3VCoWizs9SWgvgqFgISOgjdR3Tss+N90DwDVcPcd+dkRac3OkGmpe2bb7TsmtuPPCf8HLoYTyavSdT38QkzSiQhMOSnU9N9G9KUjNCKcz208VTYCMYUi7BgVEVPWm831m+MQoAxzG0lyh8Vz9OTGFSKlJFJjOCPRILXuZ+J/XTXV42ZsykaSaCrJ4KEw51jHOwsEDJinRfGIAiGTmr5iMQALRJsIsBG955b/QqlY8w7fVUv0qj6OIjtAxOkMeukB1dIMaqIkIekBP6AW9Wo/Ws/VmvS9aC1Y+c4h+lfXxDfH5mB8=</latexit><latexit sha1_base64="PTVZ8ahnVvS7lNK1/25iyEAMRLc=">AAACD3icbZC7TsMwFIadcivhFmBksWhBDKhKusBApUosjEWiF6mpqhPXaa06TmQ7SFXVN2DhVVgYQIiVlY23wWkzQOFYlj79/zmyzx8knCntul9WYWV1bX2juGlvbe/s7jn7By0Vp5LQJol5LDsBKMqZoE3NNKedRFKIAk7bwfg689v3VCoWizs9SWgvgqFgISOgjdR3Tss+N90DwDVcPcd+dkRac3OkGmpe2bb7TsmtuPPCf8HLoYTyavSdT38QkzSiQhMOSnU9N9G9KUjNCKcz208VTYCMYUi7BgVEVPWm831m+MQoAxzG0lyh8Vz9OTGFSKlJFJjOCPRILXuZ+J/XTXV42ZsykaSaCrJ4KEw51jHOwsEDJinRfGIAiGTmr5iMQALRJsIsBG955b/QqlY8w7fVUv0qj6OIjtAxOkMeukB1dIMaqIkIekBP6AW9Wo/Ws/VmvS9aC1Y+c4h+lfXxDfH5mB8=</latexit><latexit sha1_base64="PTVZ8ahnVvS7lNK1/25iyEAMRLc=">AAACD3icbZC7TsMwFIadcivhFmBksWhBDKhKusBApUosjEWiF6mpqhPXaa06TmQ7SFXVN2DhVVgYQIiVlY23wWkzQOFYlj79/zmyzx8knCntul9WYWV1bX2juGlvbe/s7jn7By0Vp5LQJol5LDsBKMqZoE3NNKedRFKIAk7bwfg689v3VCoWizs9SWgvgqFgISOgjdR3Tss+N90DwDVcPcd+dkRac3OkGmpe2bb7TsmtuPPCf8HLoYTyavSdT38QkzSiQhMOSnU9N9G9KUjNCKcz208VTYCMYUi7BgVEVPWm831m+MQoAxzG0lyh8Vz9OTGFSKlJFJjOCPRILXuZ+J/XTXV42ZsykaSaCrJ4KEw51jHOwsEDJinRfGIAiGTmr5iMQALRJsIsBG955b/QqlY8w7fVUv0qj6OIjtAxOkMeukB1dIMaqIkIekBP6AW9Wo/Ws/VmvS9aC1Y+c4h+lfXxDfH5mB8=</latexit><latexit sha1_base64="PTVZ8ahnVvS7lNK1/25iyEAMRLc=">AAACD3icbZC7TsMwFIadcivhFmBksWhBDKhKusBApUosjEWiF6mpqhPXaa06TmQ7SFXVN2DhVVgYQIiVlY23wWkzQOFYlj79/zmyzx8knCntul9WYWV1bX2juGlvbe/s7jn7By0Vp5LQJol5LDsBKMqZoE3NNKedRFKIAk7bwfg689v3VCoWizs9SWgvgqFgISOgjdR3Tss+N90DwDVcPcd+dkRac3OkGmpe2bb7TsmtuPPCf8HLoYTyavSdT38QkzSiQhMOSnU9N9G9KUjNCKcz208VTYCMYUi7BgVEVPWm831m+MQoAxzG0lyh8Vz9OTGFSKlJFJjOCPRILXuZ+J/XTXV42ZsykaSaCrJ4KEw51jHOwsEDJinRfGIAiGTmr5iMQALRJsIsBG955b/QqlY8w7fVUv0qj6OIjtAxOkMeukB1dIMaqIkIekBP6AW9Wo/Ws/VmvS9aC1Y+c4h+lfXxDfH5mB8=</latexit>

I We converted the safety verification problem into an LMI feasibilityproblem

– A convex problem: can be solved in polynomial time

I Pros:– We can use various forms of QCs to abstract any type of activation

function.

– Our method is able to capture the cross-coupling between neurons across

di↵erent layers, thereby reducing conservatism, especially for deep

networks.

– We can control the trade-o↵ between computational complexity and

conservatism by systematically including or excluding di↵erent types of

QCs.

– Requires much less memory compared to similar (SDP-based) approaches

I Cons:– Limited scalability of SDP solvers.

31

E↵ect of Number of Hidden Neurons

I Network: one-layer with architecture (2-N-2) where N = 100, 500, 1000.

I ReLU activation function

I Input set: `1-norm: X = {x : kx� x?k1 ✏} with x? = (0.5, 0.5) and✏ = 0.1

-3 -2 -1 0 1 2 3 4 5

0

1

2

3

4

5

6

7

8 OutputSDP (this paper)

41 42 43 44 45 46 47 48 49 50

22

23

24

25

26

27

28

29

30OutputSDP (this paper)

4.8 5 5.2 5.4 5.6 5.8 66.8

7

7.2

7.4

7.6

7.8

8

8.2 OutputSDP (this paper)

Figure: Increase in the number of hidden nodes

37

E↵ect of Magnitude of Perturbation

I Network: one-layer with architecture (2-500-2).


I Input set: `1-norm: X = {x : kx� x?k1 ✏} with x? = (0.5, 0.5) and✏ = 0.1, 0.4, 0.8.

-6 -4 -2 0 2 4 6 8 10

2

4

6

8

10

12

14

16


1 2 3 4 5 6 7 8

5

6

7

8

9

10


4.8 5 5.2 5.4 5.6 5.8 66.8

7

7.2

7.4

7.6

7.8

8

8.2 OutputSDP (this paper)

Figure: increase in the magnitude of input set

38

E↵ect of Number of Layers

I Network: one-layer with architecture (2-100-...-100-2) with ` = 1, 2, 4, 6, 8, 10layers.


I Input set: `1-norm: X = {x : kx� x?k1 ✏} with x? = (0.5, 0.5) and✏ = 0.1.

-7 -6 -5 -4 -3 -2

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

6.5OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

10 10.5 11 11.5 12

5.5

6

6.5

7

7.5OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

-4 -3 -2 -1 0 1 2 3 4

-2

-1

0

1

2

3

4

5

6OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

12.5 13 13.5 14 14.5

8.8

9

9.2

9.4

9.6

9.8

10

10.2

10.4

10.6 OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

3.9 4 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8

-1.6

-1.5

-1.4

-1.3

-1.2

-1.1

-1

-0.9

-0.8OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

-17.5 -17 -16.5 -16 -15.5-5.5

-5

-4.5

-4

-3.5

OutputSDP (this paper)SDR (Raghunathan et al., 2018b)

Figure: increase in the number of hidden layers 39

It is with deep sorrow that I announce the passing of our colleague and friend Oded Maler, in the evening of September 3rd, 2018.Oded was a brilliant researcher, admired for his integrity and his scientific curiosity. He made substantial and original contributions in Formal Methods. He had a broad spectrum of interests including not only computer science and mathematics, but also biology and philosophy of science. His sense of humour and his human qualities, made him a dear and respected colleague to many at home and abroad.For his colleagues at Verimag, he was a source of inspiration, an example, and the dearest friend. He leaves a huge void in our community.Joseph Sifakis

Oded Maler Barbaric Science from a Captive...

Documents

Transcript of Oded Maler Barbaric Science from a Captive...