FIDL - pdfs.semanticscholar.org€¦ · Introduction 2 FI: A systematic way to ... and SFI tool...
Transcript of FIDL - pdfs.semanticscholar.org€¦ · Introduction 2 FI: A systematic way to ... and SFI tool...
FIDLAFaultInjectionDescriptionLanguage
forCompiler-basedTools
MaryamRaiyatAliabadiKarthikPattabiraman
UniversityofBritishColumbia(UBC)Canada
Introduction
2
FI:Asystematicwaytomodelfaults
Testingthecodeagainstfaultsisa‘must’
Background-LLVM
3
LLFI
❖Widelyusedinindustry[Lattner’05]
WeiJ,ThomasA,LiG,Pattabiraman K.Quantifyingtheaccuracyofhigh-levelfaultinjectiontechniquesforhardwarefaults.InDSN’14.
GenLLFIIndexPass
Profiling Pass FaultInjectionPass
FIInstSelector
InstTypeFIInstSelector
ControllerFICustomSelectorManager
FIRegSelecto
ProfilingLib Fault InjectionLib TraceInstructionLib
Fault Injector Manager
Fault Injector HardFaultInjectors SoftFaultInjectors
Fault-free Outcome
Profiling executable
Fault injection executable
Trace executable
Faulty outcomeTrace diff
Com
pile
Tim
eRu
n Ti
me
TracePass
Traces
API Data MemoryI/O Timing Messaging
SoftFIRegSelectors
Background- LLFI[ISSRE’14 ]
4
Auto-discoveryPass
PrimaryMotivation❖ HowtomakeLLFIprogrammable?
❖ ComplexcodingVS.Simplescripts
5
#include "llvm/IR/Instructions.h"#include "llvm/IR/Constants.h"#include "llvm/IR/IntrinsicInst.h"#include <fstream>using namespace std;namespace llfi {
class FuncArgRegSelector: publicSoftwareFIRegSelector {
public:FuncArgRegSelector(int target_arg) :
pos_argument(target_arg), specified_arg(true) {};FuncArgRegSelector():pos_argument(0),
specified_arg(false) {};private:
int pos_argument;bool specified_arg;virtual bool isRegofInstFITarget(Value *reg,
Instruction *inst);virtual bool isRegofInstFITarget(Value* reg,
Instruction* inst, int pos);};class FuncDestRegSelector: public
SoftwareFIRegSelector {private:
virtual bool isRegofInstFITarget(Value *reg, Instruction *inst);
boolFuncArgRegSelector::isRegofInstFITarget(Value *reg, Instruction *inst){
if(isa<CallInst>(inst) == false){return false }else{CallInst* CI = dyn_cast<CallInst>(inst);if(this->specified_arg == true){
Challenges- Whyanewlanguage?
6
Extensibility
distinctroles
combinatorialexplosion
Solution:FIDL
❖ FIDL: FaultInjectionDescriptionLanguage
❖ Efficiently makesSFItoolprogrammable
❖Simplified and accelerated faultmodeldesign
❖ Dynamically extendsSFItool
7
FIDL- Testmanagement
❖ Asystematicwaytofaultscenariodesign❖ Flexiblymanagescombinatorialexplosion
❖ Combinesheuristicswithfaultmodel❖ Securityanalysis
8
FIDL- Extensibility
❖FIDL: anAspect-OrientedProgramming(AOP)Language
❖ AspectsinFIDLscripts❖WeavesaspectsintothesourcecodeofSFItool❖AutomaticallyextendsSFItool
9
FIDL- Highlevelabstraction
❖Encapsulation❖Hidescompilerdetails❖Simplifiesfaultmodeldesignprocess❖SeparatesrolesoftesterandSFItooldeveloper
❖ Acceleration❖ Plug&playdesign
10
FIDLFI:Aprogrammableframework
❖FIDLFI=FIDL+Compiler-basedSFItool
11
FIDL Compiler(SFIdriver)
ApplicationUnderTest(AUT)FIDLscripts
(Faultscenarios)
Compiler-basedSFIEngineFaultmodels
FIDLScript
LLFI
FIDLCompiler Extended
LLFI
{Action, [Trigger, Target]}
AspectWeaver
{Advice, Point-Cut}Aspectspecification
Codebase
FIDLFIworkflow
12
Modifiedcode
Failure_Class:Failure_Mode:New_Failure_Mode:{
Trigger:<IRinstructions>
Trigger*:<RefinedIRinstructions>
Target:<IRregisters>
Action:<Corrupt/SetValue/Freeze/Delay/Perturb>
}
13
Faultinjectormodule
FIDLScriptSpecification
Faulttriggermodule
FIDLscriptExample- Heartbleed
1 Failure_Class: Memory2 Failure_Mode: Heartbleed3 New_Failure_Mode:4 Trigger: call: [memcpy]
6 Target: memcpy :: src[2]7 Action: Perturb :: CustomInjector 8 Custom_Injector:9 *Target = *Target + 10000;
14
ApplicationinjectedbyHeartbleed
FIDLscriptExample- Heartbleed
1 Failure_Class: Memory2 Failure_Mode: Heartbleed3 New_Failure_Mode:4 Trigger: call: [memcpy]5 Trigger*: [14,44] 6 Target: memcpy :: src[2]7 Action: Perturb :: CustomInjector 8 Custom_Injector:9 *Target = *Target + 10000;
16
ExperimentalEvaluation
• Experimentalsetup– Fivesamplefaultmodels– Fourstandardbenchmarks,andtheNull-httpdwebserver
– 2000-runSFIcampaignforeveryfaultmodel
• Evaluationmetrics– Timeoverhead– Implementationoverhead– Complexity
17
ExperimentalResults:Complexity
18
• 10Xcomplexityreduction
FaultModelFIDLScript
(LOC)FFM(LOC)
OFM(LOC)
BufferOverflow 9 96 68
Memoryleak 11 71 68
DataCorruption 8 64 61
WrongAPI 11 111 109G-Heartbleed 10 112 81
OFM (OriginalFaultModel):primarilydevelopedinLLFIinC++languageFFM(FIDL-generatedFaultModel):translatedfromFIDLscripttoC++code
ExperimentalResults:Spaceoverhead
19
• 4-18%spaceoverhead
FaultModelFIDLScript(LOC)
FFM(LOC)
OFM(LOC)
BufferOverflow 9 96 68
Memoryleak 11 71 68
DataCorruption 8 64 61
WrongAPI 11 111 109G-Heartbleed 10 112 81
OFM (OriginalFaultModel):primarilydevelopedinLLFIinC++languageFFM(FIDL-generatedFaultModel):translatedfromFIDLscripttoC++code
FFM(C++)
LLFIRecompilation
FIDLCompiler
4-5millisecond
2-3second
2-3second
OFM(C++)
FIDLscript
FaultinjectionintoApplicationUnderTest
OverallPerformance
20
LLFIRecompilation
• Maximumtimeoverhead:6.7%• Averagetimeoverhead:3.9%
21
Figure1.ComparingTimeoverhead(%)ofmemoryleak’sfaultmodelacrossbenchmarks.
TF(OFM) :AverageTimeoverheadforanoriginalfaultmodel
TF(FFM):AverageTimeoverheadforaFIDL-generatedfaultmodel
Experimentalresults:Timeoverhead
0
50
100
150
200
250
300
350
mcf sad cutcp blackscholes nullhttpd
TF(OFM) TF(FFM)
PreviousworkVsFIDLFIFITool Programmability Highlevel
AbstractionExtensibility
FIG DomainSpecificLanguage(DSL)
Yes No
LFI XML-based No No
FAIL* DSL(FAIL) Yes No
PREFAIL Policy-based Yes No
EDFI CommandLineArguments
No No
FIDLFI FIDL Yes Yes
22
Summary• FIDL:Fault Injection Description Language
– Drivescompiler-basedSFItools
– DevelopsfaultmodelsinaPlug&playfashion
– Reducesthecomplexityoffaultmodelsby10timeswithnegligibletimeoverhead.
• IntegratedintoLLFIFramework(BSDlicense)
https://github.com/DependableSystemsLab/[email protected]
23