SecurityRegressionAddressingSecurityRegressionbyUnitTesting

ChristopherGrayson@_lavalamp

Introduction

WHOAMI

3

• ATL• Webdevelopment• Academicresearcher• Haxin’allthethings• (butIrlllly likenetworks)

• Founder• Redteam

@_lavalamp

• Securityregressionisahugeproblem• Lotsofinfrastructurebuiltaroundregressiontestingalready• Let’sleverageallofthatexistinginfrastructuretoimproveapplicationsecuritypostureataminimalcosttodevelopmentteams

WHY’S DIS

4

1. Background

2. DynamicSecurityTestGeneration

3. Non-dynamicSecurityTestGeneration

4. Conclusion

Agenda

5

Background

• I’vealwayslovedbreakingintothings,havebeendoingthisprofessionallysince2012• Goin,breakapp,helpclientwithremediation,checkthatremediationworked– great!• Comeback3-6monthslaterandtestagain,samevulns areback(commonlyinthesameplaces)• Offensivetestingisgoodatdiagnosing- notsolving

A Bit More on Motivation…

7

• Standardtoolinanydevelopmentteam’stoolbox• Unitteststoensurecodedoesnotregresstoapriorstateofinstability• Lotsofgreattools(especiallyintheCI/CDchain)forensuringtestsarepassingbeforedeployment

Regression Testing

8

Whynottaketheproblemofsecurityregressionanduseallofthetoolsalreadybuiltforregressiontestingtoimprovethesecuritypostureoftestedapplications?

Putting it All Together

9

• StreetArtAroundtheWorld!• WritteninDjango(standardframework,noAPI,fullpost-back)• Sametechniquesworkforanyprogramminglanguageandframeworkthatsupportintrospection• TheseexamplesrequireaframeworkthathasexplicitURLmapping

The Demo Application

10

https://github.com/lavalamp-/security-unit-testing

Dynamic Generation

• DjangorequiresuserstowriteviewsandthenexplicitlymaptheseviewstoURLrouteswheretheyareservedfrom• Viewscomefromasetofpre-definedbaseclassesthatsupportdefaultfunctionality(UpdateView,DeleteView,DetailView,FormView,etc)

Django Registered Routes

12

• Wecanuseintrospectiontoenumeratealloftheviewsregisteredwithinanapplication• Nowthatweknowtheviews,howcanwesupporttestingfunctionalitythatissuesrequeststoalloftheviewfunctionality?• EntertheRequestor class

Testing Registered Routes

13

• RequestorsmappedtoviewstheyaremeanttosendrequeststoviaPythondecorators• Singletonregistrycontainsmappingofviewstorequestors• Importingalloftheviewsautomaticallyestablishesallofthemappings

Requestor Registry Architecture

14

• Wenowcanenumeratealloftheviewsandaccessclassesthataredesignedtosubmitrequeststotheviews• Withthiscapabilitywecandynamicallygeneratetestcasesforalloftheviewsinanapplication• TestcasestakeviewclassesandHTTPverbsasargumentstoconstructors

Dynamic Test Generation

15

Ifwearerelyingonrequestorclassesbeingdefinedforallviews,thenlet’stestforit!

Testing for Requestors

16

We’vegottheabilitytotesteveryknownHTTPverbofeveryregisteredview,solet’stestforsuccessfulHTTPresponses.

Testing for Denial of Service

17

TesttoensurethatthemethodssupportedbyrequestorsmatchthemethodsreturnedbyOPTIONSrequest.

Testing for Unknown Methods

18

• Telltherequestorswhetherornotthetestedviewrequiresauthentication• Canimproveuponthisdemobycheckingforinheritanceofthe

LoginRequiredMixin• Checkthatunauthenticatedrequestisdenied

Testing for Auth Enforcement

19

Response Header Inclusion

20

WealreadybuiltoutrequestorsbasedontheOPTIONSresponse,sonowlet’smakesurethattheOPTIONSresponseincludedthecorrectHTTP

verbs.

Testing for OPTIONS Accuracy

21

TesttoensurethatCSRFtokensarerequiredforfunctioninvocationonnon-idempotentviewfunctionality.

Testing for CSRF Enforcement

22

• Wenowhaveguaranteesthat• Ourappcontainsnohiddenfunctionality• Allofourviewsareworkingasintendedgivenexpectedinput• Authenticationisbeingproperlyenforced• Securityheadersarepresent• CSRFisproperlyprotectedagainst

What Have We Gained?

23

• Thoseguaranteesaregreatandall,butcan’twejustwriteindividualunitteststotestforthem?• Inadevelopmentteamwehavemultiplepeoplecontributingcodeallthetime• Throughdynamicgeneration,thesetestswillautomaticallybeappliedtoallnewviews,providingthesameguaranteestocodethathasn’tevenbeenwrittenyet

Why Dynamic Generation?

24

• Otherthingsthatwecouldwritedynamictestsfor• Rate-limiting• FuzzingofallinputvaluestoPOST/PUT/PATCH/DELETE(introspectionintoformsusedtopowertheviews)• Properupdating,creation,anddeletionofnewmodelsbasedoninputdata

Where Can We Go?

25

Testing Other Vulns

Testforproperencodingofoutputdata!

Testing for Cross-site Scripting

27

Submittworequeststotheserver,onemakingtheSQLquerymatchnoneandanothermakingtheSQLquerymatchall,testtoseeiftheresultsmatchthe

none andall expectedresponses

Testing for SQL Injection

28

SubmitmaliciousinputandseeifHTTPredirectresponseredirectstofullURL

Testing for Open Redirects

29

Conclusion

• Initialoverheadisgreaterthanwritingindividualunittests,butnewviewsaddedtotheapplicationalsobenefitfromthetests• ProvideuswithstrongguaranteesaboutknownapplicationfunctionalityandbasicHTTP-basedsecuritycontrols

Benefits of Dynamic Generation

31

• SecurityguaranteesnowenforcedbyCI/CDintegration• TestDrivenDevelopment?Great –haveyoursecuritytesterswritefailingunitteststhatyouthenincorporateintoyourtestsuite• Anewinterfaceforhowsecurityanddevelopmentteamscanworktogetherinharmony

Benefits of Sec. Unit Testing

32

• Securityregressionisabigproblem• Wecanusethedevelopmentparadigmofregressiontestingtoaddresssecurityregression• Dynamictestgenerationcantakeusalongway• Individualtestsforindividualcasesfurtheraugmentdynamictestgenerationcapabilities

Recap

33

• SecurityUnitTestingProjecthttps://github.com/lavalamp-/security-unit-testing• Lavalamp’s PersonalBloghttps://l.avala.mp/• DjangoWebFrameworkhttps://www.djangoproject.com/

Resources

34

THANK YOU!

@_lavalampchris [AT] websight [DOT] io

github.com/lavalamp-