APattern-BasedSoftwareTestingFrameworkforExploitability...

21
Research Article A Pattern-Based Software Testing Framework for Exploitability Evaluation of Metadata Corruption Vulnerabilities Fenglei Deng, Jian Wang ,BinZhang,ChaoFeng,ZhiyuanJiang,andYunfeiSu College of Electronic Science, National University of Defense Technology, Changsha 410073, China Correspondence should be addressed to Jian Wang; [email protected] and Yunfei Su; [email protected] Received 1 July 2020; Revised 4 August 2020; Accepted 9 September 2020; Published 27 September 2020 Academic Editor: Miroslav Bures Copyright © 2020 Fenglei Deng et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. In recent years, increased attention is being given to software quality assurance and protection. With considerable verification and protection schemes proposed and deployed, today’s software unfortunately still fails to be protected from cyberattacks, especially in the presence of insecure organization of heap metadata. In this paper, we aim to explore whether heap metadata could be corrupted and exploited by cyberattackers, in an attempt to assess the exploitability of vulnerabilities and ensure software quality. Tothisend,wepropose RELAY, a software testing framework to simulate human exploitation behavior for metadata corruption at the machine level. RELAY employs the heap layout serialization method to construct exploit patterns from human expertise and decomposes complex exploit-solving problems into a series of intermediate state-solving subproblems. With the heap layout procedural method, RELAY makes use of the fewer resources consumed to solve a layout problem according to the exploit pattern, activates the intermediate state, and generates the final exploit. Additionally, RELAY can be easily extended and can continuously assimilate human knowledge to enhance its ability for exploitability evaluation. Using 20 CTF&RHG programs, we then demonstrate that RELAY has the ability to evaluate the exploitability of metadata corruption vulnerabilities and works more efficiently compared with other state-of-the-art automated tools. 1.Introduction Since cyberattacks have severely damaged the quality of software [1, 2], considerable defensive schemes have been taken to protect software security [3, 4]. In recent years, due to the deployment of mitigation mechanisms for stack-based vulnerabilities and compiler-level inspections, stack exploits have become increasingly more difficult to execute against hardened programs [5, 6]. However, there are still a large number of heap-based vulnerabilities in software written in insecure programming languages such as C and C++ [7–9]. ese heap vulnerabilities increase the possibility of mali- cious attacks and pose a serious threat to software security [10]. e threat from hacking mainly comes from heap metadata corruption [11–13]. To improve performance, the heap allocator usually places dynamically allocated user data in the same memory area as the metadata that control the behavior of the allocator [14]. ese metadata are unprotected, and vulnerabilities related to processing user data may overwrite metadata, which causes subsequent operations of the heap allocator to violate security as- sumptions [15]. Since this is an inherent defect of the heap allocator, it is not limited to any specific applications. Any program that employs the heap allocator may trigger such metadata attacks. Although developers have introduced enhanced mechanisms to protect heap metadata [16, 17], such as the 2017 patch, attackers can easily bypass the protection by slightly modifying their behavior [18]. Because of this adaptation, there have still been widespread attacks on metadata [19]. AEG (automatic exploit generation) was introduced to quickly generate exploits and improve the ability to assess the exploitability of software vulnerabilities [20–23]. In recent years, heap-based AEGs have improved, and some simple exploitations for user-data corruption can be pro- cessed [24, 25]. However, the exploitation of metadata re- quires a high degree of skill, and so this process is usually Hindawi Scientific Programming Volume 2020, Article ID 8883746, 21 pages https://doi.org/10.1155/2020/8883746

Transcript of APattern-BasedSoftwareTestingFrameworkforExploitability...

Page 1: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

Research ArticleA Pattern-Based Software Testing Framework for ExploitabilityEvaluation of Metadata Corruption Vulnerabilities

Fenglei Deng Jian Wang Bin Zhang Chao Feng Zhiyuan Jiang and Yunfei Su

College of Electronic Science National University of Defense Technology Changsha 410073 China

Correspondence should be addressed to Jian Wang jwangnudteducn and Yunfei Su suyunfeinudteducn

Received 1 July 2020 Revised 4 August 2020 Accepted 9 September 2020 Published 27 September 2020

Academic Editor Miroslav Bures

Copyright copy 2020 Fenglei Deng et al -is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

In recent years increased attention is being given to software quality assurance and protection With considerable verification andprotection schemes proposed and deployed todayrsquos software unfortunately still fails to be protected from cyberattacks especiallyin the presence of insecure organization of heap metadata In this paper we aim to explore whether heap metadata could becorrupted and exploited by cyberattackers in an attempt to assess the exploitability of vulnerabilities and ensure software qualityTo this end we propose RELAY a software testing framework to simulate human exploitation behavior for metadata corruption atthe machine level RELAY employs the heap layout serialization method to construct exploit patterns from human expertise anddecomposes complex exploit-solving problems into a series of intermediate state-solving subproblems With the heap layoutprocedural method RELAYmakes use of the fewer resources consumed to solve a layout problem according to the exploit patternactivates the intermediate state and generates the final exploit Additionally RELAY can be easily extended and can continuouslyassimilate human knowledge to enhance its ability for exploitability evaluation Using 20 CTFampRHG programs we thendemonstrate that RELAY has the ability to evaluate the exploitability of metadata corruption vulnerabilities and works moreefficiently compared with other state-of-the-art automated tools

1 Introduction

Since cyberattacks have severely damaged the quality ofsoftware [1 2] considerable defensive schemes have beentaken to protect software security [3 4] In recent years dueto the deployment of mitigation mechanisms for stack-basedvulnerabilities and compiler-level inspections stack exploitshave become increasingly more difficult to execute againsthardened programs [5 6] However there are still a largenumber of heap-based vulnerabilities in software written ininsecure programming languages such as C and C++ [7ndash9]-ese heap vulnerabilities increase the possibility of mali-cious attacks and pose a serious threat to software security[10]

-e threat from hacking mainly comes from heapmetadata corruption [11ndash13] To improve performance theheap allocator usually places dynamically allocated user datain the same memory area as the metadata that control thebehavior of the allocator [14] -ese metadata are

unprotected and vulnerabilities related to processing userdata may overwrite metadata which causes subsequentoperations of the heap allocator to violate security as-sumptions [15] Since this is an inherent defect of the heapallocator it is not limited to any specific applications Anyprogram that employs the heap allocator may trigger suchmetadata attacks Although developers have introducedenhanced mechanisms to protect heap metadata [16 17]such as the 2017 patch attackers can easily bypass theprotection by slightly modifying their behavior [18] Becauseof this adaptation there have still been widespread attackson metadata [19]

AEG (automatic exploit generation) was introduced toquickly generate exploits and improve the ability to assessthe exploitability of software vulnerabilities [20ndash23] Inrecent years heap-based AEGs have improved and somesimple exploitations for user-data corruption can be pro-cessed [24 25] However the exploitation of metadata re-quires a high degree of skill and so this process is usually

HindawiScientific ProgrammingVolume 2020 Article ID 8883746 21 pageshttpsdoiorg10115520208883746

combined with human expertise [26] Existing AEGs do notintegrate the required high-level expertise well making itdifficult to complete metadata attacks Specifically we cansubdivide the problems of AEGs in metadata attacks into thefollowing two aspects

11 Coarse-GrainedHeap Layout Arrangement -e existingAEG methods lack the expertise of the intermediate ex-ploitation process and it uses a saltatory strategy to completethe migration of memory state (MMS) For example Revery[24] jumps directly from the panic state to the memory stateincluding arbitrary address writing (AAW) However inmost cases the exploitation of heap vulnerabilities espe-cially metadata corruption needs to be closely combinedwith human expertise and it is necessary to traverse in-termediate memory states in a step-by-step strategy

12 Blindness for Driving MMS -e exploitation in thekernel [27] and the interpreter [28] can directly trigger aspecific heap operation through the exposed interface toconstruct the target memory state Heaphopper [29] andArcheap [11] search for exploitation primitives by con-necting the heap allocator to the driver program -eyanalyze the exploitability of the metadata in the allocatorwithout considering exploit generation of real programsembedding in the allocator However driving the MMS foruser-level applications is still a nontrivial problem It isnecessary to satisfy the constraints of both path and statereachability and solve for the corresponding input ExistingAEG drives MMS through a blind search and it is difficult tosolve the above constraints

To solve the above problems we propose a new solutioncalled RELAY a software testing framework that simulateshuman exploitation behavior at the machine level to facil-itate exploitability evaluation for software vulnerabilitiesRELAY employs exploit patterns converted from humanexpertise to guide MMS and it can generate exploits formetadata corruption

We propose a serialization method for heap layoutwhich transforms the human expertise of exploitation into asequence of memory states that the machine can understandand solve -e process of a metadata attack can be viewed asa combination of memory states following in a specific orderand the attack will fail even if there is only a slight change inthis order -ese memory states and their sorting processesare highly dependent on artificial knowledge such as Unlink[30]Hof [31] exploit methods Based on this informationRELAY employs a predicate-based memory characterizationmethod to depict the critical memory state of exploitationand build a relationship chain of state migration called anexploit pattern -rough building an exploit pattern we canrefine the memory status of the heap layout making thefuzzy and disordered heap layout more ordered At the sametime it also converts a single difficult migration problem ofmemory state into multiple simple problems which reducethe difficulty in solving constraints

RELAY builds exploit pattern at the logical level and thenneeds to construct a state migration path (psm) at the

machine level and drives programs to complete the MMSBased on the heap layout serialization method we employ anovel heap layout procedural method to drive the MMSSpecifically with the memory operation primitive (MOP)graph and heap allocator model RELAY first solves a set ofheap operation sequences which satisfy numerical andstructural constraints to build a psm and then approximatesthe target memory state by exploring paths generating fi-nally corresponding inputs With the heap layout proceduralmethod RELAY can effectively drive the process of ex-ploitation at a small cost

In summary this paper makes the followingcontributions

(i) We propose a new solution RELAY a softwaretesting framework for simulating human exploita-tion behavior at the machine level which canevaluate exploitability for metadata corruptionvulnerabilities In particular RELAY can be easilyextended and can continuously assimilate humanexpertise to enhance its ability for exploitabilityevaluation

(ii) We propose a serialization method of heap layoutwhich transforms the high-level expertise of ex-ploitation into an orderly memory state combina-tion that the machine can understand and solve

(iii) We propose a novel procedural method of heaplayout to achieve MMS driving and activate exploitpatterns at the machine level

(iv) We implement a prototype of RELAY and dem-onstrate its effectiveness with CTF (Capture theFlag) and RHG (Robot Hacking Game) programs

-e rest of this paper is organized as follows Section 2describes research motivation and presents the overview ofRELAY Section 3 proposes the heap layout serializationmethod to construct exploit patterns Section 4 proposes theheap layout procedural method to manipulate the heaplayout according to the exploit patterns in Section 5 weexplain the implementation of RELAY and give an experi-mental evaluation of the system Section 6 summarizes thework that is most relevant to ours finally we summarize thework and discuss the potential research orientation in thefuture in Section 7

2 Motivation Example

In this section we display exploitation for a common vul-nerability to illustrate the challenges faced by existing AEGsand introduce our solution

21 Metadata Exploitation Example -e heap allocatorusually maintains metadata (such as size presize fd and bkin ptmalloc [32]) to manage heap blocks and control heapbehavior Since the heap allocator does not fully protect themetadata if developers do not pay attention to the safetyrules when programming and invoke the heap allocatorwithout due consideration a hacker will be able to break thesoftware security by corrupting the metadata Due to the

2 Scientific Programming

above reasons metadata attacks are widely used throughoutthe real world

Security analysts have further summarized some attackmethods against metadata corruption such as FastBin attack[33] Unlink Hof and chunk overlapampFastBin attack [34]For FastBin attack we need to modify the fd pointer of thereleased heap block and fake a FastBin heap block and thentake down the forged heap block from the FastBin linked listthrough multiple allocations resulting in arbitrary addresswriting (AAW) ForUnlink we need to modify the fd and bkpointers of the freed chunk and trigger the unlink operationby merging heap blocks for an AAW For Hof we need tomodify the size of TopChunk calculate the distance betweentarget address and current TopChunk address and triggerthe allocation with a larger size Finally the system willreturn the target memory block to users as an allocatedchunk For chunk overlapampFastBin attack we need to reusethe common memory between two heap blocks and thustamper with fd pointer to continue with FastBin attack(more specific details will be described later)

It is complicated to implement these exploitationmethods including precise calculation and organized layoutarrangement But as shown above the metadata may bedamaged easily by heap overflow Use-After-Free (UAF) off-by-one [35] and other vulnerabilities due to inadequateprotection So these exploitation methods are widely usedtogether with the frequent metadata attacks -erefore thecritical issue we need to solve is how to combine thesepopular but complicated exploitation methods to strengthenthe ability of the machine to evaluate the exploitability ofvulnerabilities automatically and thus maintain the softwaresecurity

Next we will present the existing problems of AEGthrough case analysis and explore the solutions We use thecommon off-by-one vulnerability as an example which is avulnerability often used to destroy size metadata combinedwith the FastBin attack and unlink methods to cause arbi-trary code execution Figure 1 shows a typical exploitationprocess of an off-by-one vulnerability -e memory opera-tions are required at each step and the correspondingcritical memory states are shown as follows

(1) By overflowing one byte the last byte of the sizemetadata of allocated chunk 2 can be controllable

(2) By freeing chunk 2 and then reoccupying the po-sition chunk 2 can be extended and overlapped withchunk 3

(3) By releasing chunk 3 and then controlling its con-tents with the extended chunk 2 fd pointer of chunk3 can be controllable

(4) By allocating chunks multiple times memory 4(callable memory) pointed by the controlled fdpointer can be allocated We achieve an arbitraryaddress allocation in this step

(5) By writing data to chunk 4 this callable memory iscontrollable

(6) By invokingmemory 4 the program reaches the stateof control-flow hijacking

22 Challenge -e above process can represent mostmetadata attacks and they usually need to complete themigration between multiple states by multiple steps -eexisting AEG method cannot refine these intermediatememory states It is difficult to directly search for primitivessuch as control-flow hijacking and arbitrary address writing(AAW) from the panic state Besides it can be seen that theexploitation of metadata requires an in-depth understandingof the internal mechanisms of the heap allocator the defi-nition and ordering of intermediate states during exploi-tation is also based on human expertise Existing AEGs donot use human knowledge well and it cannot simulate thebehavior of artificial exploitation at the machine level

In addition for each step the program needs to trigger aspecific operation such as write malloc and free to enter anintermediate state After these operations are triggered thecorrection of the parameters or write content corresponds tothe solving of numerical constraints which can be com-pleted by the symbolic execution tool [36 37] How totrigger these heap operations corresponds to the solving ofstructural constraintsWe need to drive the program to entera specific path branch at a specific time to solve thesestructural constraints which is usually done by exploring thepath with fuzzing tools [38 39] Obviously it is not easy tocomplete the oriented path coverage However the MMSusually requires triggering these operations frequently Forexample step 4 may require triggering malloc repeatedly toremove chunk 4 from the free list which often brings with itgreater complexity and consumption Existing AEGmethods have a limited ability to solve such structuralconstraints and exploring paths blindly is usually inefficient

23 Solution Figure 2 shows RELAYrsquos framework In theserialization part of the heap layout RELAY transformshuman expertise of exploitation into an exploit pattern li-brary-e procedural part of the heap layout mainly consistsof two modules the migration path construction (MPC)module and the migration path drive (MPD) module Firstthis part takes PoC as input analyzes the current memorystate and obtains the corresponding state migration pair(rsm) from the exploit pattern library -en the MPC buildsthe psm that satisfies the structural constraints in rsm andthen the MPD is responsible for driving the psm in the realenvironment and generates the corresponding input -etwo iterate through the intermediate states solve the con-straints that satisfy path reachability and state reachabilityand generate control-flow hijacking inputs Finally theexploit generation module outputs exploits to complete thespecific attack

231 Heap Layout Serialization First this module employsthe predicate-based memory characterization method todescribe rsm -en by combining the advanced knowledgeobserved in human exploitation behavior it assembles thersm to build an exploit pattern for a typical exploitationprocess -is exploit pattern is a migration chain composedof a series of intermediate states in the exploitation processFor example the migration from the state of controllable fd

Scientific Programming 3

pointer in step 3 to the state of arbitrary address allocation instep 4 can form a section of the exploit pattern

232 Migration Path Construction Module (MPC) Firstthis part obtains a control-flow graph (CFG) through staticanalysis and based on this builds a memory operationprimitive (MOP) graph from which the memory operationgroups are extracted (MOGs) -en we establish a heapallocator model to infer the memory state corresponding tothe MOG and build the psm by comparing the targetmemory states of the migration pair Finally we utilize thetriggering probability of the path to solve the psm with thesmallest driving cost For example for a migration demandfrom state 3 to state 4 the psm can be solved as MMM (Mrepresents malloc) -is module can interact with an MPD

233 Migration Path Driver Module (MPD) After receivingthe psm A A A (A for allocation) from the MPC module

the MPD drives the psm with fuzzing at the machine leveland attempts to activate state 4 If state 4 is not activated afterdriving the MPD will reselect the psm constructed by theMPC and launch it again for the current migration pair Ifactivated the MPD will retain the input and switch to thenext migration pair 4-5 By repeatedly executing the MPCand the MPD RELAYwill activate all the intermediate statesat the machine level

234 Exploit Generation Modules -is module receives testcases that can hijack PC (program counter) and performsymbolic constraint solving on the PC and the relatedcontrollable memory space to generate the final exploits Interms of AEG for EIP-hijacking vulnerabilities such asMayhem [23] and CRAX [40] the critical part of theseschemes is that after hijacking EIP the control flow will beguided to the target position by symbolic execution andthus the PoC input is transformed into EXP input We reuse

Exppatternlibrary

Prog stateanalysis

Graphbuilding

MOGextraction

Allocatormodel

MPC

Migrationpair MPD

ExpgenerationExp

PoC

Prog

PC hijackinput

Serialization part

Procedural part

Humanexpertise

Migrationpath

Figure 2 Overview of RELAYrsquos frame PoC represents proof-of-concept

chunk1 chunk2 chunk3

chunk1 chunk2 chunk333 33 33

33 65 33

chunk133 chunk265 chunk333

chunk133 chunk265 chunk333 fd

chunk133 chunk265 chunk333 memory433

chunk133 chunk265 chunk333 memory433

1 overwrite size of chunk2

2 deallocate chunk2 and reallocate it

3 deallocate chunk3 and overwrite fd of it

4 allocate memory4 pointed to by the fd of chunk3

5 overwrite memory4

6 invoke memory4

Figure 1 Typical exploitation process for off-by-one vulnerability the number before the chunk represents its size

4 Scientific Programming

their work for our exploit generation module Specifically ifit is found that the EIP register contains a symbolic valueand we will arrange the shellcode into memory with per-missionW+X and then add a symbol constraint to make thesymbol value point to the shellcode location redirectingcontrol flow to that location and generating EXP for arbi-trary code execution Since this is a previous work we willnot repeat it in this paper

3 Serialization Method of Heap Layout

-e existing AEGs perform badly in refining the heap layoutand often ldquojumpsrdquo during the processing of the heap layoutFor example Revery [24] jumps directly from the state oftriggering a vulnerability to the state of hijacking the controlflow -e limitation is that they adopt a random explorationstrategy in the heap layout process and they are not guidedsufficiently by human expertise -erefore we need to ex-plore a method that can guide the heap layout process applythis expertise at the machine level and make the machinesolve the complex problems of metadata attacks using theway of human thinking

To this end we propose a serialization method of heaplayout combined with the way humans think about craftingexploits to decompose the problem of solving the highlycomplex process of the heap layout into multiple sub-problems with low complexity First of all we employ amachine-understandable memory descriptive language torepresent the memory attributes the data and structuralrelationships between memory objects to ensure accuratecharacterization of intermediate states -en we refine theldquofragmented knowledgerdquo of manual exploitation to modelthe migration pair and orderly combine the migration pairaccording to typical exploit methods Finally we can buildan exploit pattern that can be utilized to guide the heaplayout process

31 Predicate-Based Memory Characterization Method-is section employs a formal method of describing memorystates which can be easily translated into programminglanguages and can be implemented at the machine levelSpecifically this method characterizes the memory statethrough three elements memory objects object attributesand relationships between objects -e relationships be-tween objects include numerical and structural relations-is section first clarifies the definition of memory objects-en it describes the properties of these memory objectsand the numerical and structural relationships betweenthese objects based on predicates

311 Extended Representation of Memory ObjectsMemory objects are usually represented by a tupleONS a w where a is the address of the memory unit and w

is the width For example 0xdeadbeef 4 indicates thememory unit of consecutive 4 bytes located in the address0xdeadbeef We represent the structured memory objects

such as the heap block on the basis of the tuple -esememory objects are discussed below in the 32 bitarchitecture

Although the structure objects in the heap memory suchas heap blocks and free linked lists are complicated the basicprocessed objects are heap blocks -erefore the structureddescription of heap memory is also based on heap blocks-e heap block is divided into the allocated state and thefree-ed state and so we utilize different data structures tocharacterize them

-e heap block in the allocated state can be formalized ascA a sp sc D A M P where a represents the address ofthe chunk sp represents the size of the physical neigh-bouring chunk at the low address sc represents the size ofthe chunk D represents the data area of the chunk andAMP represents whether it is the main allocated areawhether it is mapped memory and whether the physicalneighbouring chunk at a low address is in the allocated stateAll of the elements in the formal representation of cA areONS-type memory objects (a can be regarded as a 4 bytememory block storing the chunk address) -e heap block inthe free-ed state adds a forward pointer Pfd and a backwardpointer Pbk (both are ONS-type memory objects) on the basisof cA that is cF a sp sc D A M P Pfd Pbk

Furthermore the free list can be defined asL r di dd c1 c2 cN where ci is a free-ed chunk N isof numeric type and represents the length of the free list L r

is of Boolean type and represents whether the linked list isfor random access and di and dd are only for nonrandomaccess linked lists and they are of numeric types respec-tively representing the direction of inserting and deletingnodes of the linked list (0 represents the direction from thehead of the linked list and 1 represents the direction fromthe tail of linked list) According to whether di and dd are thesame the free list for nonrandom access can be divided intoa one-way operation list (such as FastBin in ptmalloc) or atwo-way operation list (such as SmallBin)

Based on the heap block and free linked list the heapmemory can be formally described asH A LF LS LL U T R where A c1 c2 is the set ofallocated chunks in the entire memory space (A contains notonly the heap blocks allocated in the heap memory but alsothe heap blocks that attackers may forge) LFLSLLstands forFastBin SmallBin and LargeBin respectively U stands forUnsortedBin T stands for TopChunk R stands for LastRemainder (the structure of UTR is similar to a chunk inthe free-ed state so we will not go into details)

With the representation method of structured heapmemory we can complete the fine-grained description of theheap memory state For example existci isinHA(C(ciP)) de-picts the situation where the P flag of allocated memory hasoverflowed (we use C(obj) to indicate that obj are controlledby external input which is strictly defined later) andC(H Tsc) depicts that the size field of the TopChunk hasoverflowed It can be seen from the former expression thatan accurate description of the memory state must be basedon the accurate definition of the attributes of the memory

Scientific Programming 5

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 2: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

combined with human expertise [26] Existing AEGs do notintegrate the required high-level expertise well making itdifficult to complete metadata attacks Specifically we cansubdivide the problems of AEGs in metadata attacks into thefollowing two aspects

11 Coarse-GrainedHeap Layout Arrangement -e existingAEG methods lack the expertise of the intermediate ex-ploitation process and it uses a saltatory strategy to completethe migration of memory state (MMS) For example Revery[24] jumps directly from the panic state to the memory stateincluding arbitrary address writing (AAW) However inmost cases the exploitation of heap vulnerabilities espe-cially metadata corruption needs to be closely combinedwith human expertise and it is necessary to traverse in-termediate memory states in a step-by-step strategy

12 Blindness for Driving MMS -e exploitation in thekernel [27] and the interpreter [28] can directly trigger aspecific heap operation through the exposed interface toconstruct the target memory state Heaphopper [29] andArcheap [11] search for exploitation primitives by con-necting the heap allocator to the driver program -eyanalyze the exploitability of the metadata in the allocatorwithout considering exploit generation of real programsembedding in the allocator However driving the MMS foruser-level applications is still a nontrivial problem It isnecessary to satisfy the constraints of both path and statereachability and solve for the corresponding input ExistingAEG drives MMS through a blind search and it is difficult tosolve the above constraints

To solve the above problems we propose a new solutioncalled RELAY a software testing framework that simulateshuman exploitation behavior at the machine level to facil-itate exploitability evaluation for software vulnerabilitiesRELAY employs exploit patterns converted from humanexpertise to guide MMS and it can generate exploits formetadata corruption

We propose a serialization method for heap layoutwhich transforms the human expertise of exploitation into asequence of memory states that the machine can understandand solve -e process of a metadata attack can be viewed asa combination of memory states following in a specific orderand the attack will fail even if there is only a slight change inthis order -ese memory states and their sorting processesare highly dependent on artificial knowledge such as Unlink[30]Hof [31] exploit methods Based on this informationRELAY employs a predicate-based memory characterizationmethod to depict the critical memory state of exploitationand build a relationship chain of state migration called anexploit pattern -rough building an exploit pattern we canrefine the memory status of the heap layout making thefuzzy and disordered heap layout more ordered At the sametime it also converts a single difficult migration problem ofmemory state into multiple simple problems which reducethe difficulty in solving constraints

RELAY builds exploit pattern at the logical level and thenneeds to construct a state migration path (psm) at the

machine level and drives programs to complete the MMSBased on the heap layout serialization method we employ anovel heap layout procedural method to drive the MMSSpecifically with the memory operation primitive (MOP)graph and heap allocator model RELAY first solves a set ofheap operation sequences which satisfy numerical andstructural constraints to build a psm and then approximatesthe target memory state by exploring paths generating fi-nally corresponding inputs With the heap layout proceduralmethod RELAY can effectively drive the process of ex-ploitation at a small cost

In summary this paper makes the followingcontributions

(i) We propose a new solution RELAY a softwaretesting framework for simulating human exploita-tion behavior at the machine level which canevaluate exploitability for metadata corruptionvulnerabilities In particular RELAY can be easilyextended and can continuously assimilate humanexpertise to enhance its ability for exploitabilityevaluation

(ii) We propose a serialization method of heap layoutwhich transforms the high-level expertise of ex-ploitation into an orderly memory state combina-tion that the machine can understand and solve

(iii) We propose a novel procedural method of heaplayout to achieve MMS driving and activate exploitpatterns at the machine level

(iv) We implement a prototype of RELAY and dem-onstrate its effectiveness with CTF (Capture theFlag) and RHG (Robot Hacking Game) programs

-e rest of this paper is organized as follows Section 2describes research motivation and presents the overview ofRELAY Section 3 proposes the heap layout serializationmethod to construct exploit patterns Section 4 proposes theheap layout procedural method to manipulate the heaplayout according to the exploit patterns in Section 5 weexplain the implementation of RELAY and give an experi-mental evaluation of the system Section 6 summarizes thework that is most relevant to ours finally we summarize thework and discuss the potential research orientation in thefuture in Section 7

2 Motivation Example

In this section we display exploitation for a common vul-nerability to illustrate the challenges faced by existing AEGsand introduce our solution

21 Metadata Exploitation Example -e heap allocatorusually maintains metadata (such as size presize fd and bkin ptmalloc [32]) to manage heap blocks and control heapbehavior Since the heap allocator does not fully protect themetadata if developers do not pay attention to the safetyrules when programming and invoke the heap allocatorwithout due consideration a hacker will be able to break thesoftware security by corrupting the metadata Due to the

2 Scientific Programming

above reasons metadata attacks are widely used throughoutthe real world

Security analysts have further summarized some attackmethods against metadata corruption such as FastBin attack[33] Unlink Hof and chunk overlapampFastBin attack [34]For FastBin attack we need to modify the fd pointer of thereleased heap block and fake a FastBin heap block and thentake down the forged heap block from the FastBin linked listthrough multiple allocations resulting in arbitrary addresswriting (AAW) ForUnlink we need to modify the fd and bkpointers of the freed chunk and trigger the unlink operationby merging heap blocks for an AAW For Hof we need tomodify the size of TopChunk calculate the distance betweentarget address and current TopChunk address and triggerthe allocation with a larger size Finally the system willreturn the target memory block to users as an allocatedchunk For chunk overlapampFastBin attack we need to reusethe common memory between two heap blocks and thustamper with fd pointer to continue with FastBin attack(more specific details will be described later)

It is complicated to implement these exploitationmethods including precise calculation and organized layoutarrangement But as shown above the metadata may bedamaged easily by heap overflow Use-After-Free (UAF) off-by-one [35] and other vulnerabilities due to inadequateprotection So these exploitation methods are widely usedtogether with the frequent metadata attacks -erefore thecritical issue we need to solve is how to combine thesepopular but complicated exploitation methods to strengthenthe ability of the machine to evaluate the exploitability ofvulnerabilities automatically and thus maintain the softwaresecurity

Next we will present the existing problems of AEGthrough case analysis and explore the solutions We use thecommon off-by-one vulnerability as an example which is avulnerability often used to destroy size metadata combinedwith the FastBin attack and unlink methods to cause arbi-trary code execution Figure 1 shows a typical exploitationprocess of an off-by-one vulnerability -e memory opera-tions are required at each step and the correspondingcritical memory states are shown as follows

(1) By overflowing one byte the last byte of the sizemetadata of allocated chunk 2 can be controllable

(2) By freeing chunk 2 and then reoccupying the po-sition chunk 2 can be extended and overlapped withchunk 3

(3) By releasing chunk 3 and then controlling its con-tents with the extended chunk 2 fd pointer of chunk3 can be controllable

(4) By allocating chunks multiple times memory 4(callable memory) pointed by the controlled fdpointer can be allocated We achieve an arbitraryaddress allocation in this step

(5) By writing data to chunk 4 this callable memory iscontrollable

(6) By invokingmemory 4 the program reaches the stateof control-flow hijacking

22 Challenge -e above process can represent mostmetadata attacks and they usually need to complete themigration between multiple states by multiple steps -eexisting AEG method cannot refine these intermediatememory states It is difficult to directly search for primitivessuch as control-flow hijacking and arbitrary address writing(AAW) from the panic state Besides it can be seen that theexploitation of metadata requires an in-depth understandingof the internal mechanisms of the heap allocator the defi-nition and ordering of intermediate states during exploi-tation is also based on human expertise Existing AEGs donot use human knowledge well and it cannot simulate thebehavior of artificial exploitation at the machine level

In addition for each step the program needs to trigger aspecific operation such as write malloc and free to enter anintermediate state After these operations are triggered thecorrection of the parameters or write content corresponds tothe solving of numerical constraints which can be com-pleted by the symbolic execution tool [36 37] How totrigger these heap operations corresponds to the solving ofstructural constraintsWe need to drive the program to entera specific path branch at a specific time to solve thesestructural constraints which is usually done by exploring thepath with fuzzing tools [38 39] Obviously it is not easy tocomplete the oriented path coverage However the MMSusually requires triggering these operations frequently Forexample step 4 may require triggering malloc repeatedly toremove chunk 4 from the free list which often brings with itgreater complexity and consumption Existing AEGmethods have a limited ability to solve such structuralconstraints and exploring paths blindly is usually inefficient

23 Solution Figure 2 shows RELAYrsquos framework In theserialization part of the heap layout RELAY transformshuman expertise of exploitation into an exploit pattern li-brary-e procedural part of the heap layout mainly consistsof two modules the migration path construction (MPC)module and the migration path drive (MPD) module Firstthis part takes PoC as input analyzes the current memorystate and obtains the corresponding state migration pair(rsm) from the exploit pattern library -en the MPC buildsthe psm that satisfies the structural constraints in rsm andthen the MPD is responsible for driving the psm in the realenvironment and generates the corresponding input -etwo iterate through the intermediate states solve the con-straints that satisfy path reachability and state reachabilityand generate control-flow hijacking inputs Finally theexploit generation module outputs exploits to complete thespecific attack

231 Heap Layout Serialization First this module employsthe predicate-based memory characterization method todescribe rsm -en by combining the advanced knowledgeobserved in human exploitation behavior it assembles thersm to build an exploit pattern for a typical exploitationprocess -is exploit pattern is a migration chain composedof a series of intermediate states in the exploitation processFor example the migration from the state of controllable fd

Scientific Programming 3

pointer in step 3 to the state of arbitrary address allocation instep 4 can form a section of the exploit pattern

232 Migration Path Construction Module (MPC) Firstthis part obtains a control-flow graph (CFG) through staticanalysis and based on this builds a memory operationprimitive (MOP) graph from which the memory operationgroups are extracted (MOGs) -en we establish a heapallocator model to infer the memory state corresponding tothe MOG and build the psm by comparing the targetmemory states of the migration pair Finally we utilize thetriggering probability of the path to solve the psm with thesmallest driving cost For example for a migration demandfrom state 3 to state 4 the psm can be solved as MMM (Mrepresents malloc) -is module can interact with an MPD

233 Migration Path Driver Module (MPD) After receivingthe psm A A A (A for allocation) from the MPC module

the MPD drives the psm with fuzzing at the machine leveland attempts to activate state 4 If state 4 is not activated afterdriving the MPD will reselect the psm constructed by theMPC and launch it again for the current migration pair Ifactivated the MPD will retain the input and switch to thenext migration pair 4-5 By repeatedly executing the MPCand the MPD RELAYwill activate all the intermediate statesat the machine level

234 Exploit Generation Modules -is module receives testcases that can hijack PC (program counter) and performsymbolic constraint solving on the PC and the relatedcontrollable memory space to generate the final exploits Interms of AEG for EIP-hijacking vulnerabilities such asMayhem [23] and CRAX [40] the critical part of theseschemes is that after hijacking EIP the control flow will beguided to the target position by symbolic execution andthus the PoC input is transformed into EXP input We reuse

Exppatternlibrary

Prog stateanalysis

Graphbuilding

MOGextraction

Allocatormodel

MPC

Migrationpair MPD

ExpgenerationExp

PoC

Prog

PC hijackinput

Serialization part

Procedural part

Humanexpertise

Migrationpath

Figure 2 Overview of RELAYrsquos frame PoC represents proof-of-concept

chunk1 chunk2 chunk3

chunk1 chunk2 chunk333 33 33

33 65 33

chunk133 chunk265 chunk333

chunk133 chunk265 chunk333 fd

chunk133 chunk265 chunk333 memory433

chunk133 chunk265 chunk333 memory433

1 overwrite size of chunk2

2 deallocate chunk2 and reallocate it

3 deallocate chunk3 and overwrite fd of it

4 allocate memory4 pointed to by the fd of chunk3

5 overwrite memory4

6 invoke memory4

Figure 1 Typical exploitation process for off-by-one vulnerability the number before the chunk represents its size

4 Scientific Programming

their work for our exploit generation module Specifically ifit is found that the EIP register contains a symbolic valueand we will arrange the shellcode into memory with per-missionW+X and then add a symbol constraint to make thesymbol value point to the shellcode location redirectingcontrol flow to that location and generating EXP for arbi-trary code execution Since this is a previous work we willnot repeat it in this paper

3 Serialization Method of Heap Layout

-e existing AEGs perform badly in refining the heap layoutand often ldquojumpsrdquo during the processing of the heap layoutFor example Revery [24] jumps directly from the state oftriggering a vulnerability to the state of hijacking the controlflow -e limitation is that they adopt a random explorationstrategy in the heap layout process and they are not guidedsufficiently by human expertise -erefore we need to ex-plore a method that can guide the heap layout process applythis expertise at the machine level and make the machinesolve the complex problems of metadata attacks using theway of human thinking

To this end we propose a serialization method of heaplayout combined with the way humans think about craftingexploits to decompose the problem of solving the highlycomplex process of the heap layout into multiple sub-problems with low complexity First of all we employ amachine-understandable memory descriptive language torepresent the memory attributes the data and structuralrelationships between memory objects to ensure accuratecharacterization of intermediate states -en we refine theldquofragmented knowledgerdquo of manual exploitation to modelthe migration pair and orderly combine the migration pairaccording to typical exploit methods Finally we can buildan exploit pattern that can be utilized to guide the heaplayout process

31 Predicate-Based Memory Characterization Method-is section employs a formal method of describing memorystates which can be easily translated into programminglanguages and can be implemented at the machine levelSpecifically this method characterizes the memory statethrough three elements memory objects object attributesand relationships between objects -e relationships be-tween objects include numerical and structural relations-is section first clarifies the definition of memory objects-en it describes the properties of these memory objectsand the numerical and structural relationships betweenthese objects based on predicates

311 Extended Representation of Memory ObjectsMemory objects are usually represented by a tupleONS a w where a is the address of the memory unit and w

is the width For example 0xdeadbeef 4 indicates thememory unit of consecutive 4 bytes located in the address0xdeadbeef We represent the structured memory objects

such as the heap block on the basis of the tuple -esememory objects are discussed below in the 32 bitarchitecture

Although the structure objects in the heap memory suchas heap blocks and free linked lists are complicated the basicprocessed objects are heap blocks -erefore the structureddescription of heap memory is also based on heap blocks-e heap block is divided into the allocated state and thefree-ed state and so we utilize different data structures tocharacterize them

-e heap block in the allocated state can be formalized ascA a sp sc D A M P where a represents the address ofthe chunk sp represents the size of the physical neigh-bouring chunk at the low address sc represents the size ofthe chunk D represents the data area of the chunk andAMP represents whether it is the main allocated areawhether it is mapped memory and whether the physicalneighbouring chunk at a low address is in the allocated stateAll of the elements in the formal representation of cA areONS-type memory objects (a can be regarded as a 4 bytememory block storing the chunk address) -e heap block inthe free-ed state adds a forward pointer Pfd and a backwardpointer Pbk (both are ONS-type memory objects) on the basisof cA that is cF a sp sc D A M P Pfd Pbk

Furthermore the free list can be defined asL r di dd c1 c2 cN where ci is a free-ed chunk N isof numeric type and represents the length of the free list L r

is of Boolean type and represents whether the linked list isfor random access and di and dd are only for nonrandomaccess linked lists and they are of numeric types respec-tively representing the direction of inserting and deletingnodes of the linked list (0 represents the direction from thehead of the linked list and 1 represents the direction fromthe tail of linked list) According to whether di and dd are thesame the free list for nonrandom access can be divided intoa one-way operation list (such as FastBin in ptmalloc) or atwo-way operation list (such as SmallBin)

Based on the heap block and free linked list the heapmemory can be formally described asH A LF LS LL U T R where A c1 c2 is the set ofallocated chunks in the entire memory space (A contains notonly the heap blocks allocated in the heap memory but alsothe heap blocks that attackers may forge) LFLSLLstands forFastBin SmallBin and LargeBin respectively U stands forUnsortedBin T stands for TopChunk R stands for LastRemainder (the structure of UTR is similar to a chunk inthe free-ed state so we will not go into details)

With the representation method of structured heapmemory we can complete the fine-grained description of theheap memory state For example existci isinHA(C(ciP)) de-picts the situation where the P flag of allocated memory hasoverflowed (we use C(obj) to indicate that obj are controlledby external input which is strictly defined later) andC(H Tsc) depicts that the size field of the TopChunk hasoverflowed It can be seen from the former expression thatan accurate description of the memory state must be basedon the accurate definition of the attributes of the memory

Scientific Programming 5

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 3: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

above reasons metadata attacks are widely used throughoutthe real world

Security analysts have further summarized some attackmethods against metadata corruption such as FastBin attack[33] Unlink Hof and chunk overlapampFastBin attack [34]For FastBin attack we need to modify the fd pointer of thereleased heap block and fake a FastBin heap block and thentake down the forged heap block from the FastBin linked listthrough multiple allocations resulting in arbitrary addresswriting (AAW) ForUnlink we need to modify the fd and bkpointers of the freed chunk and trigger the unlink operationby merging heap blocks for an AAW For Hof we need tomodify the size of TopChunk calculate the distance betweentarget address and current TopChunk address and triggerthe allocation with a larger size Finally the system willreturn the target memory block to users as an allocatedchunk For chunk overlapampFastBin attack we need to reusethe common memory between two heap blocks and thustamper with fd pointer to continue with FastBin attack(more specific details will be described later)

It is complicated to implement these exploitationmethods including precise calculation and organized layoutarrangement But as shown above the metadata may bedamaged easily by heap overflow Use-After-Free (UAF) off-by-one [35] and other vulnerabilities due to inadequateprotection So these exploitation methods are widely usedtogether with the frequent metadata attacks -erefore thecritical issue we need to solve is how to combine thesepopular but complicated exploitation methods to strengthenthe ability of the machine to evaluate the exploitability ofvulnerabilities automatically and thus maintain the softwaresecurity

Next we will present the existing problems of AEGthrough case analysis and explore the solutions We use thecommon off-by-one vulnerability as an example which is avulnerability often used to destroy size metadata combinedwith the FastBin attack and unlink methods to cause arbi-trary code execution Figure 1 shows a typical exploitationprocess of an off-by-one vulnerability -e memory opera-tions are required at each step and the correspondingcritical memory states are shown as follows

(1) By overflowing one byte the last byte of the sizemetadata of allocated chunk 2 can be controllable

(2) By freeing chunk 2 and then reoccupying the po-sition chunk 2 can be extended and overlapped withchunk 3

(3) By releasing chunk 3 and then controlling its con-tents with the extended chunk 2 fd pointer of chunk3 can be controllable

(4) By allocating chunks multiple times memory 4(callable memory) pointed by the controlled fdpointer can be allocated We achieve an arbitraryaddress allocation in this step

(5) By writing data to chunk 4 this callable memory iscontrollable

(6) By invokingmemory 4 the program reaches the stateof control-flow hijacking

22 Challenge -e above process can represent mostmetadata attacks and they usually need to complete themigration between multiple states by multiple steps -eexisting AEG method cannot refine these intermediatememory states It is difficult to directly search for primitivessuch as control-flow hijacking and arbitrary address writing(AAW) from the panic state Besides it can be seen that theexploitation of metadata requires an in-depth understandingof the internal mechanisms of the heap allocator the defi-nition and ordering of intermediate states during exploi-tation is also based on human expertise Existing AEGs donot use human knowledge well and it cannot simulate thebehavior of artificial exploitation at the machine level

In addition for each step the program needs to trigger aspecific operation such as write malloc and free to enter anintermediate state After these operations are triggered thecorrection of the parameters or write content corresponds tothe solving of numerical constraints which can be com-pleted by the symbolic execution tool [36 37] How totrigger these heap operations corresponds to the solving ofstructural constraintsWe need to drive the program to entera specific path branch at a specific time to solve thesestructural constraints which is usually done by exploring thepath with fuzzing tools [38 39] Obviously it is not easy tocomplete the oriented path coverage However the MMSusually requires triggering these operations frequently Forexample step 4 may require triggering malloc repeatedly toremove chunk 4 from the free list which often brings with itgreater complexity and consumption Existing AEGmethods have a limited ability to solve such structuralconstraints and exploring paths blindly is usually inefficient

23 Solution Figure 2 shows RELAYrsquos framework In theserialization part of the heap layout RELAY transformshuman expertise of exploitation into an exploit pattern li-brary-e procedural part of the heap layout mainly consistsof two modules the migration path construction (MPC)module and the migration path drive (MPD) module Firstthis part takes PoC as input analyzes the current memorystate and obtains the corresponding state migration pair(rsm) from the exploit pattern library -en the MPC buildsthe psm that satisfies the structural constraints in rsm andthen the MPD is responsible for driving the psm in the realenvironment and generates the corresponding input -etwo iterate through the intermediate states solve the con-straints that satisfy path reachability and state reachabilityand generate control-flow hijacking inputs Finally theexploit generation module outputs exploits to complete thespecific attack

231 Heap Layout Serialization First this module employsthe predicate-based memory characterization method todescribe rsm -en by combining the advanced knowledgeobserved in human exploitation behavior it assembles thersm to build an exploit pattern for a typical exploitationprocess -is exploit pattern is a migration chain composedof a series of intermediate states in the exploitation processFor example the migration from the state of controllable fd

Scientific Programming 3

pointer in step 3 to the state of arbitrary address allocation instep 4 can form a section of the exploit pattern

232 Migration Path Construction Module (MPC) Firstthis part obtains a control-flow graph (CFG) through staticanalysis and based on this builds a memory operationprimitive (MOP) graph from which the memory operationgroups are extracted (MOGs) -en we establish a heapallocator model to infer the memory state corresponding tothe MOG and build the psm by comparing the targetmemory states of the migration pair Finally we utilize thetriggering probability of the path to solve the psm with thesmallest driving cost For example for a migration demandfrom state 3 to state 4 the psm can be solved as MMM (Mrepresents malloc) -is module can interact with an MPD

233 Migration Path Driver Module (MPD) After receivingthe psm A A A (A for allocation) from the MPC module

the MPD drives the psm with fuzzing at the machine leveland attempts to activate state 4 If state 4 is not activated afterdriving the MPD will reselect the psm constructed by theMPC and launch it again for the current migration pair Ifactivated the MPD will retain the input and switch to thenext migration pair 4-5 By repeatedly executing the MPCand the MPD RELAYwill activate all the intermediate statesat the machine level

234 Exploit Generation Modules -is module receives testcases that can hijack PC (program counter) and performsymbolic constraint solving on the PC and the relatedcontrollable memory space to generate the final exploits Interms of AEG for EIP-hijacking vulnerabilities such asMayhem [23] and CRAX [40] the critical part of theseschemes is that after hijacking EIP the control flow will beguided to the target position by symbolic execution andthus the PoC input is transformed into EXP input We reuse

Exppatternlibrary

Prog stateanalysis

Graphbuilding

MOGextraction

Allocatormodel

MPC

Migrationpair MPD

ExpgenerationExp

PoC

Prog

PC hijackinput

Serialization part

Procedural part

Humanexpertise

Migrationpath

Figure 2 Overview of RELAYrsquos frame PoC represents proof-of-concept

chunk1 chunk2 chunk3

chunk1 chunk2 chunk333 33 33

33 65 33

chunk133 chunk265 chunk333

chunk133 chunk265 chunk333 fd

chunk133 chunk265 chunk333 memory433

chunk133 chunk265 chunk333 memory433

1 overwrite size of chunk2

2 deallocate chunk2 and reallocate it

3 deallocate chunk3 and overwrite fd of it

4 allocate memory4 pointed to by the fd of chunk3

5 overwrite memory4

6 invoke memory4

Figure 1 Typical exploitation process for off-by-one vulnerability the number before the chunk represents its size

4 Scientific Programming

their work for our exploit generation module Specifically ifit is found that the EIP register contains a symbolic valueand we will arrange the shellcode into memory with per-missionW+X and then add a symbol constraint to make thesymbol value point to the shellcode location redirectingcontrol flow to that location and generating EXP for arbi-trary code execution Since this is a previous work we willnot repeat it in this paper

3 Serialization Method of Heap Layout

-e existing AEGs perform badly in refining the heap layoutand often ldquojumpsrdquo during the processing of the heap layoutFor example Revery [24] jumps directly from the state oftriggering a vulnerability to the state of hijacking the controlflow -e limitation is that they adopt a random explorationstrategy in the heap layout process and they are not guidedsufficiently by human expertise -erefore we need to ex-plore a method that can guide the heap layout process applythis expertise at the machine level and make the machinesolve the complex problems of metadata attacks using theway of human thinking

To this end we propose a serialization method of heaplayout combined with the way humans think about craftingexploits to decompose the problem of solving the highlycomplex process of the heap layout into multiple sub-problems with low complexity First of all we employ amachine-understandable memory descriptive language torepresent the memory attributes the data and structuralrelationships between memory objects to ensure accuratecharacterization of intermediate states -en we refine theldquofragmented knowledgerdquo of manual exploitation to modelthe migration pair and orderly combine the migration pairaccording to typical exploit methods Finally we can buildan exploit pattern that can be utilized to guide the heaplayout process

31 Predicate-Based Memory Characterization Method-is section employs a formal method of describing memorystates which can be easily translated into programminglanguages and can be implemented at the machine levelSpecifically this method characterizes the memory statethrough three elements memory objects object attributesand relationships between objects -e relationships be-tween objects include numerical and structural relations-is section first clarifies the definition of memory objects-en it describes the properties of these memory objectsand the numerical and structural relationships betweenthese objects based on predicates

311 Extended Representation of Memory ObjectsMemory objects are usually represented by a tupleONS a w where a is the address of the memory unit and w

is the width For example 0xdeadbeef 4 indicates thememory unit of consecutive 4 bytes located in the address0xdeadbeef We represent the structured memory objects

such as the heap block on the basis of the tuple -esememory objects are discussed below in the 32 bitarchitecture

Although the structure objects in the heap memory suchas heap blocks and free linked lists are complicated the basicprocessed objects are heap blocks -erefore the structureddescription of heap memory is also based on heap blocks-e heap block is divided into the allocated state and thefree-ed state and so we utilize different data structures tocharacterize them

-e heap block in the allocated state can be formalized ascA a sp sc D A M P where a represents the address ofthe chunk sp represents the size of the physical neigh-bouring chunk at the low address sc represents the size ofthe chunk D represents the data area of the chunk andAMP represents whether it is the main allocated areawhether it is mapped memory and whether the physicalneighbouring chunk at a low address is in the allocated stateAll of the elements in the formal representation of cA areONS-type memory objects (a can be regarded as a 4 bytememory block storing the chunk address) -e heap block inthe free-ed state adds a forward pointer Pfd and a backwardpointer Pbk (both are ONS-type memory objects) on the basisof cA that is cF a sp sc D A M P Pfd Pbk

Furthermore the free list can be defined asL r di dd c1 c2 cN where ci is a free-ed chunk N isof numeric type and represents the length of the free list L r

is of Boolean type and represents whether the linked list isfor random access and di and dd are only for nonrandomaccess linked lists and they are of numeric types respec-tively representing the direction of inserting and deletingnodes of the linked list (0 represents the direction from thehead of the linked list and 1 represents the direction fromthe tail of linked list) According to whether di and dd are thesame the free list for nonrandom access can be divided intoa one-way operation list (such as FastBin in ptmalloc) or atwo-way operation list (such as SmallBin)

Based on the heap block and free linked list the heapmemory can be formally described asH A LF LS LL U T R where A c1 c2 is the set ofallocated chunks in the entire memory space (A contains notonly the heap blocks allocated in the heap memory but alsothe heap blocks that attackers may forge) LFLSLLstands forFastBin SmallBin and LargeBin respectively U stands forUnsortedBin T stands for TopChunk R stands for LastRemainder (the structure of UTR is similar to a chunk inthe free-ed state so we will not go into details)

With the representation method of structured heapmemory we can complete the fine-grained description of theheap memory state For example existci isinHA(C(ciP)) de-picts the situation where the P flag of allocated memory hasoverflowed (we use C(obj) to indicate that obj are controlledby external input which is strictly defined later) andC(H Tsc) depicts that the size field of the TopChunk hasoverflowed It can be seen from the former expression thatan accurate description of the memory state must be basedon the accurate definition of the attributes of the memory

Scientific Programming 5

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 4: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

pointer in step 3 to the state of arbitrary address allocation instep 4 can form a section of the exploit pattern

232 Migration Path Construction Module (MPC) Firstthis part obtains a control-flow graph (CFG) through staticanalysis and based on this builds a memory operationprimitive (MOP) graph from which the memory operationgroups are extracted (MOGs) -en we establish a heapallocator model to infer the memory state corresponding tothe MOG and build the psm by comparing the targetmemory states of the migration pair Finally we utilize thetriggering probability of the path to solve the psm with thesmallest driving cost For example for a migration demandfrom state 3 to state 4 the psm can be solved as MMM (Mrepresents malloc) -is module can interact with an MPD

233 Migration Path Driver Module (MPD) After receivingthe psm A A A (A for allocation) from the MPC module

the MPD drives the psm with fuzzing at the machine leveland attempts to activate state 4 If state 4 is not activated afterdriving the MPD will reselect the psm constructed by theMPC and launch it again for the current migration pair Ifactivated the MPD will retain the input and switch to thenext migration pair 4-5 By repeatedly executing the MPCand the MPD RELAYwill activate all the intermediate statesat the machine level

234 Exploit Generation Modules -is module receives testcases that can hijack PC (program counter) and performsymbolic constraint solving on the PC and the relatedcontrollable memory space to generate the final exploits Interms of AEG for EIP-hijacking vulnerabilities such asMayhem [23] and CRAX [40] the critical part of theseschemes is that after hijacking EIP the control flow will beguided to the target position by symbolic execution andthus the PoC input is transformed into EXP input We reuse

Exppatternlibrary

Prog stateanalysis

Graphbuilding

MOGextraction

Allocatormodel

MPC

Migrationpair MPD

ExpgenerationExp

PoC

Prog

PC hijackinput

Serialization part

Procedural part

Humanexpertise

Migrationpath

Figure 2 Overview of RELAYrsquos frame PoC represents proof-of-concept

chunk1 chunk2 chunk3

chunk1 chunk2 chunk333 33 33

33 65 33

chunk133 chunk265 chunk333

chunk133 chunk265 chunk333 fd

chunk133 chunk265 chunk333 memory433

chunk133 chunk265 chunk333 memory433

1 overwrite size of chunk2

2 deallocate chunk2 and reallocate it

3 deallocate chunk3 and overwrite fd of it

4 allocate memory4 pointed to by the fd of chunk3

5 overwrite memory4

6 invoke memory4

Figure 1 Typical exploitation process for off-by-one vulnerability the number before the chunk represents its size

4 Scientific Programming

their work for our exploit generation module Specifically ifit is found that the EIP register contains a symbolic valueand we will arrange the shellcode into memory with per-missionW+X and then add a symbol constraint to make thesymbol value point to the shellcode location redirectingcontrol flow to that location and generating EXP for arbi-trary code execution Since this is a previous work we willnot repeat it in this paper

3 Serialization Method of Heap Layout

-e existing AEGs perform badly in refining the heap layoutand often ldquojumpsrdquo during the processing of the heap layoutFor example Revery [24] jumps directly from the state oftriggering a vulnerability to the state of hijacking the controlflow -e limitation is that they adopt a random explorationstrategy in the heap layout process and they are not guidedsufficiently by human expertise -erefore we need to ex-plore a method that can guide the heap layout process applythis expertise at the machine level and make the machinesolve the complex problems of metadata attacks using theway of human thinking

To this end we propose a serialization method of heaplayout combined with the way humans think about craftingexploits to decompose the problem of solving the highlycomplex process of the heap layout into multiple sub-problems with low complexity First of all we employ amachine-understandable memory descriptive language torepresent the memory attributes the data and structuralrelationships between memory objects to ensure accuratecharacterization of intermediate states -en we refine theldquofragmented knowledgerdquo of manual exploitation to modelthe migration pair and orderly combine the migration pairaccording to typical exploit methods Finally we can buildan exploit pattern that can be utilized to guide the heaplayout process

31 Predicate-Based Memory Characterization Method-is section employs a formal method of describing memorystates which can be easily translated into programminglanguages and can be implemented at the machine levelSpecifically this method characterizes the memory statethrough three elements memory objects object attributesand relationships between objects -e relationships be-tween objects include numerical and structural relations-is section first clarifies the definition of memory objects-en it describes the properties of these memory objectsand the numerical and structural relationships betweenthese objects based on predicates

311 Extended Representation of Memory ObjectsMemory objects are usually represented by a tupleONS a w where a is the address of the memory unit and w

is the width For example 0xdeadbeef 4 indicates thememory unit of consecutive 4 bytes located in the address0xdeadbeef We represent the structured memory objects

such as the heap block on the basis of the tuple -esememory objects are discussed below in the 32 bitarchitecture

Although the structure objects in the heap memory suchas heap blocks and free linked lists are complicated the basicprocessed objects are heap blocks -erefore the structureddescription of heap memory is also based on heap blocks-e heap block is divided into the allocated state and thefree-ed state and so we utilize different data structures tocharacterize them

-e heap block in the allocated state can be formalized ascA a sp sc D A M P where a represents the address ofthe chunk sp represents the size of the physical neigh-bouring chunk at the low address sc represents the size ofthe chunk D represents the data area of the chunk andAMP represents whether it is the main allocated areawhether it is mapped memory and whether the physicalneighbouring chunk at a low address is in the allocated stateAll of the elements in the formal representation of cA areONS-type memory objects (a can be regarded as a 4 bytememory block storing the chunk address) -e heap block inthe free-ed state adds a forward pointer Pfd and a backwardpointer Pbk (both are ONS-type memory objects) on the basisof cA that is cF a sp sc D A M P Pfd Pbk

Furthermore the free list can be defined asL r di dd c1 c2 cN where ci is a free-ed chunk N isof numeric type and represents the length of the free list L r

is of Boolean type and represents whether the linked list isfor random access and di and dd are only for nonrandomaccess linked lists and they are of numeric types respec-tively representing the direction of inserting and deletingnodes of the linked list (0 represents the direction from thehead of the linked list and 1 represents the direction fromthe tail of linked list) According to whether di and dd are thesame the free list for nonrandom access can be divided intoa one-way operation list (such as FastBin in ptmalloc) or atwo-way operation list (such as SmallBin)

Based on the heap block and free linked list the heapmemory can be formally described asH A LF LS LL U T R where A c1 c2 is the set ofallocated chunks in the entire memory space (A contains notonly the heap blocks allocated in the heap memory but alsothe heap blocks that attackers may forge) LFLSLLstands forFastBin SmallBin and LargeBin respectively U stands forUnsortedBin T stands for TopChunk R stands for LastRemainder (the structure of UTR is similar to a chunk inthe free-ed state so we will not go into details)

With the representation method of structured heapmemory we can complete the fine-grained description of theheap memory state For example existci isinHA(C(ciP)) de-picts the situation where the P flag of allocated memory hasoverflowed (we use C(obj) to indicate that obj are controlledby external input which is strictly defined later) andC(H Tsc) depicts that the size field of the TopChunk hasoverflowed It can be seen from the former expression thatan accurate description of the memory state must be basedon the accurate definition of the attributes of the memory

Scientific Programming 5

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 5: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

their work for our exploit generation module Specifically ifit is found that the EIP register contains a symbolic valueand we will arrange the shellcode into memory with per-missionW+X and then add a symbol constraint to make thesymbol value point to the shellcode location redirectingcontrol flow to that location and generating EXP for arbi-trary code execution Since this is a previous work we willnot repeat it in this paper

3 Serialization Method of Heap Layout

-e existing AEGs perform badly in refining the heap layoutand often ldquojumpsrdquo during the processing of the heap layoutFor example Revery [24] jumps directly from the state oftriggering a vulnerability to the state of hijacking the controlflow -e limitation is that they adopt a random explorationstrategy in the heap layout process and they are not guidedsufficiently by human expertise -erefore we need to ex-plore a method that can guide the heap layout process applythis expertise at the machine level and make the machinesolve the complex problems of metadata attacks using theway of human thinking

To this end we propose a serialization method of heaplayout combined with the way humans think about craftingexploits to decompose the problem of solving the highlycomplex process of the heap layout into multiple sub-problems with low complexity First of all we employ amachine-understandable memory descriptive language torepresent the memory attributes the data and structuralrelationships between memory objects to ensure accuratecharacterization of intermediate states -en we refine theldquofragmented knowledgerdquo of manual exploitation to modelthe migration pair and orderly combine the migration pairaccording to typical exploit methods Finally we can buildan exploit pattern that can be utilized to guide the heaplayout process

31 Predicate-Based Memory Characterization Method-is section employs a formal method of describing memorystates which can be easily translated into programminglanguages and can be implemented at the machine levelSpecifically this method characterizes the memory statethrough three elements memory objects object attributesand relationships between objects -e relationships be-tween objects include numerical and structural relations-is section first clarifies the definition of memory objects-en it describes the properties of these memory objectsand the numerical and structural relationships betweenthese objects based on predicates

311 Extended Representation of Memory ObjectsMemory objects are usually represented by a tupleONS a w where a is the address of the memory unit and w

is the width For example 0xdeadbeef 4 indicates thememory unit of consecutive 4 bytes located in the address0xdeadbeef We represent the structured memory objects

such as the heap block on the basis of the tuple -esememory objects are discussed below in the 32 bitarchitecture

Although the structure objects in the heap memory suchas heap blocks and free linked lists are complicated the basicprocessed objects are heap blocks -erefore the structureddescription of heap memory is also based on heap blocks-e heap block is divided into the allocated state and thefree-ed state and so we utilize different data structures tocharacterize them

-e heap block in the allocated state can be formalized ascA a sp sc D A M P where a represents the address ofthe chunk sp represents the size of the physical neigh-bouring chunk at the low address sc represents the size ofthe chunk D represents the data area of the chunk andAMP represents whether it is the main allocated areawhether it is mapped memory and whether the physicalneighbouring chunk at a low address is in the allocated stateAll of the elements in the formal representation of cA areONS-type memory objects (a can be regarded as a 4 bytememory block storing the chunk address) -e heap block inthe free-ed state adds a forward pointer Pfd and a backwardpointer Pbk (both are ONS-type memory objects) on the basisof cA that is cF a sp sc D A M P Pfd Pbk

Furthermore the free list can be defined asL r di dd c1 c2 cN where ci is a free-ed chunk N isof numeric type and represents the length of the free list L r

is of Boolean type and represents whether the linked list isfor random access and di and dd are only for nonrandomaccess linked lists and they are of numeric types respec-tively representing the direction of inserting and deletingnodes of the linked list (0 represents the direction from thehead of the linked list and 1 represents the direction fromthe tail of linked list) According to whether di and dd are thesame the free list for nonrandom access can be divided intoa one-way operation list (such as FastBin in ptmalloc) or atwo-way operation list (such as SmallBin)

Based on the heap block and free linked list the heapmemory can be formally described asH A LF LS LL U T R where A c1 c2 is the set ofallocated chunks in the entire memory space (A contains notonly the heap blocks allocated in the heap memory but alsothe heap blocks that attackers may forge) LFLSLLstands forFastBin SmallBin and LargeBin respectively U stands forUnsortedBin T stands for TopChunk R stands for LastRemainder (the structure of UTR is similar to a chunk inthe free-ed state so we will not go into details)

With the representation method of structured heapmemory we can complete the fine-grained description of theheap memory state For example existci isinHA(C(ciP)) de-picts the situation where the P flag of allocated memory hasoverflowed (we use C(obj) to indicate that obj are controlledby external input which is strictly defined later) andC(H Tsc) depicts that the size field of the TopChunk hasoverflowed It can be seen from the former expression thatan accurate description of the memory state must be basedon the accurate definition of the attributes of the memory

Scientific Programming 5

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 6: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

objects (such as controllable C) and relationships betweenmemory objects (such as ci isinHA) so we will separatelyintroduce their definition in the following sections

312 Attribute Predicate Definition We have extendedconventional memory attributes to support complex attri-bute descriptions of memory units For each memory unitwe introduced three additional attributes besides thereadable writable and executable attributes controllablecallable and chunk association as shown in Table 1

-e following three examples explain these three newattributes It can be expressed as C(HTsc) if TopChunkrsquossize field is overflowed I(0xdeadbeef 4) indicates that thedata stored in consecutive 4 bytes of memory unit located ataddress 0xdeadbeef can be directly or indirectly migrated tothe PC register (for example the data of the memory unitwhere the return address is located will be written to the PCregister when the function returns and the data in the GOTtable will be migrated to the PC if invoking the corre-sponding function) CP(0xdeadbeef 4) indicates that thedata stored in the 4 consecutive bytes of memory unit lo-cated at address 0xdeadbeef can be used as a heap blockpointer for allocation release and writing operations

313 Relation Predicate Definition -is section analyzesthe relationships between memory objects from two aspectsthe numerical relationship (cS) and the structural rela-tionship (cD) -e numerical relationship cS is used toindicate the relationship of memory objects at the numericallevel Common numerical relationships are shown inTable 2 For example it can be expressed as EQ(c1sc c2sc) ifthe size of the heap block c1 is equal to that of the heap blockc2 If the size of the heap block c meets the requirement of thefree linked list HLF it can be expressed as SZ(cscHLF)Besides representing the numerical relationship betweenobjects cD can also be used to express the relationshipbetween objects and values for example EQ(c1sc 64) in-dicates that the size of the chunk c1 is 64 bytes

-e structural relationship cS is used to indicate therelationship between memory objects at the structural level-e 9 typical structural relationships are shown in Table 3For example it can be expressed as IN(c1HLF) if the free-ed chunk c1 belongs to the FastBin list For the unstructuredmemory object mem that stores the pointer pointing to thedata area of chunk c it can be expressed as PT(mem 4 cD)An equivalence relation (IS) is used to indicate that theelements that constitute two objects are the same and thememory area occupied is completely the same that is thetwo objects are essentially the same object An above rela-tionship (HT) and below relationship (BT) are used todescribe the positional relationship of memory objects inphysical space-e forwardbackward relationship (FDBK)

is similar to the HTBT relationship -e only difference isthat the FDBK is used to indicate that the adjacency re-lationship is higher or lower in the current structured objectlevel For example FD(c1 c2) indicates that the chunk next

to the chunk c1 in the direction of high address is c2LAP(c1 c2mem) is used to indicate that there is a commonmemory object mem between c1 and c2

32 Exploit Pattern Construction Exploitation can beexpressed as the migration process of a special memorystate in the program state space -erefore we employstate-space notation to describe the exploit process State-space notation can be defined as quaternion S b O Gwhere S is the set of possible states b isin S is the initial stateof the program O is the set of possible operators and G isthe target state -is section describes the state migrationprocess at the model level by constructing the exploitpattern-e definition of the operator and the realization ofthe state migration at the machine level are described in thenext section

-e experience of artificial exploitation consists of aseries of pieces of ldquofragmented knowledgerdquo for example anldquoAAWrdquo may result in ldquocontrol-flow hijackingrdquo and ldquocon-trollable format stringrdquo may result in ldquoAAWrdquo-e process ofmanual exploitation essentially organizes these pieces ofldquofragmented knowledgerdquo in a specific order For examplethe process of implementing arbitrary code execution usingformat string vulnerabilities can be expressed as ldquoformatstring controllablerdquo⟶ ldquoAAWrdquo⟶ ldquocontrol flow hijack-ingrdquo How to represent this ldquofragmented knowledgerdquo andhow to organize it is the key to applying human expertise tothe state migration process

321 State Migration Pair We employ state migration pairs(rsm) to represent ldquofragmented knowledgerdquo rsm can beexpressed as S⟶Psm Sprime where S and Sprime are the memorystates portrayed by predicates and psm o1 o2

oN represents the migration path formed by the orderlycombination of several operations Table 4 lists severaltypical state migration pairs related to heap exploits under32 bit systems where val(x) represents the integer value ataddress x and add(o1 o2v) represents adding the objects o1and o2 (or numerical value v) on the numerical level

Table 4 does not need to ensure the completeness of thecharacterization Firstly the state migration pairs in thistable can be continuously updated according to humanexperience and secondly the source and destination statesof themigration pair do not need to be fully characterized forall memory and register objects but only focus on the criticalmemory objects structural relationships and numericalrelationships based on the human experience

322 Exploit Pattern -e artificial exploit methods can beregarded as a combination of ldquofragmented knowledgerdquo in aspecific order We combine rsm in the same order to build anexploit pattern to simulate human exploit methods

It should be noted that before constructing the exploitpattern we need to backpropagate the constraints collectedin the migration pairs to ensure the effectiveness of the

6 Scientific Programming

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 7: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

Table 3 Structural relationship between memory objects

Object relation Relation predicate DescriptionSubordinate IN(o1 o2) o1 belongs to o2Pointer PT(o1 o2) o1 pointer to o2Equivalence IS(o1 o2) o1 is same to o2Higher HT(o1 o2) o1 is higher than o2Lower BT(o1 o2) o1 is lower than o2Forward FD(o1 o2) o1 is front of o2Backward BK(o1 o2) o1 is behind o2Overlap LAP(o1 o2) o1 and o2 have the same memory mem

Table 4 Typical state migration pairs for heap vulnerabilities

Serialnumber Fragmented knowledge Migration pair

1 If callable memory is controlledit may cause control-flow hijacking C(ltp4gt)andI(ltp4gt)⟶

Psm C(PC)

2If size of top chunk is controlled

it may cause arbitrary chunk allocationin the high address direction

C(HTSc)⟶Psm IN(cHA) andC(ca) andGE(ca HTa) andLE(caadd(HTaHTSc))

3 -e data area of the allocated chunk can becontrolled by external input IN(cHA) andLE(padd(ca csc)) andGE(pcDa)⟶

PsmC(ltp 4gt )

4 If FastBin list has a controllable chunk it may causearbitrary chunk allocation IN(cHLF) andC(ca) andSZ(cscHLF)⟶

Psm IN(cprimeHA) andC(cprimea) andEQ(ca cprimea)

5 Pointer to low address minus12 bytes can becontrolled by external input

C(ltp4gt)andPT(ltp4gtltp-124gt)andW(ltp4gt)⟶

PsmC(ltp 4gt )andCP(ltp 4gt )

6If the heap block pointer is controlled a

chunk with a controllable addresscan be obtained

CP(ltp4gt)andC(ltp4gt)and SZ(ltval(val(p)+4)4gtHLF)⟶Psm IN(c HLF) and

C(ca) andEQ(caval(p))

7 Backward merge process of heap block IN(c HLs) andBCS(c c1 c2) andBCD(cc1 c2)andPT(ltp4gt c1D)⟶Psm

PT(ltp4gtltp-124gt)

8 -e writable area pointed by the heap pointercan be controlled by external input CP(ltp 4gt)andPT(ltp 4gtltpprime 4gt)andW(ltpprime 4gt)⟶

Psm C(ltpprime 4gt)

9 If size of allocated chunk is controllableit may cause chunk overlap IN(c HA) andC(csc)⟶

Psm IN(cH HA) andHT(cH c) andLap(c cH p)

10 If chunks are overlapped it may causechunk in FastBin list controllable

IN(cH HA) andHT(cH c) andLap(c cH p)⟶Psm IN(cFHLF) andC(cFa)

andSZ(cFsc HLF)Dot notation is used to indicate access to fields in an object and comma is used to separate two objects in predication

Table 1 Attribute predicate definition

Object attribute Attribute predicate DescriptionReadable R(o) Can read from the memory of object oWritable W(o) Can write to the memory of object oExecutable X(o) -e data in the memory of object o can be executableControllable C(o) -e data in the memory of object o can be controlled by inputCallable I(o) -e data in the memory of object o can migrate to PCChunk association CP(o) -e data in the memory of object o can be used as a pointer

Table 2 Numerical relationship between memory objects

Object relation Relation predicate DescriptionEqual EQ(o1 o2v) o1 equals o2numeral v

Unequal NE(o1 o2v) o1 unequals o2numeral v

Greater GT(o1 o2v) o1 is greater than o2numeral v

Greater or equal GE(o1 o2v) o1 is greater than or equal to o2numeral v

Less LT(o1 o2v) o1 is less than o2numeral v

Less or equal LE(o1 o2v) o1 is less than or equal to o2numeral v

Meet size SZ(c L) -e size of the chunk c lets the requirements of the free list L into the chain

Scientific Programming 7

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 8: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

exploit pattern For example migration pair 1 requires thatthe memory at p in the original state has a callable attributeso 3⟶ 1combination requires backpropagation I(p 4)

to the source state of 3 migration pairs ieIN(cHA)and LE(p add(ca csc))andGE(p cDa)and I(p 4)In order to facilitate expression we distinguish memory

objects in different states through the subscript (for instancep1 and p2 represent the same memory area p in differentstates) so the 3⟶ 1 combination can be expressed asthe following formula after the constraint propagation iscompleted

3⟶ 1

IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857

ΛGE langp2 c2Darang( 1113857ΛEQ langp2 p1rang( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(1)

We have established corresponding exploit patterns forcommon exploit methods FastBin attack can be expressedas 4⟶ 3⟶ 1 and the exploit pattern is as follows

fast bin exp

IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc HLF( 1113857ΛEQ c3a c2a( 1113857

⟶Psm IN c2 HA( 1113857Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(2)

Unlink can be expressed as 7⟶ 5⟶ 8⟶ 1and the exploit pattern is as follows

Unlink exp

IN c4 HLS( 1113857ΛBCS c4 c41 c421113872 1113873 ΛBCD c4 c41 c421113872 1113873ΛPT p4 4 c41D1113872 1113873

Λ EQ p4 p3( 1113857

⟶Psm CP langp3 4rang( 1113857ΛPT langp3 4rang langp3 minus 12 4rang( 1113857ΛW langp3 4rang( 1113857 ΛEQ p3 minus 12 p2( 1113857

⟶Psm CP langp2 4rang( 1113857 ΛPT langp2 4rang langp2prime 4rang( 1113857ΛW langp2prime 4rang( 1113857ΛEQ p2prime p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC) (3)

8 Scientific Programming

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 9: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

Hof can be expressed as 2⟶ 3⟶ 1 and theexploit pattern is as follows

hof exp

C HTsc( 1113857

⟶Psm IN c2 HA( 1113857ΛGE c2a HTa( 1113857Λ LE c2a add HTa HTsc( 1113857( 1113857

Λ LE p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p1 p2( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857

⟶Psm

C(PC)

(4)

-e example in Section 21 shows a typical exploitmethod for off-by-one vulnerabilities and we can call it

chunk overlapampFastBin attack which can be expressed as9⟶ 10⟶ 4⟶ 3⟶ 1

overlap exp

IN(c HA) ΛC csc( 1113857

⟶Psm

IN cH HA( 1113857ΛHT cH c( 1113857Λ Lap c cH p( 1113857

⟶Psm IN c3 HLF( 1113857ΛC c3 a( 1113857Λ SZ c3sc( 1113857ΛEQ c3a c2a( 1113857

⟶Psm

IN c2 HA( 1113857ΛC p2 add c2a c2sc( 1113857( 1113857ΛGE p2 c2Da( 1113857ΛEQ p2 p1( 1113857

⟶Psm

C langp1 4rang( 1113857Λ I langp1 4rang( 1113857⟶Psm

C(PC)

(5)

323 Guidance of Exploit Generation It can be seen that theexploit pattern can characterize themigration relationship ofthe intermediate state during the exploitation process in-cluding the data relationship and structural relationshipbetween the memory objects In order to further guideexploit generation we divide the predicates used to char-acterize the memory state into two categories numericallydependent predicates and structurally dependent predicates-e data dependency predicates refer to EQNEQ SZ andother predicates that reflect the numerical relationship be-tween memory objects structure dependency predicatesmainly include the extended memory attribute predicate(C I and CP) and IN FD and other predicates that reflectthe structural relationship between memory objects -esetwo types of predicates correspond to form two differenttypes of constraints namely numerical constraints ψD andstructural constraints ψS -e exploit pattern can transformthe problem of automatic generation of heap exploits intothe problem of solving numerical constraints ψD andstructural constraints ψS

We employed the fast bin exp exploit pattern as anexample to explain how to guide the exploit generation bysolving these two types of constraints -e exploit consists of4 steps corresponding to 4 states in fast bin exp

(1) Now the fd pointer of c3 chunk in the FastBin list iscontrollable which has satisfied the followingstructural constraints

ψS IN c3HLF( 1113857andC c3 a( 1113857 (6)

But we have not made the c3a and c2a equal whichmeans the following numerical constraints have notbeen satisfied

ψD SZ c3scHLF( 1113857andEQ c3a c2a( 1113857 (7)

In order to solve ψD we should calculate the address ofc2-e data area of c2 needs to cover p2 and p2 needs tobe callable such as malloc_hook So we can update thenumerical constraints as follows

ψD SZ c3scHLF( 1113857and LE mallc hook add c3a c3sc( 1113857( 1113857

andGE malloc hook c3Da( 1113857

(8)

We solve it by using a constraint solver(2) -e following numerical constraints have been

satisfied

Scientific Programming 9

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 10: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

ψD LE p2 add c2a c2sc( 1113857( 1113857

andGE p2 c2Da( 1113857andEQ p2 p1( 1113857(9)

-e following structural constraints require making c2become an allocated chunk

ψS IN c2HA( 1113857 (10)

If c2 is located in the Nth position from the head endof FastBin linked list (namely there are N-1 heapblocks before c2) we need according to the FIFO(first in last out) mechanism of FastBin to trigger theallocation for consecutive N times to take out c2 andthe size of the allocation needs to satisfy the re-quirements of the FastBin list

(3) Now that numerical constraints have been satisfied weonly need to solve the following structural constraints

ψS C langp1 4rang( 1113857and I langp1 4rang( 1113857 (11)

p1is malloc_hook which satisfies I(langp1 4rang) so wejust need to cover the malloc_hook to satisfyC(langp1 4rang)

(4) After controlling themalloc_hook we need to invokemalloc to transfer the data stored in the malloc_hookto the PC register which implements C(PC)

4 Procedural Method of Heap Layout

-e serialization method of heap layout divides the ex-ploitation process into an ordered set of multiple inter-mediate states at the logical level by constructing the exploitpattern and indicates the direction for the heap layout

We also need to activate the intermediate state at themachine level and implement the migration process of theheap layout

41 Framework With the exploit pattern we reduce thecomplexity of traversing the state space of the program butthe extremely large path space also places a significantburden on exploitation It is often inefficient to drive theprogram to the target state by randomly exploring theprogram path Taking the exploitation process for off-by-oneas an example in moving from state 3 to state 4 a certainnumber of heap allocation operations are required and thesize of the allocated heap block must meet the requirementsof the linked list Blindly traversing the program path willresult in a random combination of memory operations andtheir parameters which is very inefficient

-erefore we divide the heap layout process into twostages migration path construction (MPC) and migrationpath driving (MPD)-eMPC phase constructs a set of statemigration paths Psm according to the exploit pattern and theMPD phase attempts to drive the program along the psm Bybuilding the psm to guide the state migration of the programwe can reduce the complexity of traversing the path spaceand improve the driving efficiency

-e overall design of the procedural method of heaplayout is shown in Figure 3 -rough the heap layoutserialization module we build the exploit pattern -eprogram state analysis module analyzes the currentprogram state and finds rsm which source state matcheswith it We can decompose rsm into numerical constraintsψD and structural constraint ψS (as explained in theprevious section) In each round of processing an rsm thenumerical constraint solving module first corrects theinput related to ψD Next the MPC generates a Psm setaccording to the structural constraint ψS -en the MPDperforms searches according to the psm and drives thestate migration of the program In the loop we first adoptan optimization strategy For the migration path set Psmwe give priority to the one with the smallest driving costand reduce the complexity of the driving migration path asmuch as possible Secondly we add a feedback mecha-nism If the new state is not matched with the target stateafter the path is driven (the matching method will beexplained later) we give up the intermediate test case andrestore to the original state of the program remove thecurrent migration path from the set and then start a newround of retrying to drive If matched the program stateanalysis will pass the next rsm to the MPC and theMPD Byrepeatedly solving ψD and ψS RELAY can complete thewhole MSM activate all intermediate states and finallygenerate the hijacking PC input

Since the existing symbolic execution technology cansolve the numerical constraints in the exploit pattern [41]we will not repeat the process of solving ψD but explain howto automatically solve the structural constraints in the ex-ploit pattern

42 Migration Path Construction (MPC) -e MPC con-structs migration paths according to structural constraintsFirst the MPC constructs the memory operation primitive(MOP) graph through static analysis and extracts thememory operation groups (MOGs) from it -en it em-ploys the heap allocator model to solve the psm from theMOG according to the structural constraints and selects thek paths with the smallest driving cost (the relevant defi-nition will be described later) to build the optimized setPlowastsm In the MOP graph section we give a formal definitionof psm and explain why structural constraints can be solvedby using an ordered set of MOPs Based on this weconstruct a MOP graph and introduce a method to extractthe heap operation groups from it -e heap allocatormodel is relatively independent and easy to expand tosupport different heap allocators It establishes a mappingrelationship between heap operations and memory statesand it is used to examine the effect of heap operations Inorder to find the solution it takes the structural constraintsas input and employs the MOP graph to obtain all possiblegroups of heap operations -en it utilizes the heap al-locator model to solve the group of heap operations thatcan meet the structural constraints as psm and finally takesthe k paths with the smallest driving cost to form a outputset Plowastsm

10 Scientific Programming

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 11: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

421 MOP Graph -e structural constraint cannot besolved by calculation tools such as constraint solvers [42]and can only be solved by effectively combining basic op-erations on the heap memory structure We abstract thetypical heap memory structure operations into five basicmemory operation primitives allocation (A) deallocation(F) memory write (W) memory read (R) and memory call(V) which will be explained separately below

-e allocation primitives are built on the granularity offunctions such as malloc and realloc in ptmalloc whererealloc can be regarded as an allocation or a deallocation-edeallocation primitive is also built on function granularitytypically free in ptmalloc Writing and reading primitives canbe completed by an instruction (such asmov [addr] and val)basic block or external function (such as read function) Inorder to reduce the analysis complexity we define thememory writeread built on basic block granularity -ecalling primitive is built on the calling method For amemory unit with callable attributes if the target callingmethod is empty it means that the data for the memory unit(such as the return address when the function returns) canbe automatically transferred to the PC register implicitly ifthe target calling method is not empty the calling primitiveis generally triggered by invoking the data in the memoryunit in the form of a function pointer (such as invokingmalloc_hook when executing malloc)

Based on the above MOPs we can formally define thepsm and convert the problem of solving structural con-straints into a problem of solving an ordered set of MOPs-e migration path between states refers to an ordered set ofseveral MOPs expressed as psm lango1 o2 oNrang(oi isin (A F W R V)) In the case of satisfying the numericalconstraint ψD after program G executes these operationsthe memory state satisfies the structural constraint rela-tionship ψS that is ψS(G(psm)) true

In order to construct the psm we need to select a group ofMOPs that can satisfy the structural constraints from theprogram path We must first establish an understanding ofthe program path fromwhich to obtain all possible groups ofMOPs so we proposed the concept ofMOP graph-eMOPgraph represents all of the MOPs that may be traversed

during the execution of a program It represents all possibletypes and sequences of all MOPs in the form of a graph Asshown in Figure 4(b) each node in the figure represents aMOP and the directed edge indicates the possible transferdirection of the MOP

-e trigger of a MOP is bound to the program path andthe program path can be reflected by CFG so we employCFG to extract the MOP graph -e extraction process isshown in Figure 4(a) First we utilize a static analysis tool toobtain a complete CFG which includes the basic blockaddress instruction information and the jump relationshipbetween the basic blocks Next we mark the location of theMOP on the CFG as a new node For the MOP of functiongranularity we mark them in the basic block according tothe function invocation address For the MOP of basic blockgranularity we mark the corresponding first address of thebasic block We use the tuple n langTo Aorang to represent thenode where To indicates the type of MOP (5 types in totalA F W R V) and Ao represents the marked address

After marking the new node we establish a new directededge according to the connection relationship of the new nodein the CFG Specifically if there is a connectivity pathpc between the basic blockswhere the two newnodes (ni and nj)are located and there are no other new nodes in pc then we canbuild a directed edge eij langni njrang For example the basicblocks where n1 n2 are located have a connection relationshipshown in Figure 4(a) and therefore a directed edgee12 langn1 n2rang can be established -e directed edge indicatesthat the program may perform the o2 operation after the o1operation so we can append the o2 operation after the o1operation when extracting the MOG

In particular for the directed edges eij langni njrang if ni hasonly one new successor node nj and any path in CFG from ni

passes through the nj node we call elowastij langni njrang astrengthened directed edge For example since there is an3 n4 connection relationship in Figure 4(a) satisfying theabove relationship a strengthened directed edgeelowast34 langn3 n4rang can be established -e strengthened directededge means that the program will inevitably invoke the o4operation after the o3 operation so we must append the o4operation after the o3 operation when extracting the MOG

Prog stateanalysis

Middlecase

1

2

3

Exppattern

1

2

Migrationdemand

Numericalconstraint

solving

MPCMPD

Migrationdemand

1

2

Migrationpath

M

F

M

F

M

M

M

F

F

PChijackinput

PoCStructuralconstraint

solving

Figure 3 Design of the procedural part for RELAY

Scientific Programming 11

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 12: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

After traversing all of the marked new nodes we keep thenew nodes and the new directed edges remove the irrelevantbasic blocks and jump relationships to complete buildingthe MOP graph as shown in Figure 4(b)

From the MOP graph we can explore the path with thegraph traversal algorithm and extract all possible MOGsfrom it -e traversal method is as follows first we mustdetermine the starting point of the traversal Since the psm isa stitching of the paths in the test case we need to ensure thatthe selectedMOG can be appended to the original operation-is requires us to determine the starting point for thetraversal according to the corresponding program statepoint in the test case We record the memory operationexecution trace Oini lango1 o2 oNrang (oi isin (A F W

R V)) of the program in the test case and select the lastmemory operation oN We then find the node ni in the MOPgraph whose address matches with oNrsquos which means wefind out the current program memory operation in thegraph So we use this node as the starting point for exploringto ensure that the constructed psm can be spliced into thecurrent program path After this we need to determine thetraversal method Since we need to solve psm while exploring(explained later) which requires improving the efficiency ofexploring shallow paths we adopt a breadth-first traversalmethod along the directional edge (consider repeatedlytraversing nodes in the ring)

As shown in Figure 4(c) we collect MOPs correspondingto the nodes on each traversed path pk as a group O(pk) Itshould be noted that if a traversed node ni has a strengtheneddirected edge elowastij langni njrang the node ni cannot be used as theendpoint of the traversal path pk -e path pk must beextended backward to add nirsquos successor node nj to thegroup O( pk) Because some memory operations are ac-companied by noise when program triggers these opera-tions it will inevitably trigger additional memoryoperations It is reflected in the MOP graph that theseoperations have a strengthened edge We can take the noiseproblem into consideration by adding special treatment of

the strengthened edge and so the operation group extractedis more reasonable On the contrary for general directededges there is not such a strong correlation between thefront and back operations so they can be randomly com-bined without special treatment

422 Heap Allocator Model We have already explainedabove that on blindly driving the program traversing thepath space to meet structural constraints is often inefficientWe hope to find a specific path in advance and then drive theprogram along with it By reducing the complexity of thepath space we can improve the efficiency of the drive Inorder to find the psm we first extracted a series of combi-nations of operations from the MOP graph and then weneeded to find a group that satisfied the structural con-straints To this end we established a heap allocator model toinfer memory operations observed the memory state afterperforming memory operations and selected an operationgroup that allows the memory state to satisfy structuralconstraints

It should be noted that the model is not only meant toreplace the work of the heap allocator such as managing theheap memory to store data but only to imitate the char-acteristics of the heap allocator and analyze the relationshipbetween memory operation and memory state So we onlyestablished the mapping relationship between memoryoperations and memory state at the logical level and did notactually allocate or deallocate memory at the machine levelIn addition because we only focus on the critical objectsassociated with structural constraints in the heap memorythere is no need to fully characterize all heap memory

-e essence of the heap allocator is to input the memoryoperation o and output a memory state Ho so the key tobuilding a heap allocator model is to build a mapping re-lationship between the memory operation and the memorystate After the program invokes a memory operation thememory changes can be expressed as the relationship

R

AA

FR

WA

F

A

F

WA

A

F

A

n3

n4 e12elowast34

n2

n1

(a)

R

A

A

F R

W

A

F

A

F

WA

A FA

(b)

R

A

A

F R

W

A

F

A

F

WA

A FA

18

20

23

27

31

16 19P3 = ltA F A Fgt P4 = ltA Agt

P1 = ltA Fgt P2 = ltAgt

(c)

Figure 4 Constructing the MOP graph and extracting the MOG from it -e circles and lines represent the nodes and edges of the MOPgraph respectively -e blue line represents the strengthened edge -e dotted line represents the traversal path pk -e number representsthe weight of the edge called driving cost (explained later) (a) Mark nodes and edges in CFG (b) build MOP graph (c) extract MOG fromMOP graph

12 Scientific Programming

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 13: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

Hi⟶o Ho -e output state Ho is determined by theoperation o and input state Hi -erefore a binary func-tional relationship Ho f(oHi) can be established andour goal is to parse this binary function In order to expressbinary functions in an enumerable form we need to employfinite symbolic expressions to represent infinite specificvalues of memory states and memory operations To thisend we use the extended representation of the memoryobject to express the memory state and then use the unifiedoperation symbol to express the memory operation toconvert the binary function into an exhaustible segmentedform

Taking ptmalloc as an example we show how to build aheap allocator model and express it as a binary mappingrelationship For convenience the size represents the alignedparameter of allocation clowastA represents the newly allocatedchunk and clowastF represents the released chunk SK(size L)

indicates that the size is within the range of the linked listLMinGT(ci L sizeprime) represents that ci is the smallest of allchunks in the linked list L whose size is larger than sizeprimeSplit(ci cj ck) represents the split of ci into cj and ckMerge(ci cj) indicates the merge of cj into ci ChBins(L)

represents moving chunks into the linked list L fromUnsortedBins and ChSpl(ci) represents moving the splitchunk ciinto UnsortedBins MinSize represents the lowerlimit of size for UnsortedBins First we list 10 critical ex-pressions for Hi as conditions for the piecewise function

C1 IS HiLFi lang 0 0 0 c1 c2 cNrang1113872 1113873and SK sizeHiLFi

1113872 1113873

C2 IS HiLSi lang0 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLSi

1113872 1113873

C3 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andEQ c1 size( 1113857

C4 IS HiU lang1 0 1 c1 c2 cNrang( 1113857andGT c1 size + MinSize( 1113857

C5 IS HiLLi lang1 0 1 c1 c2 cNrang1113872 1113873and SK sizeHiLLi

1113872 1113873

andMinGT ckHiLLi size1113872 1113873

C6 GT HiTsc clowastF sc( 1113857

C7 SZ clowastF( 1113857HiLFi

C8 EQ clowastF P 0( 1113857and FD cL cF( 1113857

C9 BK T cF( 1113857

C10 EQ chP 0( 1113857andBK cH cF( 1113857andBK ch cH( 1113857

(12)

-en we can build the mapping relations listed inTable 5 It should be noted that we only built the mappingrelationship for the changed heap memory in the structuraldimension rather than across the entire area In order toconstruct a psm in a unified form we count the WROprimitives as part of solving the structural constraintsHowever strictly speaking these primitives will not directlybring changes in the heap memory structure so we have notlisted them in Table 5 but only built a numerical mapping inthe related memory region for them (eg marking the targetaddress of the writing primitive as controllable) In additionwe do not consider the influence of mmap [43] and

reasonably assume that the allocatedmemory will not exceedthe size of TopChunk (avoid triggering system calls to reducespeed)

For example an allocator model gets inputo malloc (16) and the current program state isHiLFi

lang 0 0 0 c1 c2rang which satisfies C1 So according tothe mapping relationship of f(oHi) the model will outputa new state Ho where the linked list changes toLF1

lang0 0 0 c1rang and returns an allocated chunk cA c2If entering a memory operation group O(pk)

o1 o2 oN first of all we will mutate their parametersFor a memory allocation primitive we will mutate its sizeFor memory deallocation primitive the mutation space of itsparameter addr contains the addresses of all the allocatedand deallocated chunks in the current state For the writingprimitive the mutation space of its parameter addr containsthe addresses to be controlled in the target state (the defi-nition of target state will be given later) and the mutationspace of width is the set [1 2 4 8 16 32 64 128] -en weinput these mutated MOPs into the heap allocator modelone by one and process them according to the mappingrelationship to obtain the final new program state

With the help of the heap allocator model after enteringthe MOG and the initial state of the program we can inferthe new state and match the new state with the target state(this matching method will be introduced later) and we canknow whether the current MOG can make the memory statesatisfy the structural constraint

423 Solving the Migration Path Our goal is to extract aMOG that satisfies the structural constraints from theprogram path to build psm To achieve this goal we shouldcomplete the following three processes extract the MOGsolve psm and consider the difficulty in driving psm -emethod of extracting MOGs from the MOP graph wasintroduced earlier Below we first introduce the calculationmethod of the degree of state difference and explain how tofind the sections that satisfy the structural constraints fromMOGs by the degree of difference -en we introduce theconcept of driving cost and explain how to use driving costto reduce the difficulty of driving psm Finally we summarizethe whole process for buildingpsm(1) Degree of State Difference We introduce the concept ofstate difference degree (dD) to measure the distance ofmemory structures between different states Since there aredifferent data structures in memory regions such as linkedlists and the set of allocated chunks we must define thedistance of these typical data structures involved in detail

-e operation for the random access linked list is un-directed and chunks can be inserted and removed from anyposition on the linked list Typical random access-linked listsinclude ptmallocrsquos LargeBin linked list and UnsortedBin Forthe two random access linked lists A and B the degree ofdifference between the structures is defined in the followingequation where LCS(A B) refers to the longest commonsubsequence distance (a special kind of the edit distance [44]which only considers insertion and deletion)

Scientific Programming 13

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 14: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

dRL(A B) 1

LCS(A B) + 1 (13)

-e direction of operation of the one-way access linkedlist is fixed so insertion and deletion can only start from oneend of the linked list For example insertion and deletion areperformed in the same direction (FILO) for the FastBin listin ptmalloc and however in different directions (FIFO) forthe SmallBin list -e definition of the distance between thetwo types of linked list structures can be expressed in thefollowing equation where LA and LB represent the length ofthe linked lists of A and B respectively for FILO-type linkedlists LAB represents the longest matching length of B⟶ A

after A and B are aligned backward in the direction of de-letion For FIFO-type linked lists LAB represents the lengthof the part that can completely match the subsequence of A

after the first same node of B and A from the insertiondirection

dSL(A B) LA + LB minus 2lowastLAB (14)

-e degree of difference between unordered sets is de-fined in the following equation where diff(B A) representsthe number of elements which are located in B but not in AFor example assuming that the target state requires 4memory units to be controllable and only one of thememoryunits is currently controlled the degree of difference be-tween the current state and the target state is 3

dset(A B) diff(B A) (15)

We total the degree of difference between the differentdata structures involved in the two states to obtain the totaldD between the states

From the structural constraints we can acquire theinitial state and corresponding target state (we can link themas a state migration demand (dsm) expressed asdsm Hi⟶ Hlowasto ) By inputting the extracted MOG andthe initial state Hi into the heap allocator model we caninfer the new stateHo and calculate the dD between the newstate and the target state If dD is 0 it indicates to the MOGthat it satisfies the structural constraints For example thereis a structural constraint for IN(c2HA) First we use thecurrent program state as the initial state Hi and the c2

chunk in the linked list is expressed as HiLF1 lang0 0 0

c1 c2 c3rang Next we ensure the target state Hlowasto Hlowasto A c2

After receiving the operation group O(pk) A A from theheap allocator the output state is Ho where the linked listchanges to LF1 lang0 0 0 c1rang and returns the chunk cA c2-e dD between the new stateHo and the target stateHlowasto is0 indicating that we have found a psm langA Arang

(2) Path Driving Cost -e complexity of branch constraintsfor program paths is different so the difficulty in exploringthese branches for a driver is also different In order toimprove the efficiency of exploration we hope to find thoseparts with low driving difficulty from the set of psm We putforward the concept of driving cost (dC) to measure thedifficulty in driving the psm Because it is difficult to directlymeasure the complexity of path branch constraints by staticanalysis we employ a dynamic method of path explorationto evaluate the difficulty in driving the path

While exploring the path using fuzzer we record thenumber of times it hit the directed edges in the MOP graphby hooking function and positioning the basic block Whilereceiving many test cases we also have a large number ofrecords on the directed edge eij in the MOP graph Wecount the total number of test cases wt and divide the totalnumber of hitting (wt) by the number of hitting the edge(wij) -en we mark the weight of the directed edge eij aswtwij(if wijis 0 then it is marked as infinity) and a MOPgraph with weights is established which can reflect thedifficulty in driving different paths If the weight of the edgeis higher it means that it is more difficult to drive along thecorresponding path By totaling the weight of the edgesincluded in the psm on the graph we can evaluate thedifficulty in driving psm

So far we can formally define the driving cost ofpsm lango1 o2 middot middot middot oN rang (oi isin (A F R W V))

dC 1113944Nminus1

i1

wt

wi(i+1)

(16)

wi(i+1) represents the weight of the directed edgeei(i+1) langni ni+1rang and ni ni+1 corresponds to the oi oi+1respectively As shown in Figure 4(c) the driving cost of the

Table 5 Typical mapping relation for a heap allocatorAFClowast2 Clowast4

o Hi Ho

A

C1 HoLFilt000c1c2 cN-1gtcA cNnotC1andC2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andC3 ChBins(HoLL HoLS) HoUNul cA c1notC1andnotC2andnotC3andC4 Split(c1cicj) andcisc size cA ci ChBins(HoLLLS) HoUNul

notC1andnotC2andnotC3andnotC4andC5 Split(ckcicj) andcisc size cA ciChSpl(cj)notC1andnotC2andnotC3andnotC4andnotC5andClowast2 HoLSilt001c2 cNgtcA c1

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andClowast4 Split(c1cicj) andcisc size cA ci ChBins(HoLL HoLS) HoUNul

notC1andnotC2andnotC3andnotC4andnotC5andnotClowast2 andnotClowast4 andC6 Split(Tcicj) andcisc size cA ciTsc cjsc

F

C7 HoLFilt000c1c2 cNcFgtnotC7andC8 Merge(cLcF) HoUlt101c1c2 cNcLgt

notC7andnotC8andC9 Merge(TcF)notC7andnotC8andnotC9andC10 Merge(cFcH) HoUlt101c1c2 cNcFgt

A allocation F deallocation reallocation can be regarded as one of these -e allocator needs to check Clowast2 and Clowast4 again after checking the previous 5conditions

14 Scientific Programming

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 15: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

path p3 langA F A Frang is 88 It is important to note if thepsmincludes an edge with an infinite weight it means that such apath cannot be triggered during the running of the programand so this path will be removed from group Psm

(3) MPCWorkflow Finally we sort out the entire process ofconstructing psm and the entire MPC workflow is shown inFigure 5 For a structural constraint we first construct aMOP graph for the program and extract a group of memoryoperations O(pk) from the graph -en we mutate theirparameters and input operations into the heap allocatormodel for inference and observe whether the structuralconstraint requirements are satisfied If satisfying thesestructural constraints the group of operations will be addedto the set of preliminary migration paths In this process weadopt a synchronous strategy of inferring while extracting toimprove efficiency and we set a time threshold for mutationFor each group of operations extracted we continuouslyperform mutations for them and then infer the results untilthe threshold is exceeded -en we extract the next group ofoperations and repeat the process For the preliminarymigration path set Psm we select the kpaths with the smallestdriving cost arrange them in the optimized set Plowastsm in orderand pass the set to the MPD

43 Migration Path Driving (MPD) -e problem of drivingthe psm is essentially a reachability problem and our purposeis to generate an input that can drive the program to reachthe target point (the call location of target memory oper-ation)-e problem of the reachability of the target point canbe decomposed into two problems control-flow arrival(CFA) and data flow arrival (DFA) For example for the F

operation in the psm the generated test cases not only needto trigger deallocation functions but also need to accuratelyset the function parameters as the chunk address is expectedto be deallocated

Currently there are many fuzzing tools that have theability to drive program reaching specific locations [45ndash47]For example AFLGo [48] uses a guided gray-box fuzz togenerate test cases that can reach the target program pointWe have made modifications on the basis of AFLGo togenerate test cases satisfying both CFA and DFA

-e modification method is as follows by constantlyoptimizing the test cases with higher path weights (ones whichinclude a target memory operation) we output test cases thatcan reach the target program point and achieve control-flowarrival Taking test cases with CFA as seeds we continuouslyapproach test cases with higher data weights (ones associatedwith target parameters) and finally output target test cases

5 Implementation and Evaluation

We implemented the prototype of Relay on top of theframework mentioned previously On the selective symbolicexecution engine S2E [49] we defined four exploit patternsincluding FastBin Unlink Hof and Overlap and imple-mented the program state analysis module -e function ofsolving numerical constraints and arranging shellcode is also

built on top of S2E MOP construction and MOG extractionare implemented on top of IDA Python [50] Also weimplemented a heap allocator model by building functionmapping relationships in Python And the MPDmodule wasbuilt on top of the fuzzing engine AFL [51] RELAY containsa total of 85 K lines of CC++ and 46 K lines of Python Itshould be noted that although we only implement fourexploit patterns currently RELAY is an extensible frame-work that will include more types of vulnerabilities andexploit methods in the future -e experimental part mainlyexplores the following two problems

(i) RQ1 does RELAY has the ability to evaluateexploitability for metadata corruptionvulnerabilities

(ii) RQ2 what factors mainly affect the efficiency ofRELAY

51 Benchmark Selection Affected by program scale run-ning environment and interaction mode the testing processfor real software is cumbersome and easily interfered byirrelevant factors -e CTF and RHG programs are specif-ically designed to test the ability of humans and machines todeal with vulnerabilities -ey can be regarded as a set ofprograms that reduce the scale of real software and irrelevantfactors but retain their vulnerability characteristics -eyfocus on human expertise for exploitation and usually re-quire high-level skill to solve problems We have selected 17CTF programs and 3 RHG programs from the recent publicevents -e reason for choosing more CTF programs is thatCTF is mainly used to evaluate the application ability ofhigh-level knowledge than the machinersquos computation ca-pability and can better test whether our system can effec-tively deal with the vulnerability in combination withartificial knowledge

-e exploitability evaluation for these target programscovers four common exploitation methods (FastBin attackUnlink Hof and chunk overlapampFastBin attack) to verifywhether our system can use the corresponding four exploitpatterns to complete assessing the vulnerability Each target

Input Program P CurrentState si NewState soMopPath pk MopGroup O(pk)StructuralConstraint ψS

Output OptimizedMigrationPath Plowast

smMopGraph larr BuildMopGraph(P)for pk isin TraverseMopGraph(si MopGraph) do

for i isin MutateThreshold doOprime(pk) larr Mutate(O(pk))so larr AllocateModel(siOprime(pk))dD larr CalcDifferDegree(so ψS)if dD larr 0 then

Psm larr append(Oprime(pk))

Plowast

sm larr SelectMinDriveCost(psm)

endend

end

O(pk) larr ExtractMopGraph(pk)

123456789

10111213

Figure 5 MPC workflow

Scientific Programming 15

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 16: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

program represents some type of programs bounded to thespecific exploitation method and the capability in dealingwith one of the target programs indicating the capability inevaluating this type of program Certainly the capability ofour system is not limited to these 20 selected programs forinstance our system also completed the exploitation for twoheap vulnerabilities on the spot in the open event ofDEFCON China [52]

In a word the target programs we selected have thefollowing features

(i) -e program comes from the public event and it isbelieved to reflect the vulnerability feature of realsoftware

(ii) -e program does not contain source code anddebug symbols which can fully restore theexploitability evaluation in the real environment

(iii) Each program contains metadata corruptionvulnerabilities

(iv) Each program represents some types of programbounded to the specific exploitation method

52 Experimental Setup All experiments were run on aLinux Ubuntu Server 1804 LTS AMD64 equipped with anIntel(R) Core(TM) i9-9980XE CPU300GHz and 64GBRAM Our experiments enabled DEP [53] but disabledASLR [54] since RELAY does not support automaticallybypassing ASLR

53Exploits byRELAY(RQ1) In this section we first displaythe experimental results and then analyze the specific casesand demonstrate the strengths of RELAY compared to otherAEG solutions

531 Overall Results As shown in Table 6 we tested a totalof 20 programs of which 16 successfully generated theexploits and 4 failed -e table details the program nameevent name and vulnerability type respectively In additionit shows the exploit pattern corresponding to the initial stateof the program detected by the program state analysismodule after inputting the PoC (the correct exploit pattern ismarked with an lowast after manual verification) and it alsorecords the total number of nodes Nm contained in theMOPgraph

We also recorded the total path length Lp of the mi-gration path corresponding to the final exploit (by addingthe number of memory operations included in each mi-gration round) and the total driving cost Dc (by adding thedriving cost corresponding to each migration round) Forequality we use the same number of test cases wt generatedby fuzzing to solve driving costs for all programs In ad-dition we recorded the elapsed time Tm from inputting PoCto outputting EXP

Finally we compared it to other state-of-the-art AEGsolutions As far as we know the current AEG solution foruser-level application vulnerabilities is the only Revery Wegot the experimental data from the Revery paper (programs

without data are marked as ldquominusrdquo) To draw a more objectivecompare conclusion we also chose another open-sourceAEG solution Rex [55] as an experimental comparisonDeveloped by the shellphish team Rex won the first place inthe Cyber Grand Challenge (CGC) [56] contest although it isnot specifically designed to deal with heap vulnerabilities

It can be seen that the time to complete exploitation isranging from 10 to 50minutes -is suggests that RELAY isan efficient AEG tool and as we know other AEG tools thatdeal with heap vulnerabilities such as Revery often needover 50minutes or even hundreds of minutes to work out-e RELAYrsquos speed is even accepted by the professionalsecurity analyst (it usually takes tens of minutes to completethe exploitation of the heap vulnerability manually) soRELAY can be put into practice to deal with heap vulner-ability for replacing the manual evaluation to some extent

We can also see that both Nm and Lp have an ap-proximately positive correlation trend with the elapsed time-is is because Nm is positively correlated to the programscale and there is a positive correlation trend between Lp

and the complexity of drivingmigration path to some extentAnd Dc has a stronger relationship with Tm and the two aregenerally positively correlated which proves that we canimprove efficiency for exploitation by choosing migrationpaths with a low driving cost (detailed in Section 52)

532 Case Study In this section we investigated theseprograms in detail and analyzed why our solution wassuccessful or not

(1) Successful Cases RELAY successfully generated ex-ploits for 16 programs covering multiple types of vulnera-bilities and the four defined exploit patterns which showsRELAY has the ability to process metadata corruption vul-nerabilities In contrast to Rex it cannot generate an exploitfor any programs listed Rex only works under the circum-stance that the control flow has been hijacked but the aboveprograms did not enter the state of control-flow hijackingafter running PoC So we conclude that Rex does not have thecapability in dealing with common heap vulnerabilitiesLikewise Revery cannot solve any listed program (for pro-grams without data we reasonably conclude that Revery failson them) Below we employ specific programs to analyze thereasons why RELAY succeeds and Revery fails

First RELAY can solve numerical constraints on specificlocations and time For example bcloud stipulates that thechunk can only be overflowed by four bytes It is necessary tomodify the size of TopChunk to a large specific value at thebeginning For simple note it needs to point fd and bk to aspecific position in the heap table to construct an AAW Byanalyzing the numerical constraints in the exploit patternRELAY can modify the target value before the heap blockstructure changes However Revery cannot perceive theseconstraint relationships And if we fail to complete thememory value modification within an effective time periodsuch as modifying fd and bk before the heap block merges ormodifying the size before splitting TopChunk it will makethe memory allocation and deallocation meaningless or evenprevent exploitation

16 Scientific Programming

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 17: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

Second RELAY can convert structural constraintsproblems to solve the memory operation sequence andtransform the state coverage problem into a specific pathexploration problem which can greatly reduce the com-plexity of the state space and the difficulty in path explo-ration For pwnme it needs to take out the fake chunk fromthe FastBin chain and then RELAY can calculate theshortest path A-A-A to complete the task However Reverydirectly uses fuzzing to search for the target state which mayrepeatedly trigger another high-frequency path A-F-A-F Onthis path we cannot quickly take out the target chunk whichreduces the efficiency of promoting heap layout and mayeven cause the program to exit after reaching the allowedmaximum number of allocations

(2) Failed Cases RELAY currently supports four modesdoes not combine the global analysis of the program suf-ficiently and arranges the corresponding fixed modeaccording to the current program state -e contradictionbetween the complexity of the program path and the fixedmode brings confusion to the exploitation If there arespecial restrictions on the program paths the migration pathobtained according to the exploit pattern may not be drivenFor example RELAY built exploit modes for 4 programs inthe table but these programs cannot be driven in a realenvironment

For example for bookstore the size of the allocation isfixed and does not belong to the FastBin range And theoverflowed content is not controlled which makes theconventional HofUnlinkFastBin attack impossible -esolution is to destroy the size metadata and cause a chunkoverlap then the content of the format string can bemodified and AAW can be triggered by the format stringvulnerability For babyheap the size of allocation cannotexceed 0times 58 and the callable fake chunk cannot be directlyconstructed on the FastBin chain But the TopChunk pointerat Main_Arena [57] can be modified with FastBin attack to

cause an arbitrary address allocation For wheelofrobots thecallable fake chunk also cannot be constructed directlyAttackers can only launch chunk overlapampFastBin attack tomodify one control variable to achieve heap overflow of anylength and then employ Unlink to achieve AAW Similarlyfor babypwn attackers also need to modify a variable thatcontrols the allowed number of allocations with FastBinattack and then the malloc_hook can be modified throughFastBin attack again

It can be seen that if there are strict restrictions on theprogram path a single exploit pattern can no longer fullyguide the exploitation And more flexible combinations ofmultiple vulnerabilities and multiple patterns are required tobypass the restricted path and reactivate the exploitation Infuture work we will consider adding some ML (machinelearning) methods to build the overall perception of theprogram paths and explore ways of organizing multiplevulnerabilities and exploit patterns to approach the targetheap layout

54 Efficiency of RELAY (RQ2) Previously we analyzed theeffectiveness of RELAY and we assess its efficiency in thissection -e working time of RELAY mainly comes fromMPC and MPD modules so we conducted an evaluation forthe performance of these two modules and analyzed thefactors that may have affected their performance

First we evaluated the impact of the introduction ofdriving cost on MPC MPD and overall speed We selected13 CTF programs and recorded the working time of theMPC and MPD modules under the two conditions of firstlyusing the driving cost (C1) and then of not using it (C2) ForC1 the MPC module introduced the time consumption ofcalculating the driving cost (mainly from the path explo-ration of fuzzing tools) and the MPD module attempts todrive the migration path of the optimization set Plowastsm in

Table 6 Overall results of RELAY

Name CTF Vul type Exp pattern Nm Lp Dc Tms Revery

Successful cases

BRHG-13 BRHG UAF fast_bin_exp 6 5 38 1283 mdashpwn14 DEFCON ChinaampBCTF Heap overflow hof_exp 8 4 32 927 mdashpwn40 DEFCON ChinaampBCTF Heap overflow unlink_exp 5 5 43 1194 mdashnote2 ZCTF 2016 Heap overflow unlink_exp 18 10 83 1427 note3 ZCTF 2016 Heap overflow unlink_exp 19 12 74 1483 fb AliCTF 2016 Heap overflow unlink_exp 32 13 96 1638

Stkof HITCON 2014 Heap overflow unlink_exp 27 12 117 1829 Simple note Tokyo westerns 2017 Off-by-one unlink_exp 21 10 99 1595 Search engine 2015 9447 CTF Double free fast_bin_exp 21 13 106 1762 mdash

Badint Defcon qualifier 2017 Heap overflow fast_bin_exp 31 13 138 2053 mdashWeqpon Delta Ctf 2019 UAF fast_bin_exp 24 12 82 1518 mdashBabyheap 0ctf 2017 Heap overflow overlap_exp 36 16 175 2431 mdashPwnme NCTF2019 Off-by-one overlap_exp 25 15 152 2218 mdashBcloud 2016 BCTF Heap overflow hof_exp 13 9 75 1364 mdash

Gyctf2020_force BUUCTF Heap overflow hof_exp 16 8 86 1487 mdashBamboobox Hitcon training Heap overflow hof_exp 12 8 69 1329 mdash

Failed cases

Bookstore 2015 hacklu Heap overflow unlink_exp 25 10 126 mdash mdashBabyheap 0ctf2018 Off-by-one overlap_exp 21 13 153 mdash mdash

Wheelofrobots 2017 insomnirsquohack Off-by-one overlap_exp 18 11 141 mdash mdashBabypwn N1ctf2019 Double free fastbin_exp 23 11 148 mdash mdash

Scientific Programming 17

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 18: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

ascending order for driving cost -ese experimental resultsare reflected on the right bar of Figure 6 for each programFor C2 the MPC module does not need to calculate thedriving cost and the MPD module attempts to drive themigration path in the set Psm with a random selectionstrategy-e results of these experiments are recorded on theleft bar of Figure 6 for each program

Figure 6 shows the experimental results From thehorizontal comparison although the introduction of drivingcost increases the working time of theMPCmodule the totalexploitation time is reduced During the experiment of C1we calculated the standard deviation σ for the weight of theMOP graph edges for each program-en we further foundthat for a program with more uneven weight distribution onthe edges in the MOP graph (ie with a larger σ) there is amore obvious reduction in the total exploitation time fromC2 to C1 -erefore we infer that the introduction of drivingcost can effectively improve the efficiency of the system andthis improvement is obvious when processing programswith an uneven distribution of complexity for branchconstraints

In addition in terms of longitudinal comparison nomatter under C1 or C2 conditions MPC has a small workingtime and there is no large fluctuation in the time for dif-ferent programs So we infer that the efficiency of theMPC isonly slightly affected by the program logic On the contraryMPD takes a lot of time to drive the migration path Wecalculated the average driving cost μ for the migration pathsduring the experiment of C1 It can be clearly seen that thehigher the driving cost the more time the MPD takes whichindicate that the performance of the MPD is greatly affectedby the program logic In conclusion we can infer that theMPC is a low-consumption and stable module Below wecontinue to discuss the performance improvements broughtby the MPC to the system

In the process of promoting the heap layout the largestdifference between RELAY and other tools such as Revery isthat RELAY adds a guide of the migration path for the heaplayout driver (ie adding the MPC module) -e key forRELAY to driving heap layout efficiently is that the MPC canextract migration paths that satisfy structural constraintsfrom the huge path space Since we lack the experimentaldata of other tools we expect to perform a self-contrastexperiment on the MPC module to observe the reductioneffect of the program path space after the MPC moduleworks so that we can evaluate the performance improvementof RELAY compared to other tools

-e path space can be expressed as a set of tuples pi diwhere path pi lango1 o2 oNrang (oi isin (A F R W V)) anddi represents the driving cost of path pi For the two pathspaces and G2 the efficiency ratio of the complete explo-ration can be expressed as follows (N M represent the totalnumber of paths for G1 and G2 respectively)

η 1113936

Mi1 di

1113936Nj1 dj

(17)

If assuming that the driving costs of all program pathsare the same this expression can be simplified to η MN

which indicates that the efficiency of exploring path space isinversely proportional to the number of paths in path space

Based on this assumption we conducted a separate teston the MPC module For each program within a 2-minutelimit we recorded the number of program paths tried byMPC (denoted byg1 and shown as the whole bars inFigure 7) and the number of migration paths that can satisfythe structural constraints (denoted by g2 and shown as bluesegments in Figure 7) We can suppose that for those toolswithout the MPC function the path space to be explored isG1 (g1 for size) then the path space of the same program tobe explored by RELAY with MPC function is G2 (g2for size)so the efficiency ratio of RELAY to other tools is roughη g1g2

As shown in Figure 7 it can be seen that after usingMPC the program path space is roughly compressed to lessthan 15 -e efficiency of exploring the path space is im-proved accordingly which means we can promote the heaplayout more effectively We accept this conclusion under theassumption that all driving costs of program paths are thesame In fact MPC can build the path space Glowast2 composed ofmigration paths with low driving costs and according to theprevious formula the efficiency of exploring space Glowast2 willbe higher than G2

6 Related Work

As described above our work focuses on assessingexploitability for metadata corruption vulnerabilities inapplications It should be noted that RELAY is the firstsystem that addresses AEG problems aimed at metadatacorruption vulnerability characteristics in user-level appli-cations As a result the works most relevant to ours is heap-based AEG solutions In the following we describe theexisting works of this type and discuss their limitations

In order to find exploit primitives for metadata cor-ruption in the allocator Repel et al [11] described the firstmethod of AEG for heap overflow -ey connect the driverto the target allocator and then use summary execution todiscover exploit primitives To explore weaknesses in theheap allocator Eckert et al [29] proposed a system Hea-pHopper for discovering primitives in the allocator andbuilding an exploit in the context of a driver connected by anallocator In order to solve the derivative problem of heapexploits Wang et al [24] described Revery a system thatemploys a layout-oriented fuzzer and control-flow stitchingtechnique to discover the exploitable state in the divergentpath instead of the crash path -eir approach can generatesome heap exploits but only in the case where their fuzzerrandomly reaches the required heap layout To facilitate theexploitation of kernel UAF vulnerabilities Wu et al [58]constructed FUZE a system employing kernel fuzzing andsymbolic execution to identify useful system calls for kernelUAF exploitation and utilize dynamic tracking and an off-the-shelf constraintrsquo solver to guide exploitation In order topromote the exploitation of kernel vulnerabilities Chen et al[27] proposed SLAKE which utilizes static and dynamicanalysis techniques to identify kernel objects and usefulsystem calls and then modeled common exploitation

18 Scientific Programming

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 19: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

methods to manipulate the slab layout In order to connectan exploitable state to trigger the execution of a code-reusepayload Wu et al [59] constructed Kepler which accepts aninput including control-flow hijacking primitive andbootstraps any kernel ROP payload by symbolically stitchingan exploitation chain with corresponding gadgets In orderto automatically generate exploits for heap overflow vul-nerability in the PHP interpreter Heelan et al [28 60]employed a pseudorandom black-box search method tomanipulate heap layout and described an automatic solutionGollum which contains a number of novel ideas includingthe mining of tests for code fragments that provide prim-itives lazy resolution of heap layouts a genetic algorithm forheap layout manipulation and a completely gray-box ap-proach to automatically generate exploits

7 Conclusion and Future Work

Given that the organization of heap metadata is usuallycomplicated it is vulnerable to variant forms of heap-relatedattacks Despite its importance fewer efforts have beenmadeto analyze whether heap metadata could be corrupted andexploited by cyberattackers To improve software quality andensure cybersecurity we proposed RELAY a software testingframework to assess the exploitability of metadata corrup-tion vulnerabilities by simulating human exploitation be-havior Experiments suggest that RELAY can effectivelyevaluate hazardous degree for programs containing met-adata corruption vulnerabilities and works more efficientlycompared to other state-of-the-art automated tools How-ever RELAY does not work properly in some cases since the

3000

2500

2000

1500

1000

500

0

Tim

e (s)

Binary name

μ = 87σ = 18

μ = 108σ = 21

μ = 137σ = 29

μ = 154σ = 31 μ = 132

σ = 42

μ = 213σ = 67 μ = 182

σ = 35

μ = 85σ = 17

μ = 94σ = 16 μ = 81

σ = 13

μ = 146σ = 52

μ = 174σ = 26

μ = 117σ = 23

MPD(C2)MPC(C2)

MPD(C1)MPC(C1)

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 6 Working time for MPC and MPD under different conditions

18001600140012001000

800600400200

0

Num

bers

of p

aths

Binary name

note

2

note

3 fb

Stko

f

Sim

ple n

ote

Baby

heap

Pwnm

e

Bclo

ud

Gyc

tf_20

20_f

orce

Bam

boob

ox

Sear

ch en

gine

Badi

nt

Weq

pon

Figure 7 Performance of MPC and efficiency improvement for RELAY

Scientific Programming 19

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 20: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

contradiction between the complexity of the program pathsand the unicity of the exploit patterns confuses the evalu-ation (detailed in the Failed Cases in Section 532) In futurework we will try to add theMLmethod to help RELAY buildthe overall concept of program paths and explore ways oforganizing multiple vulnerabilities and exploit patterns toenhance the capability of RELAY Besides we find that thecurrent discussion on the security issues in Internet of-ings (IoT) systems is very intense and we can also see thatsome excellent research results [61ndash65] have been publishedIn the course of studying the above work we have found thatthere are some similarities with our research direction in thisfield such as the path exploring and constructing method-erefore in future work we will consider integrating themethod of exploitability evaluation for vulnerabilities withthe analysis method of IoT security issues

Data Availability

All data included in this study are available from the cor-responding author upon request

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is work was supported by the National Natural ScienceFoundation of China (Grant no 61702540) and the HunanProvincial Natural Science Foundation of China (Grant no2018JJ3615)

References

[1] C Luo Bo Wang K Huang and Y Lou ldquoStudy on softwarevulnerability charac- teristics and its identification methodrdquoMathematical Problems in Engineering vol 2020 2020

[2] G Spanos and L Angelis ldquoA multi-target approach to esti-mate software vulnerability characteristics and severityscoresrdquo Journal of Systems amp Software vol 2018 2018

[3] V V D Veen N Dutt-Sharma L Cavallaro and H BosldquoMemory errors the past the present and the futurerdquo 2012

[4] B Padmanabhuni and H B K Tan ldquoDefending againstbuffer-overflow vulnerabilitiesrdquo Computer vol 44 no 11pp 53ndash60 2011

[5] TW L Szekeres andM Payer ldquoSok eternal war in memoryrdquo2013

[6] P K Kudjo J Chen M Solomon R Amankwah andC Kudjo ldquo-e effect of bellwether analysis on softwarevulnerability severity prediction modelsrdquo Software QualityControl vol 1 pp 1ndash34 2020

[7] M Mouzarani B Sadeghiyan and M Zolfaghari ldquoSmartfuzzing method for detecting stack-based buffer overflow inbinary codesrdquo Iet Software vol 10 no 4 pp 96ndash107 2016

[8] J Park Y Choo and J Lee ldquoA hybrid vulnerability analysistool using a risk evaluation techniquerdquo Wireless PersonalCommunications vol 10 2018

[9] H Liang X Pei X Jia W Shen and J Zhang ldquoFuzzing stateof the artrdquo IEEE Transactions on Reliability vol 10 2018

[10] Shellphishhow2heap 2020 httpsgithubcomshellphishhow2heap

[11] D Repel J Kinder and L Cavallaro ldquoModular synthesis ofheap exploitsrdquo in Proceedings of the 2017 Workshop onProgramming Languages and Analysis for Security vol 25ndash35New York NY USA 2017

[12] M Kharbutli X Jiang Y Solihin G Venkataramani andM Prvulovic ldquoComprehensively and effifiicently protectingthe heaprdquo Computer Architecture News vol 10 2006

[13] D Tian X Li M Chen and C Hu ldquoiCruiser an improvedapproach for concurrent heap buffer overflow monitoringrdquoIEICE Transactions on Information and Systems vol 97 no 3pp 601ndash605 2014

[14] glibc Libc 2020 httpswwwgnuorgsoftwarelibclibchtml[15] S Silvestro H Liu C Corey Z Lin and T Liu ldquoFreeguard a

faster secure heap allocatorrdquo 2017[16] A Brahmakshatriya P Kedia D P Mckee et al ldquoAn

instrumenting compiler for enforcing confifidentiality in low-level coderdquo 2017

[17] Institute of Computer Sciencetechnology Peking UniversityBeijing China Beijing Key Laboratory of Internet SecurityTechnology and Beiji ldquoUsing type analysis in compiler tomitigate integeroverflflow-to-buffer-overflflow threatrdquo Jour-nal of Computer Security vol 10 2016

[18] Shellphish how2heap-fifix for the new check 2020 httpsgithubcomshellphishhow2heapcompare58aed1ce

[19] R Jukka R Sampsa H Sami and L N Ville ldquoA case study onsoftware vulnerability coordinationrdquo Information amp SoftwareTechnology vol 10 2018

[20] P Poosankam D X Song D Brumley and J Zheng ldquoAu-tomatic patch-based exploit generation is possible techniquesand implicationsrdquo in Proceedings of the IEEE Symposium onSecurity and Privacy New York NY USA 2008

[21] S Heelan Automatic Generation of Control Flflow HijackingExploits for Software Vulnerabilities University of OxfordOxford UK 2009 httpswwwcproverorgdissertationsthesis-Heelanpdf

[22] T Avgerinos S K Cha B L T Hao and D Brumley ldquoAEGautomatic exploit generationrdquo in Proceedings of the Networkand Distributed System Security Symposium New York NYUSA 2011

[23] S K Cha T Avgerinos A Rebert and D BrumleyldquoUnleashing Mayhem on binary coderdquo in Proceedings of the2012 IEEE Symposium on Security and Privacy SPrsquo12 IEEEComputer Society Washington DC USA 2012

[24] Y Wang C Zhang X Xiang et al ldquoRevery from proof-of-concept to exploitablerdquo in Proceedings of the 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity New York NY USA 2018

[25] H Liang Y Cai H Hu P Su and D Feng ldquoAutomaticallyassessing crashes from heap overflflowsrdquo in Proceedings of theIEEEACM International Conference on Automated SoftwareEngineering New York NY USA 2017

[26] Y Shoshitaishvili A Bianchi K Borgolte et al ldquoMechanicalphish resilient autonomous hackingrdquo IEEE Security amp Pri-vacy vol 16 no 2 pp 12ndash22 2018

[27] Y Chen and X Xing ldquoSlake facilitating slab manipulation forexploiting vulnerabilities in the linux kernelrdquo in Proceedingsof the 2019 ACM SIGSAC Conference on Computer andCommunications Security pp 1707ndash1722 New York NYUSA 2019

[28] S Heelan T Melham and D Kroening ldquoAutomatic heaplayout manipulation for exploitationrdquo in Proceedings of the

20 Scientific Programming

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21

Page 21: APattern-BasedSoftwareTestingFrameworkforExploitability ...downloads.hindawi.com/journals/sp/2020/8883746.pdf · combinedwithhumanexpertise[26].ExistingAEGsdonot integrate the required

27th USENIX Security Symposium (USENIX Security 18)pp 763ndash779 New York NY USA 2018

[29] M Eckert A Bianchi R Wang S Yan C Kruegel andG Vigna ldquoHeaphopper bringing bounded model checking toheap implementation securityrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18) NewYork NY USA 2018

[30] Unlink Exploit 2020[31] House of force Exploit 2020[32] Glibc Malloc-Pthreads Malloc 2020[33] Fastbin Attack 2020 httpsgithubcomylcangelexploit

treemasterheapfastbin_attackl[34] ChunkOverlap and FastBinAttack 2020 httpsctf-wikigithub

ioctf-wikipwnlinuxglibc-heapchunk_extend_overlapping[35] CWE-193 Off-By-One Error 2020[36] C Cadar and K Sen ldquoSymbolic execution for software testing

three decades laterrdquo Communications of the ACM vol 56no 2 pp 82ndash90 2013

[37] E J Schwartz T Avgerinos and D Brumley ldquoAll you everwanted to know about dynamic taint analysis and forwardsymbolic execution (but might have been afraid to ask)rdquo inProceedings of the IEEE Symposium on Security and PrivacyNew York NY USA 2010

[38] M Bohme V-T Pham and A Roychoudhury ldquoCoverage-based greybox fuzzing as Markov chainrdquo IEEE Transactionson Software Engineering vol 45 no 5 pp 489ndash506 2017

[39] C Lemieux and K Sen ldquoFairfuzz a targeted mutation strategyfor increasing greybox fuzz testing coveragerdquo in Proceedingsof the 33rd ACMIEEE International Conference On Auto-mated Software Engineering pp 475ndash485 New York NYUSA 2018

[40] S K Huang M H Huang P Y Huang W L Chung andW M Leong ldquoCrax software crash analysis for automaticexploit generation by modeling attacks as symbolic contin-uationsrdquo in IEEE Sixth International Conference on SoftwareSecurity amp Reliability New York NY USA 2012

[41] C S Pasareanu R Kersten L Kasper and Q S PhanldquoSymbolic execution and recent applications to worst-caseexecution load testing and security analysisrdquo Advances inComputers vol 22 2018

[42] Tools of Constraint Solver 2020[43] ldquoMmap munmap map or unmap fifiles or devices into

memoryrdquo 2020[44] ldquoEdit distance Measure degree of difference between char-

acter stringrdquo 2020[45] W You P Zong K Chen X F Wang and B Liang

ldquoSemfuzz semantics-based automatic generation of proof-of-concept exploitsrdquo in Proceedings of the Acm Sigsac ConferenceLondon UK 2017

[46] Y Li S Ji C Lv et al ldquoVfuzz vulnerability-oriented evo-lutionary fuzzingrdquo 2019

[47] W You X Wang S Ma J Huang and X Zhang ldquoProfuzzeron-the-flfly input type probing for better zero-day vulnera-bility discoveryrdquo in Proceedings of the 2019 IEEE Symposiumon Security and Privacy (SP) London UK 2019

[48] M Bohme V-T Pham M-D Nguyen andA Roychoudhury ldquoDirected greybox fuzzingrdquo in Proceedingsof the 2017 ACM SIGSAC Conference on Computer andCommunications Security pp 2329ndash2344 New York NYUSA 2017

[49] V Chipounov V Kuznetsov and C George ldquoS2e a platformfor in-vivo multipath analysis of software systemsrdquo in Pro-ceedings of the Sixteenth International Conference on

Architectural Support for Programming Languages and Op-erating Systems New York NY USA 2011

[50] IDA Disassembler and Debugger 2020[51] AFL American Fuzzy Lopa 2020[52] DEFCON CHINA 2020[53] S Andersen and A Vincent Changes to Functionality in

Microsoft Windows Xp Service Pack 2 Part 3 Memory Pro-tection Technologies Microsoft TechNet 2004

[54] PaX Team ldquoAddress space layout randomization (aslr)rdquo 2003[55] ldquoRex-shellphishrsquos automated exploitation enginerdquo 2020[56] DARPA ldquoCyber grand challengerdquo 2020[57] Main Arena ldquoRoot of all managed heap memoryrdquo 2020[58] W Wu Y Chen J Xu X Xing X Gong and W Zou

ldquoFUZE towards facilitating exploit generation for kerneluse-after-free vulnerabilitiesrdquo in Proceedings of the 27thUSENIX Security Symposium (USENIX Security 18)pp 781ndash797 New York NY USA 2018

[59] W Wu Y Chen X Xing and W Zou ldquoKEPLER facili-tating control-flflow hijacking primitive evaluation for linuxkernel vulnerabilitiesrdquo in Proceedings of the 28th USENIXSecurity Symposium (USENIX Security 19) pp 1187ndash1204New York NY USA 2019

[60] S Heelan T Melham and D Kroening ldquoGollum modularand greybox exploit generation for heap overflflows in in-terpretersrdquo in Proceedings of the 2019 ACM SIGSAC Con-ference on Computer and Communications Securitypp 1689ndash1706 New York NY USA 2019

[61] D M Mena I Papapanagiotou and B Yang ldquoInternet ofthings survey on securityrdquo Information Security Journal AGlobal Perspective vol 27 no 3 pp 162ndash182 2017

[62] B S Ahmed M Bures K Frajtak and T Cerny ldquoAspects ofquality in internet of things (IoT) solutions a systematicmapping studyrdquo IEEE Access vol 7 pp 13758ndash13780 2017

[63] A Mosenia and N K Jha ldquoA comprehensive study of securityof internet-of-thingsrdquo IEEE Transactions on Emerging Topicsin Computing vol 5 no 4 pp 586ndash602 2017

[64] M Bures B S Ahmed and T Cerny ldquoInternet of thingscurrent challenges in the quality assurance and testingmethodsrdquo in Proceedings of the 9th Icatse InternationalConference on Information Science amp Applications New YorkNY USA 2018

[65] A Mosenia ldquoAddressing security and privacy challenges ininternet of thingsrdquo 2018

Scientific Programming 21