DoLTEst: In-depth Downlink Negative Testing Framework for ...

DoLTEst In-depth Downlink Negative Testing Framework for LTE Devices

CheolJun Parklowast Sangwook Baelowast BeomSeok Oh Jiho Lee Eunkyu Lee Insu Yun and Yongdae KimKorea Advanced Institute of Science and Technology (KAIST)

fermioncj hoops beomseoko jiholee ekleez insuyun yongdaekkaistackr

AbstractAn implementation flaw in LTE control plane protocols at

end-user devices directly leads to severe security threats Inorder to uncover these flaws conducting negative testing is apromising approach whose test case only contains invalid orprohibited messages Despite its importance the cellular stan-dard mostly focuses on positive test cases producing manyimplementation vulnerabilities unchecked as evidenced bymany existing vulnerabilities To fill this gap we presentDOLTEST a negative testing framework which can compre-hensively test an end-user device Enumerable test cases witha deterministic oracle produced from detailed specificationanalysis make it suitable to be used as a standard to find imple-mentation vulnerabilities We uncovered 26 implementationflaws from 43 devices from 5 different baseband manufactur-ers by using DOLTEST demonstrating its effectiveness

1 IntroductionDespite the passage of 13 years since the first release long-term evolution (LTE) remains the dominant protocol overthe newly implemented 5G network Recent reports shownearly 6 billion LTE subscriptions worldwide whereas 5Gsubscriptions only exceed 04 billion [20 25] Additionallymost 5G-capable devices still support the LTE protocol owingto backward compatibility needs and slow 5G standalone(SA) mode deployment compared to non-standalone (NSA)mode [26]

LTE security remains critical because it provides privacy-sensitive data plane services such as voice calling short mes-sage services (SMS) and internet access For these data planeservices to work efficiently and safely the LTE control planesupports basic control operations such as security controlmobility management and authentication Thus any insecuri-ties related to the control plane can affect the confidentialityintegrity and availability in the data plane

For LTE security discovering implementation vulnerabil-ities is as important as eliminating standard (design) vul-

lowastThese two authors equally contributed

nerabilities Even though researchers have uncovered sub-stantial design flaws in the LTE control plane [17 29 3032 36 38 40 41 49 51ndash53] it does not necessarily leadto secure implementations Implementation vulnerabilitieshave been continuously reported including memory corrup-tions [22 34 42 43] and non standard-compliant vulnerabil-ities [16 27 36 43 46 48 50] Although the former mainlyoriginates from development mistakes the latter involves thenature of the LTE standard document (specification) it iswritten in a natural language rather than a formal languageresulting in several ambiguities Exacerbating the issue thespecification only provides positive testing specifications (ieconformance test suites [8 9]) that mostly verify positivecases to see if valid messages are correctly handled In otherwords the conformance test mostly ignores negative test-ing which examines if invalid or prohibited messages areappropriately handled Thus this may leave many implemen-tation vulnerabilities unchecked

For this reason previous studies have attempted severalapproaches to uncover implementation vulnerabilities Earlierworks focused on a few prohibited messages to check whetherdevices drop these messages via manual testing [4552] Rup-precht et al [48] proposed the first testing framework to deter-mine whether prohibited algorithms can be selected in devicesLTEFuzz [36] presented the security testing system that ex-amines the message authentication logic for various controlplane messages in UEs and network equipment Despite theirsuccesses in discovering implementation vulnerabilities pre-vious works still fail to comprehensively cover negative casesthat are explicitly prohibited by specification This is becausethey rely on only a small part of specification We believe thatmanual yet detailed efforts to understand the specificationare unavoidable because the specification written in informaland ambiguous forms represents 13 years of LTE historyincluding endless discussions among 3GPP representatives

In this paper we propose DOLTEST a negative testingframework for LTE which can comprehensively test a userequipment (UE) (ie devices) based on specification Unlikeother works it supports an enumerable number of test cases

with a deterministic oracle that describe standard-compliantbehaviors for each negative test case Our approach inherentlyinvolves a significant amount of manual efforts to analyzethe specification to find implementation flaws that are notcompliant with it

Despite our best efforts it is nearly impossible to fullyunderstand the specification due to its ambiguities and highcomplexity To address this issue DOLTEST uses the fol-lowing approaches First we re-define UE states based ona security context which changes UErsquos protocol flows Sec-ond with respect to this new definition we generate a guide-line by carefully analyzing the specification The guidelineis a manually-written rule that specifies message types andcontents to generate negative test cases To tame ambigui-ties in the specification DOLTEST generates test cases over-approximately from the guideline We then run negative test-ing with our initial test cases against various UEs (ie 43UEs) Our basic expectation is that every test case shouldbe silently dropped because the guideline is designed to pro-duce only negative cases If any UE does not silently drop acertain case we re-check the specification and even discusswith a 3GPP representative to confirm a standard-compliantbehavior As a result we can refine these over-approximatedtest cases to obtain enumerable negative test cases with thedeterministic oracle Such a procedure for building test casesand oracle is labor-intensive however this is a one-time costand other implementations can easily adopt DOLTEST with-out re-spending such efforts that we already put Finally wehope that our test suite could be standardized to fill the gap ofnegative testing in conformance specification

Using DOLTEST we tested 43 cellular devices (from sevendevice manufacturers that use baseband processors from thetop five major baseband manufacturers) We generated a totalof 1848 test messages considering the abstracted UE statesVia manual root cause analysis we found 26 implementationflaws of which 22 were not previously reported These im-plementation flaws can directly lead to critical vulnerabilitiessuch as eavesdropping authentication bypass or informa-tion leakage We validated attack scenarios by exploiting theabove vulnerabilities including network identity amp time zonespoofing SMS injection and traffic eavesdropping

We also discuss our experience analyzing ambiguities inspecifications First we show a few cases from which wewere not able to define the standard-compliant behavior evenfrom the specification We report our discussion on threeambiguity issues with a representative from 3GPP Second wereport a few example cases where the specification does notclearly define the UErsquos behavior leading to deviant behaviorsamong our test UEs Later we show how such behavior canbe used to fingerprint baseband chipsets Finally consideringthe size of the specification one may think that the use ofNLP (Natural Language Process) could be useful to solveour problem We discuss why this could be challenging evenwith the recent progress in NLP technologies

eNB EPCUEDownlink

Uplink

MME HSS

IPNAS

RRC

Layer 2 (MAC)

Layer 1 (PHY)

AP

BP

Control plane

Data plane

S-GW P-GW Internet

IMS network

Figure 1 LTE network architecture

Contributions DOLTEST is the first comprehensive neg-ative testing framework for LTE downlink implementa-tions to find non standard-compliant vulnerabilities in UEsDOLTEST supports an enumerable number of test cases witha deterministic oracle that describes standard-compliant be-haviors for each negative test case through the manual analysison the specifications of NAS and RRC Using a total of 1848test cases we tested 43 cellular devices spanning 5 majorbaseband manufacturers to uncover 26 implementation flawsincluding 22 previously unknown ones Furthermore we dis-cuss our experience analyzing ambiguities in the specificationin detailScope DOLTEST focuses on finding implementation vulnera-bilities for downlink message processing in UEs In particularDOLTEST aims at finding non standard-compliant bugs formessage authentication without considering standard vulnera-bilities or memory corruptions DOLTEST focuses on controlplane messages in LTE (ie NAS and RRC) DOLTEST alsoadopts generic adversarial models of LTE network whichhave no knowledge of cryptographic keys to bypass integritychecks in LTEResponsible Disclosure We have responsibly disclosed allvulnerabilities that we found We reported each to the corre-sponding (device and baseband) manufacturer since 2019 andare actively working with them for patchingAvailability DOLTEST is publicly available on httpsgithubcomSysSec-KAISTDoLTEst

2 Background

21 LTE Network ArchitectureThe LTE network comprises three components a UE evolvedNode B (eNB) and evolved packet core (EPC) (see Fig 1)UE refers to any device located at the edge of a cellular net-work that provides voice and data cellular services to end-users It embeds an application processor (AP) that handlesthe operating system and application services and a basebandprocessor (BP) that is responsible for mobile communicationsincluding radiodigital signal processing identity manage-ment cryptographic protection and network authenticationeNB provides a wireless connection between the UE and theEPC as a base station Furthermore for secure and reliablecellular service it manages radio resources and wireless con-nections while offering confidentiality and integrity protectionto control plane messages and user plane data for each UEusing a dedicated Radio Resource Control (RRC) protocolEPC comprises a mobility management entity (MME) for

RRC Connection setup Complete + NAS Attach Request

RRC SecurityModeCommand ndash Complete

UE eNB EPC

Security Protected

NAS Authentication request - responseNAS Security mode command - complete

NAS Attach accept ndash complete + EPS bearer activation

RRC ConnectionRequest - Setup

Normal Cellular Service

NAS Identity request

NAS Identity request (IMEI) - response

6 0 e 2 1 3 2 01 7 0 5 0 7 5 5 0 2

Identityrequest IMEISeq EMM

Integrity protectedMAC

~~ ~~~~

EMMPlain NAS message

Message typeSecurity Component IEvalue

NAS Identity request (IMSI)- response

Figure 2 LTE control plane procedure and message structure

user authentication and keysessionidentity managementgateways (GW) for managing IP data traffic and a homesubscriber server (HSS) for storing the authentication infor-mation of the mobile subscribers (eg international mobilesubscriber identity (IMSI) and international mobile stationequipment identity (IMEI)) In particular the MME communi-cates with the UE via the non-access-stratum (NAS) protocol

22 Control Plane OperationAttach procedure The ATTACH procedure is the mandatoryinitial process for a UE to use cellular services when it ispowered on or returns from airplane mode The ATTACH pro-cedure comprises several NAS and RRC procedures (Fig 2)Initially the UE establishes a radio connection with an eNBby exchanging RRC Connection Request-Setup messages Sec-ondly the MME and UE establish the NAS security contextby performing the EMM common procedures (ie identifi-cation authentication and security mode control) Note thatthese procedures are initiated by the MME In the identifica-tion procedure the MME asks the UE to send its identifierby exchanging Identity Request-Response messages In the au-thentication and key agreement (AKA) procedure the UEand the MME mutually authenticate each other by exchang-ing Authentication Request-Response The MME then negoti-ates the security algorithms used for encryption and integrityprotection by exchanging NAS Security Mode Command-CompleteAfter establishing a security context NAS messages are en-crypted and integrity protected Thereafter the eNB andthe UE establish the RRC security context by exchangingRRC SecurityModeCommand-Complete Finally the MME sends anAttach Accept message to inform that the Attach Request isaccepted This involves the Evolved Packet System (EPS)bearer activation which is required for the data plane serviceAfter the UE replies with Attach complete message the ATTACH

procedure completesControl plane message structure Control plane messagescomprise three parts message types information elements(IEs) and security components Each message type hasits own functional purposes within the control plane pro-cedure For example an Identity Request message type isused by the NAS identification procedure and the RRC

FBS attacker MitM attacker Signal injection attacker

Figure 3 Threat models in LTE

UEInformationRequest message type is sent by eNB to requestUE reports Depending on the message type a message cancontain a single IE or multiple IEs IEs carry certain valuesaccording to the defined length and the value type When themessage is constructed it is encapsulated by security com-ponents that are used for integrity protection and encryptionThese include a security header type a message authenti-cation code (MAC) and a sequence number The securityheader type defines the level of protection in accordance withthe following values 0ndashno security protection 1ndashintegrityprotected and 2ndashIntegrity protected and ciphered1 Note thatmessages having security header type 0 are referred to as plainunprotected or unauthenticated messages whereas messageshaving other security header types (1-5) are referred to asprotected or authenticated if they have a valid MAC The mes-sage in Fig 2 shows the message type as 0x55 indicating thatthis is an Identity Request and the IE value as 0x02 meaningthat the requested identity type is IMEI Because the securityheader type has a value of 1 the message contains a MACand sequence number to provide integrity protection

23 Attack Models for UE in LTEThere are three representative active attack models in LTE net-work Fake base station (FBS) Man-in-the-Middle (MitM)and signal injection Fig 3 Because the adversaries in allthree threat models have no valid cryptographic key of thevictim they can craft only plain messages or messages with awrong MAC value However since the FBS attacker cannothelp a UE to establish the security context with the networkit can send messages to the victim UE only before AKA pro-cedure whereas MitM and signal injection attacker can sendmessages even after AKA procedure The operational logicsof the attack models are detailed in App A

3 Problems of Prior Negative Testing31 Lack of Negative Testing in SpecificationCurrently conformance specification [8] is used to test UE im-plementations This specification focuses on UErsquos correct be-haviors the UE handles valid control plane messages properlyThis mainly entails positive testing to handle non-erroneouscases Unfortunately the specification rarely includes invalidor prohibited messages (ie negative testing) although thesemessages are crucial for security For example if the UEblindly accepts messages without authentication it can lead toserious attacks such as impersonation or information leakageTherefore it is important to test whether a UE successfully

1Security header type values 3 4 5 and 12 are dedicated for certainmessage types [4]

drops such invalid messages related to securityTo demonstrate that the existing conformance specification

has insufficient negative cases for security we count the num-ber of test scenarios that explicitly define the correspondingaction to the message having prohibited IEvalue or the mes-sage without integrity protection in the current conformancespecification [8 9] We examined the specification of version1550 Among 993 test scenarios we found that only 14 nega-tive test cases 3 and 11 cases are related to RRC and NAS re-spectively Even worse the existing test scenarios only coverlimited message types For example in RRC the conformancetesting covers only two message types for security These sce-narios check whether 1) the UE uses a correct security algo-rithm for integrity check and encryption (SecurityModeCommand)and 2) the UE discards the UECapabilityEnquiry message thathas an invalid MAC Moreover these test scenarios fail tocover 1) unsafe behaviors explicitly prohibited by the specifi-cation and 2) known implementation vulnerabilities reportedby previous works [36 45 48 52]

32 Limitations of Previous WorksAlthough there have been various approaches [36 45 48 52]for negative testing to discover vulnerabilities in UE imple-mentation they all have the following limitations(1) Stateless testing Existing works only consider limitedstates in LTE network In particular they only consider stateswhere the FBS attacker has a chance to transmit the adver-sarial messages to the UE (ie before establishing a securitycontext) However recent works have shown successful at-tack scenarios using new threat models such as MitM [4953]and signal injection [59] Such attackers are more powerfulto transmit adversarial control plane messages even after theUE establishes its security context Considering that the UErsquosoperation is stateful and the advanced threat model can sendmessage at any state of UE stateful negative testing is re-quired Unfortunately existing techniques fail to reflect suchpowerful attackers in their negative testing(2) Limited coverage in negative messages Existing worksonly focus on limited components in building negative mes-sages Remember that the control plane message consists ofthree parts message type IEs and security componentsDueto its large search space the state-of-the-art tool LTEFuzzdecides to mutate its IEs by randomly substituting valuesfrom commercial network logs This approach is effectiveto avoid nonsense values that will be early-rejected How-ever these real-world logs are network-dependent they nevercontain prohibited messages or rarely include infrequentlyused messages by the network For example Security Mode

Command messages in commercial networks never contain thesecurity algorithms EIA4ndashEIA7 whose behaviors are unde-fined and reserved for future extension In addition it wouldhardly contain the prohibited security algorithm EIA0 (nullintegrity) Both cases are important to discover an integritybypass vulnerability [48] Unfortunately LTEFuzz fails to

explore messages that ordinary networks would not transmit

4 Overview41 GoalsThe main goal of DOLTEST is to build a negative testingframework that 1) can test a UE comprehensively based onspecification 2) using deterministic oracle that can interprettest results against 3) enumerable test cases We hope that ourtest framework could be standardized to complement the lackof negative testing in conformance specificationsComprehensive testing DOLTEST should have comprehen-sive test cases to thoroughly examine the prohibited or invalidsituations that the UE can encounter To this end it needs toconsider both states and test messages simultaneously becauseeach messagersquos behavior depends on the current state of theUE Moreover unlike other works DOLTEST should con-sider every part of message components message type IEand security components It enables DOLTEST to reflect themost powerful adversary from the Dolev-Yao model [19] theadversary can overhear intercept and synthesize any messageand is only limited that she has no valid cryptographic keyof the victim This includes all existing attacks in a cellularnetwork such as FBS MitM and signal injectionDeterministic oracle DOLTEST should support determin-istic oracle for each test case to device which behavior isstandard-compliant This is essential for a standardizable testsuite to understand the correct behavior for the test case with-out any exceptionEnumerable test cases Finally we want DOLTEST to gen-erate only an enumerable number of test cases This propertyallows developers to efficiently test their implementations andto easily understand the root causes of discovered bugs ThusDOLTEST is desired to choose the significant ones amongnearly infinitely many candidates for negative test cases

42 Challenges in Negative TestingTo achieve the previously mentioned goals we need to addressthe following challengesC1 Security-irrelevant state definition in specificationTo fully evaluate a UErsquos security negative testing shouldconsider its various states however existing definitions ofstates are improper for security testing In particular the ex-isting specification explicitly defines REGISTEREDDEREGISTERED

and IDLECONNECTED states for NAS and RRC respectively Un-fortunately these definitions only describe UE connectivitywithout considering their security contexts Moreover speci-fications also describe implicit states to differentiate UErsquos be-havior based on security contexts For instance a UE shouldnot reply to an Identity Request that asks for IMEI beforethe security activation implying implicit states However be-cause of this implicitness manufacturers implement thesestates arbitrarily as long as they can pass conformance testspecifications In a nutshell existing definitions of both ex-plicit and implicit states make negative testing hard to 1)enumerate test cases for stateful security testing and 2) test

Security contextbased abstraction

Abstracted state

Specification document

Diverged UE state

Manual specification analysis

SpecTest case generation guideline Test UE

EPC

Test

Mes

sage

Resp

onse Refinement

Preliminary OracleeNB

Specification analysis

Target state

Testcase Implementation

flaw analysis

Implicationanalysis

Over-the-Air testing

Test caseResponse

(EPCeNBlog)

UErsquos internallogs

Deterministic oracle buildingManual

post-analysisTest case generation

Msg typesStatements

Rule

IEvalueSeccomp

State No-SCSechdr 0 (no integrity )

Over-approximatedtest cases

Msg Type Identity ReqIE Identity Type 2Value 0 (reserved)MAC plain

DeviantBehavior

①

②

③ ④

DeterministicOracle

Test cases

3GPPPreliminary test cases

Preliminary test cases

⑤ ⑥

Figure 4 Overview of DOLTEST

multiple UEs regardless of their underlying implementationsC2 Non-enumerable negative cases It is infeasible to testall possible negative cases because there can be an enormousnumber of different LTE messages In particular the RRCand NAS have 118 message types and each message has cor-responding optional IEs with their values whose numbers aremore than a thousand Therefore considering all componentsmdash message types IEs and values mdash they comprise a hugenumber of combinations Note that the number of candidatesbecomes even larger if we perform stateful testing

Meanwhile each trial for negative testing in UEs is expen-sive Due to the nature of over-the-air testing it is hard tosend a large number of test messages Also when UE failsthe attach procedure multiple times within a certain period itdisconnects from the test network and suspends reconnectionfor a few seconds Note that such a case frequently happensin the negative testing because it can interfere the normalattach procedure due to their abnormal cases Thus we needto carefully select only essential cases for negative testingC3 Ambiguities in complicate specification It is difficultto determine the UErsquos correct behavior when receiving eachtest case even if we can enumerate all negative test casesThis is mainly because the specification has many ambigu-ous statements and descriptions of the operations are spreadover multiple documents each of which are several hundredpages long For each test case the oracle needs to determineimplementation correctness based on the corresponding be-havior However we found that specification ambiguouslydescribes whether the test case is actually prohibited or in-valid (sect62) Additionally the statements regarding a singlecontrol procedure are defined at different places over multipledocuments This makes developers misunderstand the specifi-cation even with their best efforts Therefore building oraclebased solely on the specification is challenging consideringthe ambiguities and the size of the specifications

43 Our ApproachA1 Defining security abstracted states To address C1 weabstract the implicit states relevant to security based on thespecification In particular we focus on parts in specificationthat define the actions of the UE depending on its authenti-cated states Our key intuition comes from the fact that thespecification defines the acceptance of control plane messagesdepending on the existence of a security context Thus the

four abstracted states are categorized by the existence of se-curity context in RRC and NAS Note that these states areimplementation-independent and are fully controllable at thenetwork meaning that we can use this definition for negativetesting regardless of the underlying UE implementationsA2 Specification-driven test message generation To re-duce the huge search space in negative testing DOLTESTgenerates test cases based on the specification which includesevery essential point for negative cases However it is nearlyimpossible to completely understand the specification dueto its size and complexity To address this we first extractguidelines from the specification by carefully analyzing itand DOLTEST over-approximately generates test cases basedon the guidelinesA3 Building deterministic oracle We first assume thatnegative test messages from DOLTEST should be silentlydropped Then we validate this by evaluating each messageto an extensive set of devices (see Tab 6) If we discoverdeviant behaviors from our assumption we double-check thespecification to refine the initial oracle (sect54) It is worth not-ing that this refinement is possible since DOLTEST only hasan enumerable set of test cases

5 DesignFig 4 illustrates the workflow of DOLTEST Remember thatDOLTEST aims at constructing negative test cases with de-terministic oracle To this end DOLTEST performs the fol-lowing six steps 1 First we redefine the existing implicitUE states as security abstracted states for negative testing(sect51) 2 With this new state definition we construct rulesfor test case generation called guidelines by carefully analyz-ing the specification (sect522) 3 Then DOLTEST generatestest cases according to the guidelines To address ambiguitiesin the specification DOLTEST enumerates test cases in anover-approximated manner (sect523) 4 DOLTEST conductspreliminary over-the-air testing using the over-approximatedtest cases For testing DOLTEST uses its test EPCeNB fortransmitting a test message to a UE at a specific state Throughthe testing DOLTEST collects the outputs (eg responses anda UErsquos internal logs) for each test UE Using these outputsDOLTEST checks whether the test UE shows deviant behav-iors (sect53) 5 DOLTEST refines its preliminary test cases andoracle If a UE shows a deviant behavior in the preliminarytesting we re-check the specification and even discuss with a

Attach Request

NAS Authentication Request

NAS Security Mode Command

RRC Security Mode Command

RRC Connection Reconfiguration

NAS Attach Accept

REGISTERED

EMM Information

Downlink NAS Transport

RRC Connection Release

RRC Connection Reestablishment

NAS Attach Reject

NAS GUTI Reallocation

RRC UE capability Enquiry

NAS Identity Request


RRC ueInformation Request

Service RequestTAU Request

No-SC

N-SC

NR-SC

REGI

START END NAS RRC

Figure 5 An abbreviated state machine and security ab-stracted states for RRC and NAS protocol

representative from 3GPP to understand a standard-compliantbehavior 6 If this behavior is not buggy we modify ouroracle or eliminate non-negative test cases from our over-approximated set (sect54) Otherwise we manually analyze thisimplementation flaw (sect6) and its implications (sect7) After thisrefinement process DOLTEST can finally obtain negative testcases with deterministic oracle

51 Security Abstracted StatesWe redefine states in the LTE network for negative testingbased on the presence of NAS and RRC security contextsNO-SC N-SC NR-SC and REGI As shown in Fig 5 onecan define a UErsquos state machine using control plane messagesfrom the network (left side) However this representationcan incur a state explosion if we assume that a UE can re-ceive an arbitrary message in any state Note that this is avalid assumption in adversarial settings Moreover this statemachine is independent of the existence of security contextsThus DOLTEST newly defines a UErsquos state based on securitycontexts (right side in Fig 5) This security abstracted stateallows DOLTEST to explore different authentication logicwhich is changed by the existence of security contexts In thefollowing we discuss each state in our new definitionbull NO-SC In this state only an initial radio connection is

established between the UE and eNB without any securitycontext Therefore all control plane messages in this stateremain unprotected

bull N-SC After completing the AKA procedure a UE entersthis state and the NAS security context is established Inthis state all the NAS messages exchanged between themare protected However RRC messages are still unprotectedbecause the RRC security context is not yet established

bull NR-SC The UE enters this state after RRC security con-text is established alongside the NAS security context Thusall RRC and NAS messages are protected in this state No-tably an FBS attacker cannot attack a UE in this state asshe cannot help the UE to establish the security contextwith the network

bull REGI The UE finally enters this state when it completesthe entire registration process with the LTE network Thisstate is the same with REGISTERED state defined in the speci-

Table 1 Abstracted statesNO-SC N-SC NR-SC REGI

NAS security context 7 3 3 3RRC security context 7 7 3 3

ATTACH accomplishment 7 7 7 3Reflected Threat model F M I F M I M I M Ilowast F FBS M MitM I Signal Injection

fication A UE in this state is successfully established as adata bearer (EPS bearer) hence it can use cellular servicessuch as calling and data services

Compared to the existing explicit or implicit state definitionthis new definition entails the following propertiesSecurity-oriented These abstracted states reflect the natureof LTE security where the existence of different securitycontexts allows different authentication logic for control planemessages To comprehensively evaluate such logic our firstthree states (ie NO-SC N-SC and NR-SC) reflect theimplicitly-defined states of NASRRC with various securitycontexts We further consider the state REGI to reflect theexplicit state REGISTERED in the specification Although ithas the same security contexts with NR-SC it is reasonableto consider REGI specially because it has unique behaviorsfor certain messages defined by the specification Tab 1 showshow each threat model is associated with the abstracted statesImplementation agnostic This new definition for abstractedstates helps us to accomplish implementation-agnostic testingAs these states reflect fundamental authentication proceduresfor LTE we can explicitly control transitions between statesby following standardized LTE authentication steps For in-stance we can change a UErsquos state from N-SC to NR-SC bysending an RRC SecurityModeCommand message from our testingnetwork Therefore DOLTEST enables stateful testing formultiple devices regardless of implementation (sect53)

52 Specification-driven Message GenerationDOLTEST generates test messages that are invalid or prohib-ited by specification in a selective yet approximated mannerBasically our approach is best-effort as shown in Fig 4 wefirst carefully analyze specifications to build a rule for testcase generation called a guideline

521 Manual Specification Analysis

We manually read the entire RRC and NAS specification tocreate guidelines In particular we focused on statements re-lated with message authentication because DOLTEST aimsat discovering security issues in LTE (please see our GitHubrepository for the representative statements used to makeguidelines [55]) However these statements are not solely suf-ficient we also need to refer to other parts of the specificationto create guidelines (eg need to understand correspondingIEs for messages)

To review our analysis we leverage two auxiliary methodsFirst we rechecked the paragraphs around the security-criticalwords mdash lsquoshall notrsquo lsquosecurity activationrsquo and lsquointegrity pro-

tectionrsquo mdash These words are essential to describe messageauthentication which DOLTEST focuses on Second we com-pared different versions of the specification This is based onour intuitions that 1) revision could happen due to securitypatches and 2) new features are likely to have flaws duringimplementing them

Despite our best efforts it is nearly impossible to com-pletely understand the specification Therefore the guide-line allows to specify ambiguities and DOLTEST generatestest cases by over-approximately interpreting such ambigu-ities This over-approximation often leads to include excep-tions in its test cases (ie non-negative test cases) thereforeDOLTEST re-validates these cases while constructing the or-acle (sect54) Unfortunately DOLTEST still cannot guaranteeany completeness which will be discussed in sect8

522 Guideline generation

We carefully analyze the specification and make guidelines fornegative test case generation The guidelines are handcraftedrules for test case generation that include every componentfor test cases which consist of message type IE and its val-ues security components (security header type and MAC)and states To address ambiguities in the specification theguideline allows to specify a wildcard for the component thatspecifies arbitrary value This procedure to make guidelinesis easier than understanding the specification completelyMessage type selection We first need to choose messagetypes for guidelines As described in sect1 we aim at evaluat-ing security authentication in UEs therefore we focus onthe messages that the UEs should not accept without validsecurity protection (ie integrity protection) In other wordswe exclude messages that are not protected by design (ieplain) As a result we consider 14 message types for testingboth NAS and RRC which can be found in Tab 3Construction For each message type we generate guide-lines for negative test case generation To this end we firstcarefully analyze the specification and identify statementsthat explicitly prohibit certain messages We then generateguidelines manually by transforming these statements Wehave three rules for this transformationbull If specification does not explicitly mention a certain compo-

nent the guideline adds all possible values to its candidates(ie wildcard)

bull If specification defines invalid or prohibited values for acomponent the guideline adds them to its candidates

bull If specification defines allowed values for a component theguideline adds its complement set to its candidates2

As a result we generate 17 guidelines from the specifica-tion (Tab 5)Example Fig 6 shows how we generate a guideline fromspecification using Identity request as an example As shownin the topmost part of Fig 6 the specification only allows

2For example as shown in Tab 5 if EIA1 EIA2 EIA3 are allowed weadd test cases for EIA0 and EIA 4 to 7

Except the messages hellip below no NAS signalling messages shall be processed by the UEhellipunless the network has established secure exchange of NAS messageshellip hellip- Identity request ((if requested identification parameter is IMSI)

Guideline(sect521)

Specification

Over-approximation

(sect522)

State Security Header Type Message Type IE Value

Identity Request Identity Type 2 not IMSIMAC

No-SC 0 (no integrity protected) Identity Request Identity Type 2 0 (reserved) plain

No-SC 1 (integrity protected) Identity Request Identity Type 2 0 (reserved) plainNo-SC 0 (no integrity protected) Identity Request Identity Type 2 2 (IMEI) plainNo-SC 0 (no integrity protected) Identity Request Identity Type 2 0 (reserved) broken

N-SC 0 (no integrity protected) Identity Request Identity Type 2 0 (reserved) plain

Figure 6 The guideline for the Identity Request test messagesNote that the lsquorsquo indicates the wildcard

specific messages to be processed by the UE without integritycheck One of such messages is Identity request it has an ad-ditional restriction that the messagersquos requested identificationparameter (ie Identity Type 2) should be IMSI whose valueis 1 According to the rules described above we formulatea guideline for Identity request This guideline has a com-plement set for Identity Type2 to exclude the specified value(IMSI) from its candidates Moreover we set arbitrary val-ues (ie wildcards) for remaining components state securityheader type and MAC This reflects the above specificationit says that Identity request without IMSI should be alwaysprotected In other words a UE should not accept such a mes-sage even with any combination of the remaining componentsTherefore DOLTEST considers their all possible combina-tions to comprehensively evaluate a UErsquos authentication logic

As another example the fourth guideline in Tab 5 is de-rived from the statement ldquo the E-UTRAN may configurethe UE to perform measurement reporting but the UE onlysends the corresponding measurement reports after success-ful security activationrdquo This configuration is handled by anRRCConnectionReconfiguration message with the measConfig IETherefore the guideline is used for validating whether the UEchecks integrity correctly with this message To understandthis message one should also look procedure description onmeasurement configuration (clause 55) and message structure(clause 6) in [7]

523 Over-approximated test case generation

After finishing the guideline generation DOLTEST enumer-ates every test message according to the guidelines as shownin Fig 6 In particular for making a test case DOLTESTchooses one of the candidate values in each componentFor information elements and security header componentsDOLTEST uses possible values defined in [4] and [3] ForMAC DOLTEST uses zero (ie no integrity) and random(ie broken integrity) MAC for their candidates This stepis fundamentally over-approximation because there may besome exceptions in our enumerations which are not truly neg-ative messages DOLTEST can discover these exceptions byobserving the behaviors of multiple implementations whichare discussed in sect54 Finally we generate 1848 test cases intotal from 17 guidelines which are shown in Tab 5

53 Over-the-Air TestingDOLTEST conducts the generated test cases using its over-the-air testing frameworkTesting environment As shown in Fig 4 over-the-air test-ing is composed of the test EPCeNB and the test UE Toimplement the test EPCeNB we use a modified version ofsrsLTE [24] (release 19_09) in combination with an USRPB210 as software-defined radio The test UE is equipped witha programmable SIM card (ISIM-SJA2 or SIM-SJS1-4FF)It is connected to the testing framework which uses the di-agnostic message (DM) monitoring tool [28 58] to obtainchanges in baseband and Android Debug Bridge (ADB) [11]to obtain system-wide changes During the test case execu-tion the radio interface and test UE are encapsulated into aFaraday case [54] to avoid any interference to the commercialnetwork and usersTesting procedure For each test case which consists of atest message and an abstract state DOLTEST first moves thetesting UErsquos state to the target state sends the test messageand collects responses for finding bugs To change the UErsquosstate DOLTEST follows standard authentication proceduresin LTE the EPCeNB sends 1) Security Mode Command in NASand RRC for N-SC and NR-SC respectively and 2) Attach

Accept message for REGI Then the EPCeNB sends the testmessage to UE and DOLTEST collects the UErsquos responses(if any) alongside with internal log messages retrieved viaDM and ADB The responses and logs are then used to buildthe deterministic oracle (sect54)Reducing state transition overhead For efficiency in over-the-air testing DOLTEST can reduce repetitive state transi-tion overhead Remember that state transition happens onlyby special messages such as Security Mode Command or Attach

Accept In other words we can assure a UErsquos state remains thesame if it accepts any message that is not one of these specialones Using this property DOLTEST makes a UE to executemultiple messages consecutively without going back to theinitial state NO-SC This makes DOLTEST exempt from atime-consuming procedure that reboots the UE by invokingairplane mode at every test case execution

However it does not mean that DOLTEST can send anunlimited number of test messages at once The specificationdefines that every message should be processed in a specifictime slot otherwise a UE silently releases the connection andincreases its attempt counter If this attempt counter exceedsits threshold the UE will stay in a disconnected state for12 minutes (ie T3402 [4]) resulting the delay in testingThus DOLTEST limits the number of consecutive test casesEmpirically the current prototype of DOLTEST only sendssix messages in a session to avoid this issueTesting time DOLTEST requires several hours to test all1848 test messages due to its over-the-air testing We ob-served that one session took about 10 seconds and the overalltesting took 8 hours on average this testing time varies over

the tested devices There are several reasons for such largetime consumption First DOLTEST needs to wait for secondsto determine whether the UE does not respond to a test mes-sage One of the most common reactions against negativemessages is silently rejecting the messages To distinguishthis from a slowdown in delivering responses we use a time-out mdash 2 seconds in our prototype mdash which delays the overalltesting Second the UE occasionally stops reconnecting tothe testing framework after disconnection This situation lastsfor a few minutes due to the radio environment or internallogic of the UE To escape from it we sometimes manuallyinvoked the airplane mode to reset This idle time occupiesover 70 of the overall testing

54 Deterministic Oracle BuildingDOLTEST builds a deterministic oracle to overcome theambiguities in understanding specification In particularDOLTEST first makes initial assumptions and refine themfor building the oracleInitial assumptions DOLTEST initially assumes that a UEshould silently drop all our test messages This is because weconstruct our test messages that are explicitly prohibited orinvalid based on the specificationRefinement DOLTEST refines a preliminary oracle basedon deviant behaviors in implementations Below we show ourrefinement procedure based on the type of deviant behaviorsbull If a UE returns a normal response it means either 1)

the UE has an implementation flaw or 2) the message isexceptional in the specification To confirm this we inves-tigate every part of the specification that defines how theUE should behave if it receives the test message Then wecheck whether the current response is standard-compliant ornot If the specification allows this test case as an exceptionwe eliminate this case from the over-approximated set be-cause it is actually a positive case Otherwise we considerit as an implementation flaw (sect61) and further analyze itsimplication (sect7) Finally we might not be able to figure outthe standard-compliant behavior even after checking thespecification due to its ambiguity In such a case we dis-cuss with a representative from 3GPP (sect62) to understandthe standard-compliant behavior

bull If a UE sends a reject or an error message we furthercheck whether the specification mandates to use a specificreject or an error message If so we modify the oracle toregard the correct behavior as sending that message andconsider other behaviors as non standard-compliant If astandard-compliant behavior for the test case is undefinedwe refine our oracle to ignore any reject response Note thatthis unspecified behavior could be used for device finger-printing which will be discussed in sect63

bull If a UE has no response we further check the UErsquos in-ternal logs for particular message types These messagetypes have no response inherently according to the specifica-tion Such message types are Attach reject EMM information

and RRC ConnectionRelease Since these message types canchange the UErsquos internal state we use information fromDM or ADB which is collected from DOLTESTrsquos testingframework to determine whether the messages are actuallyrejected (sect53) If the UE changes its state similar to thefirst case we investigate the specification related to our testcase and check the standard-compliant behavior In App Bwe described the detailed logic for handling these messagesExcept for these three message types other message typeshave explicit corresponding response messages when theyprocess the transmitted test messages Thus for those mes-sage types DOLTEST can determine the UErsquos rejection ofthe messages without monitoring UErsquos stateDuring the refinement procedure we successfully handle

messages that return reject responses including Security Mode

Failure EMM status or RRCConnectionReestablishmentRequestMoreover we also eliminate positive cases from our over-approximation related to Identity Request Lastly we refinedthree ambiguities in the specification (sect62) and two types ofunspecified rejecting behaviors (sect63)

6 Negative Testing ResultsWe applied DOLTEST on 43 cellular devices from five ma-jor baseband manufacturers Qualcomm Exynos MediaTekHiSilicon and Intel We include details for these devices inAppendix (Tab 6) Using the guidelines in sect52 we gener-ated 1848 test messages and conducted the test at securityabstracted states in sect51 In summary we uncovered 26 im-plementation flaws using our negative test framework (sect61)We also uncovered inconsistent error handling among de-vices (sect63) Note that such inconsistencies are legitimatebecause they are undefined in the specification however theycan also be used for device fingerprinting (sect72)Statistics Among 1848 test messages 229 of them havehelped DOLTEST to discover flaws In addition 83 test mes-sages exhibited inconsistent error handling behaviors as de-scribed in sect63 For the remaining 1536 messages all UEshandled them without any problems

61 Implementation FlawsUsing DOLTESTrsquos negative test suite we could discover 26implementation flaws We categorized implementation flawsinto nine types based on the problematic behaviors as shownin Tab 2 The numbers in column lsquoDrsquo represents the ourfindings on each flaw type

Tab 3 shows uncovered implementation flaws on each mes-sage in each security abstracted state Due to the space limitwe leave our full testing results in the Appendix (Tab 6) Ex-isting work has shown that some UEs accept a few unauthen-ticated RRCNAS messages While those bugs are patched innewer basebands we were able to find new variants thanks toour extended coverage We found a total of 22 cases amongwhich 4 cases were previously discovered in old basebandsIn addition owing to the stateful testing that considers the var-

Table 2 Description of implementation flaw typesS Security header type mishandling D L B A

S1 Accept invalid security header types for certain message types 5 0 0 0S2 Accept invalid security header type for certain UE states 3 2 0 0S3 Mishandle reserved security header type 1 0 0 0

M Message type mishandling D L B AM1 Accept prohibited message types before security activation 2 2 0 0M2 Accept unprotected messages with certain message types 6 0 0 20

after security activationI IEvalue mishandling D L B A

I1 Accept prohibited IEs 3 1 0 0I2 Accept prohibited values 3 0 0 0I3 Mishandle reserved values 3 0 2 0I4 Mishandle reserved IEs 0 0 47 0

D DOLTEST L LTEFuzz B BaseSpec A Atomic

ious threat models DOLTEST uncovered four flaw types (M2S1 S2 and S3) in the state where the UE has security con-text (ie post-AKA)

To show the effectiveness of our approach we comparedDOLTEST with the recent works mdash LTEFuzz BaseSpec andAtomic mdash on uncovering baseband implementation flawsWe directly compare with the results reported in these papersexcluding design flaws that are out-of-scope of our work Thisis because LTEFuzz [36] and Atomic [15] are not publiclyavailable and BaseSpec only supports a subset of our testeddevices Plus due to the anonymity part of the analysis codefor BaseSpec is not publicly available which makes reproduc-tion non-trivial The numbers in right four columns indicateuncovered implementation flaws for each flaw type In partic-ular the state-of-the-art negative testing tool LTEFuzz [36]can only cover three types (S2 M1 I1) finding five implemen-tation flaws This is because unlike DOLTEST LTEFuzz islimited to exploring UE states and message components (egnegative IEvalues and security header types) Other worksare specialized in finding specific types of flaws In particu-lar Atomic [15] found 20 flaws of the M2 flaw type whichare originated from mishandling three message types AlsoBaseSpec [34] targeted the implementation flaws in messageparsing and found numerous flaws of the I3 and I4 flaw typesIt uncovered 47 flaws of the I4 flaw type which are originatedfrom the mishandling of NAS messages having malformedstructure This type of flaw is out of DOLTESTrsquos testingscope The generated test messages in DOLTEST follow thedefined structure by specifications Compared to other worksDOLTEST could uncover the various types of flaws owing toits over-approximated test case generation and stateful testing

The exploitability of each implementation flaw heavily de-pends on its tested state which is directly mapped to thethreat model In particular while the implementation flawsin NO-SC could be exploited by FBS the implementationflaws in other states only can be exploited by MitM or SignalInjection Note that each threat model has its own challengesfor conducting the attack(s) The FBS attacker needs to oper-ate with higher signal strength to lure the victim Includingthe FBS requirement the MitM attacker requires to maintainradio configuration is additionally Signal injection attacker

Table 3 Implementation flaws on messages The numbers in parentheses indicate the number of flaws that have the samemessage type and states but different details which results in different implementation flaws (See sect61 for the detailed reasons)

Protocol Message State Implication StudiedNO-SC N-SC NR-SC REGI All

RRC

RRCConnectionReconfiguration I1(2)dagger I1 M2 - AKA bypass (I1) Location leak (I1M2) [36] [52]RRCConnectionRelease - M2 - Redirection attack (M2) [41]SecurityModeCommand I2daggerI3 - - Eavesdropping (I2I3) [48]UECapabilityEnquiry - M2 - Information leak (M2) [53]CounterCheck M1 M2 - Information leak (M2) -UEInformationRequest M1dagger M2 - Location leak (M1M2) [52]DLInformationTransfer - M2 - - -

NAS

Identity Request I2I3 - S1S2(2)

S3

Information leakage (S1S2I2I3) [43]Security Mode Command I3 - - Eavesdropping (I3) [48]GUTI Reallocation Command - S1 Identity spoofing (S1) Denial-of-Service (S1) [36]EMM Information - S1 - NITZ spoofing (S1) [45]Downlink NAS Transport - S1 - SMS phishing (S1) [43]Attach Reject S2I2 - S1 Denial-of-Service (S1S2I2) [52]Attach Accept - - - -

Studied Attacks using the message type was previously studied dagger Previously reported

needs to be synchronized to the legitimate eNB and its signalstrength needs to be higher than 3 dB for the capture effect

In the following we discuss implementation flaws thatDOLTEST can discover in detail

611 Mishandling security header type

S1 Accept security header types invalid for certain mes-sage types The 3GPP specification defines the value of theavailable security header types for each message For examplethe two security header types mdash integrity protected with newEPS security context (3) and security header for the ServiceRequest message (12) mdash are set to be used only for Security

Mode Command and Service Request respectively Therefore theother message types should not use these dedicated valuesaccording to the specification (clause 931 in [4])

The S1 flaws are uncovered by checking if devices acceptNAS messages having such invalid security header types Wefound that every device using Qualcomm baseband acceptsany NAS message (except two message types) without check-ing the integrity if the security header type value is set to 33

This implies that the adversary can launch all known attackson NAS (Tab 3) by using that particular security header typeConsidering its severity Qualcomm has assigned a CVE witha critical security rating (CVE-2019-2289)S2 Accept invalid security header type for certain UEstates The specification defines that a UE needs to handle se-curity header types properly according to its state Particularlyin NO-SC where the security context is not yet establisheda UE should discard a message with the security header typesmdash integrity protected (1) or integrity protected and ciphered(2) mdash because UE cannot calculate a valid MAC Moreovera UE should not accept a message with the security headertype not security protected (0) after the security context isestablished (clause 4442 in [4])

DOLTEST discovered three vulnerabilities when handling

3Because we reported this issue in early 2019 the issue was patched ondevices released after that

Identity Request and Attach Reject In particular we foundthat iPhone 6 and Galaxy Note 5 responded to any Identity

Request whose security header type is 1 even before the se-curity context is established (ie NO-SC)4 This causes in-formation leakage because these UEs send privacy-criticalidentities (eg IMEI or IMEISV) in NO-SC

Moreover we discovered that many UEs respond to themessage with security header type 0 after N-SC Our neg-ative testing shows that devices with Exynos (all) deviceswith MediaTek (all) and devices (LG and Samsung) withQualcomm baseband contain this implementation flaw Whenthese devices receive a plain Identity Request after N-SCthey respond to it with an Identity Response message that con-tains the IMSI Note that this bug only appears in Identity

Request message when Identity type 2 IE is set to the reservedvalue (0) It also violates a standard because UEs shouldnot accept plain messages after N-SC Previous work hasshown that some UEs mishandle the IE value on Identity

Request [43] However our result shows that mishandling aninvalid security header type also leads to the same problemS3 Mishandle reserved security header type The specifi-cation does not define usages for security header types from 6

to 11 which are reserved for future extension Plus if a UEreceives security header types ranging from 13 to 15 in a mes-sage it should interpret those as 12 which is dedicated to theService Request message (clause 931 in [4])

DOLTEST discovered that some UEs violate the afore-mentioned behaviors in the specification (S3) We found thatall tested UEs have no problem when the security headertypes are 6ndash11 they discard all messages with the headertypes However we found that several UEs fail to handleother reserved security header types Particularly devices withQualcomm (all produced before mid-2017) and one Huaweibaseband send an EMM Status message when they receive amessage with security header type 12 By contrast they do

4iPhone 6 with the old firmware (iOS 121) does not have this flawwhereas the same model with the latest firmware (iOS 1251) has

not respond to messages with security header type 15 In otherwords the UEs behave differently to messages with those twosecurity header types which should be the same Meanwhileother devices do not respond to messages with those securityheader types conforming to the standard

612 Mishandling message type

M1 Accept prohibited message types before security ac-tivation The cellular system is designed to expose minimalinformation before security activation Hence the specifica-tion prohibits to process of certain message types dependingon the security context Thus UEs should not accept thosemessage types in NO-SC (clause 5 Annex 6 in [7])

We discovered that many UEs accept two prohibitedmessage types in NO-SC First all devices with theExynos baseband accept a CounterCheck and respond witha CounterCheckResponse Second devices with MediaTek (1device) Qualcomm (2 devices) and Exynos (all) base-bands accept a UEInformationRequest and respond withUEInformationResponse Although M1 was widely tested in pre-vious works [365253] we newly uncovered implementationflaws on CounterCheck The flaw in UEInformationRequest waspreviously known but we were still able to find multiple un-patched devicesM2 Accept unprotected messages with certain messagetypes after security activation Once the security contextis established certain message types should only be used inintegrity-protected messages Thus UEs should discard thesemessages if their integrity check fails after NR-SC (clause5312 in [7])

DOLTEST also found that several UEs accept unprotectedmessages with zero MAC even after NR-SC In particularwe observe that devices with Qualcomm (2 devices) basebandaccept every unprotected RRC messages types after NR-SCDue to this fatal implementation flaw an attacker can spoof ev-ery NASRRC message even after NR-SC Previous studiesfound these bugs only in NO-SC

613 Mishandling IE and value

I1 Accept prohibited IEs The specification prohibits a UEto handle the messages that contain security-critical IEs inNO-SC and N-SC Thus the UE should not accept messagescontaining prohibited IE in a specific state (clause 5 andAnnex 6 in [7])

We found the I1 flaws in five test devices that acceptthree forbidden IEs These IEs are used for establish-ing signaling bearer 2 (srb-ToAddModList) data bearer (drb-ToAddModList) and requesting signal measurement report fromUE (measConfig) In particular three devices with the Qual-comm baseband incorrectly accept a message with the firsttwo IEs DOLTEST newly discovered the bug associated withthe first IE Also DOLTEST found that two devices withthe latest Exynos baseband accept measConfig IE and sendMeasurement Report to the network Interestingly even though

this vulnerability was patched in 2016 it has been introducedagain in the latest version It shows that we require negativetesting to prevent such regression bugsI2 Accept prohibited values The standard also explic-itly prohibits the use of certain IE values in N-SC (sub-clause 4442 in [4] 5312 in [7]) We uncovered threeimplementation flaws of this type in six test devices i)Identity Request message containing an IE value to requestthe UErsquos privacy-sensitive identity (IMEI and IMEISV) ii)Attach Reject whose reject cause is set to 25 iii) Security

ModeCommand in RRC which contains null integrity protectionalgorithm (EIA0)

In particular we found that the two devices with the lat-est Exynos baseband accept the EIA0 value in Security Mode

Command in RRC which is explicitly prohibited by the standardNote that the same bug was found in the USB dongle with theHuawei baseband previously [48] Of note none of the otherbasebands from the same manufacturer had this problem Wegot high severity from Samsung for this vulnerabilityI3 Mishandle reserved values The specification containsreserved values whose operation is not clearly defined Thuswe analyzed UErsquos behavior against messages with reservedvalues

Our negative testing exposed implementation flaws of I3in three IE values First UE mishandle reserved securityalgorithms in NAS and RRC The security algorithm is a3-bit value indicating the security algorithm type for the se-curity context the value from 0 to 3 have dedicated algo-rithms whereas the values from 4 to 7 are reserved We foundthat devices with Qualcomm (two) and Exynos (one) acceptsecurity mode command with these reserved values and respondwith security mode complete containing zero MAC This im-plies that those devices nullify the integrity protection on thecontrol plane which results the same security problem to theuse of prohibited value EIA0 (clause 4441 in [4] clause5312 in [7]) Second we also discovered inconsistencyamong devices in handling the Identity Request message Inparticular if a device receives the Identity Request messagewhose identity type 2 value is zero devices with Huaweiand Intel baseband had no response whereas other devicesrespond it with Identity Response containing IMSI By furtherreferring another specification document we found that thelatter is correct implementation because the reserved value 0

should be interpreted as IMSI (clause 10559 in [3])

62 Ambiguous Statements in SpecificationDuring our research we found that the specification containsambiguous or often conflicting statements which could beinterpreted in multiple ways In particular we had to dis-cuss three issues with a representative from 3GPP SA3 tounderstand the exact intention and historical meaning of thespecification We briefly summarize our discussion belowCounterCheck We found that the specification states thehandling of the CounterCheck message differently in two parts

In the RRC specification [7] the table in the appendix (An-nex 6) specifies that the UE should not accept unprotectedCounterCheck whereas the procedure description in the body(clause 5) does not prohibit it Due to this inconsistency itis ambiguous whether accepting a CounterCheck message be-fore security activation is prohibited Meanwhile anotherspecification spells out that CounterCheck and its response areintegrity protected (clause 75 in [6]) According to the rep-resentative Annex 6 is an informative annex so it doesnot mandate integrity protection of CounterCheck Howeverhe added that because the CounterCheck message is used toverify the amount of data sentreceived it is nonsense to use itbefore the data bearer activation (ie before REGI) Shouldthis prohibited Except Samsung everyone does (sect612)UECapabilityEnquiry We also found that it is ambigu-ous to determine the correct operation of UE againstUECapabilityEnquiry This is because this behavior is writtenonly on the network-side without clearly defining the UErsquosbehavior The specifications after v1590 say that E-UTRANshould retrieve UE capabilities only after AS security acti-vation for the UECapabilityEnquiry message This means thateNB can request the UErsquos radio capability information onlyin NR-SC and REGI However it is unclear that what UEshould do when it receives such a message in NO-SC Inter-estingly our test results show that all UEs respond in NO-SCMeanwhile a previous work regards such unauthenticatedUE capability [53] as a security vulnerability in terms ofinformation leakage

The representative said that the specification does not ex-plicitly prohibit the UE to transmit UE capability to eNBbefore security activation but only recommends to prohibitThis is because the term should means it is highly rec-ommended but not mandatory whereas shall means it ismandatory according to the specification drafting rule [1]Also he added that considering security implications of unau-thenticated UE capability transfer procedure the standardbody tried to prohibit the use of UECapabilityEnquiry beforesecurity activation by using the term shall However due tothe performance optimization issue some network equipmentmanufacturers opposed the prohibition Because the currentstandard allowing exceptional cases with the term should isthe result of a compromise UE capability transfer before se-curity activation is not an implementation vulnerability As aresult we exclude this message during the refinement processIdentity Request Lastly we found an ambiguous descrip-tion for the Identity Request message that contains a reservedvalue According to the NAS specification 1) a UE can re-spond to a plain Identity Request message if request iden-tification parameter is IMSI before security activation [4]Concurrently another specification [3] indicates that 2) theUE should interpret the reserved value as IMSI [3] Our test-ing revealed that UEs with Huawei or Intel baseband did notrespond to the plain Identity Request message containing thereserved value When we reported this issue to the Huawei

PSIRT (Product Security Incidence Response Team) theyargue that having no response is correct behavior becausethey think the reserved value is not exactly an IMSI parameterIf it is they argue that statement 1) should have referred thestatement 2) However according to the 3GPP representativehaving a response with IMSI is the correct behavior

63 Unspecified Behaviors in SpecificationDuring the test we observed that devices exhibit inconsis-tent behaviors when handling the same messages Unlike theprevious implementation flaws these inconsistencies are alllegitimate because they are unspecified in the specificationHowever we believe that such inconsistencies can be used tofingerprint devices which will be shown in sect72 After inves-tigating these issues we summarized them into two typesImplicit vs explicit reject UErsquos rejecting behaviors can bedivided into two types implicit reject and explicit rejectIn particular UEs can silently drop a message without anyaction if the message is prohibited or invalid (implicit re-ject) In contrast they can use certain messages to reporterrors which include Security Mode Reject EMM Status andRRCConnection ReestablishmentRequest (explicit reject)

We found that rejecting behaviors for certain messages aredifferent depending on manufacturers For example deviceswith Huawei baseband send EMM status when they receivea NAS message with invalid security header type 3 How-ever devices with old Qualcomm baseband send the messagewhen they receive a NAS message with reversed securityheader type 12 instead of 3 Meanwhile devices with Medi-aTek Exynos Intel and recent Qualcomm basebands neversend that message during our testDifferent causes in explicit reject We discovered that de-vices use different causes to report an error These causesoffer a detailed description of the erroneous circumstancesFor example we found that when devices receive NAS Security

Mode Command message with null integrity protection deviceswith MediaTek baseband use cause 23 (UE security capabil-ities mismatch) whereas others use cause 24 (security moderejected unspecified) This implies that the UEs regard thereason for the error differently for the negative messagesFundamental reasons for inconsistencies We argue thatthe fundamental reason for this inconsistent behavior amongtest devices is the lack of specifications for handling invalidor prohibited messages In other words the specification doesnot clearly define whether to use error messages or whichcauses to use when the UE receives certain invalid or prohib-ited messages Thus handling undefined error cases is up tothe developers leading to different implementations for man-ufacturers Meanwhile these inconsistencies enable devicefingerprinting attacks (sect72)

7 Attack ScenariosIn this section we introduce several representative attacksthat can be performed using the implementation flaws dis-

cussed in sect6 The complete list of possible attacks is shown inTab 3 We implemented each attack on a testbed by slightlymodifying srsLTE [24] with USRP B210X310 and validatedthem on COTS UEs

71 Attacks using Implementation Flaws711 NITZ (Network Identity and Time Zone) spoofing

In this attack an FBS attacker can manipulate the date localtime UTC offset network name and daylight saving time ofthe victim device [45]Vulnerability An attacker can bypass security protection ver-ification by altering a security header type to 3 (S1) Undernormal circumstances this message should be transmittedwith proper security protection Although we validated this at-tack on a Xiaomi Black Shark all devices with the Qualcommbaseband prior to the Snapdragon X24 are vulnerableAttack Procedure When the victim UE is connected to amalicious eNB the attacker sends an EMM Information messagewith security header type 3 containing a manipulated timeor date Although the EMM Information message contains aninvalid MAC value the victim UE accepts the messageImpact When a UE receives this message it updates its dateand time based on the message data which are controlled bythe attacker This attack is valid while the victim is connectedto the FBS However if the UE reconnects to the networkthrough Service Request this attack can be sustained becausethe EMM Information message is optional in this process unlikeAttach Request Note that we have not seen the message in theService Request process of our commercial network

712 SMS injection

The goal of this attack is to deliver an SMS with manipulatedsender ID timestamp and text content The signal injectionattacker can execute this on a victim UE connected to com-mercial networksVulnerability An attacker can perform this attack by combin-ing two vulnerabilities She first uses the vulnerability (M2)that some devices do not check security protection for RRCmessages even after the security activation Using this thesignal injection attacker can make a UE in REGI state toreceive a plain RRC message Then the attacker can bypassthe NAS security context by using the second vulnerability(S1) UEs accept some NAS messages without protection ifwe use the security header type 3 We validated this attack onGalaxy S4S5 with a Qualcomm basebandAttack Procedure The attacker first creates an SMS PDU en-coded in GSM 0340 format [2] containing malicious contentsShe then encapsulates the PDU into a plain DL Information

Transfer message containing Downlink NAS Transport with se-curity header type 3 Afterward the attacker performs a signalinjection attack to transfer the crafted message To do thisthe attacker additionally needs to know the victim UErsquos ra-dio identifier (RNTI) which can be acquired from identitymapping [32 49] using downlink sniffing tools [12 14 21]

Table 4 Different responses of devices to negative messages

Baseband Device Message

1 2 3 4 5

Intel Apple iPhone XS middot middot middot A5 middotQualcomm Xiaomi Mi Mix 2 middot A2 A4 A5 A3Exynos Samsung Galaxy S10 A1 middot A4 A5 middotMediaTek LG K50 middot middot A4 A6 middotHiSilicon Huawei Mate 20 Pro middot A3 middot A5 middot

bull Requestndash 1 CounterCheck 2 GUTI reallocation command with securityheader type 3 3 Identity Request with reserved value (0) 4 Security mode com-mand with security header type 3 and reserved value 5 EMM information withsecurity header type 12bull Responsendash middot No response A1 CounterCheckResponse A2 GUTI reallocation

complete A3 EMM status with cause 97 A4 Identity Response A5 Security modereject with cause 24 A6 Security mode reject with cause 23

Impact When a victim UE receives this message it displaysa notification the same way as in the case of a normal SMSOwing to security component mishandling the victim UE ac-cepts the attackerrsquos plain message while it is connected to thecommercial network Because the LTE device still supportslegacy SMS over NAS through the Downlink NAS Transport italso accepts malicious SMS messages

713 Eavesdropping and manipulating data traffic

In this attack a MitM attacker can eavesdrop or manipulatedata plane packets between a victim UE and networks [48]Vulnerability An attacker can set the null integrity protectionalgorithm because the UE accepts the forbidden security al-gorithm (I2) We validated this attack on Galaxy S10 Note10and A71 with the latest Exynos chipsetAttack Procedure When the victim UE is connected to themalicious eNB the attacker initiates the normal ATTACH proce-dure with the UE Although the attacker does not have a validkey she can pass mutual authentication and the NAS securityactivation procedure by relaying messages from a legitimatenetwork Thereafter the attacker sends an RRC SecurityMode

Command message with EIA0 After that the UE uses null in-tegrity protection in its subsequent communicationImpact When a device receives a SecurityModeCommand mes-sage with EIA0 it uses null ciphering and null integrity protec-tion from a subsequent communication (subclause 5312 [7])Because the data plane packets of the wireless end are trans-mitted in plain text in subsequent communication an attackercan eavesdrop or manipulate the packets Note that EIA0 inwhich an UE accepts EIA0 (Null algorithm) for the integritychecking logic is an old and well-studied issue HoweverDOLTEST uncovered that brand new devices (eg GalaxyS10Note10A71) still have the same implementation flaw

72 FingerprintingThe different behaviors of UEs against the same negativemessage enable an attacker to fingerprint the baseband of thevictim device in the wireless domain To do this an attackercan leverage two types of responses from UE non standard-compliant behavior (presence of vulnerability) (sect61) and

inconsistent error handling (sect63) During the test we foundthat more than 80 of our test messages can induce those be-haviors

As shown in Tab 4 an attacker can classify devices up tothe baseband manufacturer level using these different behav-iors Moreover it is possible to classify some devices withQualcomm Samsung and MediaTek basebands up to thechipset-level because they have chipset-dependent featuresOnce the UE is identified the attacker can further executetargeted attacks by combining known vulnerabilities at theOS-level or baseband-chipset-level [53]

Unlike previous works our fingerprinting uses inconsisten-cies in implementations Previous work used the unauthen-ticated capability information of users (Attach Request andUECapabilityInformation) as fingerprint features which are al-lowed to be exposed in the air [53] While the root cause ofprevious work is due to flaw of the standard design recentstandard tries to protect such information from being exposedbefore security activation If UE information is explicitly pro-tected in the specification those features cannot be used forfingerprinting However our features remain valid unless theimplementation inconsistency remains

8 Discussion amp LimitationsAdopting NLP A natural next step seems to adopt the NLPon analyzing the large-sized and complex specifications fortest case generation Accordingly DARPA also has issueda call for a project [18] that adopts NLP on 3GPP specifica-tion analysis Recently Chen et al presented a technique todiscover a few particular vulnerabilities by applying NLP onthe specification analysis [15] While NLP looks promisingadopting it on generating negative test cases still needs toaddress several challenges according to our detailed specifi-cation analysis First it should be capable of extracting theprohibited procedure from the sentences spread over the dis-tinct locations A single sentence does not solely contributeto extract prohibited operations During the refinement pro-cedure sect54 we had to cross-check multiple sentences tounderstand the context of the procedure and to determine theimplementation flaw (sect54) Note that such a case where thedescription of a procedure is scattered far away from eachother is one of the key limitations of the prior work [15] Sec-ond the NLP has to handle ambiguities in the specificationIn sect62 we discuss three different cases of ambiguities allof which seem challenging for NLP to resolve For examplewe observed that the specification only describes the behaviorin the network-side perspective without clearly defining theUErsquos behavior In summary unless the NLP addresses thesechallenges effectively we believe that the manual analysis onthe specification is still requiredNegative testing beyond LTE Although DOLTEST aims tofind UE implementation flaws in the LTE domain we believethat the overall philosophy and approach could be adoptedto the other layers of cellular generations (2G 3G and 5G)

after specification analysis In particular DOLTEST requiresthe following considerations of new features in 5G [5 10]the 5G-SA has 1) newly defined messages (eg REGISTRATIONREQUEST) 2) newly defined or modified IE (eg SUCI) and itsvalue and 3) newly added UErsquos state This requires changesin our security abstracted states because there is a new RRCstate (RRC INACTIVE) where the RRC security context is main-tained even after the radio connection is suspended Also onecan implement a negative testing framework based on thepublicly available 2G3G open stack with another substantialanalysis We leave it as future workLimitations First DOLTEST inherently requires a lot ofmanual effort on its building This is because the specifica-tion is written in natural language without being formallydescribed However our approach requires only one-time ef-forts and can be adopted without duplicating the laboriousmanual analysis Second DOLTEST cannot guarantee anycompleteness because of its manual analysis Therefore itwould not be surprising if someone can discover additionalflaws in baseband firmware even after applying DOLTESTThird DOLTEST suffers from inherited limitations of black-box testing As blackbox testing DOLTEST cannot under-stand the internals of a UE we simply believe that the UErejects messages if it has no explicit behaviors (eg responsesor state changes in DM or ADB) However it is possible thatbaseband vendors implement hidden behaviors even thoughwe have not found any indications for this during our testingFinally DOLTEST relies on internal logs (ie DM ADB)that are occasionally limited This is mainly because 1) thisaccess is proprietary to the baseband manufacturers (egHuawei) and 2) the access is not open to all types of UEs (egIoT devices) This characteristic limits the negative testingof DOLTEST in terms of 1) the test coverage (limited testmessage types) and 2) the root cause analysis using detailedinternal information (eg stored configuration values) Webelieve that this limitation will not apply to manufacturerswho adopt the negative testing approach as they should havefull access to the UE and its debugging capabilities

9 Related WorkImplementation flaws in LTE Researchers have widelyadopted over-the-air testing to discover implementation flawsof the LTE network [3641454852] Shaik et al [52] studiedsecurity implications when a UE blindly accepts certain pro-hibited messages due to its implementation or design flawsRupprecht et al [48] proposed the first testing frameworkfor LTE baseband and discovered that several UEs acceptinsecure security algorithms Kim et al [36] introduced asemi-automatic tool that systematically tests basic securityproperties of LTE networks both in uplink and downlinkSimilar to these works DOLTEST also aims at discoveringimplementation flaws in LTE however it widens its coverageby considering UE states and detailed message componentsbased on specification

Firmware analysis on baseband implementations is alsoan effective approach to uncover implementation vulnerabil-ities [22 34 42 57] Recently Maier et al [42] proposedemulation-based fuzzing to find memory corruption vul-nerabilities in UE implementations Kim et al [34] exam-ined the standard-compliance of the baseband software bycomparing extracted binary-embedded properties with thespecification Unlike DOLTEST these approaches requirefirmware-specific analysis Moreover static analysis (egBaseSpec [34]) can suffer from false positives due to mispre-diction of runtime environmentsLTE attack models Fake Base Station (FBS) has been apredominant attack model for exploiting design or implemen-tation vulnerabilities in LTE such as IMSI catching [17 44]fake emergency alert [38] or information leak [53] Recentworks have demonstrated the practicality of MitM attack inLTE networks [48 49] Rupprecht et al [49] successfullydemonstrate data manipulation attack by exploiting the ab-sence of integrity protection in layer 2 Moreover Yang etal [59] presented the SigOver attack which enables an ad-versary to inject a malicious message to the victimrsquos UE evenwithout a radio connection establishment To reflect theseemerging attacks in our negative testing DOLTEST adopts themost powerful attack model which is the Dolev-Yao modeland considers every attack model in our abstracted statesDesign vulnerabilities in LTE Formal verification hasshown its effectiveness in uncovering design flaws in the stan-dards [132931] Hussain et al [29] presented a model-basedadversarial testing approach on critical procedures of LTE todetect design flaws Moreover Karim et al [33] adopted aproperty-guided formal verification framework to examineLTE implementations Unlike these works DOLTEST targetsfinding non standard-compliant behaviors in implementationsSome other works also have revealed numerous design vul-nerabilities in LTE including LTE data plane [37 51] broad-cast channel [23 30 38] identity mapping logic [32 49] andVoLTE domain [35 39]Operational issues in LTE Various works have uncoveredoperational issues by analyzing the control plane proceduresin the cellular network [16272837475056] These worksspan various applications and use passive analysis on commer-cial logs Tu et al [56] proposed a signaling diagnosis tool toexamine inter-protocol interactions Hong et al [27] revealedan unsafe identity management scheme of the commercialnetwork resulting in privacy leakage Chlosta et al [16] ana-lyzed the security configuration of commercial LTE networks

10 Concluding RemarksDespite numerous implementation vulnerabilities reported alack of negative testing in specification still leaves other im-plementation vulnerabilities unchecked To address this wepresent DOLTEST a negative testing framework to uncoverthe non standard-compliant behaviors in LTE implementa-tions After in-depth analysis of the specification we build

enumerable test cases with deterministic oracle along with theover-the-air testing tool As a result we applied DOLTESTon 43 devices from five baseband manufacturers and uncov-ered 26 implementation flaws of which 22 were previouslyunknown We also found several ambiguous and unspecifiedcases in the specification that can confuse even experiencedprofessionals We recommend 3GPP to provide formally ver-ified standards possibly with sample code and negative testcases as a fundamental resolution to address this problem Un-til then applying NLP for automatic test case generation is leftas an interesting yet formidable challenge for the future work

AcknowledgementWe sincerely appreciate our shepherd Marius Muench andanonymous reviewers for valuable comments and sugges-tions This work was supported by Institute of Information ampcommunications Technology Planning amp Evaluation (IITP)grant funded by the Korea government (MSIT) (No2020-0-00428 automated security diagnosis framework for cellularnetwork protocol using formal and comparative analysis andNo2019-0-01343 regional strategic industry convergencesecurity core talent training business)

References[1] 3GPP TR 21801 v1710 Specification drafting rules 2021

[2] 3GPP TS 23040 v1530 Technical realization of the ShortMessage Service (SMS) 2019

[3] 3GPP TS 24008 v1630 Mobile radio interface Layer 3specification Core network protocols Stage 3 2019

[4] 3GPP TS 24301 v1651 Non-Access-Stratum (NAS) proto-col for Evolved Packet System (EPS) Stage 3 2020

[5] 3GPP TS 24501 v1630 Non-Access-Stratum (NAS) proto-col for 5G System (5GS) Stage 3 2019

[6] 3GPP TS 33401 v1630 Security architecture 2020

[7] 3GPP TS 36331 v15100 LTE RRC Protocol specification2020

[8] 3GPP TS 36523-1 v1550 User Equipment (UE) confor-mance specification Part 1 Protocol conformance specifica-tion 2019

[9] 3GPP TS 36523-3 v1550 User Equipment (UE) confor-mance specification Part 3 Test suites 2019

[10] 3GPP TS 38331 v1631 5G NR RRC Protocol specification2021

[11] Android debug bridge httpsdeveloperandroidcomstudiocommand-lineadb

[12] AirScope httpwwwsoftwareradiosystemscomtagairscope

[13] D Basin J Dreier L Hirschi S Radomirovic R Sasse andV Stettler A formal analysis of 5G authentication In ACMConference on Computer and Communications Security 2018

[14] N Bui and J Widmer OWL A reliable online watcher forlte control channel measurements In All Things CellularOperations Applications and Challenges 2016

[15] Y Chen Y Yao X Wang D Xu X Liu C Yue K ChenH Tang and B Liu Bookworm game Automatic discoveryof LTE vulnerabilities In IEEE Symposium on Security andPrivacy 2021

[16] M Chlosta D Rupprecht T Holz and C Poumlpper LTE se-curity disabled misconfiguration in commercial networks InConference on Security and Privacy in Wireless and MobileNetworks 2019

[17] A Dabrowski N Pianta T Klepp M Mulazzani andE Weippl IMSI-catch me if you can IMSI-catcher-catchersIn ACM Conference on Computer and Communications Secu-rity 2014

[18] Open Programmable Secure 5G (OPS-5G) httpssamgovopp6ee795ad86a044d1a64f441ef713a476

[19] D Dolev and A Yao On the security of public key protocolsIEEE Transactions on information theory 1983

[20] Ericsson2021report httpswwwericssoncomenmobility-reportdataforecastsmobile-subscriptions-outlook

[21] R Falkenberg and C Wietfeld FALCON An accurate real-time monitor for client-based mobile network data analyticsIn IEEE Global Communications Conference 2019

[22] N Golde and D Komaromy Breaking band reverse engineer-ing and exploiting the shannon baseband REcon 2016

[23] N Golde K Redon and J-P Seifert Let me answer that foryou Exploiting broadcast information in cellular networks InUSENIX Security Symposium 2013

[24] I Gomez-Miguelez A Garcia-Saavedra P D Sutton P Ser-rano C Cano and D J Leith srsLTE An Open-SourcePlatform for LTE Evolution and Experimentation In ACMInternational Workshop on Wireless Network Testbeds Experi-mental Evaluation and Characterization 2016

[25] LTE and 5G Subscribers httpsgsacomcompaperlte-and-5g-subscribers-march-2021-q4

[26] 5G implementation guidelines httpswwwgsmacomfuturenetworkswp-contentuploads2019035G-Implementation-Guideline-v20-July-2019pdf

[27] B Hong S Bae and Y Kim GUTI reallocation demystifiedCellular location tracking with changing temporary identifierIn Network and Distributed System Security Symposium 2018

[28] B Hong S Park H Kim D Kim H Hong H Choi J-PSeifert S-J Lee and Y Kim Peeking over the cellular walledgardens-a method for closed network diagnosis In IEEE Trans-actions on Mobile Computing 2018

[29] S Hussain O Chowdhury S Mehnaz and E Bertino LTEIn-spector A systematic approach for adversarial testing of 4GLTE In Network and Distributed System Security Symposium2018

[30] S R Hussain M Echeverria O Chowdhury N Li andE Bertino Privacy attacks to the 4G and 5G cellular pag-ing protocols using side channel information In Network andDistributed System Security Symposium 2019

[31] S R Hussain M Echeverria I Karim O Chowdhury andE Bertino 5GReasoner A property-directed security andprivacy analysis framework for 5g cellular network protocol InACM Conference on Computer and Communications Security2019

[32] R P Jover LTE security protocol exploits and locationtracking experimentation with low-cost software radio arXivpreprint arXiv160705171 2016

[33] I Karim S Hussain and E Bertino ProChecker An auto-mated security and privacy analysis framework for communica-tion protocol In IEEE International Conference on DistributedComputing Systems 2021

[34] E Kim D Kim C Park I Yun and Y Kim BASESPECComparative analysis of baseband software and cellular speci-fications for L3 protocols In Network and Distributed SystemSecurity Symposium 2021

[35] H Kim D Kim M Kwon H Han Y Jang D Han T Kimand Y Kim Breaking and fixing VoLTE Exploiting hiddendata channels and mis-implementations In ACM Conferenceon Computer and Communications Security 2015

[36] H Kim J Lee E Lee and Y Kim Touching the untouchablesDynamic security analysis of the LTE control plane In IEEESymposium on Security and Privacy 2019

[37] K Kohls D Rupprecht T Holz and C Poumlpper Lost trafficencryption fingerprinting LTE4G traffic on layer two InConference on Security and Privacy in Wireless and MobileNetworks 2019

[38] G Lee J Lee J Lee Y Im M Hollingsworth E WustrowD Grunwald and S Ha This is your president speakingSpoofing alerts in 4G LTE networks In ACM InternationalConference on Mobile Computing Systems (MobiSys) 2019

[39] C-Y Li G-H Tu C Peng Z Yuan Y Li S Lu and X WangInsecurity of voice solution VoLTE in LTE mobile networks InACM Conference on Computer and Communications Security2015

[40] M Lichtman R P Jover M Labib R Rao V Marojevic andJ H Reed LTELTE-A jamming spoofing and sniffing threatassessment and mitigation IEEE Communications Magazine2016

[41] H Lin LTE REDIRECTION Forcing targeted LTE cellphoneinto unsafe network In Hack In The Box Security Conference(HITBSec-Conf) 2016

[42] D Maier L Seidel and S Park BaseSAFE baseband sani-tized fuzzing through emulation In Conference on Securityand Privacy in Wireless and Mobile Networks 2020

[43] B Michau and C Devine How to not break lte cryptoIn ANSSI Symposium sur la seacutecuriteacute des technologies delrsquoinformation et des communications (SSTIC) 2016

[44] P Ney I Smith G Cadamuro and T Kohno SeaGlass en-abling city-wide IMSI-catcher detection Privacy EnhancingTechnologies 2017

[45] S Park A Shaik R Borgaonkar and J-P Seifert White rabbitin mobile Effect of unsecured clock source in smartphones InWorkshop on Security and Privacy in Smartphones and MobileDevices 2016

[46] D PAULI Samsung S6 calls open to man-in-the-middle basestation snooping The Register 12 2015

[47] S Rosen H Luo Q A Chen Z M Mao J Hui A Drakeand K Lau Discovering fine-grained RRC state dynamics andperformance impacts in cellular networks In InternationalConference on Mobile Computing and Networking 2014

[48] D Rupprecht K Jansen and C Poumlpper Putting LTE securityfunctions to the test A framework to evaluate implementationcorrectness In USENIX Workshop on Offensive Technologies2016

[49] D Rupprecht K Kohls T Holz and C Poumlpper Breaking LTEon layer two In IEEE Symposium on Security and Privacy2019

[50] D Rupprecht K Kohls T Holz and C Poumlpper Call memaybe Eavesdropping encrypted LTE calls with ReVoLTE InUSENIX Security Symposium 2020

[51] D Rupprecht K Kohls T Holz and C Poumlpper Imp4gt imper-sonation attacks in 4G networks In Network and DistributedSystem Security Symposium 2020

[52] A Shaik R Borgaonkar N Asokan V Niemi and J-P SeifertPractical attacks against privacy and availability in 4GLTEmobile communication systems In Network and DistributedSystem Security Symposium 2016

[53] A Shaik R Borgaonkar S Park and J-P Seifert New vul-nerabilities in 4G and 5G cellular access network protocolsexposing device capabilities In Conference on Security andPrivacy in Wireless and Mobile Networks 2019

[54] TESCOM TC-5910D httpentescomcokrproductproduct2code=020101

[55] Representative statements used for the guideline gener-ation httpsgithubcomSysSec-KAISTDoLTEstblobmaintable7md

[56] G-H Tu Y Li C Peng C-Y Li H Wang and S Lu Control-plane protocol interactions in cellular networks ACM SIG-COMM Computer Communication Review 2014

[57] R-P Weinmann Baseband attacks Remote exploitation ofmemory corruptions in cellular protocol stacks In USENIXWorkshop on Offensive Technologies 2012

[58] Accuver XCAL httpwwwaccuvercom

[59] H Yang S Bae M Son H Kim S M Kim and Y KimHiding in plain signal Physical signal overshadowing attackon LTE In USENIX Security Symposium 2019

AppendixA LTE Attack ModelsWe discuss representative LTE attack models in detailFBS mimics a commercial cell tower to lure nearby cellulardevices by operating at a high radio signal power After aradio connection is established between the malicious basestation and a victim device it transmits an adversarial messageto the UE Because the FBS attacker does not have validinformation to accomplish the AKA procedure with the UEshe can transmit the control plane messages only before the

UE establishes the security context Most previous attacks [2936 38 41 48 52 53] have adopted this threat model whichincludes IMSI catching UE capability hijacking and fakeemergency alertsMitM Although an MitM attacker operates a malicious basestation as an FBS attacker she establishes additional radioconnections with a legitimate eNB after a victim connectsFor a legitimate eNB the attacker impersonates the victimUE by relaying the victimrsquos messages Likewise the attackerimpersonates the legitimate eNB to the victim UE MitMattacker can transmit adversarial messages to the victim UEeven after the UE establishes the security context This isbecause the MitM attacker accomplishes the AKA procedureby relaying authentication messages of the eNB and the UEAlthough the attacker can transmit messages anytime she canonly transmit messages with invalid security protection dueto the absence of security keys Several works [48 49] haveutilized this threat modelSignal Injection is an intuitive threat model introduced bySigOver attack [59] Rather than establishing a radio connec-tion with the victim the signal injection attacker preciselyoverwrites the signal containing an adversarial message in thephysical domain Unlike the above threat models the attackerneither requires the victim UE to connect to her FBS nor torelay messages between UE and an eNB Similar to the MitMattack the signal injection attacker can inject adversarial con-trol plane messages to the victim UE at any time The attackercan pass the authentication procedure by allowing the victimUE to communicate with legitimate eNB

B Handling Silent Test Message AcceptanceIn this section we describe how we identify the test mes-sages being accepted or rejected by the UE via ADBDM ifa UE has no corresponding response (ie Attach reject EMMinformation and RRCConnectionRelease)EMM information We use ADB to determine whether thetest case of EMM information is accepted by the UE This is be-cause EMM information is used for changing NITZ informationwhich ADB can monitor by using date ADB commandAttach reject We use DM to determine whether the testcases of Attach reject are accepted by the UE The UErsquos cor-responding action to the legitimate Attach reject is to changesits EMM state to DEREGISTERED which the NAS specifi-cation defines We monitor such state changes by using DMtools which show the UErsquos current EMM state For examplewhen the state of UE is changed the chipset produces DMlogs notifying the current UE state informationRRC connection release We use DM to identify the UErsquosacceptance to the test message of RRCConnectionRelease Oncethe UE accepts the RRCConnectionRelease it destroys radio con-nection to eNB including all radio bearers and signaling radiobearers Its RRC state is then changed to RRC IDLE whichwe can monitor by DM

Table 5 Generated guidelines and the corresponding number of test cases Note that if the guideline involves two IEs (ie handover procedure by usingRRCConnectionReconfiguration) DOLTEST adds both IEs in generating test cases We omitted non-trivial IE values for brevity The represents the wildcardexplained in sect522 We used RRC specification version 15100 NAS specification version 1651

GuidelineProtocol No State Security Header Type Message Type IE MAC Reference of test cases

for each state Page

1 NA RRCConnectionReconfiguration drb-ToAddModList A6 5311 in [7] 2 68p2 NA RRCConnectionReconfiguration srb-ToAddModList SRB2 A6 5311 in [7] 2 39p3 NA RRCConnectionReconfiguration measConfig A6 5551 in [7] 2 68p

4 NA RRCConnectionReconfiguration mobilityControlInfo A6 5651 in [7] 2 918p 72psecurityConfigHO 5 NA RRCConnectionRelease A6 in [7] 2 918p6 NA SecurityModeCommand integrityProtection EIA1 EIA2 EIA3c A6 5312 in [7] 10 70p7 NA UECapabilityEnquiry A6 5632 in [7] 2 230p8 NA counterCheck A6 in [7] 2 918p9 NA UEInformationRequest A6 5652 in [7] 2 919p

RRC

10 NA DLInformationTransfer A6 in [7] 2 918p

11 Identity Request Identity Type2 IMSIc 4442 in [4] 124 50p 51p12 Security Mode Command integrityProtAlgorithm EIA1 EIA2 EIA3c 4441 4442 in [4] 155 50p13 GUTI Reallocation Command 4442 in [4] 31 50p 51p14 EMM Information 4442 in [4] 31 50p 51p15 Downlink NAS Transport 4442 in [4] 31 50p 51p16 Attach Reject EMM cause25 4442 55125 in [4] 31 50p 51p 129p

NAS

17 Attach Accept 4442 in [4] 31 50p 51p

Table 6 A full list of tested devices including device name phone vendor baseband vendor chipset model firmware version last updated date andvulnerabilities of each We used the latest firmware of a device when we started testing it and have not patched it since then to reproduce our experiments Pleaserefer to Tab 3 and sect61 for a detailed description of the implementation flaws

Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion

Last update(YYMM) Implementation flaw

1 iPhone 6 Apple Qualcomm MDM9625 72100 78004 18102101 S1S3I1 S2S3I12 iPhone 8 Apple Intel XMM 7480 40201 2103 I33 iPhone XS Apple Intel XMM 7560 10308 1902 I34 iPhone 12 Pro Apple Qualcomm Snapdragon X55 16211 2104 -5 Y9 Huawei HiSilicon Kirin 659 21C60B269S003C000 1806 S3I36 P10 Lite Huawei HiSilicon Kirin 658 21C60B268S000C000 1711 I37 P10 Huawei HiSilicon Kirin 960 21C30B323S003C000 1805 I38 Mate 10 Pro Huawei HiSilicon Kirin 970 21C10B551S000C000 1801 I39 P20 pro Huawei HiSilicon Kirin 970 21C20B369S007C000 1904 I3

10 Mate 20 pro Huawei HiSilicon Kirin 980 21C10B687S000C000 1812 I311 X401 LG Mediatek MT6750 MOLYLR11W1552MDTC01LVSFSPV1P22 1802 S2M112 X6 LG Mediatek Helio P22 MT6762 MOLYLR12AR3TC01PIESPV1P10T12 1907 S213 K50 LG Mediatek Helio P22 MT6762 MOLYLR12AR3TC01PIESPV1P26 2012 S214 G6 LG Qualcomm MSM8996 Snapdragon 821 MPSSTH201c31-00024-M8996FAAAANAZM-114234411432331 1804 S1S2S315 V35 ThinQ LG Qualcomm SDM845 Snapdragon 845 MPSSAT40c29-00057-SDM845_GEN_PACK-1 1901 S1S216 G7 ThinQ LG Qualcomm SDM845 Snapdragon 845 MPSSAT40c29-00088-SDM845_GEN_PACK-1299473 2008 S217 G8 ThinQ LG Qualcomm SM8150 Snapdragon 855 MPSSHE10c4-00104-SM8150_GEN_PACK-1 2101 S218 V50 LG Qualcomm SM8150 Snapdragon 855 MPSSHE15c4-002701-SM8150_GENFUSION_PACK-121551514 1909 S219 Oppo find X OPPO Qualcomm SDM845 Snapgragon 845 Q_V1_P14Q_V1_P14 1808 S120 Galaxy S4 Samsung Qualcomm MSM8974 Snapdragon 800 E330KKKUCNG5 1609 S1S2S3M1M2I1I2I321 Galaxy S5 Samsung Qualcomm MSM8974AC Snapdragon 801 G900VVRU1ANI2 1411 S1S3M1M2I222 Galaxy S5 A Samsung Qualcomm APQ8084 Snapdragon 805 G906LKLU1CPK2 1612 S1S2S3M2I1I2I323 Galaxy Note5 Samsung Samsung Exynos 7 (7420) N920SKSU2DQH2 1708 S2M1I224 Galaxy S6 Samsung Samsung Exynos 7 (7420) G920SKSU3EQC9 1704 S2M1I325 Galaxy Note FE Samsung Samsung Exynos 8 (8890) N935JJJU4CTJ1 2102 S2M126 Galaxy Note8 Samsung Samsung Exynos 9 (8895) N950NKOU4CRH2 1810 S2M127 Galaxy S8 Samsung Qualcomm MSM8998 Snapdragon 835 G950U1UES5CSB2 1902 S1S2S328 Galaxy Note9 Samsung Samsung Exynos 9 (9810) N960NKOU3DSLA 1912 S2M129 Galaxy S10 Samsung Samsung Exynos 9 (9820) G977NKOU2BTA2 G977NKOU4DK1 20012011 S2M1I1I2 S2M1I130 Galaxy S10 Samsung Qualcomm SM8150 Snapdragon 855 G977UVRS3YSJK 1911 -31 Galaxy A31 Samsung Mediatek Helio P65 MT6768 A315NKOU1BUA1 2102 S232 Galaxy S20 Samsung Qualcomm SM8250 Snapdragon 865 G981NKSU1CTKD 2011 -33 Galaxy A71 Samsung Samsung Exynos 9 (980) A716SKSU1ATF4 A716SKSU3BTL2 20062012 S2M1I1I2 S2M1I134 Galaxy Note20 Samsung Qualcomm SM8250 Snapdragon 865 N986NKSU1CUC9 2103 -35 Redmi 5 Xiaomi Qualcomm SDM450 Snapdragon 450 MPSSTA23c1-00522-8953_GEN_PACK-1_V042 1712 S1S336 Redmi note 4x Xiaomi Qualcomm MSM8953 Snapdragon 625 953_GEN_PACK-112263811233381 1712 S1S337 Mi Max 3 Xiaomi Qualcomm SDM636 Snapdragon 636 AT32-00672-0812_2359_46aa9a7 1807 S138 Mi 5S Xiaomi Qualcomm MSM8996 Snapdragon 821 TH20c19-0612_1733_9fe7ce8 1805 S1S339 Mi Mix 2 Xiaomi Qualcomm MSM8998 Snapdragon 835 AT20-0608_2116_6c4a86b 1805 S1S340 Black Shark Xiaomi Qualcomm SDM845 Snapdragon 845 00888-SDM845_GEN_PACK-11637131 1811 S141 POCOphone F1 Xiaomi Qualcomm SDM845 Snapdragon 845 AT40c26-144-1008_1436_e3055ba 1809 S142 ZTE Blade V8 Pro ZTE Qualcomm MSM8953 Snapdragon 625 -8953_GEN_PACK-1790911798991 1612 S1S343 ZTE Axon 7 ZTE Qualcomm MSM8996 Snapdragon 820 TH20c19-00104-M8996FAAAANAZM 1712 S1S3

Introduction

Background

LTE Network Architecture

Control Plane Operation

Attack Models for UE in LTE

Problems of Prior Negative Testing

Lack of Negative Testing in Specification

Limitations of Previous Works

Overview

Goals

Challenges in Negative Testing

Our Approach

Design

Security Abstracted States

Specification-driven Message Generation

Manual Specification Analysis

Guideline generation

Over-approximated test case generation

Over-the-Air Testing

Deterministic Oracle Building

Negative Testing Results

Implementation Flaws

Mishandling security header type

Mishandling message type

Mishandling IE and value

Ambiguous Statements in Specification

Unspecified Behaviors in Specification

Attack Scenarios

Attacks using Implementation Flaws

NITZ (Network Identity and Time Zone) spoofing

SMS injection

Eavesdropping and manipulating data traffic

Fingerprinting

Discussion amp Limitations

Related Work

Concluding Remarks

LTE Attack Models

Handling Silent Test Message Acceptance

with a deterministic oracle that describe standard-compliantbehaviors for each negative test case Our approach inherentlyinvolves a significant amount of manual efforts to analyzethe specification to find implementation flaws that are notcompliant with it

Despite our best efforts it is nearly impossible to fullyunderstand the specification due to its ambiguities and highcomplexity To address this issue DOLTEST uses the fol-lowing approaches First we re-define UE states based ona security context which changes UErsquos protocol flows Sec-ond with respect to this new definition we generate a guide-line by carefully analyzing the specification The guidelineis a manually-written rule that specifies message types andcontents to generate negative test cases To tame ambigui-ties in the specification DOLTEST generates test cases over-approximately from the guideline We then run negative test-ing with our initial test cases against various UEs (ie 43UEs) Our basic expectation is that every test case shouldbe silently dropped because the guideline is designed to pro-duce only negative cases If any UE does not silently drop acertain case we re-check the specification and even discusswith a 3GPP representative to confirm a standard-compliantbehavior As a result we can refine these over-approximatedtest cases to obtain enumerable negative test cases with thedeterministic oracle Such a procedure for building test casesand oracle is labor-intensive however this is a one-time costand other implementations can easily adopt DOLTEST with-out re-spending such efforts that we already put Finally wehope that our test suite could be standardized to fill the gap ofnegative testing in conformance specification

Using DOLTEST we tested 43 cellular devices (from sevendevice manufacturers that use baseband processors from thetop five major baseband manufacturers) We generated a totalof 1848 test messages considering the abstracted UE statesVia manual root cause analysis we found 26 implementationflaws of which 22 were not previously reported These im-plementation flaws can directly lead to critical vulnerabilitiessuch as eavesdropping authentication bypass or informa-tion leakage We validated attack scenarios by exploiting theabove vulnerabilities including network identity amp time zonespoofing SMS injection and traffic eavesdropping

We also discuss our experience analyzing ambiguities inspecifications First we show a few cases from which wewere not able to define the standard-compliant behavior evenfrom the specification We report our discussion on threeambiguity issues with a representative from 3GPP Second wereport a few example cases where the specification does notclearly define the UErsquos behavior leading to deviant behaviorsamong our test UEs Later we show how such behavior canbe used to fingerprint baseband chipsets Finally consideringthe size of the specification one may think that the use ofNLP (Natural Language Process) could be useful to solveour problem We discuss why this could be challenging evenwith the recent progress in NLP technologies

eNB EPCUEDownlink

Uplink

MME HSS

IPNAS

RRC

Layer 2 (MAC)

Layer 1 (PHY)

AP

BP

Control plane

Data plane

S-GW P-GW Internet

IMS network

Figure 1 LTE network architecture

Contributions DOLTEST is the first comprehensive neg-ative testing framework for LTE downlink implementa-tions to find non standard-compliant vulnerabilities in UEsDOLTEST supports an enumerable number of test cases witha deterministic oracle that describes standard-compliant be-haviors for each negative test case through the manual analysison the specifications of NAS and RRC Using a total of 1848test cases we tested 43 cellular devices spanning 5 majorbaseband manufacturers to uncover 26 implementation flawsincluding 22 previously unknown ones Furthermore we dis-cuss our experience analyzing ambiguities in the specificationin detailScope DOLTEST focuses on finding implementation vulnera-bilities for downlink message processing in UEs In particularDOLTEST aims at finding non standard-compliant bugs formessage authentication without considering standard vulnera-bilities or memory corruptions DOLTEST focuses on controlplane messages in LTE (ie NAS and RRC) DOLTEST alsoadopts generic adversarial models of LTE network whichhave no knowledge of cryptographic keys to bypass integritychecks in LTEResponsible Disclosure We have responsibly disclosed allvulnerabilities that we found We reported each to the corre-sponding (device and baseband) manufacturer since 2019 andare actively working with them for patchingAvailability DOLTEST is publicly available on httpsgithubcomSysSec-KAISTDoLTEst

2 Background

21 LTE Network ArchitectureThe LTE network comprises three components a UE evolvedNode B (eNB) and evolved packet core (EPC) (see Fig 1)UE refers to any device located at the edge of a cellular net-work that provides voice and data cellular services to end-users It embeds an application processor (AP) that handlesthe operating system and application services and a basebandprocessor (BP) that is responsible for mobile communicationsincluding radiodigital signal processing identity manage-ment cryptographic protection and network authenticationeNB provides a wireless connection between the UE and theEPC as a base station Furthermore for secure and reliablecellular service it manages radio resources and wireless con-nections while offering confidentiality and integrity protectionto control plane messages and user plane data for each UEusing a dedicated Radio Resource Control (RRC) protocolEPC comprises a mobility management entity (MME) for



UE eNB EPC

Security Protected







6 0 e 2 1 3 2 01 7 0 5 0 7 5 5 0 2



~~ ~~~~























Abstracted state


Diverged UE state



EPC

Test

Mes

sage

Resp

onse Refinement



Target state


flaw analysis

Implicationanalysis


Test caseResponse

(EPCeNBlog)




Msg typesStatements

Rule

IEvalueSeccomp




DeviantBehavior

①

②

③ ④

DeterministicOracle

Test cases



⑤ ⑥







Attach Request





NAS Attach Accept

REGISTERED

EMM Information




NAS Attach Reject







No-SC

N-SC

NR-SC

REGI

START END NAS RRC



























Guideline(sect521)

Specification

Over-approximation

(sect522)





































RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models




UE eNB EPC

Security Protected







6 0 e 2 1 3 2 01 7 0 5 0 7 5 5 0 2



~~ ~~~~























Abstracted state


Diverged UE state



EPC

Test

Mes

sage

Resp

onse Refinement



Target state


flaw analysis

Implicationanalysis


Test caseResponse

(EPCeNBlog)




Msg typesStatements

Rule

IEvalueSeccomp




DeviantBehavior

①

②

③ ④

DeterministicOracle

Test cases



⑤ ⑥







Attach Request





NAS Attach Accept

REGISTERED

EMM Information




NAS Attach Reject







No-SC

N-SC

NR-SC

REGI

START END NAS RRC



























Guideline(sect521)

Specification

Over-approximation

(sect522)





































RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models











Abstracted state


Diverged UE state



EPC

Test

Mes

sage

Resp

onse Refinement



Target state


flaw analysis

Implicationanalysis


Test caseResponse

(EPCeNBlog)




Msg typesStatements

Rule

IEvalueSeccomp




DeviantBehavior

①

②

③ ④

DeterministicOracle

Test cases



⑤ ⑥







Attach Request





NAS Attach Accept

REGISTERED

EMM Information




NAS Attach Reject







No-SC

N-SC

NR-SC

REGI

START END NAS RRC



























Guideline(sect521)

Specification

Over-approximation

(sect522)





































RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models


Attach Request





NAS Attach Accept

REGISTERED

EMM Information




NAS Attach Reject







No-SC

N-SC

NR-SC

REGI

START END NAS RRC



























Guideline(sect521)

Specification

Over-approximation

(sect522)





































RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models












Guideline(sect521)

Specification

Over-approximation

(sect522)





































RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models




























RRC


NAS


S3













































712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models



























712 SMS injection





1 2 3 4 5



















































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models












































































for each state Page



RRC



NAS



Name Phonevendor

Basebandvendor

Chipsetmodel

Firmwareversion




Introduction

Background







Overview

Goals


Our Approach

Design















Attack Scenarios



SMS injection


Fingerprinting


Related Work

Concluding Remarks

LTE Attack Models


DoLTEst: In-depth Downlink Negative Testing Framework for ...

Documents

Transcript of DoLTEst: In-depth Downlink Negative Testing Framework for ...