Not-Quite-So-Broken TLS: Lessons in Re …...August 12–14, 2015 • Washington, D.C. ISBN...

Open access to the Proceedings of the 24th USENIX Security Symposium

is sponsored by USENIX

Not-Quite-So-Broken TLS: Lessons in Re-Engineering a Security Protocol

Specification and ImplementationDavid Kaloper-Meršinjak, Hannes Mehnert, Anil Madhavapeddy, and Peter Sewell,

University of Cambridge

https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak

This paper is included in the Proceedings of the 24th USENIX Security Symposium

August 12–14, 2015 • Washington, D.C.

ISBN 978-1-939133-11-3

USENIX Association 24th USENIX Security Symposium 223

Not-quite-so-broken TLS: lessons in re-engineering a security protocolspecification and implementation

David Kaloper-Mersinjak†, Hannes Mehnert†, Anil Madhavapeddy and Peter SewellUniversity of Cambridge Computer Laboratory

[email protected]† These authors contributed equally to this work

Abstract

Transport Layer Security (TLS) implementations have ahistory of security flaws. The immediate causes of theseare often programming errors, e.g. in memory manage-ment, but the root causes are more fundamental: the chal-lenges of interpreting the ambiguous prose specification,the complexities inherent in large APIs and code bases,inherently unsafe programming choices, and the impos-sibility of directly testing conformance between imple-mentations and the specification.

We present nqsb-TLS, the result of our re-engineeredapproach to security protocol specification and imple-mentation that addresses these root causes. The samecode serves two roles: it is both a specification of TLS,executable as a test oracle to check conformance of tracesfrom arbitrary implementations, and a usable implemen-tation of TLS; a modular and declarative programmingstyle provides clean separation between its components.Many security flaws are thus excluded by construction.

nqsb-TLS can be used in standalone Unix applica-tions, which we demonstrate with a messaging client,and can also be compiled into Xen unikernels (spe-cialised virtual machine image) with a trusted comput-ing base (TCB) that is 4% of a standalone system run-ning a standard Linux/OpenSSL stack, with all networktraffic being handled in a memory-safe language; thissupports applications including HTTPS, IMAP, Git, andWebsocket clients and servers. Despite the dual-role de-sign, the high-level implementation style, and the func-tional programming language we still achieve reasonableperformance, with the same handshake performance asOpenSSL and 73% – 84% for bulk throughput.

1 Introduction

Current mainstream engineering practices for specifyingand implementing security protocols are not fit for pur-pose: as one can see from many recent compromises of

sensitive services, they are not providing the security weneed. Transport Layer Security (TLS) is the most widelydeployed security protocol on the Internet, used for au-thentication and confidentiality, but a long history of ex-ploits shows that its implementations have failed to guar-antee either property. Analysis of these exploits typicallyfocusses on their immediate causes, e.g. errors in mem-ory management or control flow, but we believe their rootcauses are more fundamental:

Error-prone languages: historical choices of pro-gramming language and programming style that tend tolead to such errors rather than protecting against them.

Lack of separation: the complexities inherent inworking with large code bases, exacerbated by lack ofemphasis on clean separation of concerns and modular-ity, and by poor language support for those.

Ambiguous and untestable specifications: the chal-lenges of writing and interpreting the large and ambigu-ous prose specifications, and the impossibility of di-rectly testing conformance between implementations anda prose specification.

In this paper we report on an experiment in developinga practical and usable TLS stack, nqsb-TLS, using a newapproach designed to address each of these root-causeproblems. This re-engineering, of the development pro-cess and of our concrete stack, aims to build in improvedsecurity from the ground up.

We demonstrate the practicality of the result in sev-eral ways: we show on-the-wire interoperability with ex-isting stacks; we show reasonable performance, in bothbulk transfer and handshakes; we use it in a test oracle,validating recorded packet traces which contain TLS ses-sions between other implementations; and we use it aspart of a standalone instant-messaging client. In additionto use in such traditional executables, nqsb-TLS is us-able in applications compiled into unikernels – type-safe,single-address-space VMs with TCBs that run directly

224 24th USENIX Security Symposium USENIX Association

on a hypervisor [32]. This integration into a uniker-nel stack lets us demonstrate a wide range of workingsystems, including HTTPS, IMAP, Git, and Websocketclients and servers, while sidestepping a further diffi-culty with radical solutions in this area: the large bodyof legacy code (in applications, operating systems, andlibraries) that existing TLS stacks are intertwined with.

We assess the security of nqsb-TLS also in severalways: for each of the root causes above, we discuss whyour approach rules out certain classes of associated flaws,with reference to an analysis of flaws found in previ-ous TLS implementations; and we test our authenticationlogic with a large corpus of certificate chains generatedby using the Frankencert fuzzer [8], which found flawsin several previous implementations. We have also madethe system publically available for penetration testing, asa Bitcoin Pinata, an example unikernel using nqsb-TLS.This has a TCB size roughly 4% of that of a similar sys-tem using OpenSSL on Linux.

We describe our overall approach in the remainder ofthe introduction. We then briefly describe the TLS pro-tocol (§2), analyse flaws previously found in TLS im-plementations (§3), and the result of applying our ap-proach, dubbed nqsb-TLS (§4). We demonstrate the du-ality of nqsb-TLS next by using its specification to vali-date recorded sessions (§5) and executing its implemen-tation to provide concrete services (§6). We evaluate theinteroperability, performance, and security (§7) of nqsb-TLS, describe related work (§8), and conclude (§9).

nqsb-TLS is freely available under a BSD license(https://nqsb.io), and the data used in this paper isopenly accessible [27].

1.1 Approach

A precise and testable specification for TLS In prin-ciple, a protocol specification should unambiguously de-fine the set of all implementation behaviour that it allows,and hence also what it does not allow: it should be pre-cise. This should not be confused with the question ofwhether a specification is loose or tight: a precise specifi-cation might well allow a wide range of implementationbehaviour. It is also highly desirable for specificationsto be executable as test oracles: given an implementa-tion behaviour (perhaps a trace captured from a particu-lar execution), the specification should let one computewhether it is in the allowed set or not.

In practice, the TLS specification is neither, but rathera series of RFCs written in prose [13, 14, 15]. An ex-plicit and precise description of the TLS state machineis lacking, as are some security-critical preconditions ofits transitions, and there are ambiguities in various semi-formal grammars. There is no way such prose documentscan be executed as a test oracle to directly test whether

Flow Entropy

ASN.1 Trust Anchors

Nocrypto

CSPRNGParse

Serialise TLS

Policy Config

X.509

Figure 1: nqsb-TLS is broken down into strongly sep-arated modules. The main part, in bold boxes, haspure value-passing interfaces and internals. The PRNGmaintains internal state, while Nocrypto includes C codebut has a pure effect-free interface. Arrows indicatedepends-on relationships.

implementation behaviour conforms to the specification.TLS is not unique in this, of course, and many otherspecifications are expressed in the same traditional prosestyle, but its disadvantages are especially serious for se-curity protocols.

For nqsb-TLS, we specify TLS as a collection of purefunctions over abstract datatypes. By avoiding I/O andshared mutable state, these functions can be consideredin isolation and each is deterministic, with errors re-turned as explicit values. The top-level function takes anabstract protocol state and an incoming message, and cal-culates the next state and any response messages. To doso, it invokes subsidiary functions to parse the message,drive the state machine, perform cryptographic opera-tions, and construct the response. This top-level functioncan be executed as a trace-checker, on traces both fromour implementation and from others, such as OpenSSL,to decide whether they are allowed by our specificationor not. In building our specification, to resolve the RFCambiguities, we read other implementations and testedinteroperability with them; we thereby capture the prac-tical de facto standard.

Reuse between specification and implementationThe same functions form the main part of our imple-mentation, coupled with code for I/O and to provide en-tropy. Note that this is not an “executable specification”in the conventional sense: our specification is necessar-ily somewhat loose, as the server must have the freedomto choose a protocol version and cipher suite, and thetrace checker must admit that, while our implementationmakes particular choices.

Each version of the implementation (Unix, unikernel)has a top-level Flow module that repeatedly performs I/Oand invokes the pure functional core; the trace-checkerhas a top-level module of the same type that reads in atrace to be checked offline.

2


Separation and modular structure This focus onpure functional descriptions also enables a decomposi-tion of the system (both implementation and specifica-tion) into strongly separated modules, with typed inter-faces, that interact only by exchanging pure values, asshown in Fig. 1. These modules and their interfaces arearranged to ensure that localised concerns such as binaryprotocol formats, ASN.1 grammars and certificate vali-dation are not spread throughout the stack, with no im-plicit dependencies via shared memory.

External resources are explicitly represented as mod-ules, instead of being implicitly accessed, and each sat-isfies a module type that describes collections of opera-tions over an abstract type, and that can be instantiatedwith any of several implementations. These include theNocrypto cryptography layer and our PRNG, which de-pends on an external Entropy module type.

Communication with the outside world is factored outinto an I/O component, Flow, that passes a byte sequenceto the pure core, then transmits responses and handlestimeouts, and is used by the top-level but not by the TLSengine itself. The pure TLS engine depends on someexternal data, such as the policy config and trust anchors.

Choice of language and style The structure we de-scribe above could be implemented in many differentprogramming languages, but guarantees of memory andtype safety are desirable to exclude many common secu-rity flaws (lack of memory safety was the largest singlesource of vulnerabilities in various TLS stacks through-out 2014, as shown in our §3 vulnerability analysis), andexpressive statically checked type and module systemshelp maintain the strongly separated structure that weoutlined. Our implementation of nqsb-TLS uses OCaml,a memory-safe, statically typed programming languagethat compiles to fast native code with a lightweight, em-beddable runtime. OCaml supports (but does not man-date) a pure programming style, and has a module sys-tem which supports large-scale abstraction via ML func-tors – modules that can depend on other modules’ types.In OCaml, we can encode complex state machines (§4),with lightweight invariants statically enforced by thetype checker (state machine problems were the secondlargest source of vulnerabilities). Merely using OCamldoes not guarantee all the properties we need, of course(one can write imperative and convoluted code in anylanguage); our specification and programming styles areequally important.

This is a significant departure from normal practice, inwhich systems software is typically written in C, but webelieve our evaluation shows that it is viable in at leastsome compelling scenarios (§7).

Non-goals For nqsb-TLS we are focussed on the engi-neering of TLS specifications and implementations, not

on the security protocol itself (as we recall in §3, somevulnerabilities have been found there). We are also notattempting to advance the state of the art in side-channeldefence, though we do follow current best practice. Weare focussed on making a stack that is usable in prac-tice and on security improvements achievable with betterengineering processes, rather than trying to prove that aspecification or implementation is correct or secure (see§8 for related work in that direction).

Current state The entire set of TLS RFCs [13, 14, 15]are implemented in nqsb-TLS, apart from minor rarelyused features, such as DSS certificates and anon andpre-shared keys ciphersuites. As we demonstrate in§7.1, nqsb-TLS can interoperate with many contempo-rary TLS implementations, but we are not attempting tosupport legacy options or those of doubtful utility. Weneither support SSLv3 [1], nor use RC4 in the defaultconfiguration [39]. The crypto wars are over: we havenot implemented ciphersuites to adhere to export restric-tions, which gave rise to the FREAK and Logjam attacks.

nqsb-TLS is strict (see §7.2), which results in roughly10% failing connections from legacy clients. But sinceour main goal is to provide security, we are not willing tomake compromises for insecure implementations. In ad-dition to TLS itself, we also implemented ASN.1, X.509and crypto primitives. From a practical point of view, thelargest missing part is elliptic curve cryptography.

2 TLS Background

TLS provides the twin features of authentication andconfidentiality. Clients typically verify the server’s iden-tity, the server can optionally verify the client’s identity,while the two endpoints establish an encrypted commu-nication channel. This channel should be immune fromeavesdropping, tampering and message forgery.

There have been three standardised versions of TLS,1.0, 1.1 and 1.2, while the last SSL (version 3) is still inwide usage. A key feature of TLS is algorithmic agility:it allows the two endpoints to negotiate the key exchangemethod, symmetric cipher and the message authentica-tion mode upon connecting. This triple is called a ciphersuite, and there are around 160 cipher suites standard-ised and widely supported. Together with a number ofstandardised extensions to the protocol that can be nego-tiated, this creates a large possible space of session pa-rameters. This large variation in configuration options isa marked characteristic of TLS, significantly contribut-ing to the complexity of its state machine.

Only a handful implementations of TLS are in wideuse. The three major free or open-source implementa-tions are OpenSSL, GnuTLS and Mozilla’s NSS. Mi-crosoft supplies SChannel with their operating systems,

3


while Apple supplies Secure Transport with theirs, andOracle Java runtime comes bundled with JSSE.

Structurally, TLS is a two-layered protocol. The outerlayer preserves message boundaries and provides fram-ing. It encapsulates one of five sub-protocols: hand-shake, change cipher spec, alert, application data orheartbeat. Both layers can contain fragmentation.

A TLS session is initiated by the client, which usesthe handshake protocol to signal the highest protocol ver-sion, possible extensions, and a set of ciphersuites it sup-ports. The server picks the highest protocol version itshares with the client and a mutually supported cipher-suite, or fails the handshake. The ciphersuite determineswhether the server authenticates itself, and depending onthe server configuration it requests the client to authenti-cate itself. After the security parameters for the authenti-cated encryption scheme are negotiated, the Change Ci-pher Spec message activates these, and the last hand-shake message authenticates the handshake. Either partycan renegotiate the session over the established channelby initiating another handshake.

The handshake sub-protocol contains a complex statemachine, which must be successfully traversed at leastonce. Handshake messages are independent of other sub-protocols, but some other sub-protocols are dependent ofa successful handshake. For instance, it is not possible toexchange application data before a session is established,and it is impossible to affect the use of negotiated sessionparameters while the negotiation is still in progress.

Server and client authentication is performed bymeans of X.509 certificates. Usually path validation isused: after one party presents a sequence of certificatescalled the certificate chain, the other party needs to ver-ify that a) each certificate in the chain is signed by thenext certificate; b) the last certificate is signed by oneof the trust anchors independent of connection; and c)that the first party owns the private key associated withthe first certificate in the chain by transferring a signedmessage containing session-specific data. For correct au-thentication, the authenticating party also needs to verifygeneral semantic well-formedness of the involved certifi-cates, and be able to deal with three version of X.509 anda number of extensions.

X.509 certificates are described through ASN.1, a no-tation for describing the abstract syntax of data, and en-coded using Distinguished Encoding Rules (DER), oneof the several standard encodings ASN.1 defines. A par-ticular description in the ASN.1 language coupled witha choice of encoding defines both the shape the the data-structures and their wire-level encoding. ASN.1 providesa rich language for describing structure, with a numberof primitive elements, like INTEGER and BIT STRING,and combining constructs, like SEQUENCE (a record ofsub-grammars) and CHOICE (a node joining alternative

grammars). The ASN.1 formalism can be used with acompiler that derives parsing and serialisation code forthe target language, but TLS implementations more typi-cally contain custom parsing code for dealing with X.509certificates. As X.509 exercises much of ASN.1, thisparsing layer is non-trivial and significantly adds to theimplementation complexity.

3 Vulnerability Analysis

In the past 13 months (January 2014 to January 2015),54 CVE security advisories have been published for 6widely used TLS implementations (see Table 1): 22 forOpenSSL, 6 for GnuTLS, 7 for NSS, 2 for SChannel, 2for Secure Transport, 5 for JSSE, and 10 related to errorsin their usage in the client software (excluding vulnera-bilities related to DTLS – TLS over UDP).

These vulnerabilities have a wide range of causes. Weclassify them into broad families below, identifying rootcauses for each and discussing how nqsb-TLS avoidsflaws of each kind.

General memory safety violations Most of these bugs,15 in total, are memory safety issues: out-of-boundsreads, out-of-bounds writes and NULL pointer derefer-ences. A large group has only been demonstrated tocrash the hosting process, ending in denial-of-service,but some lead to disclosure of sensitive information.

A now-notorious example of this class of bugs isHeartbleed in OpenSSL (CVE-2014-0160). Upon re-ceiving a heartbeat record, a TLS endpoint should re-spond by sending back the payload of the record. Therecord contains the payload and its length. In Heartbleed,the TLS implementation did not check if the length ofthe received heartbeat matched the length encoded in therecord, and responded by sending back as many bytesas were requested on the record level. This resulted inan out-of-bounds read, which lets a malicious client dis-cover parts of server’s memory. In April 2014, Cloud-flare posed a challenge of exploiting this bug to compro-mise the private RSA key, which has been accomplishedby at least four independent researchers.

nqsb-TLS avoids this class of issues entirely by thechoice of a programming language with automated mem-ory management and memory safety guarantees: inOCaml, array bounds are always checked and it is notpossible to access raw memory; and our pure functionalprogramming style rules out reuse of mutable buffers.

Certificate parsing TLS implementations need to parseASN.1, primarily for decoding X.509 certificates. WhileASN.1 is a large and fairly complex standard, for the pur-poses of TLS, it is sufficient to implement one of its en-codings (DER), and only some of the primitives. SomeTLS implementations contain an ad-hoc ASN.1 parser,

4


Product CVE ID Issue source

OpenSSL 2013-4353, 2015-0206, 2014-[3567, 3512, 3569, 3508, 3470, 0198, 0160] Memory management2015-0205, 2015-0204, 2014-3572, 2014-0224, 2014-3568, 2014-3511 State machine2014-8275 Certificate parsing2014-2234 Certificate validation2014-3509, 2010-5298 Shared mutable state2014-0076 Timing side-channel2014-3570 Wrong sqrt

GnuTLS 2014-8564, 2014-3465, 2014-3466 Memory management2014-1959, 2014-0092, 2009-5138 Certificate validation

NSS 2014-1544 Memory management2013-1740 State machine2014-1490 Shared mutable state2014-1569, 2014-1568 Certificate parsing2014-1492 Certificate validation2014-1491 DH param validation

SChannel 2014-6321 Memory managementSecure Transport 2014-1266 State machineJSSE 2014-6593, 2014-0626 State machine

2014-0625 Memory exhaustion2014-0411 Timing side-channel

Applications 2014-2734 Memory management2014-3694, 2014-0139, 2014-2522, 2014-8151, 2014-1263 Certificate validation2013-7373, 2014-0016, 2014-0017, 2013-7295 RNG seeding

Protocol-level 2014-1771, 2014-1295, 2014-6457 Triple handshake2014-3566 POODLE

Table 1: Vulnerabilities in TLS implementations in 2014.

combining the core ASN.1 parsing task with the defini-tions of ASN.1 grammars, and this code operates as apart of certificate validation.

Unsurprisingly, ASN.1 parsing is a recurrent source ofvulnerabilities in TLS and related software, dating backat least to 2004 (MS04-007, a remote code execution vul-nerability), and 3 vulnerabilities in 2014 (CVEs 2014-8275, 2014-1568 and 2014-1569). Two examples areCVE-2015-1182, the use of uninitialised memory duringparsing in PolarSSL, which could lead to remote codeexecution, and CVE-2014-1568, a case of insufficientlyselective parsing in NSS, which allowed the attacker toconstruct a fake signed certificate from a large space ofbyte sequences interpreted as the same certificate.

This class of errors is due to ambiguity in the specifi-cation, and ad-hoc parsers in most TLS implementations.nqsb-TLS avoids this class of issues entirely by separat-ing parsing from the grammar description (§4.4).

Certificate validation Closely related to ASN.1 pars-ing is certificate validation. X.509 certificates are nesteddata structures standardised in three versions and withvarious optional extensions, so validation involves pars-ing, traversing, and extracting information from complexcompound data. This opens up the potential for errorsboth in the control-flow logic of this task and in the in-terpretation of certificates (multiple GnuTLS vulnerabil-

ities are related to lax interpretation of the structures).In 2014, there were 5 issues related to certificate val-

idation. A prominent example in the control-flow logicis GnuTLS (CVE-2014-0092), where a misplaced goto

statement lead to certificate validation being skipped ifany intermediate certificate was of X.509 version 1.

Many implementations interleave the complicatedX.509 certificate validation with parsing the ASN.1grammar, leading to a complex control flow with sub-tle call chains. This illustrates another way in which thechoice of programming language and style can lead toerrors: the normal C idiom for error handling uses gotoand negative return values, while in nqsb-TLS we returnerrors explicitly as values and have to handle all possi-ble variants. OCaml’s typechecker and pattern-match ex-haustiveness checker ensures this at compile time (§4.3).

State machine errors TLS consists of several sub-protocols that are multiplexed at the record level: (i) thehandshake that initially establishes the secure connectionand subsequently renegotiates it; (ii) alerts that signalout-of-band conditions; (iii) cipher spec activation notifi-cations; (iv) heartbeats; and (v) application data. The ma-jority of the TLS protocol specification covers the hand-shake state machine. The path to a successful negotia-tion is determined during the handshake and depends onthe ciphersuite, protocol version, negotiated options, and

5


configuration, such as client authentication. Errors in thehandshake logic often lead to a security breach, allowingattackers to perform active man-in-the-middle (MITM)insertion, or to passively gain knowledge over the nego-tiated security parameters.

There were 10 vulnerabilities in this class. Some ledto denial-of-service conditions caused (for example) byNULL-pointer dereferences on receipt of an unexpectedmessage, while others lead to a breakdown of the TLSsecurity guarantees. An extensive study of problems inTLS state machine implementations has been done in theliterature [2, 11].

A prominent example is Apple’s “goto fail” (CVE-2014-1266), caused by a repetition of a goto state-ment targeting the cleanup block of the procedure re-sponsible for verifying the digital signature of theServerKeyExchange message. This caused the proce-dure to skip the subsequent logic and return the valueregistered in the output variable. As this variable wasinitialised to “success”, the signature was never verified.

Another typical example is the CCS Injection inOpenSSL (CVE-2014-0224). ChangeCipherSpec isthe message signalling that the just negotiated securityparameters are activated. In the TLS state machine, it islegal only as the penultimate message in the handshakesequence. However, both OpenSSL (CVE-2014-0224)and JSSE (CVE-2014-6593) allowed a CCS message be-fore the actual key exchange took place, which activatedpredictable initial security parameters. A MITM attackercan exploit this by sending a CCS during handshake,causing two parties to establish a deterministic sessionkey and defeating encryption.

Some of these errors are due to missing preconditionsof state machine transitions in the specification. In nqsb-TLS, our code structure (§4.1) makes the need to con-sider each of these clear. We encode the state machineexplicitly, while state transitions default to failure.

Protocol bugs In 2014, two separate issues in the pro-tocol itself were described: POODLE and triple hand-shakes. POODLE is an attack on SSL version 3, whichdoes not specify the value of padding bytes in CBCmode. Triple handshake [3] is a MITM attack where onenegotiates sessions with the same security parametersand resumes. We do not claim to prevent nor solve thoseprotocol bugs in nqsb-TLS, we mitigate triple handshakeby resuming sessions only if the extended master se-cret [4] was used. Furthermore, we focus on a modernsubset of the protocol, not including SSL version 3, soneither attack is applicable.

Timing side-channel leaks Two vulnerabilities were re-lated to timing side-channel leaks, where the observableduration of cryptographic operations depended on cryp-tographic secrets. These were implementation issues,

related to the use of variable-duration arithmetic oper-ations. The PKCS1.5 padding of the premaster secretis transmitted during an RSA key exchange. If the un-padding fails, there is computationally no need to de-crypt the received secret material. But omitting this stepleaks the information on whether the padding was cor-rect through the time signature, and this can be used toobtain the secret. A similar issue was discovered in 2014in various TLS implementations [34].

nqsb-TLS mitigates this attack by always computingthe RSA operation, on padding failure with a fake value.To mitigate timing side-channels, which a memory man-aged programming language might further expose, weuse C implementations of the low level primitives (§4.2).

Usage of the libraries Of the examined bugs, 10 werenot in TLS implementations themselves, but in the waythe client software used them. These included the high-profile anonymisation software Tor [16], the instant mes-senger Pidgin and the widely used multi-protocol datatransfer tool cURL.

TLS libraries typically have complicated APIs due toimplementing a protocol with a large parameter space.For example, OpenSSL 1.0.2 documents 243 symbols inits protocol alone, not counting the cryptographic parts ofthe API. Parts of its API are used by registering callbackswith the library that get invoked upon certain events. Awell-documented example of the difficulty in correctlyusing these APIs is the OpenSSL certificate validationcallback. The library does not implement the full logicthat is commonly needed (it omits name validation), sothe client needs to construct a function to perform cer-tificate validation using a mix of custom code and callsto OpenSSL, and supply it to the library. This step is acommon pitfall: a recent survey [23] showed that it iscommon for OpenSSL clients in the wild to do this in-correctly. We counted 6 individual advisories stemmingfrom improper usage of certificate validation API, whichis a large number given that improper certificate valida-tion undermines the authentication property of TLS andcompletely undermines its security.

The root cause of this error class is the large and com-plex legacy APIs of contemporary TLS stacks. nqsb-TLSdoes not mirror those APIs, but provides a minimal APIwith strict validation by default. This small API is suf-ficient for various applications we developed. OpenBSDuses a similar approach with their libtls API.

4 The nqsb-TLS stack

We now describe how we structure and develop the nqsb-TLS stack, following the approach outlined in the intro-duction to avoid a range of security pitfalls.

6


4.1 TLS Core

The heart of our TLS stack is the core protocol imple-mentation. By using pure, composable functions to ex-press the protocol handling, we deal with TLS as a data-transformation pipeline, independent of how the data isobtained or transmitted.

Accordingly, our core revolves around two functions.One (handle tls) takes the sequence of bytes seen onthe wire and a value denoting the previous state, andproduces, as new values, the bytes to reply with orto transfer to the application, and the subsequent state.Our state type encapsulates all the information abouta TLS session in progress, including the state of thehandshake, the cryptographic state for both directionsof communication, and the incomplete frames previ-ously received, as an immutable value. The other one(send application data) takes a sequence of bytesthat the application wishes to send and the previous state,and produces the sequence ready to be sent and the sub-sequent state. Coupled with a few operations to extractsession information from the state, these form the entireinterface to the core protocol implementation.

Below the entry points, we segment the records, de-crypt and authenticate them, and dispatch to the appro-priate protocol handler. One of the places where OCamlhelps most prominently is in handling of the combinedstate machine of handshake and its interdependent sub-protocols. We use algebraic data types to encode eachpossible handshake state as a distinct type variant, thatsymbolically denotes the state it represents and containsall of the data accumulated so far. The overall statetype is simply the discriminated union of these variants.Every operation that extracts information from state

needs to scrutinise its value through a form of multi-waybranching known as pattern match. This syntactic con-struct combines branching on the particular variant ofthe state present with extraction of components. Theresulting dispatch leads to equation-like code: branchesthat deal with distinct states follow directly from the val-ues representing them, process the state data locally, andremain fully independent in the sense of control flow andaccess to values they operate on. Finally, each separatelymaterialises the output and subsequent state.

This construction and the explicit encoding of state-machine is central to maintaining the state-machine in-variants and preserving the coherence of state represen-tation. It is impossible to enter a branch dedicated to aparticular transition without the pair of values represent-ing the appropriate state and appropriate input message,and, as intermediate data is directly obtained from thestate value, it is impossible to process it without at thesame time requiring that the state-machine is in the ap-propriate state. It is also impossible to manufacture a

state-representation ahead of time, as it needs to containall of the relevant data.

The benefit of this encoding is most clearly seen inCCS-injection-like vulnerabilities. They depend on ses-sion parameters being stored in locations visible through-out the handshake code, which are activated on receiptof the appropriate message. In the OpenSSL case (CVE-2014-0224), the dispatch code failed to verify whetherall of these locations were populated, which implies thatthe handshake progressed to the appropriate phase. Inour case, the only way to refer to the session parametersis to deconstruct a state-value containing them, and it isimpossible to create this value without having collectedthe appropriate session parameters.

All of core’s inner workings adhere to a predictable,restricted coding style. Information is always communi-cated through parameters and result values. Error prop-agation is achieved exclusively through results, withoutthe use of exceptions. We explicitly encode errors dis-tinct from successful results, instead of overloading theresult’s domain to mean error in some parts of its range.The type checker verifies both that each code path isdealing with exactly one possibility, and – through theexhaustiveness checker – that both forms have been ac-counted for. The repetitive logic of testing for error re-sults and deciding whether to propagate the error or pro-ceed is then abstracted away in a few higher-order func-tions and does not re-appear throughout the code.

This approach has also proven convenient when main-taining a growing code-base: when we had to add sig-nificant new capabilities, e.g. extending the TLS versionsupport to versions 1.1 and 1.2 or implementing clientauthentication, the scope of changes was localised andthe effects they had on other modules were flagged bythe type checker.

4.2 Nocrypto

TLS cryptography is provided by Nocrypto, a separatelibrary we developed for that purpose. It supports ba-sic modular public-key primitives like RSA, DSA andDH; the two most commonly used symmetric block ci-phers, AES and 3DES; the most important hash func-tions, MD5, SHA and the SHA2 family; and an imple-mentation of the cryptographically strong pseudorandomnumber generator, Fortuna [20].

One of the fundamental design decisions was to useblock-level symmetric encryption and hash cores writtenin C. For hashing, DES, and the portable version of AES,we use widely available public domain code. In addition,we wrote our own AES core using the Intel AES-NI in-structions.

There are two reasons for using C at this level. Firstly,symmetric encryption and hashing are the most CPU-

7


intensive operations in TLS. Therefore, performanceconcerns motivate the use of C. Secondly, the securityimpact of writing cryptography in a garbage-collectedenvironment is unclear. Performing computations oversecret material in this context is a potential attack vector.The garbage collector pauses might act as an amplifier toany existing timing side-channel leaks, revealing infor-mation about the allocation rate. We side-step this issueby leaving the secret material used by symmetric encryp-tion opaque to the OCaml runtime.

Such treatment creates a potential safety problem inturn: even if we manage to prevent certain classes of bugsin OCaml, they could occur in our C code. Our strategyto contain this is to restrict the scope of C code: we em-ploy simple control flow and never manage memory inthe C layer. C functions receive pre-allocated buffers,tracked by the runtime, and write their results there. Themost complex control flow in these are driving loops thatcall the compression function (in the case of hashes), orthe block transform (in the case of ciphers), over the con-tents of the input buffer. AES-NI instructions are partic-ularly simplifying in this respect, as the code consists ofa sequence of calls to compiler intrinsics.

Presently, only the AES-NI implementation of AESis protected from timing side-channel leaks, since thebulk of the cipher is implemented via constant-time ded-icated instructions. The generic code path is yet to beaugmented with code to pre-load substitution tables in anon-data-dependent manner.

More complex cryptographic constructions, like ci-pher modes (CBC, CTR, GCM and CCM) and HMACare implemented in OCaml on top of C-level primitives.We benefit from OCaml’s safety and expressive power inthese more complex parts of the code, but at the sametime preserve the property that secret material is not di-rectly exposed to the managed runtime.

Public key cryptography is treated differently. It is notblock-oriented and is not easily expressed in straight-line code, while the numeric operations it relies on areless amenable to C-level optimisation. At the same time,there are known techniques for mitigating timing leaksat the algorithmic level [28], unlike in the symmetriccase. We therefore implement these directly in OCamlusing GMP as our bignum backend and employ the stan-dard blinding countermeasures to compensate for poten-tial sources of timing side-channels.

Our Fortuna CSPRNG uses AES-CTR with a self-rekeying regime and a system of entropy accumulators.Instead of entropy estimation, it employs exponentiallagging of accumulators, a scheme that has been shownto asymptotically optimally recover from state compro-mise under a constant input of entropy of unknown qual-ity [17]. To retain purity of the system and facilitate de-terministic runs, entropy itself is required from the sys-

tem as an external service, as shown later in §6.For the sake of reducing complexity in the upper lay-

ers, the API of Nocrypto is concise and retains the ap-plicative style, mapping inputs to outputs. We did maketwo concessions to further simplify it: first, we useOCaml exceptions to signal programming errors of ap-plying cyptographic operations to malformed input (suchas buffers which are not a multiple of the block size inCBC mode, or the use of RSA keys unsuitably small fora message). Secondly, we employ a global and changingRNG state, because operations involving it are pervasivethroughout interactions with the library and the style ofexplicit passing would complicate the dependent code.

4.3 X.509

X.509 certificates are rich tree-like data structures whosesemantics changes with the presence of several optionalextensions. Although the core of the path-validation pro-cess is checking of the signature, a cryptographic oper-ation, the correct validation required by the standard in-cludes extensive checking of the entire data structure.

For example, each extension must be present at mostonce, the key usage extension can further constrainwhich exact operations a certificate is authorised for, anda certificate can specify the maximal chain length whichis allowed to follow. There are several ways in which acertificate can express its own identity and the identity ofits signing certificate. After parsing, a correct validationprocedure must take all these possibilities into account.

The ground encoding of certificates again benefitsfrom algebraic data types, as the control flow of func-tions that navigate this structure is directed by the type-checker. On a level above, we separate the validationprocess into a series of functions computing individualpredicates, such as the certificate being self-signed, itsvalidity period matching the provided time or confor-mance of the present extensions to the certificate ver-sion. The conjunction of these is clearly grouped intosingle top-level functions validating certificates in differ-ent roles, which describe the high-level constraints weimpose upon the certificates. The entire validation logicamounts to 314 lines of easily reviewable code.

This is in contrast to 7 000 lines of text in the RFC [9],which go into detail to explain extensions – such as poli-cies and name constraints – that are rarely seen in thewild. For the typical HTTPS setting, the RFC fails toclarify how to search for a trust anchor, and assumes in-stead the presence of exactly one. Due to cross sign-ing there can be multiple chains with different propertieswhich are not covered by the RFC.

nqsb-TLS initially strictly followed the RFC, but wasnot able to validate many HTTPS endpoints on the In-ternet. It currently follows the RFC augmented with

8


Mozilla’s guidelines and provides a self-contained con-densation of these which can be used to clarify, or evensupplant, the specification. We created an extensive testsuite with full code coverage, the code has been evalu-ated (see §7.2) with the Frankencert tool, and it success-fully parses most of ZMap’s certificate repositories. Inaddition, we also support signing and serialising to PEM.

The interface to this logic is deterministic (it is madeso by requiring the current time as an input). Our X.509library provides operations to construct a full authenti-cator, by combining the validation logic with the currenttime at the moment of construction, which the TLS corecan be parametrised with. We do not leave validation tothe user of the library, unlike other TLS libraries [23].Instead, we have full implementations of path validationwith name checking [42] and fingerprint-based valida-tion, and we use the type system to force the user to in-stantiate one of them and provide it to the TLS layer.

4.4 ASN.1

ASN.1 parsing creates a tension in TLS implementa-tions: TLS critically relies on ASN.1, but it requires onlya subset of DER encoding, and, since certificates are usu-ally pre-generated, needs very little in the way of writing.For the purposes of TLS, it is therefore sufficient to im-plement just a partial parser.

When implementing ASN.1, a decision has to be madeon how to encode the actual abstract grammar that willdrive the parsing process, given by various TLS andX.509-related standards. OpenSSL, PolarSSL, JSSE andothers, with the notable exception of GnuTLS, do notmake any attempts to separate the grammar definitionfrom the parsing process. The leaf rules of ASN.1 areimplemented as subroutines, which are exercised in theorder required by the grammar in every routine that actsas parser. In other words, they implement the parsers asad-hoc procedures that interleave the code that performsthe actual parsing with the encoding of the grammar to beparsed. Therefore the code that describes the high-levelstructure of data also contains details of invocation oflow-lever parsers and, in the case of C, memory manage-ment. Unsurprisingly, ASN.1 parsers provide a steadystream of exploits in popular TLS implementations.

We retain the full separation of the abstract syntax rep-resentation from the parsing code, avoiding the complex-ity of the code that fuses the two. At the same time, weavoid parser generators which output source code that ishard to understand.

Instead, we created a library for declaratively describ-ing ASN.1 grammars in OCaml, using a functional tech-nique known as combinatory parsing [21]. It exposesan opaque data type that describes ASN.1 grammar in-stances and provides a set of constants (corresponding

to terminals) and functions over them (corresponding toproductions). Nested applications of these functions tocreate data that describes ASN.1 grammars follow theshape of the actual ASN.1 grammar definitions. Inter-nally, this tree-like type is traversed at initialisation-timeto construct the parsing and serialisation functions.

This approach allows us to create “grammar ex-pressions” which encode ASN.1 grammars, and deriveparsers and serialisers. As the ASN.1-language we cre-ate is a fragment of OCaml, we retain all the bene-fits of its static type checking. Types of functions overgrammar representations correspond to restrictions in theproduction rules, so type-checking grammar expressionsamounts to checking their well-formedness without writ-ing a separate parser for the grammar formalism. More-over, type inference automatically derives the OCamlrepresentation of the types defined by ASN.1 grammars.

Such an approach also makes testing much easier. Thegrammar type is traversed to generate random inhabi-tants of the particular grammar, which can be serialisedand parsed back to check that the two directions matchin their interpretation of the underlying ASN.1 encodingand to exercise all of the code paths in both.

A derived parsing function does not interpret thegrammar data, but as its connections to component pars-ing functions are known only when synthesis takes placeat run-time, we do not retain the benefit of inliningand inter-function optimisation a truly compiled parserwould have. Nonetheless, given that it parses roughly50 000 certificates per second, this approach does notcreate a major performance bottleneck. The result isa significant reduction in code complexity: the ASN.1parsing logic amounts to 620 lines of OCaml, and theASN.1 grammar code for X.509 certificates and signa-tures is around 1 000 lines. For comparison, PolarSSL1.3.7 needs around 7 500 lines to parse ASN.1, whileOpenSSL 1.0.1h has around 25 000 in its ASN.1 parser.

5 Using nqsb-TLS as a test oracle

One use of nqsb-TLS is as an executable test oracle,an application which reads a recorded TLS session traceand checks whether it (together with some configurationinformation) adheres to the specification that nqsb-TLSembodies. This recorded session can be a packet capture(using tcpdump) of a TLS session between various im-plementations (e.g. OpenSSL, PolarSSL, nqsb-TLS), or,for basic testing, a trace generated by nqsb-TLS itself.

To do this we must deal with the looseness of the TLSspecification: a TLS client chooses its random nonce, setof ciphersuites, protocol version, and handshake exten-sions, while a TLS server picks its random nonce, theprotocol version, the ciphersuite, possibly the DH group,and possibly extensions. Our test oracle does not make

9


ASN.1 Nocrypto

TLS

X.509

Trace Checker

TraceChecker

OpenSSLPacket Traces

pcap

nqsb-TLSConfig Config CertsCerts

Predictable RNG

Figure 2: nqsb-TLS acts as a trace checker: the RNGis predictable, configuration and certificates are inputs,driven by packet traces from OpenSSL (or other stacks).

those decisions, but rather takes the parameters recordedin the given session. To make this possible, given the on-the-wire encryption, some configuration information hasto be provided to the trace checker, including private keymaterial. In addition, both records and sub-protocols canbe fragmented; our test oracle normalises the records tonot contain any fragmentation for comparison.

Figure 2 shows how nqsb-TLS can be used to buildsuch a test oracle (note that it does not instantiate theentropy source for this usage). The test oracle producesits initial protocol state from the given session. It cal-culates handle tls with its state and the record inputof the given session, together with the particular selec-tion of protocol version, etc., resulting in an output state,potentially an output record, and potentially decryptedapplication data. It then compares equality of the out-put record and the given session. If successful, it usesthe output state and next recorded input of the given ses-sion to evaluate handle tls again, and repeats to theend of the trace. It thus terminates either when the entiretrace has been accepted, which means success; or with adiscrepancy between the nqsb-TLS specification and therecorded session, which means failure and needs furtherinvestigation. Such a discrepancy might indicate an errorin the TLS stack being tested, an error in the nqsb-TLSspecification, or an ambiguity in what TLS actually is.

A first test of this infrastructure was to use a recordedsession of the change cipher spec injection (CVE-2014-0224): our test oracle correctly denied this session, iden-tifying an unexpected message. We ran our test ora-cle and validated our 30 000 interoperability traces (see§7.1) and our Pinata traces (see §7.2), and also validatedrecorded TLS sessions between various implementations(OpenSSL, PolarSSL, nqsb-TLS) using tcpdump.

While running the test oracle we discovered interest-ingly varied choices in fragmentation of messages amongexisting stacks, which may be useful in fingerprinting.

TCP/IP

Xentropyd

ASN.1

TLSFlow

TLS

Logger

X.509

NetFront

NetBackXenconsoled

domU

dom0

Certs

Config

CrunchFS

Protocols

HTTP

Conduit

IMAPXMPP

CSPRNGNocrypto

Figure 3: nqsb-TLS as a unikernel domU VM on Xen: adom0 Xentropyd proxies host entropy, config and certifi-cates are compiled in, various protocols run over TLS.

The test oracle opens up the prospect of extensive test-ing of the behaviour of different TLS implementations,especially if combined with automated test generation.

6 Using nqsb-TLS in applications

Another use of nqsb-TLS is as a TLS implementation inapplications. We ported nqsb-TLS to two distinct envi-ronments and developed a series of applications, somefor demonstration purposes, others for regular use.

6.1 Porting nqsb-TLSTo use nqsb-TLS as an executable implementation, wehave to provide it with implementations of entropy andflow (see Figure 1), and an effectful piece of code thatcommunicates via the network and drives the core.

We pay special attention to prevent common clientbugs which arise from complexity of configuring TLSstacks and correspondingly large APIs. In each instance,there is only one function to construct a TLS config-uration which can be turned into an I/O interface, thefunction does extensive validation of the requested pa-rameters, and the resulting configuration object is im-mutable. This restricts potentially error-prone interac-tions that configure TLS to a single API point.

Unix Porting nqsb-TLS to Unix was straightforward;we use the POSIX sockets API to build a flow and/dev/urandom as the entropy source. The exposedinterface provides convenience functions to read cer-tificates and private keys from files, and analogues oflisten, connect, accept, read, write, and close

for communication.

MirageOS The MirageOS variant allows nqsb-TLS tobe compiled into a unikernel VM (see Figure 3). It usesthe MirageOS OCaml TCP/IP library [31] to provide the

10


I/O flow, which is in turn coupled to Xen device driversthat communicate with the backend physical network de-vice via shared memory rings [44]. The logger outputsdirectly to the VM console, and the certificates and thesecret keys are compiled into OCaml data structures atbuild time and become part of the VM image. A keychallenge when running in a virtualised environment isproviding a suitable entropy source [18], especially in thecommon case of a VM having no access to physical hard-ware. Since specialised unikernels have very determin-istic boot-sequences that make sources of entropy evenscarcer, we had to extend MirageOS and Xen to avoidcryptographic weaknesses [25].

One way in which we solve this is by relying on dom0to provide cross-domain entropy injection. We devel-oped Xentropyd, a dom0 daemon which reads bytes from/dev/urandom and makes them available to VMs viaan inter-VM shared memory channel. The entropy de-vice is plugged in as a standard Xen device driver viaXenstore [22], and MirageOS has a frontend library thatperiodically injects entropy into the nqsb-TLS CSPRNG.

To avoid being fully reliant on dom0, we implementadditional entropy harvesting within the unikernel itself.We do this by trapping the MirageOS event loop and us-ing RDTSCP instruction to read the Time Stamp Counter(TSC) register on each external event. This provides uswith the unpredictability inherent in the ambient events.This source is augmented with readings from the CPURNG where available: we feed the results of RDSEED (orRDRAND) instruction into the entropy pool on each event.

To make the RNG more resilient, we do extra entropyharvesting at boot time. Following Whirlwind RNG [18],we employ a timing loop early in the boot phase, de-signed to take advantage of nondeterminism inherent inthe CPU by way of internal races in the CPU state. Thisprovides an initial entropy boost in the absence of Xen-tropyd and helps mitigate resumption-based attacks [18].

In an ideal scenario the entropy would be providedthrough both mechanisms, but we expect the usage torely on one or the other, depending on deployment: onan ARM board lacking high-resolution timing and CPURNG, the user is likely to have control over the hypervi-sor and be able to install Xentropyd. Conversely, in com-mercial hosting scenarios where the assistance of dom0might not be available but the extra CPU features are, weexpect the user to rely on the internal entropy harvesting.

6.2 Applications

An example application using the Unix interface is theterminal-based instant messaging client jackline usingXMPP. The XMPP protocol negotiates features, such asTLS, over a plaintext TCP connection. Jackline performsan upgrade to TLS via the STARTTLS mechanism before

authentication credentials are exchanged. The Unix portof nqsb-TLS contains an API that supports upgrading analready established TCP connection to TLS. Jackline canuse either of the authentication APIs (path and fingerprintvalidation) depending on user configuration.

tlstunnel also runs on Unix and accepts a TLS connec-tion, forwards the application data to another service viaTCP, similar to stud and stunnel. This has been deployedsince months on some websites.

The Unix application certify generates RSA privatekeys, self-signed certificates, and certificate signing re-quests in PEM format. It uses nocrypto and X.509.

The OCaml Conduit library (also illustrated in Fig. 3)supports communication transports that include TCP,inter-VM shared memory rings. It provides a high-levelAPI that maps URIs into specific transport mechanisms.We added nqsb-TLS support to Conduit so that any appli-cation that links to it can choose between the use of nqsb-TLS or OpenSSL, depending on an environment vari-able. As of February 2015, 42 different libraries (bothclient and server) use Conduit and its provided API andcan thus indirectly use nqsb-TLS for secure connections.The OPAM package manager uses nqsb-TLS as part ofits mirror infrastructure to fetch 2 500 distribution files,with no HTTPS-related regressions encountered.

7 Evaluation

We now assess the interoperability, security, and perfor-mance of nqsb-TLS.

7.1 Interoperability

We assess the interoperability of nqsb-TLS in severalways: testing against OpenSSL and PolarSSL on everycommit; successfully connecting to most of the Fortune500 web sites; testing X.509 certificates from ZMap; andby running a web server.

This web server, running since mid 2014, displays thelive sequence diagram of a successful TLS session es-tablished via HTTPS. A user can press a button on thewebsite which let the server initiate a renegotiation. Theserver configuration includes all three TLS protocol ver-sions and eight different ciphersuites, picking a proto-col version and ciphersuite at random. Roughly 30 000traces were recorded from roughly 350 different clientstacks (6230 unique user agent identifiers).

Of these, around 27% resulted in a connection estab-lishment failure. Our implementation is strict, and doesnot allow e.g. duplicated advertised ciphersuites. Also,several accesses came from automated tools which eval-uate the quality of a TLS server by trying each definedciphersuite separately.

11


Roughly 50% of the failed connections did not sharea ciphersuite with nqsb-TLS. Another 20% started withbytes which were not interpretable by nqsb-TLS. 12% ofthe failed connections did not contain the secure renego-tiation extension, which our server requires. 5% of thefailed traces were attempts to send an early change ci-pher spec. Another 4% tried to negotiate SSL version 3.2.5% contained a ciphersuite with null (iOS6).

We parse more than 99% of ZMap’s HTTPS(20150615) and IMAP (20150604) certificate repository.The remaining failures are RSASSA-PSS signatures, re-quiring an explicit NULL as parameter, and unknownand outdated algorithm identifiers.

This four-fold evaluation shows that our TLS imple-mentation is broadly interoperable with a large numberof other TLS implementations, which also indicates thatwe are capturing the de facto standard reasonably well.

Specification mismatches While evaluating nqsb-TLS we discovered several inconsistencies between theRFC and other TLS implementations:

• Apple’s SecureTransport and Microsoft’s SChanneldeny application data records while a renegotiationis in process, while the RFC allows interleaving.

• OpenSSL (1.0.1i) accepts any X.509v3 certificatewhich contains either digitalSignature or keyEnci-pherment in keyUsage. RFC [15] mandates digi-talSignature for DHE, keyEncipherment for RSA.

• Some unknown TLS implementation starts thepadding data [29] (must be 0) with 16 bit length.

• A TLS 1.1 stack sends the unregistered alert 0x80.

7.2 SecurityWe assess the security of nqsb-TLS in several ways: thediscussion of the root causes of many classic vulnerabil-ities and how we avoid them; mitigation of other specificissues; our state machine was tested [11]; random testingwith the Frankencert [8] fuzzing tool; a public integratedsystem protecting a bitcoin reward; and analysis of theTCB size of that compared with a similar system builtusing a conventional stack.

Avoidance of classic vulnerability root causes InSections 3 and 4 we described how the nqsb-TLS struc-ture and development process exclude the root causes ofmany vulnerabilities that have been found in previousTLS implementations.

Additional mitigations The TLS RFC [15] includesa section on implementation pitfalls, which contains alist of known protocol issues and common failures whenimplementing cryptographic operations. nqsb-TLS miti-gates all of these.

Further issues that nqsb-TLS addresses include:

• Interleaving of sub-protocols, except betweenchange of cipher spec and finished.

• Each TLS 1.0 application data is prepended by anempty fragment to randomise the IV (BEAST).

• Secure renegotiation [40] is required.• SCSV extension [35] is supported.• Best practices against attacks arising from mac-

then-encrypt in CBC mode are followed (no miti-gation of Lucky13 [19])

• No support for export restricted ciphersuites, thusno downgrade to weak RSA keys and small DHgroups (FREAK and Logjam).

• Requiring extended master secret [4] to resume asession.

State machine fuzzing Researchers fuzzed [11] nqsb-TLS and found a minor issue: alerts we send are not en-crypted. This issue was fixed within a day after discov-ery, and it is unlikely that it was security-relevant.

Frankencert Frankencert is a fuzzing tool which gen-erates syntactically valid X.509 certificate chains by ran-domly mixing valid certificates and random data. Wegenerated 10 000 X.509 certificate chains, and comparedthe verification result of OpenSSL (1.0.1i) and nqsb-TLSThe result is that nqsb-TLS accepted 120 certificates, astrict subset of the 192 OpenSSL accepted.

Of these 72 accepted by OpenSSL but not by nqsb-TLS, 57 certificate chains contain arbitrary data inX.509v3 extensions where our implementation allowsonly restricted values. An example is the key usage ex-tension, which specifies a sequence of OIDs. In the RFC,9 different OIDs are defined. Our X.509v3 grammar re-stricts the value of the key usage extension to those 9OIDs. 12 certificate chains included an X.509v3 exten-sion marked critical but not supported by nqsb-TLS.

Two server certificates are certificate authority certifi-cates. While not required by the path validation, bestpractices from Mozilla recommend to not accept a servercertificate which can act as certificate authority. The lastcertificate is valid for a Diffie-Hellman key exchange,but not for RSA. Our experimental setup used RSA, thusnqsb-TLS denied the certificate appropriately.

Exposure to new vulnerabilities Building nqsb-TLSin a managed language potentially opens us up to vul-nerabilities that would not affect stacks written in C.Algorithmic complexity attacks are a low-bandwidthclass of denial-of-service attacks that exploit deficien-cies in many common default data structure implementa-tions [10]. The modular structure of nqsb-TLS makes iteasy to audit the implementations used within each com-ponent. The French computer security governmental of-fice [37] assessed the security of the OCaml runtime in

12


2013, which lead to several changes (such as distinctionbetween immutable strings and mutable byte arrays).

The Bitcoin Pinata To demonstrate the use of nqsb-TLS in an integrated system based on MirageOS, andto encourage external code-review and penetration test-ing, we set up a public bounty, the Bitcoin Pinata. Thisis a standalone MirageOS unikernel containing the se-cret key to a bitcoin address, which it transmits upon es-tablishing a successfully authenticated TLS connection.The service exposes both TLS client and server on differ-ent ports, and it is possible to bridge the traffic betweenthe two and observe a successful handshake and the en-crypted exchange of the secret.

The attack surface encompasses the entire system,from the the underlying operating system and its TCP/IPstack, to TLS and the cryptographic level. The systemwill only accept connections authenticated by the cus-tom certificate authority that we set up for this purpose.Reward is public and automated, because if an attackermanages to access the private bitcoin key, they can trans-fer the bitcoins to an address of their choosing, which isattestable through the blockchain.

While this setup cannot prove the absence of securityissues in our stack, it motivated several people to readthrough our code and experiment with the service.

At the end of June 2015, there were 230 000 accessesto the website from more than 50 000 unique IP ad-dresses. More than 9 600 failed and 12 000 successfulTLS connections from 1000 unique IPs were present. Al-though we cannot directly verify that all successful con-nection resulted from the service being short-circuitedto connect to itself, there have been no outgoing trans-actions registered in the blockchain. The breakdown offailed connections is similar to §7.1. We collected 42 cer-tificates which were tried for authentication, but failed(not well formatted, not signed, not signed by our trustanchor, private key not present). A detailed analysis ofthe captured traces showed that most of the flaws in otherstacks have been attempted against the Pinata.

Trusted computing base The TCB size is a roughquantification of the attack surface of a system. We as-sess the TCB of our Pinata, compared to a similar tradi-tional system using Linux and OpenSSL. Both systemsare executed on the same hardware and the Xen hyper-visor, which we do not consider here. The TCB sizes ofthe two systems are shown in Table 2 (using cloc).

The traditional system contains the Linux kernel (ex-cluding device drivers and assembly code), glibc, andOpenSSL. In comparison, our Pinata uses a minimal op-erating system, the OCaml runtime, and several OCamllibraries (including GMP). While the traditional sys-tem uses a C compiler, our Pinata additionally uses theOCaml compiler (roughly 40 000 lines of code).

Linux/OpenSSL Unikernel/nqsb-TLSKernel 1600 48 (36)Runtime 689 25 (6)Crypto 230 23 (14)TLS 41 6 (0)Total 2560 102 (56)

Table 2: TCB (in kloc); portion of C code in parens

nqsb-TLS OpenSSL PolarSSLRSA 698 hs/s 723 hs/s 672 hs/s

DHE-RSA 601 hs/s 515 hs/s 367 hs/s

Table 3: Handshake performance of nqsb-TLS,OpenSSL and PolarSSL, using 1024-bit RSA certificateand 1024-bit DH group.

The trusted computing base of the traditional systemis 25 times larger than ours. Both systems provide thesame service to the outside world and are hardly distin-guishable for an external observer.

7.3 PerformanceWe evaluate the performance of nqsb-TLS, comparing itto OpenSSL 1.0.2c and PolarSSL 1.3.11. We use a singlemachine to avoid network effects. In the case of nqsb-TLS, we compile the test application as a Unix binary tolimit the comparison to TLS itself.

The test machine has an Intel i7-5600 Broadwell CPUand runs Linux 4.0.5 and glibc 2.21. Throughput is mea-sured by connecting the command line tool socat, linkedagainst OpenSSL, to a server running the tested imple-mentation, and transferring 100 MB of data from theclient to the server. This test is repeated for varioustransmission block sizes. Handshakes are measured byrunning 20 parallel processes in a continuous connectingloop and measuring the maximum number of successfulconnection within 1 second; the purpose of parallelism isto negate the network latency.

Throughput rates are summarized in Figure 4. With16 byte blocks, processing is dominated by the protocoloverhead. This helps us gauge the performance impactof using OCaml relative to C, as nqsb-TLS implementsprotocol logic entirely in OCaml. At this size, we run atabout 78% of OpenSSL’s speed.

At 8196 bytes, performance becomes almost entirelydominated by cryptographic processing. All three im-plementations use AES-NI, giving them roughly compa-rable speed. OpenSSL’s performance lead is likely dueto its extensive use of assembly, and in particular thetechnique of stitching, combining parts the of the ciphermode of operation and hashing function to saturate theCPU pipeline. PolarSSL’s performance drop compared

13


Figure 4: Scaling of throughput with applicationdata size for nqsb-TLS, OpenSSL and PolarSSL, usingAES 256 CBC SHA.

to nqsb-TLS is likely a consequence of our usage of sim-ple software pipelining in the AES-NI code. nqsb-TLSreaches about 84% of OpenSSL’s speed in this scenario.

Handshake performance, summarized in Table 3, isroughly similar. We attribute OpenSSL’s advantage totheir use of C in the protocol handling, and PolarSSL’sdisadvantage to our use of faster bignum routines pro-vided by GMP. The comparatively smaller cost nqsb-TLS pays for DH is a result of picking shorter exponents,matched to the security strength of the group modulus.

We ran miTLS 0.8.1 through Mono 4.0.1.44 on thesame test machine. Using the bundled HTTP server, weachieve a peak throughput of 19 MB/s for a file transferusing the same cipher suite. As the Mono cryptographyprovider only contains C# AES implementations, we ex-clude this implementation from further analysis. We donote, however, that the throughput ratio between miTLSand OpenSSL is similar to the one its authors report [5].

The exact numbers are likely to vary with the choice ofcipher suite, which places different weights on hashingand cipher performance, the CPU generation, which isutilised to a fuller extent by OpenSSL, and the testingscenario. The broad picture is that our usage of OCamlfor all but the lowest-level cryptographic primitives is, initself, not taking a prohibitive toll on performance.

8 Related Work

Security proofs Several research groups [36, 26, 24,12, 38] have modelled and formally verified securityproperties of TLS. Because TLS is a complex protocol,most models use a simplified core, and formalising eventhese subsets is challenging work which is not very ac-cessible to an implementer audience. Additionally, these

models need to be validated with actual implementations,to relate to the de facto standard, but this is rarely done(to do so, some kind of trace checker or executable needsto be developed). Some of these models formalised thehandshake protocol, but omitted renegotiation, in whicha security flaw was present until discovered in 2009.

miTLS The miTLS [5] stack is developed in F7 withthe possibility to extract executable F# code. It is botha formalisation of the TLS protocol and a runnable im-plementation. This formalisation allowed its developersto discover two protocol-level issues: alert fragmentationand triple handshake. As an implementation, it dependson the Common Language Runtime for execution, anduses its services for cryptographic operations and X.509treatment (including ASN.1 parsing). In contrast, nqsb-TLS cannot be used for verifying security properties ofTLS, but provides a test oracle and a fast runnable imple-mentation which is easily deployable. It compiles to na-tive code and implements the entire stack from scratch,making it self-contained. It can be used e.g. in Mira-geOS, which only provides the bare TCP/IP interfacesand has no POSIX layer or cryptographic services.

Language-oriented approach to security Mettler etal. propose Joe-E [33], a subset of Java designed to sup-port the development of secure software systems. Theystrongly argue in favour of a particular programmingstyle to facilitate ease of security reviews.

Our approach shares some of the ideas in Joe-E. Bydisallowing mutable static fields of Java classes, they ef-fectively prohibit globally visible mutable state and en-force explicit propagation of references to objects, whichserve as object capabilities. They also emphasise im-mutability to restrict the flows of data and achieve bettermodularity and separation of concerns.

The difference in our approach is that we use im-mutability and explicit data-passing not only on the mod-ule (or class) boundaries but pervasively throughout thecode, aiming to facilitate code-review and reasoning onall levels. A further difference is that Joe-E focusses theproposed changes in style on security reviews only, aim-ing to help the reader of the code ascertain that the codedoes not contain unforeseen interactions and faithfullyimplements the desired logic. In contrast, we employ afully declarative style. Our goals go beyond code review,as large portions of our implementation are accessible asa clarification to the specification, and we have an exe-cutable test oracle.

Finally, there is the difference between host languages.Java lacks some of the features we found to be most sig-nificant in simplifying the implementation, chiefly theability to encode deeply nested data structures and tra-verse them via pattern-matching, and to express local op-erations in a pure fashion.

14


Brittle implementations of cryptography systemsSchneier et al.’s work [43] discovered several root causesfor software implementing cryptographic systems, whichexplicitly mentions incorrect error handling and flawedAPI usage. We agree with their principles for softwareengineering for cryptography, and extend this further byproposing our approach: immutable data, value-passinginterfaces, explicit error handling, small API footprint.

TLS implementations in high-level languages Sev-eral high-level languages contain their own TLS stack.Oracle Java ships with JSEE, a memory-safe implemen-tation. However its overall structure closely resemblesthe C implementations. For example, the state machineis built around accumulating state by mutations of sharedmemory locations, the parsing and validation of certifi-cates are not clearly separated, and the certificate vali-dation logic includes non-trivial control flow. This re-sulted in high-level vulnerabilities similar in nature to theones found in C implementations, such as CCS Injection(CVE-2014-0626), and its unmanaged exception systemled to several further vulnerabilities [34].

There are at least two more TLS implementations infunctional languages, one in Isabelle [30] and one inHaskell. Interestingly, both implementations experimentwith their respective languages’ expressivity to give theimplementations an essentially imperative formulation.The Isabelle development uses a coroutine-like monad todirectly interleave I/O operations with the TLS process-ing, while the Haskell development uses a monad stackto both interleave I/O and to implicitly propagate the ses-sion state through the code. In this way both implementa-tions lose the clear description of data-dependencies andstrong separation of layers nqsb-TLS has.

Protocol specification and testing There is an ex-tensive literature on protocol specification and testing ingeneral (not tied to a security context). We build in par-ticular on ideas from Bishop et al.’s work on TCP [6, 41],in which they developed a precise specification for TCPand the Sockets API in a form that could be used as atrace-checker, characterising the de facto standard. TCPhas a great deal of internal nondeterminism, and soBishop et al. resorted to a general-purpose higher-orderlogic for their specification and symbolic evaluation overthat for their trace-checker. In contrast, the internal non-determinism needed for TLS can be bounded as we de-scribe in §5, and so we have been able to use simple purefunctional programming, and to arrange the specificationso that it is simultaneously usable as an implementation.We differ also in focussing on an on-the-wire specifi-cation rather than the endpoint-behaviour or end-to-endAPI behaviour specifications of that work. In contrast tothe Sockets API specified in POSIX, there is no API forTLS. Every implementation defines its custom API, and

many have a compatibility layer for the OpenSSL API.

9 Conclusion

We have described an experiment in engineering criticalsecurity-protocol software using what may be perceivedas a radical approach. We focus throughout on struc-turing the system into modules and pure functions thatcan each be understood in isolation, serving dual rolesas test-oracle specification and as implementation, ratherthan traditional prose specifications and code driven en-tirely by implementation concerns.

Our evaluation suggests that it is a successful exper-iment: nqsb-TLS is usable in multiple contexts, as testoracle and in Unix and unikernel applications, it has rea-sonable performance, and it is a very concise body ofcode. Our security assessment suggests that, while it isby no means guaranteed secure, it does not suffer fromseveral classes of flaws that have been important in pre-vious TLS implementations. In this sense, it is at leastnot quite so broken as some secure software has been.

In turn, this indicates that our approach has value.As further evidence of that, we applied the same ap-proach to the off-the-record [7] security protocol, usedfor end-to-end encryption in instant messaging proto-cols. We engineered a usable implementation and re-ported several inconsistencies in the prose specification.The XMPP client mentioned earlier uses nqsb-TLS fortransport layer encryption, and our OTR implementationfor end-to-end encryption.

The approach cannot be applied everywhere. The twoobvious limitations are (1) that we rely on a languageruntime to remove the need for manual memory manage-ment, and (2) that our specification and implementationstyle, while precise and concise, is relatively unusual inthe wider engineering community. But the benefits sug-gest that, where it can be applied, it will be well worthdoing so.Acknowledgements Parts of this work were sup-ported by EPSRC Programme Grant EP/K008528/1(REMS: Rigorous Engineering for Mainstream Systems)and by the European Unions Seventh Framework Pro-gramme FP7/20072013 under the User Centric Network-ing project (no. 611001). We also thank IPredator(https://ipredator.se) for lending bitcoins for ourPinata and hosting it, and the MirageOS team and theanonymous reviewers for their valuable feedback.

References[1] BARNES, R., THOMSON, M., PIRONTI, A., AND LANGLEY, A.

Deprecating secure sockets layer version 3.0. RFC 7568, 2015.

[2] BEURDOUCHE, B., BHARGAVAN, K., DELIGNAT-LAVAUD, A.,FOURNET, C., KOHLWEISS, M., PIRONTI, A., STRUB, P.-Y.,

15


AND ZINZINDOHOUE, J. K. A messy state of the union: Tamingthe composite state machines of TLS. In Security and Privacy(2015), IEEE.

[3] BHARGAVAN, K., DELIGNAT-LAVAUD, A., FOURNET, C.,PIRONTI, A., AND STRUB, P.-Y. Triple handshakes and cookiecutters: Breaking and fixing authentication over TLS. In Securityand Privacy (2014), IEEE.

[4] BHARGAVAN, K., DELIGNAT-LAVAUD, A., PIRONTI, A., LAN-GLEY, A., AND RAY, M. Transport Layer Security (TLS) SessionHash and Extended Master Secret Extension, Apr. 2015.

[5] BHARGAVAN, K., FOURNET, C., KOHLWEISS, M., PIRONTI,A., AND STRUB, P.-Y. Implementing TLS with verified crypto-graphic security. In Security and Privacy (2013).

[6] BISHOP, S., FAIRBAIRN, M., NORRISH, M., SEWELL, P.,SMITH, M., AND WANSBROUGH, K. Rigorous specification andconformance testing techniques for network protocols, as appliedto TCP, UDP, and Sockets. In SIGCOMM (Aug. 2005).

[7] BORISOV, N., GOLDBERG, I., AND BREWER, E. Off-the-recordcommunication, or, why not to use PGP. In WPES (2004), ACM.

[8] BRUBAKER, C., JANA, S., RAY, B., KHURSHID, S., ANDSHMATIKOV, V. Using Frankencerts for automated adversarialtesting of certificate validation in SSL/TLS implementations. InSecurity and Privacy (2014), IEEE.

[9] COOPER, D., SANTESSON, S., FARRELL, S., BOEYEN, S.,HOUSLEY, R., AND POLK, W. Internet X.509 Public Key Infras-tructure Certificate and Certificate Revocation List (CRL) Profile.RFC 5280, May 2008.

[10] CROSBY, S. A., AND WALLACH, D. S. Denial of service viaalgorithmic complexity attacks. In USENIX Security (2003).

[11] DE RUITER, J., AND POLL, E. Protocol state fuzzing of TLSimplementations. In USENIX Security (2015).

[12] DIAZ, G., CUARTERO, F., VALERO, V., AND PELAYO, F. Auto-matic verification of the TLS handshake protocol. In Symposiumon Applied Computing (2004), ACM.

[13] DIERKS, T., AND ALLEN, C. The TLS Protocol Version 1.0.RFC 2246, Jan. 1999. Obsoleted by RFC 4346.

[14] DIERKS, T., AND RESCORLA, E. The Transport Layer Security(TLS) Protocol Version 1.1. RFC 4346, Apr. 2006. Obsoleted byRFC 5246.

[15] DIERKS, T., AND RESCORLA, E. The Transport Layer Security(TLS) Protocol Version 1.2. RFC 5246, Aug. 2008.

[16] DINGLEDINE, R., MATHEWSON, N., AND SYVERSON, P. Tor:The second-generation onion router. In USENIX Security (2004).

[17] DODIS, Y., SHAMIR, A., STEPHENS-DAVIDOWITZ, N., ANDWICHS, D. How to eat your entropy and have it too – optimalrecovery strategies for compromised RNGs. Cryptology ePrintArchive, Report 2014/167, 2014.

[18] EVERSPAUGH, A., ZHAI, Y., JELLINEK, R., RISTENPART, T.,AND SWIFT, M. Not-so-random numbers in virtualized Linuxand the Whirlwind RNG. In Security and Privacy (2014), IEEE.

[19] FARDAN, N. J. A., AND PATERSON, K. G. Lucky thirteen:Breaking the TLS and DTLS record protocols. In Security andPrivacy (2013), IEEE.

[20] FERGUSON, N., AND SCHNEIER, B. Practical Cryptography,1 ed. John Wiley & Sons, Inc., 2003.

[21] FROST, R., AND LAUNCHBURY, J. Constructing natural lan-guage interpreters in a lazy functional language. The ComputerJournal (Apr. 1989).

[22] GAZAGNAIRE, T., AND HANQUEZ, V. OXenstored: An efficienthierarchical and transactional database using functional program-ming with reference cell comparisons. In ICFP (2009), ACM.

[23] GEORGIEV, M., IYENGAR, S., JANA, S., ANUBHAI, R.,BONEH, D., AND SHMATIKOV, V. The most dangerous code inthe world: Validating SSL certificates in non-browser software.In CCS (2012), ACM.

[24] HE, C., SUNDARARAJAN, M., DATTA, A., DEREK, A., ANDMITCHELL, J. C. A modular correctness proof of IEEE 802.11Iand TLS. In CCS (2005), ACM.

[25] HENINGER, N., DURUMERIC, Z., WUSTROW, E., AND HAL-DERMAN, J. A. Mining your Ps and Qs: Detection of widespreadweak keys in network devices. In USENIX Security (2012).

[26] JAGER, T., KOHLAR, F., SCHAGE, S., AND SCHWENK, J. Onthe security of TLS-DHE in the standard model. In CRYPTO(2012).

[27] KALOPER-MERSINJAK, D., MEHNERT, H., MADHAVAPEDDY,A., AND SEWELL, P. Supplementary material doi: 10.5281/zen-odo.19160, June 2015.

[28] KOCHER, P. Timing attacks on implementations of Diffie-Hellman, RSA, DSS, and other systems. In CRYPTO. 1996.

[29] LANGLEY, A. A TLS ClientHello padding extension, Feb. 2015.[30] LOCHBIHLER, A., AND ZUST, M. Programming TLS in Is-

abelle/HOL. In Isabelle (2014).[31] MADHAVAPEDDY, A., MORTIER, R., ROTSOS, C., SCOTT, D.,

SINGH, B., GAZAGNAIRE, T., SMITH, S., HAND, S., ANDCROWCROFT, J. Unikernels: Library operating systems for thecloud. In ASPLOS (2013), ACM.

[32] MADHAVAPEDDY, A., AND SCOTT, D. J. Unikernels: The riseof the virtual library operating system. Commun. ACM 57, 1 (Jan.2014), 61–69.

[33] METTLER, A., WAGNER, D., AND CLOSE, T. Joe-e: A security-oriented subset of Java. In NDSS (2010).

[34] MEYER, C., SOMOROVSKY, J., WEISS, E., SCHWENK, J.,SCHINZEL, S., AND TEWS, E. Revisiting SSL/TLS implementa-tions: New Bleichenbacher side channels and attacks. In USENIXSecurity (2014).

[35] MOELLER, B., AND LANGLEY, A. TLS Fallback Signaling Ci-pher Suite Value (SCSV) for Preventing Protocol Downgrade At-tacks. RFC 7507, Apr. 2015.

[36] MORRISSEY, P., SMART, N. P., AND WARINSCHI, B. A mod-ular security analysis of the TLS handshake protocol. In ASI-ACRYPT (2008).

[37] NATIONALE DE LA SECURITE DES SYSTEMESD’INFORMATION, A. Etude de la securite intrinseque deslangages fonctionnels, 2013.

[38] PAULSON, L. C. Inductive analysis of the internet protocol TLS.ACM Transactions on Information and System Security 2 (1999).

[39] POPOV, A. Prohibiting RC4 Cipher Suites. RFC 7465, Feb. 2015.[40] RESCORLA, E., RAY, M., DISPENSA, S., AND OSKOV, N.

Transport Layer Security (TLS) Renegotiation Indication Exten-sion. RFC 5746, Feb. 2010.

[41] RIDGE, T., NORRISH, M., AND SEWELL, P. A rigorous ap-proach to networking: TCP, from implementation to protocol toservice. In FM (May 2008).

[42] SAINT-ANDRE, P., AND HODGES, J. Representation and Verifi-cation of Domain-Based Application Service Identity within In-ternet Public Key Infrastructure Using X.509 (PKIX) Certificatesin the Context of Transport Layer Security (TLS). RFC 6125,Mar. 2011.

[43] SCHNEIER, B., FREDRIKSON, M., KOHNO, T., AND RIS-TENPART, T. Surreptitiously weakening cryptographic systems.Cryptology ePrint Archive, Report 2015/097, 2015.

[44] WARFIELD, A., HAND, S., FRASER, K., AND DEEGAN, T. Fa-cilitating the development of soft devices. In USENIX AnnualTechnical Conference (2005).

16

Not-Quite-So-Broken TLS: Lessons in Re …...August 12–14, 2015 • Washington, D.C. ISBN...

Documents

Transcript of Not-Quite-So-Broken TLS: Lessons in Re …...August 12–14, 2015 • Washington, D.C. ISBN...