A Restricted Functional Language for File System...

CDSLA Restricted Functional Language

for File System Verification

Liam O’ConnorFP-Syd, October 2013

Wednesday, 16 October 13

Abstract Spec

Low Level Spec

C Implementation

Abstract Spec

Low Level Spec

C ImplementationP

Abstract Spec

Low Level Spec

C ImplementationP

Purely Functional

High Level, Nondeterministic

Efficient

Abstract Spec

Low Level Spec

C ImplementationP

Purely Functional

High Level, Nondeterministic

“Interesting”

“Boring”

Efficient

Typical OS

include3%

kernel1%

arch19%

drivers59%

sound5%

net sound fs driversarch kernel include

Typical OS

arch19%

drivers59%

sound5%

Typical OS

sound5%

Economy of Scale

liamoc@duvel:~$ cat /proc/filesystems | wc -l31

liamoc@tstvm:~$ cat /proc/filesystems | wc -l49

Economy of Scale

We don’t want a cathedral, we want a factory!

Economy of Scale

We don’t want a cathedral, we want a factory! DSL!

Our Approach

Abstract Spec

DSL Spec

C Implementation

DSL Implementation

generates

Wishlist

• Our DSL needs to:– Establish key verification properties:

• Type/Memory Safety, Termination, Totality

– Compile to efficient C code• Destructive updates, resource disposal, no excessive copying, etc.

– Be capable of expressing code for FS operations• Create file, rename file, etc.

Wishlist

• Our DSL needs to:– Establish key verification properties:

• Type/Memory Safety, Termination, Totality

– Compile to efficient C code• Destructive updates, resource disposal, no excessive copying, etc.

– Be capable of expressing code for FS operations• Create file, rename file, etc.

• We do NOT need to express everything in DSL– Can use abstraction– Define once, verify once (manually)– These components should be used in every file system

�, x : ⌧ ` t : ⇢

� ` �(x :: ⌧). t : ⌧ ! ⇢

� ` a : ⌧ ! ⇢ � ` b : ⌧

� ` a b : ⇢

Simply typed λ-calculus

x : ⌧ 2 �

� ` x : ⌧

�, x : ⌧ ` t : ⇢

� ` �(x :: ⌧). t : ⌧ ! ⇢

� ` a : ⌧ ! ⇢ � ` b : ⌧

� ` a b : ⇢

� ` P

x : ⌧,� ` P

�1�2 ` P

�2�1 ` P

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

Simply typed λ-calculus

x : ⌧ 2 �

� ` x : ⌧

� ` t : ⌧ �, x : ⌧ ` t

0 : ⇢

� ` let x :: ⌧ = t in t

0 : ⇢

� ` f : ⌧ ! ⇢ � ` x : ⌧

� ` f(x) : ⇢

First Order Language

x : ⌧ 2 �

� ` x : ⌧

� ` P

x : ⌧,� ` P

�1�2 ` P

�2�1 ` P

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

Memory Management

Verification Implementation

Expressivity

Memory Management

Expressivity

Memory Management

Expressivity

Memory Management

Expressivity

Memory Management

• Automatic memory management (GC) is too big a burden– Many static auto-MM techniques are also either inefficient or unsafe

Memory Management

• But what about manual memory management?– let x = allocData () x’ = updateData x _ = free x in x’

Memory Management

• But what about manual memory management?– let x = allocData () x’ = updateData x _ = free x in x’

• But that’s unsafe/inefficient/terrible!– Types to the rescue!

x : ⌧ ` x : ⌧

�1 ` t : ⌧ �2, x : ⌧ ` t

0 : ⇢

�1�2 ` let x :: ⌧ = t in t

0 : ⇢

�1 ` f : ⌧ ! ⇢ �2 ` x : ⌧

�1�2 ` f(x) : ⇢

Linear, First Order Language

� ` P

x : ⌧,� ` P

�1�2 ` P

�2�1 ` P

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

Safety!?

– let x = allocData y = x _ = free x in y

– let y = allocData in ()

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

� ` P

x : ⌧,� ` P

– let x = allocData () x’ = updateData x _ = free x in x’

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

Note: CDSL core syntax, not surface syntax.

Safety!?

• This example seems safe, but rejected by linear types:– let x = allocData () x’ = updateData x _ = free x in x’

Safety!?

• But, updateData is not expressible!– It has to free x or it would be using dereliction– It could destructively update it...

Safety!?

• But, updateData is not expressible!– It has to free x or it would be using dereliction– It could destructively update it...

– let x = allocData () x’ = updateData! x in x’

Two Semantics

• Value Semantics– Imagine everything is passed by value– There is no heap (free is a no-op)– Everything is immutable– Great for reasoning

Two Semantics

• Update Semantics– Some things are stored in the heap– Destructive updates actually overwrite memory– Free actually deallocates memory– Great for implementation

Two Semantics

• Update Semantics– Some things are stored in the heap– Destructive updates actually overwrite memory– Free actually deallocates memory– Great for implementation

Linear Types allow for both views!

Unboxed types

• Some things we do want passed by value– Unboxed types, integers, small structs, etc.– They shouldn’t be linear!– Functions shouldn’t be linear either, or we could only call

them once.

– Simple solution: • allow dereliction and contraction for certain types.

T• T]

Case study: Buffer interface

– make : () -> .Buf– free : .Buf -> ()– length : .Buf -> (#U32, .Buf)

– serialise : (.Obj, .Buf) -> (.Obj, .Buf)

– deserialise : .Buf -> (.Obj, .Buf)

Shareable Values

• We need (nonlinear) “look, don’t touch” references.– make : () -> .Buf– free : .Buf -> ()– length : *Buf -> #U32– serialise : (*Obj, .Buf) -> .Buf– deserialise : *Buf -> .Obj

�1, y : T⇥ ` e : ⌧ �2, x : ⌧, y : T• ` e

0 : ⇢

�1�2, y : T• ` let! (y) x :: ⌧ = e in e

0 : ⇢

Shareable values

• Unsafe again– let buf = make () in let! (buf) buf’ = buf in let _ = free buf in length buf’

�1, y : T⇥ ` e : ⌧ �2, x : ⌧, y : T• ` e

0 : ⇢

�1�2, y : T• ` let! (y) x :: ⌧ = e in e

0 : ⇢

Shareable values

Shareable reference leaks!

�1, y : T⇥ ` e : ⌧ �2, x : ⌧, y : T• ` e

0 : ⇢

�1�2, y : T• ` let! (y) x :: ⌧ = e in e

0 : ⇢

Shareable values

Shareable reference leaks!

�1, y : T⇥ ` e : ⌧ �2, x : ⌧, y : T• ` e

0 : ⇢

�1�2, y : T• ` let! (y) x :: ⌧ = e in e

0 : ⇢

⇢ safe for T

Control Flow

• This should be allowed, but it isn’t.– let x = alloc () in if condition then update(x) else x

• This is unsafe–let x = alloc () in if condition then free(x) else ()

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

Control Flow

• This should be allowed, but it isn’t.– let x = alloc () in if condition then update(x) else x

• This is unsafe–let x = alloc () in if condition then free(x) else ()

x : ⌧, x : ⌧,� ` P

x : ⌧,� ` P

�1 ` c : Bool] �2 ` t : ⌧ �2 ` e : ⌧

�1�2 ` if c then t else e : ⌧

• Our one higher order conceit– iteration schema are (external) higher order functions

– for loops are higher order function application (plus a lambda)

� ` e : (Arr T )•� ` map e : (T ! T ) ! (Arr T )•

�1 ` i : (⌧ ! ⇢) ! � �2, x : ⌧ ` s : ⇢

�1�2 ` for x in i do s : �

� ` e : (Arr T )⇥� ` fold e : (T ! ✏) ! ✏

� ` i : (⌧ ! ⇢) ! � � ` e : �

� ` i with e : ((⌧, �) ! (⇢, �)) ! (�, �)

• Multiply all array elements by 2 (destructively)

– let arr’ = for x in map(arr) do x * 2

• Sum up an array of integers:

–let sum = for (x,y) in fold(arr) with 0 do (x + y)

• Both at the same time

– let arr’, sum = for (x,y) in map(arr) with 0 do (x*2,x + y)

• Unsafe again..– let y = alloc () in for x in map(arr) do let _ = free(y) in x

�1 ` i : (⌧ ! ⇢) ! � �2, x : ⌧ ` s : ⇢

�1�2 ` for x in i do s : �

• Unsafe again..– let y = alloc () in for x in map(arr) do let _ = free(y) in x

�1 ` i : (⌧ ! ⇢) ! � �2, x : ⌧ ` s : ⇢

�1�2 ` for x in i do s : �

�2 does not contain any linear types

Error Handling

• C has error-handling via a return-code convention– We can do better!

Error Handling

• Solution– Add a separate syntactic layer, statements, above the

expression layer.• Move let!, for, let, and if on to the statement level.

– (and anonymous products)

– Statements are different from expressions in that they can evaluate to multiple values and they can fail.

Error Handling

• Solution– Add a separate syntactic layer, statements, above the

expression layer.• Move let!, for, let, and if on to the statement level.

– (and anonymous products)

– Statements are different from expressions in that they can evaluate to multiple values and they can fail.

s : ⌧s fails ⌧f

s : ⌧ss : fails ⌧f

Error Handling

for each i: �i ` ei : ⌧i�i ` return ei : ⌧i

Error Handling

�c : ec : err] for each i: �i ` ei : ⌧i�c�i ` fail ec ei : fails ⌧i

Error Handling

�1 : e : Bool] �2 : st : Tt �2 : se : Te

�1�2 ` if e then st else se : Tt t Te

fails ⌧u ⌧t

Error Handling

�1 : e : Bool] �2 : st : Tt �2 : se : Te

�1�2 ` if e then st else se : Tt t Te

Subtyping!⌧t fails ⌧u

Error Handling

• Let (and Let!) only deal with the success case!

let x = fail(EINVAL, 3) ???

– We add binding (and let!) forms for failure cases too.– The most interesting form is for the possible-failure case,

which is also a branching construct:

handle s (x. ss) (c x. sf )

Error Handling

• Let (and Let!) only deal with the success case!

let x = fail(EINVAL, 3) ???

– We add binding (and let!) forms for failure cases too.– The most interesting form is for the possible-failure case,

which is also a branching construct:

handle s (x. ss) (c x. sf )

We force you to handle your error cases!

Datatypes

• CDSL supports product (record) and sum (tagged union) types– .{ field1 : .T, field2 : .U}– .< tag1 : T, tag2 : U >

Datatypes

• Product types are complicated:

Datatypes

let sum = operation(x.field1, x.field2)

Datatypes

let sum = operation(x.field1, x.field2) ✗

Record Types

• Need to smash open a record into its constituent fields

let token { f1, f2 } = open rec f1’, f2’ = update(f1, f2) in close token {f1 = f1’, f2 = f2’ }

Record Types

• Need to smash open a record into its constituent fields

let token { f1, f2 } = open rec f1’, f2’ = update(f1, f2) in close token {f1 = f1’, f2 = f2’ }

for destructive update

Generating Purely Functional Specs

SimpleObj = { a : #U8 , b : #U8, c : .Foo }

simpleobj_example (so : .SimpleObj) : .SimpleObj fails .SimpleObj= { buf <- buf_create(42) handle code { fail (code, so) } ; buf,i <- let! (so) simpleobj_serialise(buf,so,0) handle (code, buf) { free(buf); fail (code, so) } ; so2 <- simpleobj_new('_',0) handle code { free(buf); fail (code, so) } ; so2 <- let! (buf) simpleobj_unserialise(buf, so2, 0) handle (code, so2) { free(buf,so2); fail (code, so) } ; ok <- let!(so, so2) return (so.a == so2.a && so.b == so2.b) ; free(buf) ; if not(ok) then { free (so2); fail (32,so) } else { free (so); return (so2) } }

SimpleObj = { a : U8 , b : U8, c : Foo }

simpleobj_example (so : SimpleObj) : SimpleObj fails SimpleObj= { buf <- buf_create(42) handle code { fail (code, so) } ; buf,i <- let! (so) simpleobj_serialise(buf,so,0) handle (code, buf) { free(buf); fail (code, so) } ; so2 <- simpleobj_new('_',0) handle code { free(buf); fail (code, so) } ; so2 <- let! (buf) simpleobj_unserialise(buf, so2, 0) handle (code, so2) { free(buf,so2); fail (code, so) } ; ok <- let!(so, so2) return (so.a == so2.a && so.b == so2.b) ; free(buf) ; if not(ok) then { free (so2); fail (32,so) } else { free (so); return (so2) } }

simpleobj_example (so : SimpleObj) : SimpleObj fails SimpleObj= { buf <- buf_create(42) handle code { fail (code, so) } ; buf,i <- simpleobj_serialise(buf,so,0) handle (code, buf) { free(buf); fail (code, so) } ; so2 <- simpleobj_new('_',0) handle code { free(buf); fail (code, so) } ; so2 <- simpleobj_unserialise(buf, so2, 0) handle (code, so2) { free(buf,so2); fail (code, so) } ; ok <- return (so.a == so2.a && so.b == so2.b) ; free(buf) ; if not(ok) then { free (so2); fail (32,so) } else { free (so); return (so2) } }

simpleobj_example (so : SimpleObj) : SimpleObj fails SimpleObj= { buf <- buf_create(42) handle code { fail (code, so) } ; buf,i <- simpleobj_serialise(buf,so,0) handle (code, buf) { fail (code, so) } ; so2 <- simpleobj_new('_',0) handle code { fail (code, so) } ; so2 <- simpleobj_unserialise(buf, so2, 0) handle (code, so2) { fail (code, so) } ; ok <- return (so.a == so2.a && so.b == so2.b)

; if not(ok) then { fail (32,so) } else { return (so2) } }

simpleobj_example (so : SimpleObj) : (Err, SimpleObj) + SimpleObj= case buf_create(42) of Inl code -> Inl (code,so) Inr buf -> case simpleobj_serialise(buf,so,0) of

Inl (code,buf) -> Inl (code,so) Inr (buf,i) -> case simpleobj_new('_',0) of Inl code -> Inl (code,so) Inr so2 -> case simpleobj_unserialise(buf, so2, 0) of Inl (code,so2) -> Inl (code,so) Inr so2 -> let ok = (so.a == so2.a && so.b == so2.b) in if not(ok) then Inl (32,so)

else Inr (so2)

Current Status

• We have– A paper about our overall project (not just CDSL) in PLOS

this year.– A working (but unverified) compiler to C– Formalised type system + dynamics on paper– Formalised dynamic semantics in Isabelle– Some outdated safety proofs in Agda– A good feeling about proof work remaining to be done– A prototype of another DSL for disk (de-)serialisation that

generates CDSL– A syntax headache

A Restricted Functional Language for File System...

Documents

Transcript of A Restricted Functional Language for File System...

The Master of Philosophy in Applied Epidemiology (MAE ... OConner MAE... · Epidemiology (MAE) Program: Using a blended face-to-face and online model to teach field epidemiology O’Connor

United States Historyinternet.savannah.chatham.k12.ga.us/schools/nhs/staff/OConner/Sh… · The 2015 AP U.S. History Exam is 3 hours and 15 minutes in length. There are two sections.

2013 Marketing Program Update. 2012/2013-updates 2013 Product Innovation 2013 Marketing 2013 Calendar Agenda 2.

· XLS file · Web view · 2014-07-0218 2013. 1953 159 2013. 1955 37 2013. 1948 84 2013. 1928 40 2013. 1947 127 2013. 1912 208 2013. 1952 55 2013. 1900 31 2013. 1922 119 2013.

2014 Annual Report Abridged Version - Credit Suisse€¦ · 2014 2014 2014 2014 2014 2014 2014 2014 2014 2014 2013 2013 2013 2013 2013 2013 2013 2013 2013 2013 Key Figures At year-end,

ILLINOIS · 10 February 25, 2013 March 8, 2013 11 March 4, 2013 March 15, 2013 12 March 11, 2013 March 22, 2013 13 March 18, 2013 March 29, 2013 14 March 25, 2013 April 5, 2013 15

Niël Pretorius, Chief Executive Officer Deutsche …...Dec 2012 Jan 2013 Feb 2013 Mar 2013 Apr 2013 May 2013 Jun 2013 Jul 2013 Aug 2013 Project approval Strip and clear redundant

204370Orig1Orig2s000 · 12/13/2012 · February 28, 2013 June 11, 2013 September 10, 2013 November 7, 2013 March 6, 2013 June 14, 2013 September 16, 2013 (2) November 14, 2013 March

Marva Ellis Rhonda Penny Ann Schwab Eva Thierens Education 7204T Dr. OConner-Petruso Spring 2012 The Earths Atmosphere.

Impact of Early Hypertension Control on CV Events in Adults with Type 2 Diabetes OCONNER

Through Knowledge SharingEmpowering People · 2013. 2. 15. · June 2013 January 2013 February 2013 March 2013 April 2013 May 2013 July 2013 August 2013 September 2013 October 2013

Economics 2013 Newmont Waihi Gold Economics 2013 2013.

minutes-132 of the 132nd SEIAA... · 1208/2013 121 1/2013 1213/2013 1215/2013 1218/2013 1219/2013 1220/2013 1222/2013 1223/2013 1224/2013 1227/2013 1228/2013 1229/2013 1230/2013 1231/2013

WSR Spring 2013 - Perry Johnson Registrars, Inc. · 2016. 10. 11. · March 20, 2013 March 27, 2013 April 3, 2013 April 17, 2013 May 15, 2013 May 22, 2013 June 12, 2013 June 26, 2013

CONCORDIA MARITIME AB (PUBL)€¦ · 2012-nov. 2012-dec. 2013-jan. 2013-feb. 2013-mar. 2013-apr. 2013-maj. 2013-jun. 2013-jul. 2013-aug. 2013-sep. 2013-okt. 2013-nov. 2013-dec . 2014-jan.

RFP DistributedFeb, 2013 Vendor responsesMar, 2013 Vendor presentationsMay, 2013 Vendor selectedJune, 2013 Contract finalizedOct, 2013 Project initiatedNov,

Research in Architecture International Post-Master’s ... · 04 10 2013 11 10 2013 18 10 2013 25 10 2013 08 11 2013 15 11 2013 22 11 2013 29 11 2013 06 12 2013 13 12 2013 20 12 2013

Ipsos Poll Conducted for Reuters Core Political Data · 2016. 10. 26. · FEB 19-2013 12-2013 2-2013 23-2013 14-2013 4-2013 25-2013 16-2013 6-2013 27 2013 17-2013 8-2013 29 2013 19-2013

ILLINOIS · iv 21 May 13, 2013 May 24, 2013 22 May 20, 2013 May 31, 2013 23 May 28, 2013 June 7, 2013 24 June 3, 2013 June 14, 2013 25 June 10, 2013 June 21, 2013

1 Proposed new uses for the Ceilometer Network Christine Chiu Ewan OConner, Robin Hogan, James Holmes University of Reading.