Ownership and Immutability in Generic Java (OIGJ)

35
Ownership and Immutability in Generic Java (OIGJ) Yoav Zibin + , Alex Potanin * Paley Li * , Mahmood Ali ^ , and Michael Ernst $ + IBM * Victoria,NZ ^ MIT $ U. of Washington

description

Yoav Zibin + , Alex Potanin * Paley Li * , Mahmood Ali ^ , and Michael Ernst $ + IBM * Victoria,NZ ^ MIT $ U. of Washington. Ownership and Immutability in Generic Java (OIGJ). Ownership + Immutability. Our previous work OGJ: added Ownership to Generic Java - PowerPoint PPT Presentation

Transcript of Ownership and Immutability in Generic Java (OIGJ)

Page 1: Ownership and Immutability in Generic Java (OIGJ)

Ownership and Immutability in Generic Java (OIGJ)

Yoav Zibin+, Alex Potanin*

Paley Li*, Mahmood Ali^, and Michael Ernst$

+IBM *Victoria,NZ ^MIT $U. of Washington

Page 2: Ownership and Immutability in Generic Java (OIGJ)

2/19

Ownership + Immutability

Our previous work OGJ: added Ownership to Generic Java IGJ: added Immutability to Generic Java

This work OIGJ: combine Ownership + Immutability The sum is greater than its parts

Immutable cyclic data-structures (e.g., lists, trees) IGJ could not type-check Java collections OIGJ can, without any code changes

Page 3: Ownership and Immutability in Generic Java (OIGJ)

3/19

Problem 1: Representation exposure

Internal representation leaks to the outside private doesn’t offer real protection!Real life example!class Class { private List signers; public List getSigners() { return this.signers; }}

http://java.sun.com/security/getSigners.htmlBug: the system thinks that code signed by one identity is signed by a different identity

Forgot to copy signers!

Page 4: Ownership and Immutability in Generic Java (OIGJ)

signerNsigner2signer1

entryNentry2

4/19

Solution for Representation Exposure

Ownership: owner-as-dominator Class should own the list signers No outside alias can exist Ownership can be nested: note the tree structure

Class

signers

entry1 …

X

X

X

Page 5: Ownership and Immutability in Generic Java (OIGJ)

5/19

Problem 2: Unintended Modification

Modification is not explicit in Java Can Map.get() modify the map? for (Object key : map.keySet())

map.get(key);throws ConcurrentModificationExceptionfor the following mapnew LinkedHashMap(100, 1, true)

Reorders elements according to last-accessed (like a cache)

Page 6: Ownership and Immutability in Generic Java (OIGJ)

6/19

Solution: Immutability

Object immutability mutable / immutable

Reference immutability mutable / immutable / readonly

class Student { @Immutable Date dateOfBirth; … void setTutor(@ReadOnly Student tutor) @Mutable { … }}

Method may modify the this object

Example in IGJ syntax

Page 7: Ownership and Immutability in Generic Java (OIGJ)

7/19

Raw vs. Cooked

Creation of an immutable object Raw state: Fields can be assigned Cooked state: Fields cannot be assigned When does an object become cooked?

Traditionally An object is cooked when its constructor

finishes

Page 8: Ownership and Immutability in Generic Java (OIGJ)

8/19

OIGJ’s novel idea

Connect ownership & immutability An object is cooked when its owner’s

constructor finishes The outside world will not see this cooking

phase The complex object and its representation

become immutable simultaneously Enables creation of immutable cyclic

structures

Page 9: Ownership and Immutability in Generic Java (OIGJ)

9/19

Cooking LinkedList (1 of 2)

Goal: No refactoring

1 : LinkedList(Collection<E> c) {2 : this(); // Initializes this.header3 : Entry<E> succ = this.header, pred = succ.prev;4 : for (E e : c) {5 : Entry<E> entry = 6 : new Entry<E>(e,succ,pred);7 : // An entry is modified after its constructor finished8 : pred.next = entry; pred = entry;9 : }10: succ.prev = pred;11: }

1 : LinkedList(Collection<E> c) {2 : this(); // Initializes this.header3 : Entry<E> succ = this.header, pred = succ.prev;4 : for (E e : c) {5 : Entry<E> entry = 6 : new Entry<E>(e,succ,pred);7 : // An entry is modified after its constructor finished8 : pred.next = entry; pred = entry;9 : }10: succ.prev = pred;11: }

Page 10: Ownership and Immutability in Generic Java (OIGJ)

10/19

Cooking LinkedList (2 of 2)

1 : LinkedList(@ReadOnly Collection<E> c) @Raw {2 : this(); // Initializes this.header3 : @This @I Entry<E> succ = this.header, pred = succ.prev;4 : for (E e : c) {5 : @This @I Entry<E> entry = 6 : new @This @I Entry<E>(e,succ,pred);7 : // An entry is modified after its constructor finished8 : pred.next = entry; pred = entry;9 : }10: succ.prev = pred;11: }

1 : LinkedList(@ReadOnly Collection<E> c) @Raw {2 : this(); // Initializes this.header3 : @This @I Entry<E> succ = this.header, pred = succ.prev;4 : for (E e : c) {5 : @This @I Entry<E> entry = 6 : new @This @I Entry<E>(e,succ,pred);7 : // An entry is modified after its constructor finished8 : pred.next = entry; pred = entry;9 : }10: succ.prev = pred;11: }

Page 11: Ownership and Immutability in Generic Java (OIGJ)

11/19

OIGJ Annotations

Immutability hierarchy

ReadOnly – no modification

Raw – object under construction

Ownership hierarchyWorld – anyone can

accessThis – this owns the

object

World

This

ReadOnly

Raw

Mutable

Immutable

Page 12: Ownership and Immutability in Generic Java (OIGJ)

12/19

Immutability

1:class Foo {2: // An immutable reference to an immutable date.

@Immutable Date imD;3: // A mutable reference to a mutable date.

@Mutable Date mutD;4: // A readonly reference to any date.

@ReadOnly Date roD = ... ? imD : mutD;4: // A reference to a date with the same immutability as this.

@I Date sameI;5: // Can be called on any receiver; cannot mutate this. int readonlyMethod() @ReadOnly {...}6: // Can be called only on mutable receivers; can mutate this. void mutatingMethod() @Mutable {...}7:}

1:class Foo {2: // An immutable reference to an immutable date.

@Immutable Date imD;3: // A mutable reference to a mutable date.

@Mutable Date mutD;4: // A readonly reference to any date.

@ReadOnly Date roD = ... ? imD : mutD;4: // A reference to a date with the same immutability as this.

@I Date sameI;5: // Can be called on any receiver; cannot mutate this. int readonlyMethod() @ReadOnly {...}6: // Can be called only on mutable receivers; can mutate this. void mutatingMethod() @Mutable {...}7:}

Page 13: Ownership and Immutability in Generic Java (OIGJ)

13/19

Ownership

1: // A date with the same owner and immutability as this@O @I Date sameD;

2: // A date owned by this; it cannot leak.@This @I Date ownedD;

3: // Anyone can access this date.@World @I Date publicD;

1: // A date with the same owner and immutability as this@O @I Date sameD;

2: // A date owned by this; it cannot leak.@This @I Date ownedD;

3: // Anyone can access this date.@World @I Date publicD;

class LinkedList<E> { @This @I Entry<E> header; ...}class Entry<E> { @O @I Entry<E> next, prev; ...}

class LinkedList<E> { @This @I Entry<E> header; ...}class Entry<E> { @O @I Entry<E> next, prev; ...}

Page 14: Ownership and Immutability in Generic Java (OIGJ)

14/19

Formalization: Featherweight OIGJ

Novel idea: CookersEvery object in the heap is of the form:

y is the owner of xz is the cooker of x, i.e., x becomes

cooked when the constructor of z finishesFormalism tracks the ongoing constructorsSubtyping rules connect cookers and

owners

xFoo<y,Immutable >xFoo<y,Mutable> or z

Proved soundness and type preservation

Page 15: Ownership and Immutability in Generic Java (OIGJ)

15/19

Case studies

Implementation uses the Checkers FrameworkOnly 1600 lines of code (but still a

prototype)Requires type annotations available in JSR

308 Java Collections case study

77 classes, 33K lines of codeOnly 85 ownership-related annotationsOnly 46 immutability-related annotations

Page 16: Ownership and Immutability in Generic Java (OIGJ)

16/19

Case studies conclusions

Verified that collections own their representation

Method clone does not typecheck It’s broken: shallow copy breaks ownership Suggestion: sheep clone

Ownership-aware copy: between shallow and deep copy

Compiler-generated clone that nullifies fields, and then calls a user-defined copy method

Page 17: Ownership and Immutability in Generic Java (OIGJ)

17/19

Previous Work

Universes Relaxed owner-as-dominator to owner-as-modifier

Reference immutability: C++’s const Javari

Initialization & Immutability X10, Frozen objects, Flexible Initialization

Ownership & Immutability JOE3 , MOJO, Delayed types

Page 18: Ownership and Immutability in Generic Java (OIGJ)

18/19

Future work

Inferring ownership and immutability annotations

Further case studies Extending OIGJ

owner-as-modifier uniqueness or external uniqueness ownership transfer concurrency

Page 19: Ownership and Immutability in Generic Java (OIGJ)

19/19

Conclusions

Ownership Immutability Generic Java (OIGJ)Simple, intuitive, static, backward

compatibleConnected ownership & immutabilityCase study showing usefulnessFormal proof of soundnessLink:

http://code.google.com/p/ownership-immutability/

Questions?

Page 20: Ownership and Immutability in Generic Java (OIGJ)
Page 21: Ownership and Immutability in Generic Java (OIGJ)

21/19

Contributions

No refactoring of existing code Prototype implementation

No syntax changes (uses type-annotations in JSR 308) No runtime overhead Backward compatible

Verified encapsulation of Java collections Few annotations

More Flexible Cyclic immutable structures, the factory and visitor

design patterns Formalization and soundness proof

Page 22: Ownership and Immutability in Generic Java (OIGJ)

22/19

OIGJ typing rules

Ownership nesting Field access Field assignment Method invocation Method guards Raw parameter

can be used only in method guards (see the paper for all rules, such as

inner classes, covariant subtyping)

Page 23: Ownership and Immutability in Generic Java (OIGJ)

23/19

Ownership example in OGJ

This-owned fields/methods can be accessed only via this

class Class { @This List signers; // This-owned field public @This List getSigners1() { return this.signers; } public @World List getSigners2() { return new @World LinkedList(this.signers); } public void example(Class other) { this.signers = …; // Ok other.signers = …; // Illegal this.getSigners1(); // Ok other.getSigners1(); // Illegal other.getSigners2(); // Ok}

Page 24: Ownership and Immutability in Generic Java (OIGJ)

24/19

OIGJ syntax: fields (1 of 2)

Two new generic parameters were added

1:class Foo<O extends World, I extends ReadOnly> {2: // An immutable reference to an immutable date.

Date<O,Immut> imD = new Date<O,Immut>();3: // A mutable reference to a mutable date.

Date<O,Mutable> mutD = new Date<O,Mutable>();4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable.

Date<O,ReadOnly> roD = ... ? imD : mutD;5: // A date with the same owner and immutability as this

Date<O,I> sameD;6: // A date owned by this; it cannot leak.

Date<This,I> ownedD;7: // Anyone can access this date.

Date<World,I> publicD;

1:class Foo<O extends World, I extends ReadOnly> {2: // An immutable reference to an immutable date.

Date<O,Immut> imD = new Date<O,Immut>();3: // A mutable reference to a mutable date.

Date<O,Mutable> mutD = new Date<O,Mutable>();4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable.

Date<O,ReadOnly> roD = ... ? imD : mutD;5: // A date with the same owner and immutability as this

Date<O,I> sameD;6: // A date owned by this; it cannot leak.

Date<This,I> ownedD;7: // Anyone can access this date.

Date<World,I> publicD;

Page 25: Ownership and Immutability in Generic Java (OIGJ)

25/19

OIGJ syntax: methods (2 of 2)

Method guard <T extends U>? has a dual purpose: The method is included only if T extends U Inside the method, the bound of T is U.

8 : // Can be called on any receiver; cannot mutate this. <I extends ReadOnly>? int readonlyMethod(){...}9 : // Can be called only on mutable receivers; can mutate this. <I extends Mutable>? void mutatingMethod(){...}10: // Constructor that can create (im)mutable objects. <I extends Raw>? Foo(Date<O,I> d) {11: this.sameD = d;12: this.ownedD = new Date<This,I>();13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...);14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...);15: }

8 : // Can be called on any receiver; cannot mutate this. <I extends ReadOnly>? int readonlyMethod(){...}9 : // Can be called only on mutable receivers; can mutate this. <I extends Mutable>? void mutatingMethod(){...}10: // Constructor that can create (im)mutable objects. <I extends Raw>? Foo(Date<O,I> d) {11: this.sameD = d;12: this.ownedD = new Date<This,I>();13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...);14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...);15: }

Page 26: Ownership and Immutability in Generic Java (OIGJ)

26/19

LinkedList Example

1 : class Entry<O,I,E> {2 : E element;3 : Entry<O,I,E> next, prev; …4 : }5 : class LinkedList<O,I,E> {6 : Entry<This,I,E> header; …7 : <I extends Raw>? LinkedList(8 : Collection<?,ReadOnly,E> c) {9 : this(); this.addAll(c);10: }11: <I extends Raw>? void addAll(12: Collection<?,ReadOnly,E> c) {13: Entry<This,I,E> succ = this.header,14: pred = succ.prev;15: for (E e : c) {16: Entry<This,I,E> en =17: new Entry<This,I,E>(e,succ,pred);18: pred.next = en; pred = en; }19: succ.prev = pred;20: }

1 : class Entry<O,I,E> {2 : E element;3 : Entry<O,I,E> next, prev; …4 : }5 : class LinkedList<O,I,E> {6 : Entry<This,I,E> header; …7 : <I extends Raw>? LinkedList(8 : Collection<?,ReadOnly,E> c) {9 : this(); this.addAll(c);10: }11: <I extends Raw>? void addAll(12: Collection<?,ReadOnly,E> c) {13: Entry<This,I,E> succ = this.header,14: pred = succ.prev;15: for (E e : c) {16: Entry<This,I,E> en =17: new Entry<This,I,E>(e,succ,pred);18: pred.next = en; pred = en; }19: succ.prev = pred;20: }

Page 27: Ownership and Immutability in Generic Java (OIGJ)

27/19

Ownership nesting

List<This, I,Date<World,I>> l1; // Legal nestingList<World,I,Date<This, I>> l2; // Illegal!

List<This, I,Date<World,I>> l1; // Legal nestingList<World,I,Date<This, I>> l2; // Illegal!

The main owner parameter must be inside any other owner parameter.

public static Object<World,ReadOnly> alias_l2;public static Object<World,ReadOnly> alias_l2;

It’s illegal because we can store l2 in this variable:

Page 28: Ownership and Immutability in Generic Java (OIGJ)

28/19

Field access/assignment

1: class Foo<O extends World, I extends ReadOnly> {2: Date<This,I> ownedD; // this-owned field3: Date<O,I> sameD;4: <I extends Mutable>? void bar(Foo<This,I> other) {5: this.ownedD = …; // Legal: assign via this6: other.ownedD = …; // Illegal: not via this7: other.sameD = …; // Legal: not this-owned8: }

1: class Foo<O extends World, I extends ReadOnly> {2: Date<This,I> ownedD; // this-owned field3: Date<O,I> sameD;4: <I extends Mutable>? void bar(Foo<This,I> other) {5: this.ownedD = …; // Legal: assign via this6: other.ownedD = …; // Illegal: not via this7: other.sameD = …; // Legal: not this-owned8: }

this-owned fields can be accessed/assigned only via this

Page 29: Ownership and Immutability in Generic Java (OIGJ)

29/19

Field assignment

1 : class Foo<O extends World, I extends ReadOnly> {2 : Date<O,I> sameD;3 : <I extends Raw>? void bar(4 : Foo<?,Mutable> mutableFoo,5 : Foo<?,ReadOnly> readonlyFoo,6 : Foo<?,I> rawFoo1,7 : Foo<This,I> rawFoo2) {8 : mutableFoo.sameD = …; // Legal: object is Mutable9 : readonlyFoo.sameD = …; // Illegal: object is not Raw nor Mutable10: rawFoo1.sameD = …; // Illegal: object is not this nor this-owned11: rawFoo2.sameD = …; // Legal: object is Raw and this-owned12: this.sameD = …; // Legal: object is Raw and this13: }

1 : class Foo<O extends World, I extends ReadOnly> {2 : Date<O,I> sameD;3 : <I extends Raw>? void bar(4 : Foo<?,Mutable> mutableFoo,5 : Foo<?,ReadOnly> readonlyFoo,6 : Foo<?,I> rawFoo1,7 : Foo<This,I> rawFoo2) {8 : mutableFoo.sameD = …; // Legal: object is Mutable9 : readonlyFoo.sameD = …; // Illegal: object is not Raw nor Mutable10: rawFoo1.sameD = …; // Illegal: object is not this nor this-owned11: rawFoo2.sameD = …; // Legal: object is Raw and this-owned12: this.sameD = …; // Legal: object is Raw and this13: }

1) A field can be assigned only if the object is Raw or Mutable.

2) If it is Raw, then the object must be this or this-owned.

Page 30: Ownership and Immutability in Generic Java (OIGJ)

30/19

Method invocation

1 : class Foo<O extends World, I extends ReadOnly> {2 : Date<This,I> m1() { … } // Parameter is this-owned 3 : <I extends Raw>? void m2() { … }4 : <I extends Raw>? void bar(5 : Foo<?,I> rawFoo1,6 : Foo<This,I> rawFoo2) {7 : this.m1(); // Legal: object is this 8 : rawFoo2.m1(); // Illegal: object is not this9 : rawFoo1.m2(); // Illegal: object is not this nor this-owned10: rawFoo2.m2(); // Legal: both Raw and object is this-owned11: this.m2(); // Legal: both Raw and object is this12: }

1 : class Foo<O extends World, I extends ReadOnly> {2 : Date<This,I> m1() { … } // Parameter is this-owned 3 : <I extends Raw>? void m2() { … }4 : <I extends Raw>? void bar(5 : Foo<?,I> rawFoo1,6 : Foo<This,I> rawFoo2) {7 : this.m1(); // Legal: object is this 8 : rawFoo2.m1(); // Illegal: object is not this9 : rawFoo1.m2(); // Illegal: object is not this nor this-owned10: rawFoo2.m2(); // Legal: both Raw and object is this-owned11: this.m2(); // Legal: both Raw and object is this12: }

Method invocation is the same as field access/assignment:1)If any parameter is this-owned, then the receiver must be this.2)If the guard is Raw and the object is Raw, then the receiver must be this or this-owned.

Page 31: Ownership and Immutability in Generic Java (OIGJ)

31/19

Method guards

1: class Foo<O extends World, I extends ReadOnly> {2: <I extends Raw>? void rawM() { … }3: <I extends Mutable>? void bar(4: Foo<?,ReadOnly> readonlyFoo,5: Foo<?,I> mutableFoo) {6: readonlyFoo.rawM(); // Illegal: ReadOnly is not a subtype of Raw // The bound of I in this method is Mutable7: mutableFoo.rawM(); // Legal: Mutable is a subtype of Raw8: this.rawM(); // Legal: Mutable is a subtype of Raw9: }

1: class Foo<O extends World, I extends ReadOnly> {2: <I extends Raw>? void rawM() { … }3: <I extends Mutable>? void bar(4: Foo<?,ReadOnly> readonlyFoo,5: Foo<?,I> mutableFoo) {6: readonlyFoo.rawM(); // Illegal: ReadOnly is not a subtype of Raw // The bound of I in this method is Mutable7: mutableFoo.rawM(); // Legal: Mutable is a subtype of Raw8: this.rawM(); // Legal: Mutable is a subtype of Raw9: }

Guard “<T extends U>?” has a dual purpose:1)The receiver’s T must be a subtype of U.2)Inside the method, the bound of T is U.

Conditional Java (cJ) proposed method guards for Java

Page 32: Ownership and Immutability in Generic Java (OIGJ)

Challenge: Cyclic Immutability

Cooking a cyclic data-structure is complicatedMany objects must be raw simultaneously to

manipulate backward pointersThen everything must become immutable

simultaneously

OIGJ’s novel idea:Prolong the cooking phase by using

ownership informationEnables creation of immutable cyclic

structures

Page 33: Ownership and Immutability in Generic Java (OIGJ)

33/19

Outline

Ownership Motivating problem Owner-as-dominator

Immutability Motivating problem Class immutability, object immutability, readonly

references Challenge: cyclic immutability OIGJ language Formalism Case studies

Page 34: Ownership and Immutability in Generic Java (OIGJ)

34/19

Ownership: Owner-as-dominator

Dominators in graph theory Given: a directed rooted graph X dominates Y if any path from the root to Y

passes X Owner-as-dominator

Object graph; roots are the static variables An object cannot leak outside its owner, i.e., Any path from a root to an object passes its

owner Conclusion: No aliases to internal state

Page 35: Ownership and Immutability in Generic Java (OIGJ)

35/19

Ownership

1: // A date with the same owner and immutability as this@O @I Date sameD;

2: // A date owned by this; it cannot leak.@This @I Date ownedD;

3: // Anyone can access this date.@World @I Date publicD;

4: // Constructor that can create mutable or immutable objects. Foo(@O @I Date d) @Raw {5: this.sameD = d;6: this.ownedD = new @This @I Date ();7: }

1: // A date with the same owner and immutability as this@O @I Date sameD;

2: // A date owned by this; it cannot leak.@This @I Date ownedD;

3: // Anyone can access this date.@World @I Date publicD;

4: // Constructor that can create mutable or immutable objects. Foo(@O @I Date d) @Raw {5: this.sameD = d;6: this.ownedD = new @This @I Date ();7: }