Running Pharo on the GemStone VM

James FosterVP of Finance & Operations, GemTalk Systems LLC

ESUG 2017 – Maribor, Slovenia4 September 2017

Agenda

• GemStone/S Introduction• Replacing Base Class Libraries• Questions

2

Limitations of traditional Smalltalks• Object space (image) must fit in (virtual) RAM• Object space visible to one (single-threaded) VM• Sharing objects between VMs is difficult

– Convert to non-object format (binary, XML, SQL)– No built-in object identity for multiple exports

• Object state is lost when VM exits

3

Welcome to the magical world of GemStone

• Object space limited by disk, not RAM

• Object space shared across multiple VMs on multiple hosts

• Transactional persistence

• Image from http://www.flickr.com/photos/laffy4k/182219003/

• Creative Commons License http://creativecommons.org/licenses/by-sa/2.0/ 4

GemStone/S Architecture

5

Gem Types

• Linked Gem– Gem in application process space

• Remote Procedure Call (RPC) Gem– GCI library in application space– Gem has separate process

6

Application &GCI Library

Gem

OS Process

GemApplication & GCI Library TCP/IP

OS Process 1 OS Process 2

One-Machine Process Locations (Linked Gem)

7

Application & GCI Library Gem

StoneSPC

Repository

One-Machine Process Locations (RPC Gem)

8

Application & GCI Library

Gem

StoneSPC

Repository

NetLDI

Two-Machine Process Locations (Gem on Stone Host)

9

Application & GCI Library

Client Host

NETWORK

Gem

StoneSPC

Repository

Stone HostNetLDI

Two-Machine Process Locations (Gem Remote from Stone)

10

Application & GCI Library

Gem

Gem Host

Remote SPC

PageServer

NETWORK

NetLDIStone

SPC

Repository

Stone Host

Page Server

NetLDI

Three-Machine Process Locations

11

Application & GCI Library

Client Host

NETWORK

Gem

Gem Host

Remote SPC

PageServer

NETWORK

NetLDIStone

SPC

Repository

Stone Host

Page Server

NetLDI

Shared Page Cache

12

Agenda

• GemStone/S Introduction• Replacing Base Class Libraries• Questions

13

Replacing Base Class Libraries

• Smalltalk was intended to allow exploration• Portability

– ANSI– Grease

• Can we go to a lower level?

• One approach to learn more

14

Option 1: Complete Replacement

• Not possible in most Smalltalks– Creating methods requires code

• GemStone schema editing with Topaz– Build process starts with a few base classes– But no methods for any class (initially)

• Challenges– Difficult to debug (most tools are internal)– VM has knowledge of schema

15

Option 2: Parallel Class Hierarchy

• GemStone’s namespace model allows separate hierarchy

16

Pharo Global Lookup

17

Smalltalk(a SmalltalkImage)

a SystemDictionary a GlobalVariable(Association)

globals

a CompiledMethod

a Symbolkey

an Objectvalue

Image

GemStone/S Global Lookup

18

AllUsers(a UserProfileSet) a UserProfile

a SymbolList(Array)

Repository

a SymbolDictionary a SymbolAssociation

symbolList

a CompiledMethod

a Symbolkey

an Objectvalue

Unique or Shared SymbolDictionary Instances

19

AllUsers

Bob

a SymbolList

Repository

1. UserGlobalsCarol

a SymbolList

2. Globals

1. UserGlobals

3. Published

SymbolDictionary instances

Unique or Shared SymbolAssociation Instances

20

1. UserGlobals

assoc

#Arraykey

Arrayvalue

2. Globals

1. UserGlobals

assockey

value

Bob’s SymbolList Carol’s SymbolList

Unique Keys

21

1. UserGlobals

assoc

#Arraykey

Arrayvalue

2. Globals

1. UserGlobals

assockey

value

Bob’s SymbolList Carol’s SymbolList

#List

Unique Values

22

1. UserGlobals

2. Globals

1. UserGlobals

Bob’s SymbolList Carol’s SymbolList

assoc

#OrderedCollectionkey

OrderedCollectionvalue

assoc

key

OrderedCollectionvalue

Problems with Parallel Class Hierarchy

• GemStone’s namespace model allows separate hierarchy

• But complications exist:– Literals– Classes known to the VM

23

Complication: Literals (and their superclasses)

• Array: #()• BlockClosure: []• Boolean: true, false• ByteArray: #[1 2 3]• Character: $a• Float: 1.23• SmallInteger: 42• String: ‘Smalltalk’• Symbol: #Array• UndefinedObject: nil

24

Complication: Classes Known to the VM

• Behavior, Class, Metaclass• Exception, MessageNotUnderstood, ZeroDivide, …• Pragma• Process• ProcessorScheduler

• So, we need to use (some) base classes

25

Problem: Conflicting Implementations

• Array>>printOn:– Pharo

• #(1 2 3) printString '#(1 2 3)'

– GemStone• #(1 2 3) printString 'anArray( 1, 2, 3)'

26

Option 3: Parallel Methods in GemStone Classes

• Each GemStone class has a collection of MethodDictionary instances– Methods are compiled into an “environment”– Message sends to same environment (by default)

• Method environments:0 = GemStone/S (default)1 = Maglev (reserved for Ruby)2+ are for others

– We use 2 for Pharo

27

28

Array Array class

a MethodDictionary a MethodDictionary

Pharo Method Dictionaries

ArrayedCollection

ArrayedCollection class

a MethodDictionary a MethodDictionary

SequenceableCollection

SequenceableCollection class

a MethodDictionary a MethodDictionary

29

Array Array class

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

GemStone Method Dictionaries - 1Sequenceable

CollectionSequenceable

Collection class

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

30

Array Array class

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

GemStone Method Dictionaries - 2Sequenceable

CollectionSequenceable

Collection class

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

ArrayedCollection

ArrayedCollection class

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

Sample Messages• Implicit

– “in env 0” #(1 2 3) printString 'anArray( 1, 2, 3)'– “in env 2” #(1 2 3) printString '#(1 2 3)'

• Explicit– #(1 2 3) @env0:printString 'anArray( 1, 2, 3)'– #(1 2 3) @env2:printString '#(1 2 3)’

31

Array

methoddicts

a MethodDict [0]

nil

a MethodDict [2]

printOn: aStream“for GemStone”

printOn: aStream“for Pharo”

Problems with Using GemStone Classes

• GemStone class might have different schema– OrderedCollection in Pharo

• Instance variables: (array firstIndex lastIndex)

– OrderedCollection in GemStone• No instance variables

– Since OrderedCollection is not known to the compiler, we don’t have to use GemStone’s class

32

Options

1. Replace all classesBut schema is wrong in some casesAnd we would like to use tools during development

2. Use parallel classes exclusivelyBut compiler & VM know about some GemStone classes

3. Use parallel methods exclusivelyBut schema is different for some classes

4. HybridUse GemStone classes when necessary

Use parallel classes otherwiseUse method environment 2 for everything

33

Tools

• Topaz– GemStone’s command-line interface

• GemBuilder for Smalltalk (GBS)– GemStone’s GUI IDE for VA & VW

• Others– tODE– gt4gemstone– Jade

34

Issues

• Incomplete Class Types– GemStone has in-memory Ephemerons, but no

Weak references– No 32-bit objects

• GemStone compiler is in VM– Not as easy to experiment with new syntax

• Low-level objects may need extensive rewrite– BlockClosure, Process, ProcessorScheduler, etc.

35

Demo• http://files.pharo.org/get-files/60/pharo-minimal.zip• http://downloads.gemtalksystems.com/pub/GemStone64/3.4.0-Alpha4/• https://github.com/jgfoster/PharoGs/tree/james/james

36

Questions?

37

GemTalk Systems LLC15220 NW Greenbrier Pkwy., Suite 240

Beaverton, Oregon, 97006

Voice & Fax: +1 503 766 4714

james.foster@gemtalksystems.com

James G. FosterVPofFinance&Operations

www.gemtalksystems.com

®