scala-gopher: async implementation of CSP for scala

48
https://github.com/rssh/scala-gopher . Ruslan Shevchenko <[email protected]> @rssh1

Transcript of scala-gopher: async implementation of CSP for scala

https://github.com/rssh/scala-gopher

.Ruslan Shevchenko <[email protected]>@rssh1

https://github.com/rssh/scala-gopher

• CSP = Communicating Sequential Processes.• Tony Hoar.

• classical paper from 1978• book: http://www.usingcsp.com/ (1985)

• Languages• Occam (1983) for specialized hardware [transputers]• Go (2007) - got popular:

• Rob Pike (C, Plan9)• in Google

https://github.com/rssh/scala-gopher

Channel (buffered)

Channel (unbuffered)

Write (“blocked”) Read (“blocked”)

Computation blocks are connectedby channels

// in INMOS transputers - physically

Flow A Flow B

Flow A Flow B

Flow A Flow B

Flow A Flow B

Flow A Flow B

Flow A Flow B

Flow A Flow B

Flow A Flow B

https://github.com/rssh/scala-gopher

❖ Simple mental model

❖ Ability to “coordinate” between input/output sinks

go func(){ for { select { case x <- inX: var y <- inY if (x == y) { fmt.fprintf(“Bingo!”) } case q <- inQuit: break; } }()}

// Go

https://github.com/rssh/scala-gopher

❖ Simple mental model

❖ Ability to “coordinate” between import

go func(){ for { select { case x <- inX: var y <- inY if (x == y) { fmt.fprintf(“Bingo!”) } case q <- inQuit: break; } }()}

waits

// Go

https://github.com/rssh/scala-gopher

https://github.com/rssh/scala-gopher

• Not ‘emulation of go in scala’ but

• ‘CSP constructs in scala-way'

• Asynchronous

• build on top of Akka and SIP 22 - async

scala-gopher:

// ‘block’ is not block // ‘wait’ is not wait

https://github.com/rssh/scala-gopher

go func(){ for { select { case x <- inX: var y <- inY if (x == y) { fmt.fprintf(“Bingo!”) } case q <- inQuit: break; } }()}

select.forever { case x: inX.read => val y = inY.read if (x == y) { Console.println(“Bingo!”) } case q: inQuit.read => currentFlow.exit(())}

Go Scala

https://github.com/rssh/scala-gopher

go func(){ for { select { case x <- inX: var y <- inY if (x == y) { fmt.fprintf(“Bingo!”) } case q <- inQuit: break; } }()}

select.forever { case x: inX.read => val y = inY.read if (x == y) { Console.println(“Bingo!”) } case q: inQuit.read => currentFlow.exit(())}

Go Scala3 constructions (common combination)

https://github.com/rssh/scala-gopher

select.forever { case x: inX.read => val y = inY.read if (x == y) { Console.println(“Bingo!”) } case q: inQuit.read => currentFlow.exit(())}

Scala

// select.forever: basic construct

https://github.com/rssh/scala-gopher

select.forever { case x: String if (x==(inX|1inX2).read) => val y = inY.read if (x == y) { Console.println(“Bingo!”) } case q: Boolean if (q==inQuit.read) => currentFlow.exit(())}

Scala

https://github.com/rssh/scala-gopher

forever: apply(choice: PartialFunction[Any,Unit]): Future[Unit]

once: apply[T](choice: PartialFunction[Any,T]): Future[T]

Unblocked:

go[T]: Future[T]

selector:

“Blocked”: (must be inside ‘go’ or ‘async’ ):

forever: foreach(choice: PartialFunction[Any,Unit]): Unit

once: foreach[T](choice: PartialFunction[Any,T]): T

https://github.com/rssh/scala-gopher

Inside Partial Function:

❖ reading from Input[A] and do something

❖ Channel[A], Future[A], user input

❖ writing to Output[A] and do something

❖ Channel[A], user output

❖ do something on idle.

“ do something” wrapped in async block,

so we can ‘block’ there

https://github.com/rssh/scala-gopher

Input[A]:

❖ unblocked:

❖ aread, atake, aforeach

❖ ‘blocked:’

❖ read, take, foreach ….

❖ composition

❖ map, filter, zip, or, dup

❖ construction

❖ channel, future, collection, own implementation

open: ‘wait’closed: throw exception on reading after ‘last’ element

https://github.com/rssh/scala-gopher

Output[A]:

❖ unblocked:

❖ awrite, awriteAll,

❖ ‘blocked:’

❖ write, writeAll ….

❖ composition

❖ — (?)

❖ construction

❖ channel, actor, own implementation

open: ‘wait’ closed: throw exception on writing element

https://github.com/rssh/scala-gopher

Channel[A]:

❖ Input[A] + Output[A]

❖ garbage-collected.

❖ gopherApi.makeChannel

❖ can be buffered

https://github.com/rssh/scala-gopher

Timeouts:

val (inReady, inTimeouts) = in.withInputTimeouts(30 seconds) for(s <- selector.forever) { case a:inReady.read => Console.println(s“received ${a}”) case t:inTimeouts.read => Console.println(“timeout occurred”) }

Input => withInputTimeoutsOutput => withOutputTimeouts

https://github.com/rssh/scala-gopher

go[T](T :=> Future[T])

❖ async + defer/recover

❖ defer(callback: =>Unit): Unit

❖ recover(handler: PartialFunction[Throwable,T]):Boolean

go { val source = fetchSource(url) val csv = parseCsv(source) val svFile = new FileOutputStream(csv.name, append=true) defer{ val r = recover{ case FileNotFoundException => signal(“file not found”) } if (!r) svFile.close() } for(s <- csv.data) svFile.print(s)}

[finally][catch]

https://github.com/rssh/scala-gopher

go[T](T :=> Future[T])

go { val source = fetchSource(url) val csv = parseCsv(source) val svFile = new FileOutputStream(csv.name, append=true) defer{ val r = recover{ case FileNotFoundException => signal(“file not found”) } if (!r) svFile.close() } for(s <- csv.data) svFile.print(s)}

https://github.com/rssh/scala-gopher

goScope[T](T :=> T)

goScope { val in = new FileInputStream(inf) defer { in.close() } val out = new FileOutputStream(outf) defer{ out.close() } out.getChannel().transferFrom(in.getChannel, 0,Long.MaxValue)}

https://github.com/rssh/scala-gopher

❖ All “essential” functionality of CSP model

❖ Fully asynchronous implementation

❖ In normal language, with generic, typing, ..

❖ …. more: scala is object-oriented.

https://github.com/rssh/scala-gopher

❖ Set of input and output ports.

❖ Some internal state

❖ Functionality for signal transformations

Reusable block:

Transputer:

https://github.com/rssh/scala-gopher

trait Bingo extends SelectTransputer {

val inX = InPort[Int]()

val inY = InPort[Int]()

val out = OutPort[Boolean]()

loop {

case x: inX.read =>

val y = inY.read

Console.println(s"Bingo checker, received ${x}, ${y}”)

out.write(x==y)

}

}

https://github.com/rssh/scala-gopher

trait Acceptor extends SelectTransputer {

val inA = InPort[Boolean]()

@volatile var (nBingos, nPairs) = (0,0)

loop {

case x: inA.read =>

Console.println(s"acceptor: ${nPairs} ${nBingos} ${x}")

if (x) {

nBingos += 1

}

nPairs += 1

}

}

https://github.com/rssh/scala-gopher

val inX = gopherApi.makeChannel[Int]()

val inY = gopherApi.makeChannel[Int]()

val bingo = gopherApi.makeTransputer[Bingo]

val acceptor = gopherApi.makeTransputer[Acceptor]

bingo.inX connect inX

bingo.inY connect inY

bingo.out connect acceptor.inA

(bingo + acceptor).start()

https://github.com/rssh/scala-gopher

❖ Like ‘actors’ but works with ports

❖ We can ‘wait’ inside

❖ Connected via channels.

❖ Restartable

Transputers

// in remind of INMOS transputers

Select

select.foreach {

……………

}

Par

P1

P2 P3

P1 + P2 + P3

Replicate

Mapping

gopherApi.replicate[X](5)

Replicate

Mapping

gopherApi.replicate[X](5).in.distribute(_ % 10)

Element e from in will be directed to (e%10) instance of X

https://github.com/rssh/scala-gopher

❖ Select: [handle one transformation]

❖ Parallel[ transputers are run in parallel]

❖ Replicated

❖ [ run in parallel N instances.]

❖ [ custom logic for port shuffling ]

Transputer Supervisor => ActorSystem

Transputers

Implementation

Implementation

sealed trait Continuated[T]

case class ContRead[A,T]( callback , input, flow) extends Continuated[T]case class ContWrite[A,T]( callback , output, flow) ..case class ContSkip[T](callback, flow) …case class Done[T]( value , flow) ……..case object Never ……….

// simular to Iteratee

Implementation

sealed trait Continuated[T]

case class ContRead[A,T]( callback , input, flow) extends Continuated[T]

Implementation

sealed trait Continuated[T]

case class ContRead[A,T]( function: ContRead[A,B] => Option[ ContRead.In[A] => Future[Continuated[T]] ] , input, flow) extends Continuated[T]

Implementation

sealed trait Continuated[T]

case class ContRead[A,T]( function: ContRead[A,B] => Option[ ContRead.In[A] => Future[Continuated[T]] ] , input, flow) extends Continuated[T]

Value, Skip, Failure

Read as Protocol: (check, if available, read, return next step)

Implementation

Dispatch Actor

wait in channel read/write

FlowFlow

wait in select

scala-gopher

❖ CSP within scala ecosystem

❖ Channels complementary to RxStreams

❖ ‘async rw/async callback’

❖ Transducers complementary to Actors

❖ ‘transform streams/event reactions’

❖ Can be used together.

scala-gopher

❖ https://github.com/rssh/scala-gopher

❖ Ruslan Shevchenko

❖ @rssh1

❖ Questions ?