Cloud-scale Event Processing using Reactive Extensions...
36
Transcript of Cloud-scale Event Processing using Reactive Extensions...
Cloud-scale Event Processing
using Reactive Extensions (Rx)
Johan Laanstra & Tom Verhoeff
Events are not first-class objects
[RunOnClient]public event EventHandler<MouseEventArgs> MouseMoved;
// Runs in cloudpublic void CloudCanvas(){
MouseMoved += (o, e) => { /* do stuff */ };}
An electric eel…
delegates
Action a = new Action(Foo); // explicit creation of delegate instanceAction b = Foo; // method group conversionAction c = () => { … }; // creates anonymous method
void Foo() { … }
event Action Bar // metadata that refers to …{
add { … } // add accessorremove { … } // remove accessor
}
http://en.wikipedia.org/wiki/First-class_citizen
interface IObservable<out T>{
IDisposable Subscribe(IObserver<T> observer);}
interface IObserver<in T>{
void OnNext(T value);void OnError(Exception error);void OnCompleted();
}
Notification grammar
OnNext* (OnError | OnCompleted)?
“Gang of four” bookAddison-Wesley
Composition
Composition
interface IScheduler{
IDisposable Schedule(Action work);…
}
static class Observable{
static IObservable<T> Where<T>(this IObservable<T> source, Func<T, bool> f);static IObservable<R> Select<T, R>(this IObservable<T> source, Func<T, R> p);…
}
Function compositionwww.Wikipedia.org
Function composition
www.Wikipedia.org
public static class Observable{
public static IObservable<T> Merge<T>(this IObservable<T> xs, IObservable<T> ys){
return Create<T>(observer =>{
var gate = new object();
return new CompositeDisposable{
xs.Subscribe(x => { lock (gate) { observer.OnNext(x); } }, …),ys.Subscribe(y => { lock (gate) { observer.OnNext(y); } }, …),
};});
}}
First-class: can build
extension methods
Composition of
resource management
Await Stephen KleeneKleene star (closure)
www.Wikipedia.org
Func<T> f
var x = wait f();
IEnumerable<T> xs
foreach (var x in xs) {f(x);
}
Task<T> t
var x = await t;
IObservable<T> xs
xs.Subscribe(x => {f(x);
});
On
eM
an
y
Synchronous Asynchronous
Code-as-data
Alan KayHomoiconicity, 1969
www.Wikipedia.org
IEnumerable<T> xs
from x in xs where f(x) …
IQueryable<T> xs
from x in xs where f(x) …
IObservable<T> xs
from x in xs where f(x) …
IQbservable<T> xs
from x in xs where f(x) …
Co
de
Data
Pull Push
Events
scalable abstraction
Satya Nadella“Cloud-first, mobile-first”
www.businessinsider.com
Cortana
Cloud and device
from w in weatherwhere w.City == “Seattle”select w
flights.Where(f => f.Code == “BA49”).DelaySubscription(departure – 24 * H).TakeUntil(arrival + 24 * H).Select(f => f.Status)
userLocation.Sample(1 * M).DistinctUntilChanged().Select(here =>
traffic(here, meetingLocation).TakeUntil(meeting – minimumTime).Select(t => Timer(meeting – t.EstimatedTime)).StartWith(Timer(meeting – estimatedTime)).Switch())
.Switch()
.Select(_ => “Leave now for ” + subject)
.DelaySubscription(meeting – estimatedTime)
Temporal query
operators
Device-side
event stream
// Insert cloud-side observable sequence// of time-to-leave timer here
Cloud-side
event stream
Higher-order
query operators
Remember
tier-splitting?
IReactiveProcessing
Bing cloud
IRP
Partner1 cloudIRP
Partner2 cloudIRP
Tablet
IRP
Phone
IRP
Node
IRP
Node
IRP Node
IRP
distributed
recover
density
reliability
Scalable
checkpointing
Acknowledge / replay
Unit of pausing
Proxies
Definitions
Metadata
Hyper-cube
Sync Async
Intr
insi
cEx
trin
sic
Higher dimensionsCalabi-Yau manifold
www.Wikipedia.org
Proxies
Definitions
Metadata
Hyper-cube
Synchronous versus asynchronous
Extrinsic versus intrinsic identities
Code-as-data versus code
Reliable versus non-reliable
Etc.
Higher dimensionsCalabi-Yau manifold
www.Wikipedia.org
AsyncEx
trin
sic
E.g. this is the
client-side API
Need string theory
context
proxies
var conn = new ReactiveServiceConnection(endpoint);var ctx = new ClientContext(conn);
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
Explicit identifiers
provided for artifacts
extrinsic identifiers
Async
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
var subscription = await traffic.Where(t => t.Road == “I-90”).Select(t => new HttpData { Uri = myService,
Body = t.ToString() }).SubscribeAsync(http, mySubUri);
Explicit identifiers
provided for artifacts
SubscribeAsync expression tree
serialized
IRP
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
var subscription = await traffic.Where(t => t.Road == “I-90”).Select(t => new HttpData { Uri = …, Body = … }).SubscribeAsync(http, mySubUri);
Invoke
rx://subscribe Invoke
rx://select Invoke
rx://where trafficUri
httpUri
λt => new { bing://http/uri = …,
bing://http/body = … }
λ t => t.Road == “I-90”
Unbound parameters
processed by binder
Structural typing to
reduce coupling
onion
GitHub
Service
Host
Data model
Query engine
Operators
Your feedback is important!
Scan the QR Code and let us know via the TechDays App.
Laat ons weten wat u van de sessie vindt via de TechDays App!
Scan de QR Code.
Bent u al lid van de Microsoft Virtual Academy?! Op MVA kunt u altijd iets nieuws leren over de laatste technologie van Microsoft. Meld u vandaag aan op de MVA Stand. MVA biedt 7/24 gratis online training on-demand voor IT-Professionals en Ontwikkelaars.
