Post on 20-Sep-2018
A#1R Vers Perf A#1R
Introduction III
Radu NicolescuDepartment of Computer Science
University of Auckland
19 July 2018
1 / 25
A#1R Vers Perf A#1R
1 A#1R Short FAQ
2 FP LINQ versatility
3 Performance issues
4 Assignment #1R F#
2 / 25
A#1R Vers Perf A#1R
A#1R Short FAQ
• Read regex format? Please see the revised/extended handout
#2: @”\s+” or ”\\s+”
• Output file? Write to stdout, i.e. using C#:Console.WriteLine() , F#: printfn , node.js: console . log .
The markers will redirect such outputs to actual files:
1 jbon007 . exe b . t x t 7 > jbon007 . l o g 2> jbon007 . e r r23 jbon007 . exe b . t x t 7 1> jbon007 . l o g 2> jbon007 . e r r45 jbon007 . exe b . t x t 7 1> jbon007 . l o g 2>&1
PS. In Unix parlance, everything is a file, including stdout...
4 / 25
A#1R Vers Perf A#1R
A#1R Short FAQ
• Select first?
• Pipeline of fluent method chaining: LINQ-like high-ordermethods can be arranged in any order, as long as the outputon one is still compatible with the input of the next.
• Alternate query syntax: The order is relevant, but we haven’tdiscussed this yet .
5 / 25
A#1R Vers Perf A#1R
FP LINQ versatility
Essentially the high-level same code can be used for:
• Processing in memory sequences, e.g. arrays, lists, ...
• In LINQ, all sequences are covered by the super-interfaceIEnumerable
• Processing data base tables, views, ...
• SQL Server, Azure SQL, MySQL, NoSQL, MongoDB,CosmosDB, ...
• Processing web services
• SOAP based (XML)• REST based (XML/ATOM or JSON)
7 / 25
A#1R Vers Perf A#1R
Web Services REST
• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to buildhighly scalable distributed systems
• REST API is simply an Application Programming Interfacethat adheres to the above principles
• How many APIs for Web programming? Almost 20,000registered!
• Sorry! We are only able to look at a few, such as: Web API,ODATA, ...
• OData is just one of the many ways to build a REST API –but quite high-level, close to SQL
• We use several languages and platforms, so hopefully conceptswill surface
8 / 25
A#1R Vers Perf A#1R
FP LINQ versatility – Demo code
Look, ma, no SQL, no HTTP, no URL, ...
1 var r e s = Ord er s2 . OrderBy ( o => o . Order ID )3 . Take ( 5 )4 . S e l e c t ( o => new {5 o . OrderID ,6 o . Customer . CompanyName ,7 EmployeeName =8 o . Employee . F i rstName + ” ” + o . Employee . LastName ,9 } ) ;
1011 r e s . Dump( ” r e s ” ) ;
9 / 25
A#1R Vers Perf A#1R
FP LINQ versatility – Demo code
Same result from both a local SQl database and from a remoteODATA service:
10 / 25
A#1R Vers Perf A#1R
FP LINQ versatility – URL
Magic: FP, LINQ, Lambdas⇒ Auto-generated URL – GET request
1 ” h t t p : / / s e r v i c e s . odata . org / Northwind / Northwind . s v c /” +2 ” Ord er s ( ) ?” +3 ”$ o r d e r b y=OrderID &” +4 ”$top=5 &” +5 ”$expand=Customer , Employee &” +6 ”$ s e l e c t=OrderID , Customer /CompanyName , Employee / FirstName , Employee /LastName” ;
11 / 25
A#1R Vers Perf A#1R
FP LINQ versatility – SQL code
Magic: FP, LINQ, Lambdas ⇒ Auto-generated SQL code
1 −− Region Parameter s2 DECLARE @p0 NVarChar (1000) = ’ ’3 −− EndRegion4 SELECT [ t1 ] . [ Order ID ] , [ t2 ] . [ CompanyName ] , ( [ t3 ] . [ F i rstName ] + @p0 ) + [ t3 ] . [ LastName ] AS [ EmployeeName ]5 FROM (6 SELECT TOP ( 5 ) [ t0 ] . [ Order ID ] , [ t0 ] . [ CustomerID ] , [ t0 ] . [ EmployeeID ]7 FROM [ O rd er s ] AS [ t0 ]8 ORDER BY [ t0 ] . [ Order ID ]9 ) AS [ t1 ]
10 LEFT OUTER JOIN [ Customers ] AS [ t2 ] ON [ t2 ] . [ CustomerID ] = [ t1 ] . [ CustomerID ]11 LEFT OUTER JOIN [ Employees ] AS [ t3 ] ON [ t3 ] . [ EmployeeID ] = [ t1 ] . [ EmployeeID ]12 ORDER BY [ t1 ] . [ Order ID ]
12 / 25
A#1R Vers Perf A#1R
Data Parallelism – First, sequential versions
• Classical for loop: sum the “worked” results over a given array
1 long For ( long [ ] a ) {2 var s = 0L ;3 foreach ( var n i n a ) {4 s = s + Work ( n ) ;5 }6 return s ;7 }
• Equivalent (still) sequential C# LINQ:
1 long Seq ( long [ ] a ) {2 return3 a . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 }
• Which one is faster?14 / 25
A#1R Vers Perf A#1R
Data Parallelism – Look, ma, no threads!
• Equivalent parallel C# LINQ:
1 long Par ( long [ ] a ) {2 return3 a . A s P a r a l l e l ( ) . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 }
• Just insert . AsParallel () and the code will automatically scaleto use all available cores!
• No worries about semaphores, monitors, synchronised regions,locks, deadlocks, ...
• All this critical stuff is well encapsulated under the high-orderfunctions...
• How fast are all these?
15 / 25
A#1R Vers Perf A#1R
Data Parallelism – Look, ma, no threads!
• Linqpad results on a lab machine, with 4 cores – results areabout twice faster with optimised compiled code
1 . . . For d u r a t i o n : 15905 ms f o r ( ; ; )23 . . . Seq d u r a t i o n : 15802 ms LINQ45 . . . Par d u r a t i o n : 4104 ms LINQ parallel
• Your conclusions?
16 / 25
A#1R Vers Perf A#1R
JS – reputed very slow
• Classical for loop
1 funct ion For ( a ) {2 var s = 0 ;3 var l e n = a . l e n g t h ;4 f o r ( var i = 0 ; i < l e n ; i ++) {5 s = s + Work ( a [ i ] ) ;6 }7 return s ;8 }
• Equivalent sequential JS LINQ:
1 funct ion Seq ( a ) {2 var s =3 a . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 return s ;5 }
17 / 25
A#1R Vers Perf A#1R
node.js performance
• Node.js results on the same lab machine, with 4 cores:
1 . . . For d u r a t i o n : 16601.501 ms f o r ( ; ; )23 . . . Seq d u r a t i o n : 16573.154 ms LINQ JS
• Well, JS is not for number crunching, but not too bad...
• Your conclusions?
18 / 25
A#1R Vers Perf A#1R
How to measure performance, the FP way
• Encapsulate all measurement details in one single genericfunction – once for all!
1 s t a t i c T Durat ion<T> ( s t r i ng cap , Func<T> f ) {2 GC . C o l l e c t ( ) ;3 C o n s o l e . W r i t e L i n e ( // NB: string interpolation4 $”\ r \n . . . { cap} s t a r t ” ) ;5 var t i m e r = new Stopwatch ( ) ;6 t i m e r . S t a r t ( ) ;7 var r e s = f ( ) ;8 t i m e r . Stop ( ) ;9 C o n s o l e . W r i t e L i n e (
10 $” . . . { cap} d u r a t i o n : ” +11 ”{ t i m e r . E l a p s e d M i l l i s e c o n d s } ms” ) ;12 return r e s ;13 }
19 / 25
A#1R Vers Perf A#1R
How to invoke Duration
• Problem: Duration’s parameter f is a generic function (ok!)...but one that takes no arguments (empty parameter list)
• How to call Duration to measure a function that needsarguments, such as Seq (a), etc?
• Solution: call Duration to measure a no-arg lambda that inturn calls the required function!
1 var f = D u r a t i o n ( ” For ” , ( ) => For ( a ) ) ;2 var s = D u r a t i o n ( ” Seq ” , ( ) => Seq ( a ) ) ;3 var p = D u r a t i o n ( ” Par ” , ( ) => Par ( a ) ) ;
• For example, given () => For (a), Duration will call(() => For (a)) (), and finally For (a)!
• Thus, courtesy of lambdas and closures, our Durationfunction is really general purpose and applicable in all cases!
20 / 25
A#1R Vers Perf A#1R
Assignment #1R F#
• Structure defined by indentation: Strict offside rules! Spacesnot tabs!
• Very powerful type inference: F# is statically/strongly typed,but most types are inferred!
• Usual exceptions: types of the C# API may still need to bespecified
• True pipelines |> (cf. Unix shell) – no method chainingsurrogates
• F# lambdas: fun a −> v cf. C#: a => v
• No function return: the last expression of the logical branch isthe return value
• Parentheses required only where really needed
22 / 25
A#1R Vers Perf A#1R
Assignment #1R F# – Command-line
• From Environment
1 l e t a r g s = Envi ronment . GetCommandLineArgs ( )
• If you want a main entry point – you chose its name
1 [<E n t r y P o i n t >]2 l e t main a r g s =3 . . .
• [<EntryPoint>] is an F# attribute
23 / 25
A#1R Vers Perf A#1R
Assignment #1R F# – Read
1 open System2 open System . IO3 open System . Text . R e g u l a r E x p r e s s i o n s45 t ry6 l e t fname = ” r e a d−t e s t . t x t ”7 l e t t e x t = F i l e . ReadAl lText fname8 l e t r g x = Regex @”\ s+”9 l e t words =
10 r g x . R e p l a c e ( t e x t , ” ” ) . Trim ( ) . S p l i t ( ’ ’ )1112 words |> Array . i t e r ( fun w −> p r i n t f n ”%s ” w)1314 with ex −>15 p r i n t f n ”∗∗∗ E r r o r : %s ” ex . Message16 Envi ronment . Ex i tCode <− 1
24 / 25
A#1R Vers Perf A#1R
Assignment #1R F# – Frequencies (LP)
1 l e t F r e q u e n c i e s ( words : l i s t <s t r ing >) =2 words3 |> Seq . groupBy ( fun w −> w. ToUpper ( ) )4 |> Seq . map ( fun ( k , s ) −> ( Seq . l e n g t h s , k ) )5 |> Seq . s o r t B y D e s c e n d i n g ( fun ( c , k ) −> c )6 // . . .78 l e t Main ( ) =9 l e t words = [ ”dd” ; ”DD” ; ” ccc ” ; . . . ]
10 l e t f r s = F r e q u e n c i e s words11 f r s . Dump ” f r s ”1213 Main ( )
• F#: powerful pattern matching, everywhere, even for thefunction parameters (see map, sortBy lambdas) –cf. more limited C#: deconstructions, JS: destructuring.
25 / 25