Collaboration and ManagementDana Fisman

1

Principles of Programming Languages

www.cs.bgu.ac.il/~ppl172

Slides by Dana Fisman Based on lectures notes by Michael Elhadad Incorporating comments from Mira Balaban

2

What is a Programming Paradigm?

A paradigm is a way of thinking

A programming paradigm is a way of programming

o it recommends preferred practices o it discourages (or makes impossible) risky practices

A programming paradigm influences

o code readability o program performance o ability to reason about the program (is it correct?) o reusability of the program o fast development o etc.

3

Programming Paradigms

o Imperative Control flow is an explicit sequence of commands

o Declarative States the result you want, not how to get it

o Structured Programs have clean, goto-free, nested control structure o Procedural Imperative programs organized around hierarchies of nested procedural calls

o …

goto

There are multiple programming paradigms, among which:

H:- B1,…,B

n

4

Programming Paradigmso …

o Functional Computation proceeds by (nested) function calls that avoid any global state mutation and through the definition of function composition

o Object-Oriented

Computation is effected by sending messages to objects; objects encapsulate state and behavior

o Event-Driven Control flow is determined by asynchronous actions in reaction to events

o Logic (Rule-based) Programmer specifies a set of facts and rules, and an engine infers the answers to questions

f ( g(x) )

5

Classifications of Languages Objected Oriented: C++, Java

Functional: Haskel, ML, Scheme

Many languages do not follow a particular paradigm

Some Languages are multi-paradigm: Javascript, Python

6

What does a paradigm determines?

Dimensions of variability across programming paradigms

o Control flow o Code organization o Performance o Coupling and reuse o Testing and verification o Syntax and readability o Domain

7

Why should we learn this?

Understanding these principles of programming languages will help us

o learn new languages o compare existing languages o choose the right language for a given task o choose the right way to implement a given task o build our own language when needed.

Functional Programming

8

abbreviated FP

9

Mostly characterized by the lack of state during computation o A computation is not a sequence of states created by commands/

statements

o Rather it is a sequence of expressions that result from the evaluation of sub-expressions

Functional Programming Expressions & Values

3 + (if a>10 then (2 * 3) else (4 * 5));

In most IP computation will be preformed by intermediate states

if a>10 then temp := … else temp := …

3 + temp

In FP computation is a sequence of evaluations of sub-expressions

expr0 // given expr1 // res of evaluation expr2 // res of evaluation expr3 // res of evaluation expr4 // final

10

Example

3 + (if a>10 then (2 * 3) else (4 * 5)); // Computation proceeds in steps: // step 1 - Evaluation: (2 * 3) ==> 6 // step 2 - Substitution: ==> 3 + (if a>10 then 6 else + (4 * 5)) // step 3 - Evaluation: (4 * 5) ==> 20 // step 4 - Substitution: ==> 3 + (if a>10 then 6 else + 20) // step 5 - Substitution of a: ==> 3 + (if 8>10 then 6 else + 20) // step 6 - Evaluation: (8>10) ==> false // step 7 - Substitution: ==> 3 + (if false then 6 else + 20) // step 8 - Evaluation: if false then 6 else + 20 ==> 20 // step 9 - Substitution: ==> 3 + 20 // step 10 - Evaluation: 3 + 20 ==> 23 // end: there are no more sub-expression to evaluate ==> 23 // the value is returned

Functional Programming Expressions & Values

3 + (if a>10 then (2 * 3) else (4 * 5));

11

Functional ProgrammingoComputations in FP have no side effects

o In the previous examples, we didn’t ask the program to print a result

o Instead we evaluated an expression (which was displayed at the end)

oThe only result of a functional computation is the computed value

oNo additional changes can take place during computation

o If side-effects are required (opening files, printing, etc.) they are usually deferred to the end.

oFP does not support variable assignments or state mutations

No Side-Effects

12

Functional Programming No Side-Effects

In IP commands/statements are executed, and through their effect, state is modified (mutated).

vs.

Functional Programming

In FP expressions are evaluated and become values

Imperative Programming

13

Functional Programming

FP requires function to be first class citizens (higher order function)

This means functions are treated like any other value: ◦They can be passed as arguments to other functions ◦Functions can be returned as a result of a function ◦Functions can be anonymous (have no name)

Higher Order Functions

14

Functional Programming By the end of the lesson we will see

Characteristics of FP

◦Functions and values (rather than states and statements) ◦No side effects ◦Higher order functions

Advantages

◦Verification ◦Parallelism ◦Abstraction

in a nutshell

15

Using Javascript to illustrate FP Javascript and Typescript

o Javascript is a multi-paradigm language, it supports OOP and FP o The primitives of the language sometimes contradict good practices recommended by FP ◦ e.g. they allow easy mutations of variables and arrays ◦ but we will learn to avoid these

o Javascript is used extensively for front-end user interfaces in web and mobile applications

o Recent frameworks such as Angular2 and React rely extensively on FP o We will use Typescript (= version ES 7 of Javascript = EcmaScript 2016)

We are asked to write a program to display a number value square

The basic tool we used is a command/statement

16

From Imperative to Procedural

==> 0

console.log(0 * 0);

We are now asked to to display a square of a range of integer values from 0 to 4

The basic tool we used is a sequence of commands This kind of program is referred to as “level 0” program

17

From Imperative to Procedural

==> 0 1 4 9 16

console.log(0 * 0)console.log(1 * 1)console.log(2 * 2)console.log(3 * 3)console.log(4 * 4)

Structured programming starts with a critique of this “level 0” program o Code repetition o Cannot be easily adapted to different values of parameters o The nature of the task is not reflected in the program

Structured programming improves on this by introducing the following program contracts: o Variables to capture parameters o Use arrays to separate data and task execution o Use loop control flow structure to reflect the nature of the task

18

Structured Programmingconsole.log(0 * 0)

console.log(1 * 1)

console.log(2 * 2)

console.log(3 * 3)

console.log(4 * 4)

let numbers = [0, 1, 2, 3, 4];for (let i = 0; i < numbers.length; i++) { console.log(numbers[i] * numbers[i]);}

==> 0 1 4 9 16

The programming constructs introduced are o For-loop o Array o Variables

Now it is easy to change the code to apply it to different parameters

19

Structured Programming

let numbers = [8, 9, 10, 11, 12];for(let i = 0; i < numbers.length; i++) { console.log(numbers[i] * numbers[i]);}

==> 64 81 100 121 144

Procedural programming starts with a critique of this structural programming o We have to duplicate the code for each run with different parameters o The coupling between the code and the parameters is accidental

- Which variables correspond to parameters? - Can other parts of the code change these variables?

Procedural programming improves on this by introducing o Procedures

- With well defined interface of input parameters, output parameters, and expected behavior

o Local variables - Variables that are defined only in the scope of the procedure

20

Procedural Programming

The program constructs introduced are o function and o let which defines local variables for a block of statements

21

Procedural Programmingfunction printSquares(numbers) { for (let i = 0; i < numbers.length; i++) {

console.log(numbers[i] * numbers[i]); }}

printSquares([0, 1, 2, 3, 4]);printSquares([8, 9, 10, 11, 12]);

==>0149166481100121144

Procedures (also called functions) have a well defined interface o Name o Input variables o Return value

Why is naming a function important? o It provides an abstraction ◦ the name replaces a complex sequence of commands ◦ it can be re-used without understanding its details, only its interface

22

Procedural Programming

Suppose we are now asked to print the cubes instead of the squares? o We can define a new function computing what we want to do for each

element o And modify printSquares() to use that function

23

Procedural Programming

// We use a library function Math.powfunction cube(number) { return Math.pow(number, 3);}

function printCubes(numbers) { for(let i = 0; i < numbers.length; i++) { console.log(cube(numbers[i])); }}printCubes([0, 1, 2, 3, 4]);printCubes([8, 9, 10, 11, 12]);

==>0182764512729100013311728

Abstraction barriers:

printCubes() > cube() > Math.pow()

printSquares() > squares()

24

Procedural Programming

To support such barriers some programming languages introduce the concepts such as modules or packages

25

Is the program correct? We would like to prove that the procedure is correct according to its specification. Functions that just print (such as printCubes) are hard to verify.

To cope with this we will refactor the program in 2 stages o Data transformation - Receives and input parameter - Returns a value

o Data output - prints the transformed values

26

Is the program correct?function cubes(numbers) { for (let i = 0; i < numbers.length; i++) { numbers[i] = cube(numbers[i]); }}

function printArray(a) { for (let i = 0; i < a.length; i++) { console.log(a[i]); }}

function printCubes2(numbers) { cubes(numbers); printArray(numbers);}

// Test

printCubes2([0,1,2,3,4]);

==> 018 27 64

27

Writing a test module

==>

'all ok'

// The assert library is used to perform testsconst assert = require(‘assert');

function testCubes() {

// Empty list let numbers = []; cubes(numbers); assert.ok(numbers.length == 0);

// Invariant cubes are not modified numbers = [0,1]; cubes(numbers); assert.deepEqual([0,1], numbers, "invariant cubes");

// Regular numbers = [2]; cubes(numbers); assert.deepEqual(numbers, [8], "cube(2) === 8”);

return "all ok";}

testCubes();

28

Procedural Paradigm - ProsWe obtained a nice version of our program:

o Code is organized in layers of abstraction [printCubes > [printArrays, cubes] > power]

o Structured loops (for) are used to iterate over arrays, in a manner that reflects the task

o It can be tested

These good features were enabled by the facilities of the programming language we use:

o It is easy to define arrays, name them, initialize them, pass them, access their elements

o It is easy to define functions o Functions can invoke other functions (given the interfaces is known) o It is easy to test functionalities using facilities like assert.

The programming language encouraged us to organize our program in a good manner.

29

Procedural Paradigm - ConsWhen we scale to larger and more interesting programs, we face new types of problems

o The flow of variables across functions must be explained: - are parameters passed by value (in parameters) or by reference (inout

parameters)? - How returned values are shared between the caller and the called

function?

o The responsibilities around data structures must be clarified: - Which functions can only read data, and which functions can read and

modify data? - When is data allocated and freed?

We will return to these issues when we discuss variables and scopes. These aspects have motivated the development of the Object Oriented Paradigm.

30

Procedural Paradigm - ConsFor now, we will focus on the following issues of the procedural paradigm

o Procedural programming encourages shared state with mutation which makes concurrency difficult.

o Procedural programming commits early to a step by step way to implement operations which prevents performance optimizations.

o Procedural programming makes it difficult to create functional abstractions that are highly reusable.

o Procedural programming makes it difficult to reason about code because of shared state and mutations.

We will see how these issues are addressed by FP.

31

The Concurrency issue Suppose we run the procedure cube in two concurrent threads (using an executor as we learned in SPL) on the same array numbers.

in procedural programming

function cubes(numbers) { for (let i = 0; i < numbers.length; i++) { numbers[i] = cube(numbers[i]); }}

let n89 = [8, 9];

// In Thread 1:

cubes(n89);

// In Thread 2:

cubes(n89);

If the 2 threads are interleaved in an unfortunate sequence of events - the following scenario can occur:

Thread 1 Thread 2

numbers[1]=64

numbers[2]=81 numbers[1]=64

numbers[2]=6561

Many other unexpected values can occur

The Concurrency issue Cause of problem: uncontrolled access to shared variables

How can one solve this lack of concurrency safety?

in procedural programming

Solution

immutable data structures

enforce mutual-exclusion using locks

Hard to support

in procedural

programming

default choice in

procedural

programming

leads to problems

of liveness

(possibility of

deadlocks,

starvations, etc.)

solves also unsafe

computation due to

mutation in the

absence of threads

the solution

adopted by FP!

How do we iterate over array elements?

Using a counter variable i Can we also iterate backwards?

Does it matter? 33

Declarative vs. Procedural

function cubes(numbers) { for (let i = 0; i < numbers.length; i++) { numbers[i] = cube(numbers[i]); }}

In FP we prefer to abstract away the way the array is traversed

We want a simple way to prescribe: “apply a certain operation on all elements of an array”.

This abstract operation is called map.

34

Declarative vs. Procedural

function cubes3(numbers) { return numbers.map(cube);}

cubes3([0,1,2]);

==> [ 0, 1, 8 ]

f(x1) f(x2) f(x3) f(x4) f(x5)

f f f f f

x1 x2 x3 x4 x5

array.map(f)

An alternative way :

35

Declarative vs. Proceduralfunction cubes3(numbers) { return numbers.map(cube);}

cubes3([0,1,2]);

==> [ 0, 1, 8 ]

import {map} from ‘ramda';

function cubes4(numbers) { return map(cube, numbers);}

cubes3([0,1,2]);

array m

ap

method

map function

ramda is a typescript

package providing FP

facilities

array Function

Functionarray

36

Functional Abstractions

function cubes(numbers) { … }

function printArray(a) { … }

function printCubes2(numbers) { cubes(numbers); printArray(numbers);}

Suppose now we are required to print the exponential value of all elements in a list of numbers.

exponent

exponent

printExponent

map(cube, numbers) Math.exp(…)

This abstraction is

enabled thanks to

the ability to pass a

function as a

parameter

console.log(

)

console.log( map(Math.exp, numbers) )

37

How would you name this function?

Should we always insist to give a

function a name?

Receives: xReturns: x*x

Receives: a = a1, a2, …, anReturns: (a1 + a2 + … + an)*(1/n)

Receives: xReturns: 327(x*x) + 53

In FP it is possible to define a function without giving it a name.

In which case it is called an anonymous function or a lambda function

The syntax in Javascript is:

Examples:

Example of use:

38

Anonymous functions

( <parameters> ) => <expression>

(x) => x*x

(x,y) => x*x + 2*x*y + y*y

map(x => x*x, [0,1,2,3]) ==> [ 0, 1, 4, 9 ]

Another possible syntax in Javascript is:

Example:

Example of use:

39

Anonymous functions

function (<parameters>) {<body>}

function (x) {return x * x;}

map (function (x) {return x * x;}, [0,1,2,3]) ==> [ 0, 1, 4, 9 ]

Suppose we want to apply a function to a list of numbers, and then keep only the values that are even

40

More functional abstractions

function isEven(n) { return n % 2 == 0;}

function mapAndKeepEven(f, a) { let fa = a.map(f); let res = []; for (let i = 0; i < fa.length; i++) { if isEven(fa[i]) { res = res.concat(fa[i]);} }

return res; }

mapAndKeepEven((x)=>x*x, [0,1,2,3,4,5])

==> [ 0, 4, 16 ]

The pattern of iterating over an array and keeping only elements that satisfy a certain condition is extremely useful.

FP languages include a function to make this operation easy to use - most often called filter.

41

More functional abstractions - Filter

x1 x2 x4

x1 x2 x3 x4 x5

p?(x1) p?(x2) p?(x3) p?(x4) p?(x5)Yes Yes YesNo No

filter(p? , a)

==> [ 2, 4 ]

[1, 2, 3, 4].filter(isEven)

filter(isEven, [1,2,3,4])

==> [ 2, 4 ]

Back to our example

More concisely

More abstractly

Usage

42

More functional abstractions function mapAndKeepEven(f, a) { let fa = a.map(f); return fa.filter(isEven);}

function mapAndKeepEven(f, a) { return a.map(f).filter(isEven);}

function mapAndFilter(f, pred, a) { return a.map(f).filter(pred);}

mapAndFilter(cube, isEven, [0,1,2,3,4])

==> [ 0, 8, 64 ]

We can simply chain map and filter, without a pre-defined encapsulating function

Or using the functions filter and map of ramada package (rather than the filter and map functions of array)

43

More functional abstractions

[0,1,2,3,4,5].map((x)=>x*x).filter((x)=>x%2==0)

==> [ 0, 4, 16 ]

R.filter(isEven, R.map(cube, [0,1,2,3,4,5]))

==> [ 0, 8, 64 ]

import * as R from 'ramda';

44

More functional abstractions - compose

compose (f , g) f ( g(x) )

let evenCubes = R.compose(R.filter(isEven), R.map(cube))evenCubes([0,1,2,3,4])

Such composition is also a very general pattern, that we would like to be able do easily

==> [ 0, 8, 64 ]

45

important aspects of FP o Functions can receive functions as parameters

- including anonymous functions - for example: map, filter, compose

o Functions can return functions as computed value

- for example compose returns a new function as a computed value

- the returned function might depend on inputs, and thus can only be computed at runtime

46

Reasoning about the code Is my program correct?

Will it behave correctly on all inputs?

How much memory its computation takes?

How much steps its computation involves?

Is it equivalent to this simpler/otherwise possibly better program?

47

Reasoning about the code For instance, we might ask whether the following two programs are equivalent?

R.compose(R.filter(isEven), R.map(cube))

R.compose(R.map(cube), R.filter(isEven))Potentially

preforms less

operations

Suppose they are equivalent, should we prefer one over the other?

When are two functions f and g equivalent?

In mathematics f g if o They have the same domain, call it D o They have the same range o For every x in D the following holds : f(x) = g(x)

48

Function EquivalenceWorking Notes - Learning Emergent Scenarios

Dana, David, Assaf and Gera

...

Abstract. ⌘

1 Definitions

Let T = (⌃A,⌃B , Q, q0, �, ⌘) be a deterministic transducer, where ⌃A is its input alphabet, ⌃B is itsoutput alphabet, Q is the set of states � : Q⇥⌃A ! Q is the transition function, and ⌘ : Q ! ⌃B is theoutput function.1 The run of T on a given word w = a1, a2, . . . is a sequence of states q0, q1, q2, . . . suchthat qi+1 = �(qi, ai) for all i < |w|. The observation on a word w = a1, a2, . . . denoted ⌘(w) is a sequenceof outputs b0, b1, b2, . . . such that bi = ⌘(qi) for all i < |r| where r = q0, q1, . . . is the run of T on w.

Definition 1 (Abstractions).

– An abstraction of an observation is a function f : ⌃B⇤ ! ⌃C

⇤mapping sequences of outputs to

words over an alphabet ⌃C . An abstraction need not be length-preserving.

– An input-aware abstraction is a function f : (⌃A ⇥⌃B)⇤ ! ⌃C⇤

mapping sequences of input and

output pairs to words over an alphabet ⌃C .

Certain special cases are of interest. The below definitions are applied to abstraction for simplicity of

presentation but can be applied to an input-aware abstraction as well.

1. If for all � 2 ⌃B⇤

we have that |f(�)| = |�| we say that f is length preserving.

2. A length-preserving abstraction f is said to be a state abstraction if there exists a function fs :⌃B ! ⌃C such that f(b1b2b3 . . .) = fs(b1) fs(b2) fs(b3) . . ..

3. An abstraction is said to be punctual if |f(�)| = 1 for all � 2 ⌃⇤.

4. An abstraction is said to be multi-viewed if ⌃C = ⌃1 ⇥ ⌃2 · · · ⇥ ⌃k and there exists abstractions

f1, f2, . . . , fk such that f(�) = (f1(�), f2(�), . . . , fk(�)).

Definition 2 (Abstract Transducer). Let T = (⌃A,⌃B , Q, q0, �, ⌘) be a transducer and f : ⌃B⇤ !

⌃C⇤

an abstraction. We say that transducer A = (⌃A,⌃C , Q0, q00, �0, ⌘0) is the f abstraction of T if for

every ↵ 2 ⌃A we have ⌘0(↵) = f(⌘(↵)).

We are interested in questions of the following form, assuming an unknown transducer T with outputfunction ⌘, and an unknown abstraction f.

1. Finding an abstraction from inputs, logs and user-defined-abstractions:Given a sequence of triples (↵1,�1, �1), (↵2,�2, �2), (↵3,�3, �3), . . . where ↵i 2 ⌃⇤

A, �i = ⌘(↵i) and�i = f(�i) find f.

1We can play with other types of automata, but let’s start with this one. In particular, it might be interesting

to let ⌘ be defined from Q to ⌃⇤B (or from Q⇥⌃A to ⌃⇤

B) so that if some inputs do not cause an observable

behavior then ⌘(q) = ✏ and some inputs may yield a sequence of observable behaviors.

Domain

f

Range

49

Function Equivalence In FP a pure function has no side effects. Basically, the same definition applies. But we have to take into account, that o a computation can throw an exception o a computation may not terminate

Thus, we define that f and g are equivalent if o Whenever f(x) is evaluated to a value, then g(x) is evaluated to the same value

o If f(x) throws and exception, so does g(x) o If f(x) does not terminate, so does g(x)

50

Are they equivalent? How do we check equivalence?

We have to go over all values x of the domain D

There could be infinitely many such values (e.g. if the domain is integers)

To prove that a program is correct, we have to use the semantics of the programming language in which it is written.

The semantics of a programming language is the set of definitions providing to each operator of the language, its meaning.

51

Proving correctness The semantics of pure functional programming language is much easier to develop than that of procedural programs with side effects.

This is because it can be based on an inductive process of evaluation of expressions which only focuses on the structure of the input expression.

We will develop such tools in the course, relying extensively on types and the technique of structural induction.

The property of functional programs which makes this process easy is called referential transparency

52

Referential Transparency Referential transparency means that o the value of a program (called an expression in FP) depends only on its sub-expressions, and

o that if you substitute a sub-expression in an expression by another expression that is equivalent, then the resulting expression is equivalent to the original.

Some consequences of referential transparency are that o if one evaluates an expression twice, one obtains the same result o The relative order in which one evaluates (non-overlapping) sub-expressions of a program makes no difference to the value of the program.

o This property enables optimization methods such as parallel evaluation of sub-expressions to speed up code.

53

Proving Function Equivalence We want to check if the following two expressions are equivalent?

f = R.compose(R.filter(isEven), R.map(cube))

g = R.compose(R.map(cube), R.filter(isEven))

First we need to establish what the domain is. What is its type?

It is a finite array of integer values a = [a1,...,an]

We want to check that f(a) = g(a) for all arrays a

54

Proving Function Equivalence We develop f and g following their definitions

f(a) = filter(isEven, map(cube, a)) = filter(isEven, [cube(a1),...,cube(an)])

g(a) = map(cube, filter(isEven, a)) = map(cube, filter(isEven, [a1,...,an]))

To continue we need to apply the definition of filter

filter(pred,[a1,...,an]) =

[ai1,…,aik] ∣ 1 ≤ i1 < i2 < … < ik ≤ n,

∀j ∈ {i1,…,ik}, pred(aj) is true and ∀j ∉ {i1,…,ik}, pred(aj) is false.

Proving Function Equivalence We obtain

To prove the equivalence, we must then prove that

f(a) = filter(isEven, map(cube, a)) = filter(isEven, [cube(a1),…,cube(an)]) = [cube(ai1 ), … , cube(aik )] ∣ ∀j ∈ {i1,…,ik}, isEven(cube(aj))

g(a) = map(cube, filter(isEven, a)) = map(cube, filter(isEven, [a1,…,an])) = map(cube,[aj1,…,ajm]

∣ ∀j ∈ {j1,…,jm}, isEven(aj))

= [cube(aj1 ), … , cube(ajm)]

∣ ∀j ∈ {j1,…,jm}, isEven(aj)

∀i∈N, isEven(i) = isEven(cube(i))

Proof is enabled due to

o application of reduction steps

o where each step follows the formal semantics of the reduced sub-expression

o in particular that of filter and map