MapReduce - Functional Programming

Post on 21-Jul-2016

199 views 2 download

description

a brief introduction of Functional programming using Map Reduce algorithm.

Transcript of MapReduce - Functional Programming

Map Reduce

Map

Double the contents of an array

var a = [1, 2, 3];for (int I = 0; I < a.length; i++){a[ i ] = 2*a[ i ];}

Output:a = [2, 4, 6];

Double the contents of an arrayvar a = [1, 2, 3];

for (int i = 0; i < a.length; i++){

a[ i ] = 2*a[ i ];}

var a = [1, 2, 3];for (int i = 0; i < a.length; i++){

a[ i ] = fn(a[ i ]);}

where fn(x){ return 2*x;}

Double the contents of an arrayvar a = [1, 2, 3];

for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

} where fn(x){ return 2*x;}

function map(fn, a){for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

}where fn(x){ return 2*x;}

Double the contents of an arrayfunction map(fn, a){

for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

}where fn(x){ return 2*x;}

map(function(x){ return 2*x;},a}General Form

Reduce

Sum the contents of an array

function sum(a){ var s= 0; for(int i=0; i<a.length; i++){ s = s + a[ i ]; } return s;}

Sum the contents of an arrayfunction sum(a){ var s= 0; for(int i=0; i<a.length; i++){

s = s + a[ I ]; } return s;}

function sum(a){ var s= 0; for(int i=0; i<a.length; i++){

s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

Sum the contents of an arrayfunction sum(a){ var s= 0; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

function reduce(fn, a, init){ var s= init; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

Sum the contents of an arrayfunction reduce(fn, a, init){ var s= init; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;} where fn(a, b){ return a+b;}

General form

reduce(function(a,b){ retun a+b;}, a, init);

Benefits

In case of big arrays, code running on multiple machines, one need not to rewrite the code for computing but just replace the implementation of map and reduce as required.

Microsoft Word Document

map(function(x){ return 2*x;},a}

reduce(function(a,b){ return a+b;}, a, init);

What is hadoop?

Hadoop is parallel, distributive, fault tolerant and network topology aware implementation of MapReduce algorithm.