2012-06-25 - MapReduce auf Azure
47
-
Upload
johannes-hoppe -
Category
Technology
-
view
2.262 -
download
0
description
Der Map-Reduce Algorithmus begegnet uns bei vielen NoSQL Datenbanken. Wann immer große Datenmengen aggregiert werden, ist er das Mittel der Wahl. In einem Crashkurs wird die Theorie erläutert und gezeigt, wie in Dokumenten bei MongoDB, RavenDB und Hadoop gesucht wird. Mithilfe des MapReduce Modells von Hadoop werden Probleme wie das Durchzählen von Daten, Logfile-Analysen und Graphenanalysen demonstriert. Als Programmiersprache wird C# eingesetzt.
Transcript of 2012-06-25 - MapReduce auf Azure
0067011990999991950051507004...9999999N9+00001+99999999999... 0043011990999991950051512004...9999999N9+00221+99999999999... 0043011990999991950051518004...9999999N9-00111+99999999999... 0043012650999991949032412004...0500001N9+01111+99999999999... 0043012650999991949032418004...0500001N9+00781+99999999999...
(1950, 0) (1950, 22) (1950, -11) (1949, 111) (1949, 78)
(key, value)
(1950, 0) (1950, 22) (1950, -11) (1949, 111) (1949, 78)
(1949, [111, 78]) . (1950, [0, 22, -11])
(1949, 111) (1950, 22)
(1949, [111, 78]) . (1950, [0, 22, -11])
(1949, 111) (1949, 78) (1950, 0) (1950, 20) (1950, 10)
(1950, 25) (1950, 15) (1950, 0)
(1949, 111) (1950, 20)
(1950, 25)
(1949, [111]) . (1950, [20, 25])
js> runJar('apps/hadoop-streaming.jar', '-files "asv://server/Mapper.exe, asv://server/Reducer.exe" -mapper "Mapper.exe" -reducer "Reducer.exe" -input "inputFile" -output "outputFile"')
141.8.147.14 - - [22/Jun/2013:00:00:35 +0200] ↓ "GET /catalog/images/footbag_freaks08freestyle2_medium.jpg ↓ HTTP/1.1" 200 4379 "-" ↓ "Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)"
IP Address, Date, Request, Code, Size, Country, Referer, UserAgent 141.8.147.14, […] , Mozilla/5.0
{ "IP Address" : "141.8.147.14", "Date" : "[22/Jun/2013:00:00:35 +0200]", "Request" : "GET /catalog/images/footbag_freaks08freestyle2_medium.jpg HTTP/1.1", "Code" : 200, "Size" : 4379, "Country" : "Germany", "Referer" : "-", "UserAgent" : "Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)" }
http://highlyscalable.wordpress.com/2012/02/01/mapreduce-patterns/
![Azure MapReduce - Indiana University Bloomingtonsalsahpc.indiana.edu/.../slides/0728/Azure_MapReduce.pdf · 2010-07-29 · Apache Hadoop [24] /(Google MR) Microsoft Dryad [25] Twister](https://static.fdocuments.us/doc/165x107/5ec9aa8362c48e455a33a399/azure-mapreduce-indiana-university-2010-07-29-apache-hadoop-24-google-mr.jpg)
