Install JDK. JDK is required for Spark ... -...
Transcript of Install JDK. JDK is required for Spark ... -...
![Page 1: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/1.jpg)
![Page 2: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/2.jpg)
InstallJDK.JDKisrequiredforSpark. WewillbeusingSpark-1.4.1. DownloadSparkfrom.h@p://spark.apache.org/downloads.htmlScreenshotinnextslide
![Page 3: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/3.jpg)
![Page 4: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/4.jpg)
ExtracttheArchiveandcdtothenewlycreateddirectory. Invokeshellusing./bin/spark-shell
![Page 5: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/5.jpg)
Scalais-objectoriented,funcKonalandstaKcallytyped. ProvideshigherlevelofabstracKon. Recommendedbook:ProgramminginScala:AComprehensiveStep-by-StepGuide,2ndEdiKonbyMarKnOdersky MarKnOderskyisthecreatorofScala.
![Page 6: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/6.jpg)
FuncKonsarefirstclassciKzens. Itisavaluejustlikeanintegerorstring.UnlikefuncKonpointersinC/C++ PassfuncKonsasargumentstootherfuncKons,returnthemasresults. DefineafuncKoninsideanotherfuncKon. DefinefuncKonswithoutgivingthemaname.Example:x=>x+1,moreconcisewayis(_+1) SprinklecodewithfuncKonliterals.
![Page 7: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/7.jpg)
FuncKonsmapinputvaluestooutputvaluesratherthanchangedatainplace.
s.replace(‘a’,’b’)yieldsanewstringobject ImmutabledatastructuresarecornerstoneoffuncKonalprogramming. Methodsshouldn’thaveanysideeffects. SomefuncKonalprogramminglanguagesevenprohibitsideeffects.ButScaladoesallowit.
![Page 8: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/8.jpg)
StaKcTypingisdesirablebutannoyingwhenyouhavetospecifytypesredundantly. Ex:intinc(inty){returny+1) Ifcompilerwassmartitcouldinc(inty)(y+1). Weshallseelaterhoweventheinputtypecanbeinferred.Sotitcanbewri@ensimplyas(_+1)
![Page 9: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/9.jpg)
![Page 10: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/10.jpg)
ClustercompuKngframeworksletsyouwriteparallelcomputaKonsusingahighlevelsetofoperators. HadoopMapReduce,Dryad,SparkareclustercompuKngframeworks. WordCounttodemonstrate–HigherleveloperaKons.
![Page 11: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/11.jpg)
MoreopKmalwhenyoureuseintermediatedataovermulKplecomputaKons. DatareuseiscommoninMachineLearningandgraphalgorithms. InteracKvedatamining. Handlesbothbatch,interacKveandstreamingapplicaKonswithinoneframework SupportsJava,ScalaandPython HigherlevelofabstracKon.
![Page 12: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/12.jpg)
Data-CollecKonofelements. Distributed-DividedintoparKKonsandtheparKKonscanbespreadouttoresideondifferentmachinesinthecluster. Resilient–Maintainslineageandifyoulosedataitcanbere-computed. Couldbepersistedindiskorcanresideinmemory.
![Page 13: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/13.jpg)
TransformaKon islazy createsnewdatasetfromexisKngone. examples:filter,map.
AcKons count,take ComputesallRDD’sinlineage.
![Page 14: Install JDK. JDK is required for Spark ... - files.meetup.comfiles.meetup.com/19103823/SparkIntroJan5.pdf · Scala is - object oriented, funcKonal and stacally typed. Provides higher](https://reader035.fdocuments.us/reader035/viewer/2022062317/5f0a2ae57e708231d42a5671/html5/thumbnails/14.jpg)