Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO?...
Transcript of Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO?...
![Page 1: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/1.jpg)
Apache PredictionIO End-to-End Machine Learning Server
with Apache Spark
![Page 2: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/2.jpg)
What is Machine Learning?
![Page 3: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/3.jpg)
What is Machine Learning? Examples of machine learning applications:
• Facial recognition software • E-mail spam detectors • Automated personalized recommendations
![Page 4: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/4.jpg)
What is Apache PredictionIO?
![Page 5: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/5.jpg)
What is Apache PredictionIO?
Video Streaming
E-mail Service
Image Data
Video Rec. Engine
Spam Detection
Engine
Facial Recognition
Engine
Event Server
Data
Query via REST
Predicted Result
![Page 6: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/6.jpg)
An Engine Template is the basic skeleton of an engine.
Recommendation Engine Template
Text Classification Engine Template
Item Recommendation
Engine
User Recommendation
Engine
Semantic Analysis Engine
Spam Detector Engine
Engine user usage: quick, ready, and customizable ML solutions
![Page 7: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/7.jpg)
• DASE - the “MVC” for Machine Learning • Data: Data Source and Data Preparator • Algorithm(s) • Serving • Evaluator
Engine Building an Engine with Separation of Concerns (SoC)
![Page 8: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/8.jpg)
A. Train deployable predictive model(s) B. Respond to dynamic query C. Evaluation
Engine Functions of an Engine
![Page 9: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/9.jpg)
Engine A. Train predictive model(s)
class DataSource(…) extends PDataSource def readTraining(sc: SparkContext) ==> trainingData
class Preparator(…) extends PPreparator def prepare(sc: SparkContext, trainingData: TrainingData) ==> preparedData
class Algorithm1(…) extends PAlgorithm def train(prepareData: PreparedData) ==> Model
$ pio train
![Page 10: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/10.jpg)
Engine A. Train predictive model(s)
Event Server
Algorithm 1 Algorithm 3Algorithm 2
PreparedDate
Engine
Data Preparator
Data Source
TrainingDate
Model 3Model 1Model 2
![Page 11: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/11.jpg)
class Algorithm1(…) extends PAlgorithm def predict(model: ALSModel, query: Query) ==> predictedResult
class Serving extends LServing def serve(query: Query, predictedResults: Seq[PredictedResult]) ==> predictedResult
B. Respond to dynamic queryEngine
Query via REST
![Page 12: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/12.jpg)
B. Respond to dynamic queryEngine
Algorithm 1Model 1
Serving
Mobile App
Algorithm 3Model 3
Algorithm 2Model 2
Predicted Results
Query (input)
Predicted Result (output)
Engine
![Page 13: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/13.jpg)
Engine DASE Factory
object RecEngine extends IEngineFactory { def apply() = { new Engine( classOf[DataSource], classOf[Preparator], Map("algo1" -> classOf[Algorithm1]), classOf[Serving]) } }
![Page 14: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/14.jpg)
Running on Production
• Install PredictionIO$ bash -c "$(curl -s http://install.prediction.io/install.sh)"
• Start the Event Server $ pio eventserver
• Deploy an Engine$ pio build; pio train; pio deploy
• Update Engine Model with New Data $ pio train; pio deploy
![Page 15: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/15.jpg)
Production Features
• Model Updating and Versioning • Offline and Online Evaluation • Multiple Engine Variants • Query and Prediction Tracking
![Page 16: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/16.jpg)
Universal Recommender https://templates.prediction.io
![Page 17: Apache PredictionIO End-to-End Machine Learning Server · PDF fileWhat is Apache PredictionIO? Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial](https://reader031.fdocuments.us/reader031/viewer/2022030415/5aa0a0237f8b9a6c178e54e9/html5/thumbnails/17.jpg)
More Information
Website: http://predictionio.incubator.apache.org/
Github: https://github.com/apache/incubator-predictionio