Albert Gavino on FOSS in Data Science and Analysis
-
Upload
cp-union -
Category
Technology
-
view
64 -
download
1
Transcript of Albert Gavino on FOSS in Data Science and Analysis
From regular data to BIG data
Reg
ular
dat
aB
IG d
ata
Statistical modeling
Machine Learning
Deep Learning / A.I.
Traditional Modern
Trends in Data Science Domains
Data Science Domain Current Status in the Local Setting
Statistics Traditional
Natural Language Processing (NLP) Entered the market
Predictive Analytics / Machine Learning Entered the market
Visualization / Dashboards Entered the market
Image Processing (openCV) Exploration
Internet of Things (IoT) Exploration
Artificial Intelligence/ Deep Learning Exploration
DS/Big Data Applications to the field of Study
Agriculture Climate forecast modeling to help farmers manage plantations (e.g. corn yields)
Medical field Image processing for chest x rays, retina images for diabetic patients
Linguistics Natural Language Processing (NLP) for dialects and Sentiment Analysis applications
Economics/Finance Predicting good stock options based on good economic indicators. (e.g. effect of Elections on PSE)
Sample Field of Study Specific Applications
Engineering Internet of Things (IoT) application to Big Data
Building a Data Science Team
Data Scientist Data Engineer/Dev Ops
Statistician Viz Expert
R, Python, Spark ML
Hadoop, Spark Core, Spark stream
SAS, SPSS, R, Matlab
Tableau, Cognos D3, Javascript
Neural NetsRandom Forest
RDD, dataframes,SQLContext
Linear RegressionK-means clustering
visualization GIS maps
DS
ro
le
Pro
g La
ngua
ge
Sam
ple
ou
tput
Data Science Team Composition
The rise of the DS notebooks
Notebook Options:● Running locally on a machine (e.g. jupyter)● Running on the cloud (e.g. databricks)
DS Trends Slides Summary
Choose domains which are good to focus on (e.g. data science, dev-ops, viz, statistics, ML)
It is good to learn a programming language such as R or Python and good to combine with SQL skills
Machine Learning Ensemble methods are still quite popular with DS crowd-sourcing platforms.
DS Notebooks are convenient to use