Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview...

22
Overview of Machine Learning Methods Prasanna Balaprakash Mathema5cs and Computer Science Division & Leadership Compu5ng Facility Argonne Na5onal Laboratory Argonne Training Program on Extreme-Scale Compu5ng (ATPESC) Aug 5th, 2017

Transcript of Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview...

Page 1: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

OverviewofMachineLearningMethods

PrasannaBalaprakashMathema5csandComputerScienceDivision&

LeadershipCompu5ngFacilityArgonneNa5onalLaboratory

ArgonneTrainingProgramonExtreme-ScaleCompu5ng(ATPESC)Aug5th,2017

Page 2: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Acknowledgments

hPp://adilmoujahid.com/posts/2016/06/introduc5on-deep-learning-python-caffe/hPps://en.fabernovel.com/insights/tech-en/ai-for-dummieshPps://sebas5anraschka.com/blog/2016/model-evalua5on-selec5on-part2.htmlhPp://scoP.fortmann-roe.com/docs/BiasVariance.htmlhPp://videolectures.net/deeplearning2015_vincent_machine_learning/

2

Page 3: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Bornfromtheambi5ousgoalofar5ficialintelligence•  Twohistoricallyopposedapproaches

Machinelearning

3

Page 4: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Traditional Programming

DataProgram

Output

AbundantData

OutputProgram

Givescomputerstheabilitytolearnwithoutbeingexplicitlyprogrammed

MachineLearning

Machinelearning

4

Page 5: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Ar7ficialIntelligence

MachineLearning

DeepLearning

DeepNeuralNetworks

Machinelearning

5

Page 6: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Magic, no, more like gardening •  Seeds = Algorithms •  Nutrients = Data •  Gardener = You •  Plants = Programs

InML,dataasalistofexamples(orturnitintoone)•  ideallymanyexamples•  preferablywitheachexampleavector(orfirstturnitintoone!)

Machinelearning

6

Page 7: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

7

Machinelearning

Page 8: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Supervisedlearning

8

Page 9: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Classifica5on

9

Page 10: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

10

Classifica5on

Page 11: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Regression

11

0 20 40 60

0 20 40 60 X

Y

?

11

Page 12: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Supervisedlearning:Genericframework

•  3elements–  func7onfamily–  lossfunc7on

•  measurehowwronglythemodelpredicts

–  searchforthebestfunc7on•  mathema7calop7miza7on

12

Page 13: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Biasvariancetradeoff

13

•  Allsupervisedlearningalgorithmsseektoreducebiasandvarianceinadifferentway

•  Nofreelunch:nosinglealgorithmwillworkwellonalldataset

Page 14: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

14

Machinelearning

Page 15: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Clustering

15

•  Noexplicitpredic5ontarget•  Usetheinherentstructuresinthedatatobestorganizethedataintogroups

ofmaximumcommonality(e.g.k-Means)

Page 16: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Dimensionreduc5on

16

•  Noexplicitpredic7ontarget•  Exploittheinherentstructureinthedatatosummarizeordescribedata

usinglessinforma7on(e.g.PrincipleComponentAnalysis)

Page 17: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Genera5veAdversarialNetworks

17

Page 18: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Genera5veAdversarialNetworks

18

Page 19: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

19

Machinelearning

Page 20: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

Reinforcementlearning

20

Page 21: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

21

Machinelearning

Page 22: Overview of Machine Learning Methodspress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Overview of Machine Learning Methods Prasanna Balaprakash Mathemacs and Computer Science Division

ThankYou

22

www.mcs.anl.gov/~pbalapra


Analyzing Parallel Program Performance using HPCToolkitpress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · 2 Acknowledgments • Current funding — DOE Exascale Computing Project

Analyzing Parallel Program Performance using HPCToolkitpress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · 2 Acknowledgments • Current funding — DOE Exascale Computing Project

Multiobjective Optimization of Power, Prasanna Balaprakash ... · Prasanna Balaprakash, Paul Hovland (lead), Stefan Wild Argonne National Laboratory Ananta Tiwari San Diego Supercomputer

Multiobjective Optimization of Power, Prasanna Balaprakash ... · Prasanna Balaprakash, Paul Hovland (lead), Stefan Wild Argonne National Laboratory Ananta Tiwari San Diego Supercomputer

Sandia National Laboratoriespress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020...4 Our First Workflow This process of collaborating via Git is called the Centralized Workflow •

Sandia National Laboratoriespress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020...4 Our First Workflow This process of collaborating via Git is called the Centralized Workflow •

ATPESC 2016 - Argonne National Laboratorypress3.mcs.anl.gov/atpesc/files/2016/08/Thompson_430aug...memscriptand tvscript • Command line invocation to run TotalView and Memoryscape

ATPESC 2016 - Argonne National Laboratorypress3.mcs.anl.gov/atpesc/files/2016/08/Thompson_430aug...memscriptand tvscript • Command line invocation to run TotalView and Memoryscape

ATPESC 2017 - MCSpress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · point you will see your source and can set breakpoints. • Some points to consider… –You don’t see your

ATPESC 2017 - MCSpress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · point you will see your source and can set breakpoints. • Some points to consider… –You don’t see your

AMReX: Building a Block-Structured AMR Application (and More)press3.mcs.anl.gov/atpesc/files/2019/08/ATPESC_2019... · 2019-08-06 · AMR Application (and More) Presented to ATPESC

AMReX: Building a Block-Structured AMR Application (and More)press3.mcs.anl.gov/atpesc/files/2019/08/ATPESC_2019... · 2019-08-06 · AMR Application (and More) Presented to ATPESC

Large Scale Visualization with ParaViewpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020...–Billions of AMR cells (2008). –6.33 billion unstructured cells in Catalyst (2016).

Large Scale Visualization with ParaViewpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020...–Billions of AMR cells (2008). –6.33 billion unstructured cells in Catalyst (2016).

HYPRE: High Performance Preconditionerspress3.mcs.anl.gov/atpesc/files/2015/03/fastmath-hypre-atpesc-Falg… · HYPRE_StructGridSetExtents(grid, ilo1, iup1); Set grid extents for

HYPRE: High Performance Preconditionerspress3.mcs.anl.gov/atpesc/files/2015/03/fastmath-hypre-atpesc-Falg… · HYPRE_StructGridSetExtents(grid, ilo1, iup1); Set grid extents for

Debugging and Profiling HPC Applicationspress3.mcs.anl.gov/atpesc/files/2019/08/ATPESC... · An interoperable toolkit for debugging and profiling The de-facto standard for HPC development

Debugging and Profiling HPC Applicationspress3.mcs.anl.gov/atpesc/files/2019/08/ATPESC... · An interoperable toolkit for debugging and profiling The de-facto standard for HPC development

Interconnects - press3.mcs.anl.govpress3.mcs.anl.gov/atpesc/files/2020/07/ATPESC-2020-Track-1-Talk-r-Raffenetti...Mathematics and Computer Science Division Argonne National Laboratory.

Interconnects - press3.mcs.anl.govpress3.mcs.anl.gov/atpesc/files/2020/07/ATPESC-2020-Track-1-Talk-r-Raffenetti...Mathematics and Computer Science Division Argonne National Laboratory.

Michael A. Heroux, James M. Willenbringpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020-Track-7-Talk … · Computing (ATPESC), August 2020, online. DOI: 10.6084/m9.figshare.12719834

Michael A. Heroux, James M. Willenbringpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC-2020-Track-7-Talk … · Computing (ATPESC), August 2020, online. DOI: 10.6084/m9.figshare.12719834

Git Introduction - Argonne National Laboratorypress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC...Git Introduction Presented to ATPESC 2017 Participants Alicia Klinvex Sandia National

Git Introduction - Argonne National Laboratorypress3.mcs.anl.gov/atpesc/files/2017/08/ATPESC...Git Introduction Presented to ATPESC 2017 Participants Alicia Klinvex Sandia National

Techniques for Debugging HPC Applicationspress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC... · 4 | TotalView by Perforce © Perforce Software, Inc. totalview.io What is Debugging?

Techniques for Debugging HPC Applicationspress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC... · 4 | TotalView by Perforce © Perforce Software, Inc. totalview.io What is Debugging?

omp common core ATPESC 2017press3.mcs.anl.gov/atpesc/files/2017/08/ATPESC...•Please follow these simple rules –Do the exercises that we assign and then change things around and

omp common core ATPESC 2017press3.mcs.anl.gov/atpesc/files/2017/08/ATPESC...•Please follow these simple rules –Do the exercises that we assign and then change things around and

Argonne Training Program on Extreme-Scale Computingpress3.mcs.anl.gov/atpesc/files/2018/08/ATPESC_2018_Track-4_7_8 … · Argonne Training Program on Extreme-Scale Computing ATPESC

Argonne Training Program on Extreme-Scale Computingpress3.mcs.anl.gov/atpesc/files/2018/08/ATPESC_2018_Track-4_7_8 … · Argonne Training Program on Extreme-Scale Computing ATPESC

Software Refactoring and Documentationpress3.mcs.anl.gov/atpesc/files/2019/08/ATPESC_2019... · 2019-08-09 · 7 ATPESC 2019, July 28 –August 9, 2019 Reasons for refactoring •Once

Software Refactoring and Documentationpress3.mcs.anl.gov/atpesc/files/2019/08/ATPESC_2019... · 2019-08-09 · 7 ATPESC 2019, July 28 –August 9, 2019 Reasons for refactoring •Once

Putting it All Togetherpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC... · Why don’t people “just” use software libraries. Frustration! It can be really frustrating to not

Putting it All Togetherpress3.mcs.anl.gov/atpesc/files/2020/08/ATPESC... · Why don’t people “just” use software libraries. Frustration! It can be really frustrating to not

W3203 Discrete%Mathemacs% Logic%and%Proofs%vovsha/w3203/notes/w3203_logic.pdf · W3203 Discrete%Mathemacs% % Logic%and%Proofs% Spring2015% Instructor:%Ilia%Vovsha% % hCp://vovsha/w3203%

W3203 Discrete%Mathemacs% Logic%and%Proofs%vovsha/w3203/notes/w3203_logic.pdf · W3203 Discrete%Mathemacs% % Logic%and%Proofs% Spring2015% Instructor:%Ilia%Vovsha% % hCp://vovsha/w3203%

Silicon Photonics for Extreme Scale Computing Challenges ...press3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Silicon Photonics for Extreme Scale Computing ... Photonic Computing

Silicon Photonics for Extreme Scale Computing Challenges ...press3.mcs.anl.gov/atpesc/files/2017/08/ATPESC... · Silicon Photonics for Extreme Scale Computing ... Photonic Computing

20160801 0930 ATPESC Argonne Reinders

20160801 0930 ATPESC Argonne Reinders