2013 QMOD Presentation
-
Upload
brandon-theiss-pe -
Category
Education
-
view
438 -
download
1
description
Transcript of 2013 QMOD Presentation
About Me• Academics
– MS Industrial Engineering Rutgers University – BS Electrical & Computer Engineering Rutgers University – BA Physics Rutgers University
• Awards– ASQ Top 40 Leader in Quality Under 40– ASQ National Education Quality Excellence Award Finalist– IIE Early Career Achievement Award Winner 2013
• Professional– Principal Industrial Engineer -Medrtonic– Master Black belt- American Standard Brands– Systems Engineer- Johnson Scale Co
• Licenses– Licensed Engineer – State of Vermont– Registered to Practice before the US Patent and Trademark Office
• Certifications– ASQ Certified Manager of Quality/ Org Excellence Cert # 13788 – ASQ Certified Quality Auditor Cert # 41232 – ASQ Certified Quality Engineer Cert # 56176 – ASQ Certified Reliability Engineer Cert #7203 – ASQ Certified Six Sigma Green Belt Cert # 3962– ASQ Certified Six Sigma Black Belt Cert # 9641– ASQ Certified Software Quality Engineer Cert # 4941
• Publications– Going with the Flow- The importance of collecting data without holding up your processes- Quality Progress March
2011– "Numbers Are Not Enough: Improved Manufacturing Comes From Using Quality Data the Right Way" (cover story).
Industrial Engineering Magazine- Journal of the Institute of Industrial Engineers September (2011): 28-33. Print
Learning Objectives
• Apply Six Sigma to the Teaching of Six Sigma• Create Practitioner Academic Partnerships • Uniquely Apply SPC Charts• Use Statistical Hypothesis testing to improve
learning outcomes
Motivation• Teaching the tools, techniques and Methods of Lean Six
Sigma is inherently difficult in academic setting.• When taught in a industrial setting students have a
common motivation (the improved welfare of the company), similar levels of education and knowledge of domain specific information. Students are encouraged to learn by applying the material to their daily activities.
• This is not possible in an academic setting particularly in a mixed environment that includes everything from undergraduate juniors through senior PhD researchers.
• In addition undergraduate students tend either lack professional or have experience in Fields that are not traditionally thought of as benefiting or implementing Six Sigma (waitressing, check out clerk etc.)
5
Putting some numbers to the motivation
• Lean Six Sigma is a commonly adopted business improvement technique which integrates, the scientific method, statistics and defect reduction to obtain tangible results.
• Within 50 miles of Rutgers there are 2,249 active job listings for the phrase “six sigma green belt”
• Non University Affiliated Classes are available however are prohibitively expensive for most students ~$2,000.
• ASQ de facto industry standard for Greenbelt Certification
• Current Industrial Engineering Undergraduate and Graduate programs do not prepare students to effectively implement the Six Sigma toolkit.
• Salary Report indicates Certified Green belts earn $12,000 more per year
Class Demographics• 71 Students Registered
– 57 At Student Tuition Rate ($296)– 14 At Professional Tuition Rate ($495)
Junior Yea
r
Senior Y
ear
BA/BS
Some G
rdudate
MA/MS/J
D
PhD/PE
0.0%5.0%
10.0%15.0%20.0%25.0%30.0%35.0%40.0%
Highest Accademic Grade Completed
24222018161412108642
20
15
10
5
0
Years Of Work Exprience
Frequency
3
Histogram of Years Of Work Exprience
Solution• The beauty of the Six Sigma Methodology is that it can be applied to any
process.• The definition of a process is quite broad and can be reduced to any
verb- noun combination.• Therefore the collective process which the class studied and improved
was toPass [the]
ASQ Certified Six Sigma Green
Belt Exam
• Therefore the foundational Six Sigma Concept of DMAIC (Define Measure Analyze Improve Control) represents both the material covered in the course as well as the pedagogical method used for instruction
Theoretical Pedagogical Support• Knowles' Theory of Andragogy is based upon six fundamental assumptions
related to motivation in adult learning:– Adults become aware of their "need to know" and make a case for the value of learning.
(Need to Know)• the students in the class have self selected to enroll in the course, which was
marketed solely to prepare students to pass the certification exam.– Adults approach learning with different experiences furthermore the richest resource for
learning resides in adults themselves (Foundation).• for each topic introduced there is a tactical application applying the tool or
technique to the process of preparing for the ASQ CSSGB exam.
– Adults need to be responsible for their decisions on education; They need to be seen and treated as capable and self-directed. (Self-concept).
• no formal grades are given in the course and no homework is assigned. Students are responsible for gauging what additional preparation was required in addition to the 3-hour weekly course
Theoretical Pedagogical Support (continued)
– Adults are most interested in learning subjects having immediate relevance to cope effectively with the present real-life situations (Readiness).
• the course culminated in students taking the actual ASQ CSSGB exam, thus for the 11 weeks of the course students were constantly driving towards a relevant and timely goal
– Adult learning is problem-centered rather than content-oriented they want to learn what will help them perform tasks or deal with problems they confront in everyday situations and those presented in the context of application to real-life (Orientation).
• each activity throughout the course is driven at solving a problem with only a mild suggestion of which tool to use.
– Adults respond better to internal versus external motivators (Motivation). • the motivation for the students to take the course was entirely self originating.
About the Course & Partnership• Offered as a Non-Credit extracurricular course
at Rutgers University in Piscataway NJ• Co-Sponsored by the Rutgers Student Chapter
of the Institute for Industrial Engineers (IIE) and the Princeton NJ section of American Society for Quality (ASQ)
• Open and advertised to all members of the Rutgers Community (students, staff and faculty) as well as the surrounding public
• Objective of the course was to train students to pass the June 2nd 2012 administration of the ASQ Certified Six Sigma Green Belt Exam
Course Syllabus1. Introduction, Sample Exam2. Review Exam, Define 13. Define 2, Measure 14. Measure 2, Measure 3 5. Measure 4, Sample 50 Question Exam6. Review Exam, Analyze 1
7. Analyze 2, Analyze 38. Improve 1, Sample 50 Question Exam9. Review Exam, Control 110. Sample 100 Question Exam11. Review Exam, Additional Questions
Define Measure Analyze Improve Control• Project Definition• Team Dynamics• Brainstorming• Process Mapping
• Measurement Systems
• Histograms• Box Plots• Dot Plots• Probability
Plots• Control Charts
• Inferential Statistics
• Confidence Intervals
• Hypothesis Tests
• Regression Analysis
• Pareto Charts• Process
Capability• Lean
Pre Test• On the first night of classes students were given
an introductory survey of Six Sigma by means of a worked example applying DMAIC to the Starbucks Experience from a Customers Prospective.
• Students were then given a copy of the Certified Six Sigma Green Belt Handbook by Roderick A. Munro
• Then given a 50 Question Multiple Choice Test representative of the ASQ CSSGB Exam
• The Test was administered on two successive nights (Monday and Tuesday)
Measurement System
• An Apperson GradeMaster™ 600 Test Scanner was utilized which enabled test to be scored and returned immediately upon student submission at the exam site.
• In addition all of each answer to every question was downloaded to connected computer enabling further detailed analysis
MONDAY RESULTS
Test Scores
84.00%72.00%60.00%48.00%36.00%
9
8
7
6
5
4
3
2
1
0
Test Scores
Frequency
Mean 0.5589StDev 0.1177N 35
Histogram of Test ScoresNormal
Test for Normality
1.00.90.80.70.60.50.40.30.2
99
95
90
80
70
605040
30
20
10
5
1
Test Score
Perc
ent
Mean 0.5589StDev 0.1177N 35AD 0.396P-Value 0.352
Probability Plot of Test ScoreNormal - 95% CI
Is process in Control?
343128252219161310741
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Observation
Indiv
idual V
alu
e
_X=0.5589
UCL=0.9468
LCL=0.1709
I Chart of Test Score
Is the Process Capable?
0.840.720.600.480.36
LSL
LSL 0.78Target *USL *Sample Mean 0.558857Sample N 35StDev(Within) 0.120985StDev(Overall) 0.117718
Process Data
Cp *CPL -0.61CPU *Cpk -0.61
Pp *PPL -0.63PPU *Ppk -0.63Cpm *
Overall Capability
Potential (Within) Capability
PPM < LSL 971428.57PPM > USL *PPM Total 971428.57
Observed PerformancePPM < LSL 966214.72PPM > USL *PPM Total 966214.72
Exp. Within PerformancePPM < LSL 969849.40PPM > USL *PPM Total 969849.40
Exp. Overall Performance
WithinOverall
Process Capability of Test Scores
overall standard deviation for the entire study
overall standard deviation for the entire study if special cause eliminated
based on variation within subgroups
Are there bad questions?
464136312621161161
1.0
0.8
0.6
0.4
0.2
0.0
Sample
Pro
port
ion
_P=0.441
UCL=0.693
LCL=0.189
1
1
1
1
1
1
11
11
P Chart of Wrong
Does the order the exams are turned in effect the score?
3330272421181512963
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Index
Test
Sco
re
MAPE 15.9381MAD 0.0840MSD 0.0124
Accuracy Measures
ActualFits
Variable
Trend Analysis Plot for Test ScoreLinear Trend Model
Yt = 0.5018 + 0.00317*t
TUESDAY RESULTS
Test Scores
Test for Normality
Is the process in Control?
28252219161310741
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
Observation
Indiv
idual V
alu
e
_X=55.93%
UCL=84.62%
LCL=27.25%
1
I Chart of Scores
Is the process capable?
Are there Bad Questions?
464136312621161161
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Sample
Pro
port
ion
_P=0.441
UCL=0.717
LCL=0.164
1
11
11
1
1
1
P Chart of Incorrect
272421181512963
0.9
0.8
0.7
0.6
0.5
0.4
Index
Sco
res
MAPE 13.9747MAD 0.0779MSD 0.0100
Accuracy Measures
ActualFits
Variable
Trend Analysis Plot for ScoresLinear Trend Model
Yt = 0.5614 - 0.000138*t
Does the order exams are turned in effect test scores?
COMBINED RESULTS
Combined Test Scores
0.840.720.600.480.36
20
15
10
5
0
Combined
Frequency
Mean 0.5591StDev 0.1099N 64
Histogram of CombinedNormal
Test Scores
0.84
0.72
0.60
0.48
0.36
9
8
7
6
5
4
3
2
1
0
84.0
0%
72.0
0%
60.0
0%
48.0
0%
36.0
0%
9
8
7
6
5
4
3
2
1
0
Monday
Frequency
TuesdayMean 0.5589StDev 0.1177N 35
Monday
Mean 0.5593StDev 0.1018N 29
Tuesday
Histogram of Monday, TuesdayNormal
Is there a difference Between Classes?
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Monday Tuesday
Boxplot of Monday, Tuesday
Is there a statistical Difference?Anova: Single Factor
SUMMARYGroups Count Sum Average Variance
Monday 35 19.56 0.558857 0.013857Tuesday 29 16.22 0.55931 0.010357
ANOVASource of Variation SS df MS F P-value F crit
Between Groups 3.26E-06 1 3.26E-06 0.000265 0.987056 3.995887Within Groups 0.76114 62 0.012276
Total 0.761144 63
Is the variation different?
34
464136312621161161
1.0
0.8
0.6
0.4
0.2
0.0
Sample
Pro
port
ion
_P=0.441
UCL=0.627
LCL=0.255
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
P Chart of Wrong
What Can we See from the Out of Control Points?
Brainstorming Techniques• At the beginning of class students were asked as a group to brainstorm
ideas for why they failed the pre-test– Only 4 ideas were proposed
• Students were taught the different brainstorming techniques contained in the CSSGB Body of Knowledge– Nominal Group Technique– Multi-Voting– Affinity Diagrams – Force Field Analysis– Tree Diagrams– Cause and Effect Diagrams
• Students were then broken up into 6 different groups, assigned one of the brainstorming techniques and given the task to brainstorm why they failed the pre-test
Brainstorming Techniques Continued
• Students then presented their results to the Group
Brainstorming Results
Cause and Effect (Fishbone)
Affinity Diagram
Brainstorming Results
Tree Diagram
Force Field Analysis
Brainstorming Results
Multi-Voting
Nominal Group Technique
Brainstorming Continued
• Students then told to return to their groups and apply their “favorite” of the brainstorming techniques to the task how can you Pass the midterm exam
• Students Found the positive formulation of the task much more challenging and most groups stayed with the same technique they used for the Negative version.
Team Dynamics
• The 3rd weeks lesson began with an introduction of the Tuckman cycle of team dynamics
• Students were asked to reflect upon their experience in the brainstorming activity to see if their experiences paralleled those predicted by the model
Process Mapping
• The second portion of the 3rd Class was spent introducing the process mapping strategies in the CSSGB BoK– SIPOC (Suppliers Inputs Process Outputs Customers)– Process Mapping– Value Stream
Mapping
Process Mapping Continued
• Students were again divided into 6 groups. Each group was assigned a map type and told to Map the Exam Taking Process at either a Micro or Macro Level
• Micro Level Groups Handled the Physical steps of taking the exam such as reading the question, locating the answer and filling in the bubbles
• Macro Groups Handled the all of the preparation leading up to taking the exam• The point was to emphasize that the same tools techniques and methods can be used
on the very micro level (an operator tightening a bolt) to the very macro level (the operations of a fortune 500 company)
44
SIPOC at a even higher level
Input• Students• Body of
Knowledge• Instructor• Textbook• Facilities
Supplier• ASQ Princeton• ASQ Corporate
• Rutgers University
Output• Knowledge• Certification
Customers• Future Employers• Current Employers• Students• Rutgers University• ASQ Princeton• Rutgers IIE
Educate Students in Six
Sigma
ProcessIdentify Educational
Shortcoming
Create Course
Develop Methodology
Locate Students
Teach Students
Administer Test
Control Charts• Class 4 Introduced Students to the Control Charts Covered in the CSSGB
BoK– I-MR– X Bar-R– X Bar- S– P– NP– U– C
• Students were emailed prior to class a Microsoft Excel Workbook containing the test results and told to bring their laptops to class
• Students were asked to do the following by hand (with Excel helping for the calculations):– I-MR Chart for Test Scores– P Chart testing for “Bad Questions”– NP Chart testing for “Bad Questions”– C Chart for the number of wrong responses per exam– U Chart for the number of wrong responses per exam
Control Charts Results
NP Chart
C Chart
Midterm Analysis
Midterm Exam Results
Pre Class Exam Results
Comparison
Does a T-Test Indicate there was improvement?
t-Test: Two-Sample Assuming Unequal Variances
Mid PreMean 0.607234 0.561702Variance 0.014373 0.01111Observations 47 47Hypothesized Mean Difference 0df 91t Stat 1.955429P(T<=t) one-tail 0.0268t Critical one-tail 1.661771P(T<=t) two-tail 0.0536t Critical two-tail 1.986377
Does ANOVA Indicate there was Improvement?
Anova: Single Factor
SUMMARYGroups Count Sum Average Variance
Pre Total 64 35.78 0.559063 0.012082Mid Total 53 31.72 0.598491 0.013705
ANOVASource of Variation SS df MS F P-value F crit
Between Groups 0.045069 1 0.045069 3.516685 0.06329 3.923599Within Groups 1.473823 115 0.012816
Total 1.518892 116
Change in Scores
Is the Change in Control?
-15
-10
-5
0
5
10
15
C Chart of Change in # of Correct Responses
UCL = 8.29
LCL = -3.74
Mid= 2.28
Is the change in Scores Significant?
t-Test: Paired Two Sample for Means
Mid PreMean 0.607234043 0.561702Variance 0.014372618 0.01111Observations 47 47Pearson Correlation 0.689206844Hypothesized Mean Difference 0df 46t Stat 3.475995635P(T<=t) one-tail 0.000560995t Critical one-tail 1.678660414P(T<=t) two-tail 0.00112199t Critical two-tail 2.012895599
Not all Material on the Exam has been Covered in Class
Midterm Comparison
Pre Test Comparison
Comparison of Results for Material that has been Covered
Mid CoveredPre Covered
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Subscripts
Covere
d S
core
s
Boxplot of Covered Scores
Comparison of Covered Material
0.90.80.70.60.50.40.3
12
10
8
6
4
2
0
0.90.80.70.60.50.40.3
Pre Covered
Frequency
Mid CoveredMean 0.5785StDev 0.1252N 64
Pre Covered
Mean 0.6516StDev 0.1174N 53
Mid Covered
Histogram of Pre Covered, Mid CoveredNormal
Does ANOVA Indicate there was improvement?
Anova: Single Factor
SUMMARYGroups Count Sum Average Variance
Pre Covered 64 37.02632 0.578536 0.015686Mid Covered 53 34.53333 0.651572 0.013785
ANOVASource of Variation SS df MS F P-value F crit
Between Groups 0.154648 1 0.154648 10.43065 0.001616 3.923599Within Groups 1.70503 115 0.014826
Total 1.859678 116
Comparison of Results for Material that has not been Covered
Mid Not CoveredPre Not Covered
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Subscripts
Sco
res
Boxplot of Scores
Comparison of Material Not Covered
Does ANOVA indicate the Exam was harder?
Anova: Single Factor
SUMMARYGroups Count Sum Average Variance
Pre Not Covered 64 31.83333 0.497396 0.01785Mid Not Covered 53 27.5 0.518868 0.024926
ANOVASource of Variation SS df MS F P-value F crit
Between Groups 0.013367 1 0.013367 0.635003 0.427168 3.923599Within Groups 2.420698 115 0.02105
Total 2.434065 116
Is the Exam Taking Process Capable?
Control Charts with Minitab
• Students were emailed a Microsoft Excel Workbook with the Mid-Term data set
• It was heavily suggested that students purchase the Minitab academic license and bring their laptops to class.
• Students then divided themselves into groups around those who purchased the software and created the analysis control charts on the preceding slides.
Hypothesis Testing Exercises• In week 8 students were introduced to the hypothesis tests covered in
CSSGB BoK– Z Test– Student T– Two Sample T (known variance)– Two Sample T (unknown variance)– Paired T Test– ANOVA– Chi Squared T– F Test
• Students were emailed a data set containing both the Pre-Test and Mid-Term data and asked to perform each of the listed test using either Minitab or Microsoft Excel. The emphasis was placed on the conclusions from the data
Confidence Intervals
• Not all students took the Mid-Term that took the pre-test.
• This enabled students to utilize inferential statistics to draw conclusions about the population parameters (mean and variance particularly)
• By using the class data set provided students were able to calculate their confidence in the overall population parameters for the average test score as well as the standard deviation of the entire class
Was the Pre-Test a Predictor of the Mid Term Scores?
Improve-Control“Improve” and “Control” phase represent a small fraction of the material covered on the ASQ CSSGB exam
Within the Body of Knowledge there are the following :
• Gantt Chart• Activity Network Diagrams• Critical Path Method• Program (or Project) Evaluation and
Review Technique.
Final Exam Analysis
Exam Scores
Doesn’t Look Normal
It’s Bi-Modal!
Did the scores Improve?
Was The Difference Significant?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre 64 35.78 0.559063 0.012082
Mid 47 28.54 0.607234 0.014373
Final 40 30.43 0.76075 0.020084
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 1.029282 2 0.514641 34.534 4.91E-13 3.057197
Within Groups 2.205562 148 0.014902
Total 3.234844 150
Individual Improvement
Variable N N* Mean StDev Minimum Q1 Median Q3Change 36 0 0.1939 0.1419 -0.0600 0.0675 0.2000 0.2875
Was the Individual Improvement Significant?
t-Test: Paired Two Sample for Means
Final Pre
Mean 0.750556 0.556667
Variance 0.019743 0.010023
Observations 36 36
Pearson Correlation 0.342582
Hypothesized Mean Difference 0
df 35
t Stat 8.199954
P(T<=t) one-tail 5.8E-10
t Critical one-tail 1.689572
P(T<=t) two-tail 1.16E-09
t Critical two-tail 2.030108
Where there Hard Questions?
Pareto Chart on Topic
Count 3 3 2 2 2 1 1 1Percent 20.0 20.0 13.3 13.3 13.3 6.7 6.7 6.7Cum % 20.0 40.0 53.3 66.7 80.0 86.7 93.3 100.0
Question Topic
FMEA
Contro
l Cha
rts
Confide
nce Inter
val
Team
s
Proc
ess C
apab
lityEr
ror
Hypo
thes
is
Basic
Stats
16
14
12
10
8
6
4
2
0
100
80
60
40
20
0
Count
Perc
ent
Pareto Chart of Question Topic
Initial Process Capability
Final Process Capability
Results
• Ruba Amarin• Margit Barot• Miriam Bicej• Matthew Brown• Salem El-Nimri• William Ewart• Elizabeth Fuschetti• Robert Gaglione• Thomas Hansen• Tarun Jada• Javier Jaramillo• Michael Kagan• Anoop Krishnamurthy • Timothy Lin
• Helen Liou• Rebecca Marzec• Charles Ott• Sneha Patil• Eugene Reshetov• Matthew Rodis• Thomas Schleicher• Dante Triana• Albert Tseng• Bond Wann• Paul White• Sun Wong• Shih Yen• Jacob Ziegler
28 out of 37 Students that took the June 2nd Exam Passed the June 2nd Exam
Nationally 788 out of 1160 individuals passed the exam
Was the Result Significant?
Rutgers ASQ
Results
• Students test scores improved on average 19.4%
• 76% of Students Passed the exam compared to 68% National Average
• Increased ASQ Princeton Membership by 62 members
• Largest Ever Fund Raiser for the Rutgers IIE
86
Added Benefit
• From the funds generated by the course Rutgers was able to send 21 Students to the national IIE Conference in Orlando (shown above)
87
It took a team
• Nate Manco– ASQ Princeton Education Chair
• Richard Herczeg – ASQ Princeton Section President
• Jeff Metzler– Rutgers IIE President
• Dr. James Luxhoj– Rutgers Industrial and Systems Eng
• Brandon Theiss– Instructor
• Cindy Ielmini– Rutgers Industrial and Systems Eng
Lessons Learned
• Using the passing the exam process as a class example for the implementation of the tools and techniques of Six Sigma is an effective methodology
• There is demand for teaching Six Sigma in an academic setting
• The joint venture between Rutgers and ASQ is feasible and mutually beneficial.
• Having a diverse student population increases the overall performance of the group.
• Students need to be adequately qualified to sit for ASQ exam prior to taking the course.
89
We are sharing the Results
• Presented results at Institute of Industrial Engineers Lean and Six Sigma Conference
• Will be presented at the ASQ International Conference on Quality
90
Progress continues onward
• Course Scheduled to Run again in the Spring through Official Continuing Education Office
• First of its kind joint meeting with ASQ Princeton and Rutgers IIE in which the course results were presented.
Questions?
• Contact info– Brandon Theiss– [email protected]– Connect to me on LinkedIn