The Ergonomics of Learning The Design and Usability of the Learning Environment Are Key to Student...

The Ergonomics of Learning

The Design and Usability of the Learning Environment Are Key to Student Learning

Outcomes

Thomas J. Smith, Ph.D., CHFPResearch Associate, Human Factors

School of KinesiologyCollege of Education and Human Development

University of MinnesotaMinneapolis, MN

WORLD USABILITY DAY---USABILITY PROFESSIONALS ASSOCIATION MINNESOTA CHAPTER & UNIV. OF MINNESOTA---11/10/11

Topics

• Definitions & Two Key Questions

• Purpose and Rationale

• Design Factors Reliably Shown to Have a Strong Influence On Student Academic Achievement

• Design Factors Shown to Have an Equivocal or Weak Influence On Student Academic Achievement

• Conclusions

• Usability Analysis of Learning Environments

• How Can We Introduce Principles and Practices of Ergonomics and Usability Into Education?


‘Ergonomics/Human Factors (E/HF)’

That branch of human science concerned with the interaction of human behavior and performance, and the

design of the performance environment

‘Educational Ergonomics’

That branch of E/HF concerned with the study of how and why student learning is influenced by the design of learning

environments


"The main challenge in the science of human learning is to understand the requirements of

educational design at all levels“

(K.U. Smith and M.F. Smith (1966). Cybernetic Principles of Learning and Educational Design. New York: Holt,

Rinehart & Wilson, p. 478)


What design factors define a ‘learning environment’?

• School Equipment Designs(i.e., desks, chairs, backpacks, uniforms, etc.)

• Job & Task Demands(i.e., learning tasks, teaching tasks. etc.)

• Workspace Designs(i.e., workstation layouts, classroom space designs, etc.)

• School Environments(i.e., of classrooms, laboratories, libraries, staffrooms,

playgrounds, plus physical environmental conditions of noise, lighting, heating/cooling, etc.)

• School Macroergonomics(i.e., curricula, learning/teaching methods & pedagogy,

school-day design and scheduling, study/PE schedules, rules, security, departmental structures, governance, policies, etc.)

• Learning Aids/Technology• (i.e., books, audiovisual systems, computers, internet, etc.)


Two Key Questions (with reference to K-12 education)

1. Does the design and usability of learning environments strongly influence learning outcomes of K-12 students (a

phenomenon termed context specificity of learning)?

YES!


Ergonomics of Learning - Summary of Effect of Different Design Factors on Student Learning

Ergonomic Design FactorLearning Effect –

Strength of Evidence

Class Size Mixed

Classroom Technology Mixed

Community Socioeconomics Strong

Cooperative Learning Strong

Early Childhood Education Strong

Homework Load Weak

Informal Learning Outside Classroom Strong

More Classroom Time Strong

Online Instruction Weak

Physical Environment of Classroom Strong


Ergonomics of Learning - Summary of Effect of Different Design Factors on Student Learning (continued)

Ergonomic Design FactorLearning Effect – Strength

of Evidence

Performance Standards for Students, Teachers & Administrators

Strong

Physical Activity/Fitness Level Strong

School District Size Weak

Student Exposure to Stress & Abuse Strong

Student Health & Nutrition Strong

Student Self-Esteem Strong

Student Sleep Quality & Quantity Strong

Teacher Degree Level Weak

Teaching Quality Necessary, but not Sufficient

Teacher Subject Matter Competency Strong


2. Do K-12 Educators Recognize the Critical Influence That Good Design has on Educational System

Performance and Student Learning Outcomes?

NO!

“Factors of human design long have been ignored in experimental psychology. It has been believed that

learning could be studied as a general process.” (K.U. Smith and M.F. Smith, 1966, p. 1)

“The application of HF/E principles and practices, and the implementation of ergonomics programs, have achieved proven success in improving performance, productivity,

competitiveness, and safety and health in most occupational sectors. However, the benefits that the application of HF/E science might bring to promoting student learning have yet to be widely recognized.”

(Smith, 2007)


Purpose and Rationale

• This presentation introduces evidence for the conclusion that different designs of learning environments underlie almost all proposed solutions for improving academic achievement of K-12 students.

• However, evidence regarding the impact of the usability of these different designs on student learning outcomes is limited.


Selected Design Factors Reliably Shown to Have a Strong Influence On Student Academic Achievement --- Summary of Supporting Evidence

• Cooperative Learning

• Early Childhood Education

• Environmental Design of School Classrooms

• Longer School Days and/or School Year

• Community Socioeconomic Status

• Teaching Quality


Early Childhood Education

(Mervis, J. (2011). Past successes shape effort to expand early intervention. Science, 333, p. 953)


Special Issue of Science Devoted to Investing Early in Education

(vol 333, 8/19/11, pp. 951-983)

• Hines et al., Laying the foundation for lifetime learning, p. 951

• Mervis, Past successes shape effort to expand early intervention, pp. 952-956

• Mervis, Giving children a head start is possible---but it’s not easy, pp. 956-957

• Mervis, A passion for early education, pp. 957-958

• Diamond & Lee, Inteventions shown to aid executive function development in children 4 to 12 years old, pp. 959-964

• Dickinson, Teachers’ language practices and academic outcomes of preschool children, pp. 964-967

• Clements & Sarama, Early childhood mathematics intervention, pp. 968-970

• Klahr, Zimmerman & Jirout, Educational interventions to advance children’s scientific thinking, pp. 971-975

• Barnett, Effectiveness of early educational intervention, pp. 975-978

• Gormley, From science to policy in early childhood education, pp. 978-981

• Shonkoff, Protecting brains, not simply stimulating minds, pp. 982-983


Environmental Design of School Classrooms

Classroom Type and General Cognitive Scores on McCarthy’s Scale (N = 75)

Mean General Cognitive Scores

Classroom Type Classroom Rating 1 Classroom Rating 2

3 year olds’ classroom 105.75 80.14

Note: Classroom rating: 1 = adequate or above, 2 = inadequate

(Maxwell, L.E. (2007). Competency in child care settings : the role of the physical environment. Environment & Behavior, 39, 229)


Findings regarding the beneficial effects of good environmental design of school classrooms on

academic performance of children are not yet as conclusive as more definitive findings from studies of

office workers

Table 2

(Leaman, A., and Bordass, B. (2006). Productivity in buildings: the ‘killer’ variables. In D. Clements-Croome (Ed.), Creating the productive workplace (2nd Ed.) (Chap. 10, p. 153). London: E&FN Spon.)


Table 2

Smith, T.J., and Orfield, S.J. (2007). Occupancy quality predictors of office worker perceptions of job productivity (pp. 539-543). In Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society)


Longer School Days and/or School Year

the ‘Summer Slide’

(St. Paul Pioneer Press, 9/14/11, p. 3D)


Longer School Days and/or School Year (continued)

(Star Tribune, 9/28/10, p. A4)


Longer School Days and/or School Year (continued)

• Success of KIPP (Knowledge is Power Program) schools is based on longer school days

• A KIPP school day goes from 0725 to 1700

• On average, KIPP kids spend 50-60 percent more time learning than kids in conventional schools

• In the Bronx KIPP school (almost all low income kids), 84 percent of 8 th-grade kids are performing at or above grade level

• This matches performance of 8th grade kids in wealthy suburban schools

• Today, more than 50 KIPP schools in U.S.

(Gladwell, M. (2008). Outliers. New York: Back Bay Books)


The positive effect of a longer school day on learning is readily understood from the perspective

of the law of practice

(Schmidt, R.A. (1988). Motor Control and Learning (pp. 458-459). Human Kinetics: Champaign, IL)


Community Socioeconomic Status


Basic skills test scores, averaged for both math and reading, for Minnesota 8 th graders by school district for the years 1996, 2002, 2006 and 2009, in relation to the percentage of low income students, for 49 different districts in the Minneapolis/St. Paul (Minnesota, USA) metropolitan area, based on publicly reported data.

How Do We Account for This Dramatic Effect of Community Socioeconomic Status on Student

Academic Achievement?

• Relative to wealthier communities, communities with a lower socioeconomic status are characterized by a series of cultural design conditions that have been shown to adversely affect student academic achievement:

- higher levels of student abuse and stress

- lower parental involvement in student schooling

- lower parental educational levels

- lower/negative peer support for student achievement

- less healthy student behavioral / lifestyle habits [i.e., nutrition (too much or too little); lower involvement in regular physical activity; substance abuse; violence]


Teachers Matter Most - Supportive Claims

• ‘A growing body of research shows that the quality of the teacher in the classroom is the most important schooling factor predicting student outcomes’ (Goldhaber & Anthony, 2004).

• ‘Budget, curriculum, class size---none has a greater effect on a student than his or her teacher’ (The Economist, 2011).

• The U.S. Federal No Child Left Behind (Bush)and Race to the Top (Obama) initiatives both emphasize teacher accountability as the key to improving schools.


Teachers Matter Most - Supportive Claims (continued)

• The basic assumption of the Bill and Melinda Gates Foundation (2010) is that provision of high quality teaching will ensure student success.

• The agenda of the National Commission on Teaching and America’s Future (NCTAF) (Darling-Hammond, 1997; NCTAF, 1996). rests on what this entity considers the single most important strategy for achieving America’s educational goals, namely recruiting, preparing and supporting excellent teachers.

• The international supremacy of Finnish schools is attributed to how they prepare and manage their teaching corps (Charlemagne, 2006; Kaiser, 2005).


Teachers Matter Most - A More Balanced Perspective

• Among the foregoing design factors shown to positively impact student performance, only use of cooperative learning strategies is under direct control of the teacher.

• Holding teachers accountable for mediating the impact on student learning of a series of design factors that they cannot control may be deemed at best ineffective, and more pointedly a recipe for failure on the part of teachers and students alike.

• This consideration supports a balanced conclusion that teaching quality represents a necessary, but not sufficient, contributor to student achievement.


Selected Design Factors Shown to Have an Equivocal or Weak Influence On Student Academic Achievement

• Class Size

• Informal Learning

• School Size

• Classroom Technology


Class Size Matters - Supportive Claims

http://www.classsizematters.org/


Class Size Matters - A More Balanced Perspective

(Biddle, B.J. & Berliner, D.C. (2002). What research says about small classes & their effects, San Francisco, CA: WestEd. (www.WestEd.org/ policyperspectives)

• Pro: This research leaves no doubt that small classes have an advantage over larger classes in reading and math in the early primary grades (Jeremy Finn & Charles Achilles,1990, p. 573)

• Con: There is no credible evidence that across-the-board reductions in class size boost pupil achievement (Chester Finn & Michael Petrilli, 1998, p. 2)

• Pro: Large reductions in school class size promise learning benefits of a magnitude commonly believed not within the power of educators to achieve (Gene Glass, Leonard Cahen, Mary Lee Smith, & Nikola Filby, 1982, p. 50)

• Con: This article has concentrated on the limited task of reviewing the evidence on ... reducing class size. The surprising finding is that the evidence does not offer

much reason to expect a systematic effect from overall class size reduction policies (Eric Hanushek, 1999, p. 158)

• Pro: Taken together, these studies ... provide compelling evidence that reducing class size, particularly for younger children, will have a positive effect on student

achievement (Dan Murphy & Bella Rosenberg — writing as representatives of The American Federation of Teachers, 1998, p. 3)

• Con: There’s no evidence that smaller class sizes alone lead to higher student achievement (Nina Shokraii Rees & Kirk Johnson — writing as representatives of the Heritage Foundation, 2000, p. 1)


Class Size Matters - General Conclusions by Biddle & Berliner (2002)

• Long-term exposure to small classes in the early grades generates substantial advantages for students in American schools, and those extra gains are greater the longer students are exposed to those classes.

• Evidence for the possible advantages of small classes in the upper grades and high school is so far inconclusive.


Informal Learning

The average K-12 student spends only 21% of waking hours in a school classroom


(Bransford, J.D., Brown, A.L., & Cocking, R.R. (Eds.) (2000). How People Learn. Brain, Mind, Experience, and School (p. 148).

Washington, D.C.: National Academy Press)

Informal Learning

Over the lifetime of the average American, only 5 percent of the time is spent in the classroom


(Falk, J.H., & Dierking, L.D. (2010). The 95 percent solution. School is not where most Americans learn most of their science.

American Scientist, 98, 486-493)

Informal Learning - General Conclusions by Falk & Dierking (2010)

• Our goal is to suggest that what goes on in the 95 percent of a citizen’s life may be equally important, and possibly more important, to increasing science literacy.

• We propose that it is time to seriously question whether, in the 21st century, schooling should continue to be viewed as the most important mechanism for advancing interest in and understanding of science.

• Insufficient data exist to conclusively demonstrate that informal learning experiences currently contribute more to understanding of science than in-school experiences, but a growing body of evidence points in this direction.

• There certainly are insufficient data to refute the claim that informal learning is vitally important.


Informal Learning – Closing Thought


“I never let my schooling interfere with my education”

Mark Twain

School Size


Average Math Scores for 5th Graders in Relation to Overall School Enrollment for 1,662 Pennsylvania Schools

Average Math Scores for 11th Graders in Relation to Overall School Enrollment for 1,662 Pennsylvania Schools

(Wainer, H. (2007). The most dangerous equation. American Scientist, 95(3), 249-256)

Classroom Technology - Positive Effects


•Potential positive effects of computer use by children include enhanced cognitive development, reduced barriers to social interaction, and enhanced fine motor skills and visual processing

(Straker, 2009, based on worldwide survey of children’s computer use)

(Science, 1/9/09, special section on education & technology)

Classroom Technology - Negative Effects


•Potential negative effects of computer use by children include inappropriate content, exposure to violence, bullying, internet ‘addiction’, displacement of moderate/vigorous physical activity, exposure to junk food advertising, sleep displacement, vision problems, and musculoskeletal problems (Straker, 2009, based on worldwide survey of children’s computer use)

•Regular computer use by children may compromise higher-order cognitive processes, namely those subserving abstract vocabulary, mindfulness, reflection, inductive problem solving, critical thinking, and imagination skills (Science, 1/9/09, special section on education & technology)

Classroom Technology - Negative Effects (continued)


Usability


The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or

component [IEEE 90]

Usability of Design Factors and Conditions That Impact Learning Outcomes of K-12 Students


What do we know about the usability of different design factors known to impact learning outcomes of K-12

students?

Have Methods and Techniques of Usability Analysis Been Applied to Ergonomic Design Factors Known to Impact K-12

Student Learning Outcomes

Ergonomic Design Factor Known to Impact Learning Outcomes Documented Record of

Usability Analysis

Class Size No

Classroom Technology Yes

Community Socioeconomics Limited Application of Community Ergonomics Concepts

Cooperative Learning No

Early Childhood Education No

Informal Learning Outside Classroom No

More Classroom Time No

Online Instruction Yes


Ergonomics of Learning - Summary of Effect of Different Design Factors on Student Learning (continued)

Ergonomic Design Factor Known to Impact Learning Outcomes

Documented Record of Usability Analysis

Physical Activity/Fitness Level No

Physical Environment of Classroom Yes

School District Size No

Student Health & Nutrition No

Teaching Quality No

Teacher Subject Matter Competency No


Conclusions


• The foregoing analysis supports five broad conclusions:

1. Student learning outcomes, and more broadly the edifice of K-12 education itself, are largely defined and structured in terms of an extensive system of design factors and conditions.

2. Nevertheless, the usability of most of these design factors (i.e., the ease which educators and students can interact with these factors) has not been explored.

3. The time is long overdue for the educational system to acknowledge the central role of both ergonomic design factors, and of the usability of these factors, as major influences on student academic achievement.

4. K-12 educators and administrators should recognize and accept the key principle of evidence-based investment in usable learning environment designs.

5. That is, they should emphasize allocation of resources to the positive impact design factors addressed above, but should treat investment in factors with equivocal or weak impact with more caution and skepticism.

What Strategies Can be Deployed to Achieve These Goals?


• Ergonomic Programs for schools, school districts, and educational systems

• Usability Analysis of the designs developed and deployed by these programs

Relative Strengths and Shortcomings of Ergonomic Program Intervention versus Usability Analysis as

Alternative Strategies for Improving the Design Quality of K-12 Learning Environments

A Speculative Evaluation


Macroergonomic Consideration

Prediction for Ergonomic Program

InterventionPrediction for

Usability AnalysisBarriers to implementation Intermediate to high Low to intermediate

Likelihood of involvement by students in design analysis Low Low to intermediate

Likelihood of involvement by teachers in design analysis Intermediate Intermediate to high

Likelihood of involvement by institutional managers/ administrators in design analysis

Intermediate to high Intermediate to high

Likelihood of acceptance by students Intermediate High

Likelihood of acceptance by teachers High Intermediate

Likelihood of acceptance by institutional managers/administrators High Low

Likelihood of long-term institutional investment Intermediate High

Macroergonomic demands on institution Intermediate to high Low

Type of professional expertise needed Professional ergonomist Usability professional

Likelihood of system health and safety benefits High Low to Intermediate

Likelihood of educational systems performance benefits Intermediate to High Intermediate to High

Likelihood of student learning benefits Intermediate High

Cost Intermediate Low

Likelihood of achieving a favorable cost/benefit return Low to intermediate Intermediate


How Can We Effectively Promote Recognition and Acceptance of Ergonomic Design of Learning

Environments, and of Usability Analysis of These Environments, In a Systematic and Comprehensive

Manner, by the K-12 Educational Community?

A Guerilla Strategy for Educational Ergonomics and Usability Analysis


Key Principles of a Guerilla Ergonomics and Usability Strategy for Educational Systems (with apologies to Phil Jacobs)

• Ignore academicians in colleges of education• Adopt a bottom-up approach• Build networks of support and identify key allies,

preferably at School District levels• Focus on end users, namely teachers and students ---

this may entail intervention despite management• Build tactical teams of these end users• Avoid the “E” and “U” words --- emphasize positive

performance outcomes such as better learning, more effective and efficient systems performance, improved health and safety, and lower costs

• The bottom line --- successful ergonomic and/or usability interventions in education may require guerilla tactics


Questions?


The Ergonomics of Learning The Design and Usability of the Learning Environment Are Key to Student...

Documents

Transcript of The Ergonomics of Learning The Design and Usability of the Learning Environment Are Key to Student...