Sem-Tesis-2.pdf

8/11/2019 Sem-Tesis-2.pdf

1/6

Learning of content knowledge and development of scientific reasoningability: A cross culture comparison

Lei Baoa

Department of Physics, The Ohio State University, Columbus, Ohio 43210

Kai FangDepartment of Physics, Tongji University, Shanghai 200092, China

Tianfang CaiDepartment of Physics, Beijing Jiaotong University, Beijing 100044, China

Jing WangDepartment of Physics, The Ohio State University, Columbus, Ohio 43210

Lijia YangDepartment of Physics, National University of Defense Technology, Hunan 410073, China

Lili CuiDepartment of Physics, University of Maryland at Baltimore County, Baltimore, Maryland 21250

Jing Han and Lin DingDepartment of Physics, The Ohio State University, Columbus, Ohio 43210

Ying LuoDepartment of Physics, Beijing Normal University, Beijing, 100875, China

Received 16 April 2008; accepted 7 August 2008

We report the results of our study of the connections between students learning of physics content

knowledge and the development of general scientific reasoning abilities. In particular, we seek to

determine whether and to what extent content learning affects the development of general reasoning

abilities. Pre-college-instruction data of first-year college students in the United States and China

were collected using the FCI, BEMA, and Lawsons classroom test of scientific reasoning. We find

that the rigorous learning of physics knowledge in middle and high schools has made a significant

impact on the ability of students in China to solve physics problems, but this knowledge does not

seem to have a direct effect on their general scientific reasoning ability, which is determined to be

at the same level as that of the students in the United States. 2009 American Association of PhysicsTeachers.

DOI: 10.1119/1.2976334

I. INTRODUCTION

In most traditional education settings it is expected thatconsistent and rigorous content learning will help developstudents general reasoning abilities. However, there has notbeen much widely tested and accepted empirical evidence for

the interactions between content learning in physics and sci-ence and mathematics in general and the development ofgeneral reasoning abilities, although a few recent studieshave started to address students scientific reasoningabilities.

1,2Several assessment studies have suggested a posi-

tive correlation between students abilities in scientific rea-soning and mathematics and their achievement of content

learning in physics.

35

A theory on learning dynamics hasbeen proposed that provides a framework for understandingand modeling the interactions between general ability andcontent learning.

6,7The theory also predicts several relations

that can be empirically tested.The possible interactions between content learning and

general reasoning abilities involve at least three basic ques-tions: Does content learning affect the development of gen-eral reasoning abilities? Does training in reasoning affectcontent learning? How does the interaction occur in contextand is affected by the context? In this paper we report on theresults of study on the first question.

The research design makes use of the differences that existnaturally among different education systems. Students fromdifferent regions and education systems often go through a

very different curriculum in science and mathematics during

their middle and high school years.

Physics courses start in China in the 8th grade and con-

tinue through grade 12, providing 5 years of consistent train-ing on most topics in introductory physics. The courses are

algebra based with a serious emphasis on developing con-ceptual understanding and the skills to solve challengingproblems. As a result, college level physics courses in China

usually assume that students have developed a good under-standing of most introductory-level physics concepts and put

more emphasis on calculus-based physics problem solvingskills.In contrast, education in physics in the United States is

more flexible. High school students may choose to take onesemester of physics and can also take more advanced APphysics courses. The total course time and the emphasis onconceptual development and problem-solving skills are verydifferent compared to that in China.

In addition to curriculum structure the environment forscience teachers is also very different. In the United Statesthe supply of well qualified science teachers is limited inmany states. It is not uncommon for a high school science

1118 1118Am. J. Phys. 77 12, December 2009 http://aapt.org/ajp 2009 American Association of Physics Teachers


2/6

teacher to teach several different science courses. This ver-satility may sometimes limit the capability of a teacher tomaster the teaching of a specific discipline.

In China several science topics are regularly taught inmiddle and high school. A non-exhaustive list includes phys-ics 5 years, chemistry 34 years, geology 56 years,

biology and health science 56 years, and several one- or

two-semester courses in technology and engineering areassuch as computers and electronics, depending on local sup-port. Teachers of these science courses are very specialized.

Except in schools in some under-developed areas, it is notcommon for a chemistry teacher to teach a physics course orvice versa.

Most of the science teachers in China graduate with bach-elor andmore recentlymaster degrees from the correspond-ing departments of a normal university, which is a compre-hensive university with an emphasis on producing qualityteachers. Usually, more than half of the graduates from nor-mal universities become K12 teachers. The level of under-graduate education on content knowledge in a discipline de-partment of a normal university is equivalent to therequirements of the same department at any of the compre-hensive universities in China. Therefore, most school teach-ers in urban areas are well equipped with the necessary con-

tent knowledge and pedagogical skills. Thanks to theculturally based tradition of emphasis on education and thewide range of investments in the education infrastructure,China produces about ten million high school graduates an-nually. These students go through the national college admis-sion examination and approximately half of them continuewith some kind of higher education, although only a fewpercent will enter the top tier universities.

Because of the strong competition, Chinese students gothrough rigorous instruction in problem solving in all subjectareas throughout most of their K12 school years. Thesestudents are good at solving complicated content-based prob-lems. The question of whether this education can bring thesestudents to a high level of general skill and ability that goes

beyond the specific content areas and problem types is atopic of much interest that hasnt been well studied.

In this paper we compare first year college students in theUnited States and China in their knowledge of introductorymechanics, electricity and magnetism E&M, and in theirscientific reasoning abilities based on their performance onstandardized tests of physics concepts and scientific reason-ing abilities. The results provide a baseline for further quan-titative and qualitative studies and initial evidence on howthe middle and high school curricula might affect measuresof student content knowledge and reasoning ability.

II. RESEARCH DESIGN

The decision to compare American and Chinese studentswas made not only because the student population in Chinais an interesting and less studied group, but also because themiddle and high school science and mathematics curricula inChina are consistently controlled and enforced by a detailednational standard, which is much different from that in theUnited States. This difference provides a convenient researchdesign, in which we treat the education of physics knowl-edge in middle and high schools as an independent variable.

Although the primary variable is the teaching and learningof physics content knowledge, the interpretation of the re-sults can be extended to content knowledge in science and

mathematics in general. If there exists any effect from thetraining of content knowledge on the development of scien-tific reasoning abilities, such an effect should be consideredas the result of learning in science and mathematics in gen-eral.

We chose physics because it is a core part of the sciencecurricula, it is conceptually and logically sophisticated, and itis often considered to be related to scientific thinking ability.Another reason is that there are readily available standard-ized research-based instruments to probe student understand-

ing of physics concepts.If student performance on standardized tests in math andother science topics can be compared, the first year collegestudents in China should score better on these content ori-ented tests, because these students have to score well in re-lated topics on the very challenging college entrance exami-nation. An example of the level of math problems can befound in a recent BBC report.

8

A. Research context

Data were collected from one U.S. and two Chinese uni-versities. All are major comprehensive universities of similarranking in their respective countries. We will refer to the two

Chinese universities as C1 and C2, and the U.S. university asU1. All the students tested are first year science and engi-neering majors enrolled in calculus-based introductory phys-ics courses. The tests were administered before any collegelevel instruction of the relevant content topics.

Three standardized tests were used, including the FCI,9

BEMA,10

and Lawsons scientific reasoning test Lawsontest.

11The students in China used the Chinese versions,

which were translated by physicists fluent in both languages.The translated versions were also piloted with a small groupof undergraduate and graduate students N20 to remove

language issues.The FCI and BEMA are well known instruments in the

physics education community. The Lawson test is less

widely used and is designed to probe student ability to applyaspects of scientific and mathematical reasoning, includingconservation of mass and volume, proportional reasoning,control of variables, probabilistic reasoning, correlationalreasoning, and hypothetico-deductive reasoning.

11A brief

description of the test questions of the three instruments isincluded in the Appendix.

B. Data collection

Test data were collected from over 3000 students. The FCIdata from U1 are an accumulated pool collected from 2005to 2007. The rest of the data were collected during 2007.Table I lists the population groups tested.

III. RESULTS AND ANALYSIS

Student performance on the tests is summarized in TableII. There are significant differences on the FCI and BEMAbetween Chinese and U.S. students. In contrast, the perfor-mance on the Lawson test is almost identical with the per-formance of the U.S. students being slightly higher. The de-tails of the statistical analysis are given in TableIII.

The data show statistically significant variations of the twopopulations on the FCI and BEMA. To determine whether astatistically significant difference is a difference of practical

1119 1119Am. J. Phys., Vol. 77, No. 12, December 2009 Bao et al.


3/6

concern, we use a statistical measure, the effect size, to

evaluate the size of any observed effects. The effect size isusually computed as the ratio between the difference of two

means and the pooled standard deviation of two

populations.12

An effect size of 0.8 or greater is often con-

sidered large, meaning that the two populations tested are

categorically different. By comparing the effect size, we seethat there are categorical differences in performance on the

tests of physics content knowledge. In contrast, the perfor-

mance of the two populations on the test of more general

reasoning abilities is similar. This result indicates thatcourses on content knowledge do not have an obvious impact

on student development of general scientific reasoning abili-ties.

To see the differences in more detail the distribution of

scores of the U.S. and Chinese students on the three tests is

plotted in Figs.13.The data for the FCI and BEMA results

from the two Chinese universities are combined. From theFCI results shown in Fig. 1 we see that the American stu-

dents have a rather flat distribution in the middle range

0.250.75 starting from the chance level, which is about1

4of the total score. This result is consistent with the educa-

tion system in the U.S., which produces students with di-

verse experiences in physics learning: some had AP physicsand had studied most of the mechanics concepts, some had

regular elective physics courses and had studied some basic

ideas and elements at different levels, and some had no ex-

posure to a formal physics course.

The Chinese students had all completed an almost identi-

cal nationally required physics curriculum spanning 5 yearsfrom the 8th grade to the 12th grade. This type of back-

ground produces a narrow distribution that peaks near the

90% score.For the BEMA test see Fig. 2 the American students

have a narrow distribution centered a bit above the chancelevel, indicating that most students didnt have much expo-sure to E&M concepts in high school. The Chinese students

also scored lower than they did on the FCI, with the scoredistribution centered around 70%. The lower BEMA score ofthe Chinese students might be due to the fact that E&M is a

more difficult subject compared to mechanics and that someof the topics in the BEMAfor example, Gauss law are nottaught in the high school curriculum in China.

In the Appendix we see that all the topics in the FCI areincluded in the Chinese middle- and high-school physics cur-ricula, which are introduced at grade 8 and repeated at ahigher level at grade 10. Most of the content topics tested inthe BEMA are introduced in grade 9 and repeated at a higherlevel in grades 10 and 11.

In summary, the FCI and BEMA results imply that courses

in middle and high schools on the related physics contentdirectly impact student performance on these tests. Consis-tent and rigorous training can raise students to a fairly high

performance level, and a flexible course structure with vary-ing degree of emphasis produces a population distributedover a wide range of performance levels.

The results of the Lawson test show a completely differentpattern. The distributions of the Chinese and American stu-dents are almost identical. The results suggest that the largedifferences between the education systems in the U.S. andChina do not seem to cause much variation in the scientificreasoning abilities measured by the Lawson test.

Table I. Sample sizes of students taking the different tests.

Universities N Tests

U1 1950 FCI

235 BEMA

646 Lawson

C1 212 FCI

211 BEMA

206 Lawson

C2 122 FCI

120 BEMA

Table II. Results of the FCI, BEMA, and Lawson tests.

FCI test results 30 items

Classes U1N=1950 C1N=212 C2N=122

Mean 14.8 49.5% 25.585.0% 26.6 88.7%

Median 14.0 46.7% 26.086.7% 27.0 90.0%

Std. Dev. 5.8 19.3% 4.214.0% 3.311.0%

BEMA test results 31 items

Classes U1 N=235 C1N=211 C2N=120

Mean 9.7 31.3% 19.964.2% 21.1 68.1%

Median 9.0 29.0% 21.067.7% 22.0 71.0%

Std. Dev. 2.9 9.4% 4.112.6% 3.812.3%

Lawson test results 24 items

Classes U1 N=646 C1N=206

Mean 18.3 76.3% 17.974.6%

Median 19.0 79.2% 19.079.2%

Std. Dev. 4.2 17.5% 3.815.8%

Table III. Statistical analysis of student performances on the FCI, BEMA,

and Lawson tests. The p-value is computed based on a two-tailed t-test and

the effect size is computed with the Hedges method see Ref.12 for details

on the p-value and effect size evaluation.

C1 versus U1 C1 versus C2

FCI p-value 0.000 0.009

Effect size 1.89 0.28

BEMA p-value 0.000 0.008

Effect size 2.89 0.30

Lawson p-value 0.201

Effect size 0.10

Table IV. Correlations between test results on the Lawson test, FCI, and

BEMA.

Classes LawsonFCI LawsonBEMA FCIBEMA

C1N= 80 0.12 0.17 0.70

U1 N=111 0.20



4/6

To further explore the possible connection between educa-tion backgrounds and student development of content knowl-edge and reasoning abilities, we collected data on the threetests from students in one class in a Chinese university C1.The correlations between student scores on different tests areshown in TableIV. We also included one result from a U.S.class for the correlation between student test scores of theLawson test and the FCI. The correlations between scores on

the Lawson test and the FCI or BEMA are small. A largecorrelation is observed between student scores on the FCIand BEMA. This result provides further evidence that stu-dent learning of content knowledge in current education set-tings as measured by the FCI or BEMA does not have anobvious connection to their development of general reason-ing ability as measured by the Lawson test. Similar educa-tion on content knowledge has produced highly correlatedlearning outcomes in the content areas of mechanics andE&M. These results are consistent with the theoretical pre-

dictions about the correlation between measures of generalability and content knowledge.6

Note that this study does not address the question ofwhether and to what extent training of reasoning ability mayimpact content learning, which requires a completely differ-ent research design.

IV. DISCUSSION

Our results suggest that teaching and learning of contentknowledge in physics and science and mathematics in gen-eral in current traditional education settings doesnt affectthe development of general scientific reasoning abilities. Al-

though there are many additional uncontrolled variables suchas social and cultural factors, the result of this study is suf-ficiently robust so that uncontrolled variations shouldnt af-fect the conclusion. The fact that both the U.S. and Chinesestudents have almost identical performances on the reasoningtest indicates that on this ability measure, the effect of un-controlled variables does not seem to make a noticeable dif-ference.

From a more general perspective the performance differ-ences between content knowledge which is explicitlytaught and reasoning ability which is not directly taughtindicate that explicit training might have a powerful effect onstudent learning, although the effectiveness of this trainingmight vary with its format and the background of the student

population. A question for future study is whether directtraining of reasoning abilities will make a difference.

A possible limitation of this study is that the Lawson testis relatively simple. The average score from typical collegestudents is over 70%, suggesting possible ceiling effects. Amore detailed analysis of this instrument will be given infuture work. We can make a simple analysis of the ceilingeffects by comparing the distributions shown in Fig. 3. Wesee that the two populations in the lower half of the distri-bution are very similar. Because the scores of the lower halfof the students are far from the ceiling, the similarity of thesetwo subpopulations provides further evidence that bothpopulations are at about the same level regarding their per-formance on the Lawson test.

A historically held belief is that education in physics,which has a beautiful structure of logical and mathematicalrelations, improves the reasoning ability of students. Our re-sult suggests that the teaching of physics content in the tra-ditional format alone is not enough to improve student gen-eral reasoning ability. Therefore, we have to reconsider oureducational goals and methods, because the teaching formatand what is explicitly emphasized in courses appear to havea dominant effect on what students can gain from thesecourses. The result from this research and its implications arein support of current developments in education, which aimtoward the constructivism paradigm.

Fig. 1. FCI college entrance score distribution30 items of U.S. and Chi-

nese first year college students. The large difference between Chinese and

U.S. students shows the result of the different physics curricula in middle

schools and high schools in the two countries.

31

Fig. 2. BEMA college entrance score distribution 31 items of U.S. and

Chinese first year college students. The result is similar to the FCI result,

indicating that the physics curricula in middle schools and high schools had

made similar impact to students learning in mechanics and E&M.

24

Fig. 3. Lawson test college entrance score distribution 24 items of U.S.

and Chinese first year college students. The result shows that on the measure

of scientific reasoning the two populations perform almost identically, which

indicates that the differences in the current middle school and high school

curricula in physics or science and mathematics in general do not affect

students development of scientific reasoning.



5/6

Note added in proof.A short report on the general resultsof this study has been published in the journal Science. Thispaper provides additional details and in-depth discussions ofthe related research.

13

ACKNOWLEDGMENTS

We wish to thank all the teachers who helped with thisresearch. Special thanks to Zuyuan Wang, Junjian Mao, andYewen Zhang from Tongji University; Yingjian Qi, Commu-nication University of China; Su Yang, Beijing Jiaotong Uni-versity; Liming Li, Tsinghua University; and Tianxing Cai,Beijing Technology and Business University. We also thankthe anonymous reviewers whose careful comments signifi-cantly improved the paper. This work is supported in part

by the National Science Foundation under Grant Nos. DUE-0633473 and DUE-0618128. Any opinions, findings, andconclusions or recommendations expressed in this materialare those of the authors and do not necessarily reflect theviews of the National Science Foundation.

APPENDIX: PRE-COLLEGE PHYSICS

CURRICULUM IN CHINA

Middle and high schools students in China who choosescience or engineering as their majors and who plan to go toa university enroll for about 5 years in physics courses in-cluding 2 years of courses in the middle school and 3 yearsin high school. The curriculum is designed around six con-tent themes, which include force and motion, thermodynam-ics, electricity and magnetism, waves, optics, and nuclearphysics. Tables V and VI summarize the detailed content

Table V. Summary of content topics of the mechanics courses in Chinese

middle schools and high schools.

Categories Concepts

Kinematics in one

dimension

Reference frame, displacement

Velocity, average velocity, instantaneous

velocity, speed

Acceleration

Motion with uniform acceleration

Graphical analysis

Falling bodiesAcceleration due to gravity

Force Concept of force and vector

Force of gravity

Elasticity and deformation

Vector summation of forces

Kinetic friction and static friction

Newtons laws of motion Newtons first law of motion, inertia system

Newtons second law of motion

Newtons third law of motion

Simple application of Newtons laws

Units and SI system

Curvilinear motion Vector of velocity in curvilinear motionProjectile motion

General analysis of 2D motion

Uniform circular motion

Centripetal acceleration, centrifugal phenomena

Newtons law of universal gravitation

Momentum Momentum

Law of conservation of momentum

Center of mass

Work and energy Work

Kinetic energy

Potential energy

Mechanical energy and conservation laws

Simple harmonic

oscillations

Simple harmonic motion

Amplitude, period, frequency of oscillation

Energy in the simple harmonic oscillation

Simple pendulum

Forced vibration, resonance

Table VI. Summary of content topics of the E&M courses in Chinese

middle schools and high schools.

Categories Conce pts

Electric field Electric charges and charge conservation

Coulombs law

Electric field, field lines

Electric potential, electric potential energy

Parallel-plate capacitor, capacitance

DC circuit Ohms law, voltage, resistance

Phenomena of semiconductor and superconductor

Magnetism Magnetic field and field lines

Electric current and magnetic field

Amperes law

Electromagnetism Phenomena of electromagnetism

Faradays law of induction

Induction due to a moving conductor in magnetic field

Lenzs law

Phenomenon of self-induction

AC circuit Functions of resistance, capacitance and inductance in

AC circuit

Electric generator

Theory of transformer

Table VII. Content topics in FCIbased on the 1995 version.

Topics Items

Constant acceleration/free fall 1, 3

Projectile motion 2, 12, 14

Projectile motion/Force analysis 13

Newtons third law 4, 15, 16, 28

Circul ar mot ion/Newtons first l aw 5, 6, 7, 18

2-D motion/Newtons first law 8, 23, 24

Vector sum of velocity 9

Newtons first law 10

Newtons first law/Force analysis 11, 17, 25, 26, 27, 30

Kinematics/Motion diagrams 19, 20

2-D motion/Constant acceleration 21, 22

Force analysis 29



6/6

areas included in mechanics and E&M courses based on the2007 guidelines published by the Ministry of Education ofthe Peoples Republic of China.

For comparison the content domains addressed in BEMAand Lawson test are summarized in Tables VIIIX.

aElectronic mail: [email protected]

1A. Boudreaux, P. S. Shaffer, P. R. L. Heron, and L. C. McDermott,

Student understanding of control of variables: Deciding whether or not a

variable influences the behavior of a system, Am. J. Phys. 762, 163

1702008.2

E. Etkina, A. Van Heuvelen, Suzanne White-Brahmia, David T. Brookes,

Michael Gentile, Sahana Murthy, David Rosengrant, and Aaron Warren,

Scientific abilities and their assessment, Phys. Rev. ST Phys. Educ.

Res. 2, 020103-115 2006.3

D. E. Meltzer, The relationship between mathematics preparation and

conceptual learning gains in physics: A possible hidden variable in di-

agnostic pretest scores, Am. J. Phys. 7012, 125912682002.4

V. P. Coletta and J. A. Phillips, Interpreting FCI scores: Normalized

gain, reinstruction scores, and scientific reasoning ability, Am. J. Phys.

7312, 117211792005.5 R. R. Hake, Relationship of individual student normalized learning gains

in mechanics with gender, high-school physics, and pretest scores on

mathematics and spatial visualization, Physics Education Research Con-

ference, Boise, ID; August 2002; www.physics.indiana.edu/~hake/

PERC2002h-Hake.pdf.6

L. Bao, Dynamic models of learning and education measurement,

arXiv.org/abs/0710.1375.7

D. E. Pritchard, Y. Lee, and L. Bao, Mathematical learning models that

depend on prior knowledge and instructional strategies, Phys. Rev. ST

Phys. Educ. Res. 4, 010109-18 2008.8

BBC News report, Mathematicians set Chinese test, news.bbc.co.uk/

2/hi/uk_news/education/6589301.stm.9

FCI stands for Force Concept Inventory, which 1995 version is a

30-question multiple choice test covering basic concepts in Newtonian

mechanics. See D. Hestenes, M. Wells, and G. Swackhamer, Force con-

cept inventory, Phys. Teach. 30

3, 1411581992. The test used is the1995 version.10

BEMA is the short name for Brief Electricity and Magnetism Assess-

ment, which is a 31-question multiple choice test covering basic con-

cepts in introductory E&M. See L. Ding, R. Chabay, B. Sherwood, and

R. Beichner, Evaluating an electricity and magnetism assessment tool:

Brief electricity and magnetism assessment, Phys. Rev. ST Phys. Educ.

Res. 2, 010105-172006.11

A. E. Lawson, The development and validation of a classroom test of

formal reasoning, J. Res. Sci. Teach. 151, 1124 1978. Test used in

study: Classroom Test of Scientific Reasoning, revised ed. 2000.12

Ap-value of 0.05 or smaller indicates a statistically significant difference

between the mean scores of two populations. The effect size can be

approximately interpreted as the signal to noise ratio, in which the signal

is the difference between two means and the noise is the standard devia-

tion of the test results. A widely accepted guideline by Cohen defines 0.2

as small, 0.5 as medium, and 0.8 as large. If the effect size is larger than0.8, the two population groups tested are often considered categorically

different, meaning that they belong to two clearly distinguishable levels

categorieson an assessment scale. For more details on computing effect

size and guidelines on interpreting effect sizes, see L. V. Hedges and I.

Olkin, Statistical Methods for Meta-Analysis Academic, San Diego,

1985; and J. Cohen, Statistical Power Analysis for the Behavioral Sci-

ences Erlbaum, Hillsdale, NJ, 1988, 2nd ed.13

See L. Bao et al., Learning and Scientific Reasoning, Science

3235914, 586587 2009.

Table VIII. Content topics in BEMA.

Topics Concepts Items

Electric field Coulombs law 1, 2, 3

Electric field, field lines 4, 5, 6

Electric potential 14, 15, 16

Polarization of dielectrics 7

Gauss law 18

E-field in a conductor 19

Electric DC

current

DC circuit 10, 11, 17

RC circuit 13

Model of resistivity 12

Microscopic model of current 8, 9

Magnetism Magnetic field and field line 21, 22

B-field from current loops 24

Charge in B and E cross fields 26, 27

Magnetic force on a charge 20, 23

Force between two parallel currents 25

Force on moving chargesHall effect 30

Electromagnetic

induction

Induction 28, 29, 31

Table IX. Content topics in the Lawson test ver. 2000.

Topics Items

Conservation of mass and volume 1, 2, 3, 4

Proportional thinking 5, 6, 7, 8

Control of variables 9, 10, 11, 12, 13, 14

Probabilistic thinking 15, 16, 17, 18

Correlational thinking 19, 20

Hypothetico-deductive reasoning 21, 22, 23, 24


Sem-Tesis-2.pdf

Documents

Transcript of Sem-Tesis-2.pdf