Jan Alexis NielsenDepartment of Science EducationUniversity of Copenhagen

Presentation at the ESERA Summer School, June 26, 2018, Jyväskylä, Finland

A challenge for practice and a topic for research.

Generic competences in science teaching.

Generic Learning Goals

Policy-makers (in many parts of the world) increasingly call for science teaching to foster student learning and competence development related to:

• Inquiry• Modelling• Argumentation • Technology • Engineering • Innovation• Entrepreneurship• Communication• Science Practices, • 21st century skills

27/06/2018 2

Learning goals that fall under these areas can be called generic: they are targets of teaching in multiple disciplines or subjects, and there is often no school discipline that they belong exclusively to.

Key definitions

A generic learning goal is a target of teaching in multiple disciplines or subjects, and does not exclusively belong to any one school discipline.

A competence is a readiness to act based on relevant knowledge and skills in a way that meets the challenges of a given situation (cf. Blomhøj & Jensen, 2003).

27/06/2018 3

Example 1: Socioscientific Argumentation Competence

27/06/2018 4

Disciplinary knowledge/skills

Other relevant knowledge/skills

Societal issues

Paraphrased learning goal:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills

27/06/2018 5

Physics

Biology

Chemistry

Geograhy

Uni-disciplinary: Generic goal is woven

into the disciplines

In Denmark (year 10-12), SSI argumentationcompetence is a goal of most (science) disciplines on par with core disciplinary skills/knowledge

Example 2: Inquiry Competence

27/06/2018 6

Formulate questions

Plan and carry out investigations

Analyse data

Communicate and discuss results

Disciplinary knowledge/skills

Paraphrased learning goal:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.

27/06/2018 7

Physics

Biology

Chemistry

Geography

Cross-disciplinary:Generic goal is explicitly

cross-disciplinary

In Denmark (year 7-9), inquiry competence is one of four overarching competence goals for all science disciplines – these goals complement disciplinary knowledge/skills

27/06/2018 8

Physics

Biology

Chemistry

Geography

Cross-disciplinary:Generic goal is explicitly

cross-disciplinary

Physics

Biology

Chemistry

Geograhy

Uni-disciplinary: Generic goal is woven

into the disciplines

Activity 1:From your perspective as researchers, what do you think could challenge the inclusion of the two learning goals in the science disciplines in a curriculum?

It may help to talk about how a teacher could teach for these learning goals in your own discipline.

One in each group types in your ideas on www.menti.comuse code 86 64 70

27/06/2018 9

SSI argumentation:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills.

Inquiry:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.

Main thematic challenges in the research

27/06/2018 10

Conceptual unclarity: It is unclear (even to researchers) what these goals actually mean

Primacy of content: A focus on disciplinary content often eclipses a focus on these goal

Issues about assessment: These goals may be difficult for teaches to assess (formatively and summatively)

Main thematic challenges in the research

27/06/2018 11

SSI argumentation Inquiry

Conceptual unclarity: It is unclear (even to researchers) what these goals actually mean

Primacy of content: A focus on disciplinary content often eclipses a focus on these goal

Issues about assessment: These goals may be difficult for teaches to assess (formatively and summatively)

Research on SSI argumentation: Conceptual unclarity

27/06/2018 12

• Many scholars have focused on whetherstudents use science evidence in SSI argumentation; but it is beginning to emerge that SSI argumentation should be treated as being much richer (Nielsen, 2013b, 2017)

• Students can use science as strategic rhetoric devices or to ‘win the argument’ (Nielsen, 2011; 2012).

• We lack an operational concept of SSI argumentation that outlines what it means to balance facts and values in legitimate ways.

Changes from somatic gene therapy are not heritable

Factual premise

We should allow somatic gene therapy!

Value-laden statement

It is OK to make changes to the body that are not heritable

Value premise

Research on SSI argumentation: Primacy of content

Discipline Discipline

Socioscientific issues

Socioscientific Issues

Disciplinary content

Competences

Scenario 1 (the curriculum)

Scenario 2 (our respondents)• SSIs are often implemented in a

superficial fashion (e.g. Lazarowitz & Bloch, 2005).

• Teachers’ narratives often reflect a content-centred understanding of SSI – where SSIs merely play an instrumental role in motivating the disciplinary content (Tidemand & Nielsen, 2017).

Research on Inquiry: Conceptual unclarity

27/06/2018 14

• While ‘inquiry’ has been one of the most central notions of science education for at least 20 years, inquiry is not close to being uniformly defined

• A recent review identified no less than 19 distinct skills/sub-competences that are regularly associated with inquiry competence in multifarious combinations (Bernholt et al., 2013).

www.assistme.ku.dk 15 October 2013 31

Table 9: Scheme for the evaluation of the literature

Column Content Literature author(s) General information about the investiga-tion/ analysis

year country design (Survey, Intervention, Evaluation, Case Study, Meta-analysis) domain (Science, Technology, Mathematics) sample(s) size (N) sample characteristics: grade (school type) sample characteristics: age

Content focus of the investigation/ analysis (either as focus of the intervention/learning environment/curricula or as focus of the assessment)

scientific inquiry/science process skills diagnosing problems/ identifying questions searching for information considering alternative or multiple solutions creating mental representations constructing and using models formulating hypotheses planning investigations constructing prototypes finding structures or patterns researching conjectures collecting and interpreting data evaluating results searching for alternatives/ modifying designs constructing and critiquing arguments or explanations/ argumentation/ reasoning/ using evidence debating with peers/ communication searching for generalizations dealing with uncertainty knowledge/ achievement/ understanding/ conceptual change problem solving other

(Bernholt et al, 2013)

Research on Inquiry: Primacy of content

• In inquiry-based teaching the teacher-role becomes complex and focus is on guiding students through processes (e.g. Crawford, 2000).

• The complexity can lead teachers to stick to a role of covering content when teaching inquiry (e.g. Nielsen, 2017).

MotivatorGuide

Diagnostician

Innovator

ExperimenterResearcher

Modeler

Mentor

Collaborator

Learner

Teacher roles in inquiry teaching, after Crawford (2000)

Main thematic challenges in the research

27/06/2018 16

Socioscientific argumentation competence

Inquiry competence

Conceptual unclarity Tendency to forget that SSI argumentation involves much more than the use of disciplinary knowledge

Multifarious definitions and descriptions of a wide variety of subordinate skills

Primacy of content SSI is often used as a means to an end (framing disciplinary content), rather than an end in itself

Uncertainty about the role of teacher-as-guide may lead teachers to focus on ‘delivering’ content

Issues about assessment

27/06/2018 17

Student activity

Data

Judgement of level

Report of level

Summative

Next step in learning

Formative

(after Harlen, 2013)

Assessment

Justification in terms of assessment criteria

Assessment: one of the most important factors in education

• Washback effect: Over time the goings-on in teaching will become more and more aligned with what is being measured at in summative assessment (Harlen, 2007).

27/06/2018 18

Activity 2:Choose one of the two learning goals. From your perspective as researchers, what are your ideas about how to summatively assess the chosen learning goal?

Discuss what you think the key assessment criteria are and then come up with ideas for which kind of student activity is best suited for assessing the learning goal.

One in each group types in your ideas on www.menti.comSSI: code 26 14 39Inquiry: code 76 88 51(you can write more than one idea)

27/06/2018 19

SSI argumentation:Students should be able to engage in argumentation and make informed decisions about societal issues by drawing on relevant knowledge and skills.

Inquiry:Students should be able to formulate questions, plan and carry out investigations, analyse data, and communicate and discuss results by drawing on relevant knowledge and skills.

Research on SSI argumentation: Issues about assessment

• Many science teachers feel unable to handle and give feedback on students’ SSI discussions (Bryce & Gray, 2004)

• There is a general lack of effective assessment criteria for assessing SSI argumentation (Levinson & Turner, 2001);

• Examinations rarely test SSI argumentation validly (if at all) (Tidemand & Nielsen, 2017).

• Most assessment frameworks for SSI argumentation are constructed for assessing written arguments – and would not lead to valid assessment of dialogic argumentation (Nielsen, 2013b)

27/06/2018 21

Research on SSI argumentation: Issues about assessment

Student activity

Data

Judgement of level

Report of level

Summative

Next step in learning

Formative

Justification in terms of assessment criteria

Lack of support for teachers to translate the competence to criteria

$

Typical summative assessment activities are not ideal for this competence

Lack of guides for teacher-judgement

Research on Inquiry: Issues about (summative) assessment

• Large body of research into summative assessment of inquiry competences using multiple-choice, constructed-response or open-ended items (Bernholt et al., 2013).

• But it can be argued that this type of activity offers only a reduced vision of students’ inquiry learning – primarily focusing on student understanding (e.g. Hume & Coll, 2010)

www.assistme.ku.dk 15 October 2013 63

Figure 2: Formative assessment item on dominance relationships (Hickey & Zuiker, 2012, p. 24)

To assess students’ understanding of key concepts, concept maps instead of items are often used for a summative assessment. For example, Brandstädter, Harms, and Großschedl (2012) investigate concept maps as an assessment tool for system think-ing in biology education. As the process of the concept map development is quite com-plex, some approaches use computer-assisted methods (e. g. Schaal, Bogner, & Gir-widz, 2010).

On the other hand, concept maps can be used for formative assessment. In this case, the focus lies on checking students’ progress in understanding key concepts at several times during a treatment (e. g. Furtak et al., 2008). The analysis of concept maps can be organised by rubrics as shown in Table 29 (e. g. Nantawanit, Panijpan, & Ruen-wongsa, 2012).

In general, it is important to train students in the procedure of making a concept map (Nantawanit et al., 2012). One possible way is the think-pair-share method: First, stu-dents make an individual map, then, they build a map in a small group, and finally, they construct a concept map as a class (e. g. Furtak et al., 2008). Another common method is to give the concepts and linking words to the students (see Figure 3). Both ap-proaches have a more formative than summative character.

(Hickey & Zuiker, 2012, p. 24)

Research on Inquiry: Issues about (summative) assessment

• It may be, that using performanceassessment is the best way to secure valid summative assessment of inquiry competence (e.g. Barron & Darling-Hammond, 2008).

• But performance assessment is more complex and time-consuming – and requires substantial support for teachers (Bernholt et al., 2013).

http://ictflwinterfest-2012.wikispaces.com/file/view/ATM_cartoon.jpg

Research on Inquiry: Issues about (formative) assessment

• We have knowledge about good practices for formative assessment for inquiry – e.g. using structured whole-class discussions as assessment conversations (Ruiz-Primo & Furtak, 2007).

• But even experienced teachers need support in translating inquiry competence into assessment criteria that suits their practice (Nielsen, Tidemand & Dolin, 2018).

• Knowledge of good practices are only slowly being implemented in national assessment systems (Nielsen, 2017).

thinking requires the teacher to judge differences between students’ responses, explanations, ormental models so the critical dimensions relevant for their learning can be made explicit (e.g.,teacher compares students’ responses according to the evidence provided or responds to studentsby askingwhich explanation is more scientifically acceptable based on the information provided).Using information from assessment conversations involves taking action on the basis of studentresponses to help students move toward learning goals (e.g., helping students to reach consensuson a particular explanation based on scientific evidence). The range of student conceptions atdifferent points of a unit should determine the nature of the conversation; therefore, more than oneiteration of the cycle of eliciting, recognizing, and using may be needed to reach a consensus thatreflects the most complete and appropriate understanding.

Assessment conversations can be thus be described in the context of classroom talk moves(Christie, 2002) as ESRU cycles—the teacher Elicits a question; the Student responds; the teacherRecognizes the student’s response; and then Uses the information collected to support studentlearning.2 This model of informal formative assessment is depicted in Figure 1.

We distinguish the ESRU model from the IRE/F sequence in three ways: First, the elicitingquestions used to initiate a sequence have the potential for providing information on the evolvingstatus of student conceptions and understanding about scientific inquiry skills and habits of mind.For example, teachers can ask questions about understanding key points (e.g., clarificationquestions about vague statements or lifting questions to improve the level of discussion). Second,recognizing students’ responses in diverse ways (e.g., revoicing, rephrasing) is consideredfundamental because it indicates to the student that his/her contribution has been heard andaccepted into the ongoing classroomnarrative (Scott, 1998). Recognizing students’ responses alsoprovides the opportunity for the teacher to pull out the essential aspects of student participationand to act on them, and also provides the student the opportunity to evaluate the correctness of theteacher’s interpretation of their contribution.

Third, our conception of using comes from the formative assessment and the scientific inquiryliterature (Black & Wiliam, 1998; Clarke, 2000; Duschl, 2000, 2003; Duschl & Gitomer, 1997;National Research Council, 2001; Ramaprasad, 1983; Sadler, 1989, 1998). Use, in our sequence,is more specific than the meaning assigned to feedback in the IRF sequence. Within ‘‘use,’’ ateacher can provide students with specific information on actions they may take to reach learninggoals: ask another question that challenges or redirects the students’ thinking; modelcommunication; promote the exploration and contrast of students’ ideas; make connectionsbetween new ideas and familiar ones; recognize a student’s contribution with respect to the topicunder discussion; or increase the difficulty of the task at hand. Clearly, evaluations by themselves(e.g., ‘‘good’’ or ‘‘excellent’’) cannot be part of our ESRU pattern unless they are embedded in

Figure 1. The ESRU model of informal formative assessment.

INFORMAL ASSESSMENT PRACTICES 61

Journal of Research in Science Teaching. DOI 10.1002/tea

Ruiz-Primo & Furtak’s (2007, figure 1, p. 61) ESRU model of structuring formative assessment in classroom conversations.

27/06/2018 25

Research on Inquiry: Issues about assessment

Student activity

Data

Judgement of level

Report of level

Summative

Next step in learning

Formative

Justification in terms of assessment criteria

Multifarious skills that need to be translated into criteria

$

Typical summative assessment activities are not ideal for this competence

Teachers need substantial support, may increase in performance assessment

Main thematic challenges in the research

27/06/2018 26

Socioscientific argumentation competence

Inquiry competence

Conceptual unclarity Tendency to forget that SSI argumentation involves much more than the use of disciplinary knowledge

Multifarious definitions and descriptions of a wide variety of subordinate skills

Primacy of content SSI is often used as a means to an end (framing disciplinary content), rather than an end in itself

Uncertainty about the role of teacher-as-guide may lead teachers to focus on ‘delivering’ content

Issues about assessment There is a lack of effective formative and summative assessment practices

We have knowledge of good formative assessment practice, but changing assessment practices requires support

Generic learning goals face challenges in practice

27/06/2018 27

Primacy of content Issues about assessment

Conceptual confusion

Future research needs to investigate the direction of the relations in order to provide strategies for professional development solutions.

How to welcome a generic learning goal to the curriculum!

27/06/2018 28

Combining ideas from two research projects on generic learning goals, may be a path to follow

“Upper secondary school rethought” (innovation competence) (Inquiry competence)

How to welcome a generic learning goal to the curriculum!

Conceptual clarity: Experienced teachers formulate overarching assessment criteria for “new” generic learning goals together with researchers and other stakeholders (Nielsen, 2015a).

Micro-level insight into how the learning goal can be operationalised into constructs that can inform teaching and assessment (Nielsen, Tidemand & Dolin, 2018)

27/06/2018 29

Assessment practice: Experiencedteachers systematically implement different assessment methods for assessing the generic learning goal (Dolin, Nielsen & Tidemand, 2017).

Enables deep teacher understanding of criteria and assessment practice for the learning goal (Nielsen, 2015b)

Solid basis for changing wider assessment culture so as to better accommodate the learning goal (Nielsen, Tidemand & Dolin, 2018).

Thank you very much!

janielsen@ind.ku.dk

ReferencesAbd El Khalick, F., Boujaoude, S., Duschl, R. A., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A., . . . Taylor, P. C. (2004). Inquiry in science

education: International perspectives. Science Education, 88(3), 397–419.Barron, B. & Darling-Hammond, L. (2008). Teaching for meaningful learning: A review of research on inquiry-based and cooperative learning. In L.

Darling-Hammond, B. Barron, P. D. Pearson, A. H. Schoenfeld, E. K. Stage, T. D. Zimmermann, G. N. Cervetti, & J. Tilson (Eds.), Powerful Learning. What we know about teaching for understanding. San Francisco: Jossey-Bass.

Bernholt, S., Rönnebeck, S., Ropohl, M., Köller, O., & Parchmann, I. (2013). Report on current state of the art in formative and summative assessment in IBE in STM-Part 1. ASSIST-ME Report Series(1).

Blomhøj, M., & Jensen, T. H. (2003). Developing mathematical modelling competence: conceptual clarification and educational planning. Teaching Mathematics and its Applications, 22(3), 123-139. doi:10.1093/teamat/22.3.123

Christenson, N., & Chang Rundgren, S.-N. (2015). A Framework for Teachers’ Assessment of Socio-scientific Argumentation: An example using the GMO issue. Journal of Biological Education, 49(2), 204-212. doi:10.1080/00219266.2014.923486

Crawford, B. A. (2000). Embracing the essence of inquiry: New roles for science teachers. Journal of Research in Science Teaching, 37(9), 916–937.Day, S. P., & Bryce, T. G. K. (2010). Does the Discussion of Socio-Scientific Issues require a Paradigm Shift in Science Teachers' Thinking?

International Journal of Science Education, 33(12), 1675-1702. doi:10.1080/09500693.2010.519804Dolin, J., Black, P., Harlen, W., & Tiberghien, A. (2018). Exploring Relations Between Formative and Summative Assessment. In J. Dolin & R. Evans

(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy (pp. 53-80). Cham: Springer International Publishing.

Dolin, J., Nielsen, J. A., & Tidemand, S. (2017). Evaluering af naturfaglige kompetencer. Acta Didactica Norge, 11(3), 1-28. Duschl, R. A. (2007). Quality Argumentation and Epistemic Criteria. In S. Erduran & M. P. Jiménez-Aleixandre (Eds.), Argumentation in Science

Education (pp. 159-175): Springer Netherlands. Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and Quasi- Experimental Studies of Inquiry-Based Science Teaching: A

Meta-Analysis. Review of Educational Research, 82(3), 300–329. Harlen, W. (2007). Assessment of Learning. London: Sage Publications Ltd.Harlen, W. (2013). Assessment and Inquiry-Based Science Education: Issues in Policy and Practice. Trieste: IAP.Hickey, D. T., & Zuiker, S. J. (2012). Multilevel Assessment for Discourse, Understand- ing, and Achievement. Journal of the Learning Sciences, 21(4),

522–582.Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and Achieve- ment in Problem-Based and Inquiry Learning: A Response to

Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.Hume, A., & Coll, R. K. (2010). Authentic student inquiry: The mismatch between the intended curriculum and the student-experienced curriculum.

Research in Science & Technological Education, 28(1), 43–62.

27/06/2018 31

ReferencesIrwin, A., & Wynne, G. (1996). Misunderstanding science. Cambridge: Cambridge University Press.Kolstø, S. D. (2001). ‘To trust or not to trust, . . .’- pupils’ ways of judging information encountered in a socio-scientific issue. International Journal

of Science Education, 23(9), 877-901.Lazarowitz, R., & Bloch, I. (2005). Awareness of Societal Issues Among High School Biology Teachers Teaching Genetics. Journal of Science

Education and Technology, 14(5-6), 437-457. doi:10.1007/s10956-005-0220-4Lee, H., & Witz, K. G. (2009). Science Teachers' Inspiration for Teaching Socio-scientific Issues: Disconnection with reform efforts. International

Journal of Science Education, 31(7), 931-960. doi:10.1080/09500690801898903Levinson, R. (2006). Towards a Theoretical Framework for Teaching Controversial Socio-scientific Issues. International Journal of Science Education,

28(10), 1201-1224. doi:10.1080/09500690600560753Levinson, R., & Turner, S. (2001). The teaching of social and ethical issues in theschool curriculum, arising from developments in biomedical

research: a research study of teachers. London: The Welcome Trust.Lindahl, M. G., & Linder, C. (2013). Students’ Ontological Security and Agency in Science Education—An Example from Reasoning about the Use of

Gene Technology. International Journal of Science Education, 35(14), 2299-2330. doi:10.1080/09500693.2011.618516Millar, R., & Osborne, J. (1998). Beyond 2000: science education for the future. London: King's College London.Nielsen, J. A. (2011). Co-opting Science: A preliminary study of how students invoke science in value-laden discussions. International Journal of

Science Education, 34(2), 275-299. doi:10.1080/09500693.2011.572305Nielsen, J. A. (2013a). Dialectical Features of Students’ Argumentation: A Critical Review of Argumentation Studies in Science Education. Research in

Science Education, 43(1), 371-393. doi:10.1007/s11165-011-9266-xNielsen, J. A. (2013b). Delusions About Evidence: On Why Scientific Evidence Should Not Be the Main Concern in Socioscientific Decision-Making.

Canadian Journal for Science, Mathematics, and Technology Education, 13(4), 373-385. doi:10.1080/14926156.2013.845323Nielsen, J. A. (Ed.) (2017). Litteraturstudium til arbejdet med en national naturvidenskabsstrategi. København: Institut for Naturfagenes Didaktik.Nielsen, J. A., Tidemand, S., & Dolin, J. (2018). Transforming Assessment Research: Recommendations for Future Research. In J. Dolin & R. Evans

(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy. (pp. 279-290). Cham: Springer.Nordenbo, S., Allerup, P., Andersen, H. L., Korp, H., & Dolin, J. (2009). Pædagogisk brug af test – en systematisk review. København: Danmarks

pædagogiske universitetsforlag og Dansk Clearinghouse for Uddanelsesforskning.Orlander Arvola, A., & Lundegård, I. (2011). ‘It’s Her Body’. When Students’ Argumentation Shows Displacement of Content in a Science Classroom.

Research in Science Education, (Advance online publication), 1-25. Retrieved from http://dx.doi.org/10.1007/s11165-011-9237-2 doi:10.1007/s11165-011-9237-2

Ratcliffe, M., & Grace, M. (2003). Science education for citizenship: teaching socio-scientific issues. Maidenhead: Open University Press.

27/06/2018 32

ReferencesRuiz-Primo, M. A., & Furtak, E. M. (2006). Informal Formative Assessment and Scien- tific Inquiry: Exploring Teachers' Practices and Student

Learning. Educational As- sessment, 11(3-4), 205–235.Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring Teachers' Informal Formative As- sessment Practices and Students' Understanding in the

Context of Scientific Inquiry. Journal of Research in Science Teaching, 44(1), 57–84.Rönnebeck, S., Nielsen, J. A., Olley, C., Ropohl, M., & Stables, K. (2018). The Teaching and Assessment of Inquiry Competences. In J. Dolin & R. Evans

(Eds.), Transforming Assessment: Through an Interplay Between Practice, Research and Policy. (pp. 27-52). Cham: Springer.Sadler, T. D., & Zeidler, D. L. (2003). The morality of socioscientific issues: Construal and resolution of genetic engineering dilemmas. Science

Education, 88(1), 4-27. doi:10.1002/sce.10101Simonneaux, L. (2014). Questions Socialement Vives and Socio-scientific Issues: New Trends of Research to Meet the Training Needs of Postmodern

Society. In C. Bruguière, A. Tiberghien, & P. Clément (Eds.), Topics and Trends in Current Science Education: 9th ESERA Conference Selected Contributions (pp. 37-54). Dordrecht: Springer Netherlands.

Tidemand, S., & Nielsen, J. A. (2017). The role of socioscientific issues in biology teaching: from the perspective of teachers. International Journal of Science Education, 39(1), 44-61. doi:10.1080/09500693.2016.1264644

Zuiker, S., & Whitaker, J. R. (2014). Refining Inquiry with Multi-Form Assessment: Formative and summative assessment functions for flexible inquiry. International Journal of Science Education, 36(6), 1037-1059. doi:10.1080/09500693.2013.834489

27/06/2018 33