BERA Representations Paper · Web viewPreliminary analysis of the products has shown that the...

17:23 24/05/23

Draft, currently under revision

Pupil Attainment and the New Technologies:

some methodological design issues in the Impact2 Project

Di Mavers, Bridget Somekh, Peter Scrimshaw, Colin Harrison, Kay Hawe, Tony Fisher and Cathy Lewin

Paper presented at the British Educational Research Association Conference, Cardiff University, 7-10 September 2000.

Abstract

The ImpacT2 Project is designed to look at the ways in which using the new networked technologies in home and school affect pupil attainment. The study uses both qualitative and quantitative methods and covers some 2000 pupils in 60 primary and secondary schools.This paper looks at the theoretical origins of, and rationale for, the qualitative research methods being used. In particular we discuss the use of concept mapping to explore teacher and pupil conceptualisations of ICT, and how the roles of teachers and pupils as co-researchers have been built into the design.

Key words: ICT, Action Research, Qualitative research methods, concept-mapping, images

Impact2: Antecedents The origins of much of the approach to qualitative research in Impact2 can be traced back a considerable distance. In his seminal book ‘An Introduction to Curriculum Research and Development’ Lawrence Stenhouse explored the role of the teacher, both in terms of what that role actually was at that time and also what it could and arguably should become. He suggested that:

‘…the outstanding characteristics of the extended professional is a capacity for autonomous professional self-development through systematic self-study, through the study of the work of other teachers and through the testing of ideas by classroom research procedures.’ (L. Stenhouse (1975) p144)

This characterisation, and the exploration of the topic in his book, of course left much room for further debate. In that early stages however the action research movement in the UK showed a substantial (but not exclusive) preference for:

of 19

17:23 24/05/23

The use of qualitative rather than quantitative methods of data collection and analysis

Control of the research resting with teachers and/or researchers rather than with funders or any other outside agencies

Researching non-standard forms of pupil learning and teacher activity rather than, for instance, mainstream subject teaching or easily examinable forms of attainment

The research being conceptualised and presented in terms of practice rather than theory.

By the mid-80s, several distinctly different versions of educational action research could be identified (Elliot 1988), a situation which has continued since. While the Impact2 project in some respects fits the original profile summarised above, like several recent projects it represents a distinct evolution from Stenhouse’s position, rather than a straightforward continuation of it.

In methodological terms, Impact2 has two largely distinct components, one quantataive (which we will not consider here) and the other qualitative. The latter has itself grown out of the experiences of Impact2 team members on other projects. Of these we will briefly consider the PALM project, the SLANT project and the REPRESENTATIONS project.

The PALM project was a qualitative study of ways in which teachers could use computers to promote pupil autonomy. This was a classic example of a non-standard research topic, where conventional methods of pupil assessment would have little relevance. The emphasis upon qualitataive methods also fitted the profile. The project approach to the teacher/researcher relationship was to move away from Elliot’s position, in which the researchers acted as enablers for the teachers (Elliot 1988), towards a model in which researchers and teahcers acted more as co-learners. This transition was not fully achieved, because as Somekh (xx 1995?) has pointed out the existence of separate teacher and researcher discourses was a major obstacle to establishing such a relationship.

The SLANT project took a rather different form. It was directed towards exploring the effects of computers upon the classroom talk of teahcers and pupils in primary and middle schools. As its main publication indicates (Wegerif and Scrimshaw 1997) this ESRC-funded project was from the start conceptualised in a strongly theoretical way. Its methodology was based upon an underlying neoVygotskian theory of teaching and learning. Over this was constructed both an explicit formulation of how this theory helped to explain classroom practices (Edwards and Mercer 1987) and suggested strategies for curriculum evaluation and development (Mercer and Scrimshaw 1993). The project, by its later stages was also departing from the ‘standard’ action research paradigm in another way too, by systematically and selectively employing both qualitative and quantitative methods of data collection and analysis (Wegerif 1997). This development has since been taken further in another evaluation project (Lewin, Scrimshaw, Mercer and Wegerif, 2000) and the methodological implications explored in more detail (Scrimshaw and Lewin 2000).

of 19

17:23 24/05/23

The strong theoretical emphasis in practice meant that the SLANT project was strongly researcher-led, although the teachers very largely selected the specific topics they wanted to research in their individual classrooms, and some of them also became authors or co-authors of articles about the project (e.g. Scrimshaw and Perkins 1997, Dawes 1997, Watson 1997).

Finally, in designing the ImpacT2 evaluation, we were also able to draw upon the experience of the REPRESENTATION Project1, funded by the EU Multimedia Task Force, 1998-2000. Like ImpacT2, REPRESENTATION was concerned with the impact of ICT on children’s learning.

The Representation project aimed: to benchmark and track the developments in children’s representations of

computers and computer systems to develop a computer-based concept-mapping tool which could be used to

explore these new modes of representation. (based on Representation, 1998, p. 2, ‘Goal and Rationale’)

REPRESENTATION used a form of action research derived from Lewin’s early model which incorporated: reconnaissance by external researchers; hypothesis development; intervention; and testing/validation. The research design, as set out in the contractual work programme, did not include teachers’ involvement in identifying research questions, collecting data, analysing and interpreting that data or writing reports. In practice, teachers in the six countries with participating schools played different roles in REPRESENTATION depending on the research traditions of the lead university, but in no country was their involvement comparable with that of teachers in the PALM project. It was not intended to be so, and there was not sufficient funding to support teachers in undertaking their own research. Nevertheless, REPRESENTATION was unusual within the Multimedia Task Force projects for its close involvement with schools and teachers. The focus of all the empirical research was the collection of data in schools, and this was fed back to the software developers and informed the process of developing a new computer-based concept mapping tool. This close involvement with schools meant that there was flexibility in the research methods, which were adapted to suit the needs of the education systems in each country. For example, in the UK, the project worked with teachers in ways that would assist them in developing work for their pupils with the new National Grid for Learning and developing their own ICT competences to meet government targets.

The Impact2 Project The action research aspects of Impact2 in part grew out of our experiences on these earlier projects, and others.

1 Representation is co-ordinated by Kathy Kikis-Papadakis, FORTH, Institute of Applied and Computational Mathematics, Greece. The partners are INRP, France; Orfeus, Denmark; MAC, Ireland; University of Crete, University of Amsterdam, University Open of Catalunya and University of Huddersfield. Associated partners are IUFM de Creteil, France; and the University of Mons-Hainaut, Belgium.

of 19

17:23 24/05/23

ImpacT2 was commissioned to ‘evaluate the impact of ICT on pupil attainment’ because government wanted to know the measurable outcomes from extensive investment of public monies in ICT, in particular in setting up the National Grid for Learning and resourcing schools to use it in teaching. In a preliminary study, the team addressed the methodological problem of establishing the relationship between networked technology and attainment (McFarlane et al. 2000). It was proposed to focus upon ‘two parallel strands of development’:

The first strand (…) explores ways of using established tests and instruments to examine the relationship between networked technology and attainment (…) The second strand is more exploratory. (…) There are some fundamentally new empirical and conceptual challenges involved in attempting to identify a causal relationship between networked technology and attainment. (op cit. P. 4)

Strand two learning was assumed to be different in its nature from strand one learning, being more exploratory, self-directed and often situated outside formal educational settings. When exposure to networked technologies impacts on this kind of learning the outcomes may not be reflected in standardised tests of pupils’ attainment. Nevertheless, it is in gains in this kind of learning that ICT may have the potential to make its most significant contribution to education. Here with the two strands and their different research methodologies, Impact2 is in some respects similar to SLANT.

To investigate ‘strand two learning’, ImpacT2 adopted a Socially Contextualised Integrated Model of Learning (see Appendix One). Based on this model, it was ‘suggested that a concept mapping task could go some way to providing an instrument with which we could explore pupils’ knowledge and learning in this area, and that such a task could be used as both a baseline and an outcome measure’. This technique we will discuss in some detail.

Using concept mapping to assess non-traditional learning

ImpacT2 was able to draw on the REPRESENTATION Project for experience in using such a concept mapping task. Preliminary work during the reconnaissance phase of REPRESENTATION developed a paper-based concept mapping instrument and tested it with one class of ten year old pupils in Calderdale, West Yorkshire, in July 1999 (Crawford et al. 1999). A refined version of this paper-based instrument was then used in all six countries to benchmark 10-12 year old pupils’ representations of ICT in November 1999, and re-administered to track developments in their representations in June 2000 (Pearson and Somekh 2000). An analytical procedure was used to ‘score’ the concept maps, based upon the work of Hogg, Johnston and Crawford (Hogg et al. July 1997) and other phenomenographic research (Marton 1994).

Concept mapping is a way of externalising internal cognitive processes by making them visible in a concrete representation. Novak and Gowin first suggested concept mapping as a means of assisting learning (Novak and Gowin 1984). Drawing on Ausubel’s theory about the importance of ‘advance organisers’ in providing structures for learning

of 19

17:23 24/05/23

(Ausubel and Robinson 1969), concept mapping was at first seen as a way of enabling learners to develop propositions as part of the process of knowledge construction. On the other hand, Buzan used concept mapping, or mind mapping, as a tool for reflection and metacognition. He refers to ‘the infinite patterns, the infinite Maps of the Mind’ and to the concept map (or Mind Map) as ‘your external mirror of your own radiant thinking (that) allows you to access this vast thinking powerhouse…’ (Buzan 1993).

The notion that individual minds store knowledge and understanding in their own unique body of interlocking schema, suggests that learning is a process of making connections between new concepts and existing mental schema and in this way constructing knowledge through active engagement rather than receiving it and storing it through a more passive process of assimilation. In more recent learning theories, within which the ImpacT2 SCIM-L model is subsumed, internal mental representations and human behaviour are seen as integrally related to one another, as part of activity systems that also involve interaction with other people, so that the process of learning is situated in a concrete setting rather than disembodied as a separate mental process (Brown, Collins et al. 1989; Lave and Wenger 1991, p. 33; Engeström 1999). However, the basic notion that internal representations are unique to individuals remains unchallenged. We are part of larger inter-locking systems, inter-relating with others, drawing on the patterns of behaviour and traditions of our culture and history; but we are also individuals making sense of our experience, constructing what we know and are coming to know, through our own internal representations (Salomon 1993).

Concept maps are particularly useful as a way of exploring the acquisition of knowledge and understanding about a new tool, or set of tools in the case of ICT. Activity theory draws on the work of Vygotsky to locate meaning making in the process of mediated action. All human actions are mediated in the sense that we could not accomplish them without access to primary artifacts, such as wheels, levers, tin openers, pens, books and computers, and secondary artifacts, such as language, symbols and signs, and a large number of cognitive procedures through which we order and organise our mental processes (Wertsch 1998). Cole builds on Wartofsky’s analysis (Wartofsky 1979) to suggest that secondary artifacts also include mental representations of physical objects (primary artifacts) which enable us to predict or imagine how they can be used, so that our use of the tool depends not simply on having access to it, but also in knowing in advance how it might be used. Lack of well developed secondary artifacts of ICT would, for example, be likely to prevent individuals or groups from using them competently and confidently even when they are available for use. In an extension of this idea, Wenger (1998, pp. 57-62) talks about the importance of moving from the abstract to the concrete, through a process that he calls ‘reification’. As used by Wenger, reification means the construction of a representation or secondary artifact of an abstract concept to make it usable in mediated action. Concept maps provide external mirrors of this internal cognitive process of reification. Although concept maps are often produced by using words to represent concepts and linking these with verbs to create propositions, another way of producing concept maps is to draw images or icons that represent objects and link these merely by drawing lines. This form of concept mapping is particularly useful in capturing an individual’s current understandings of complex inter-related concepts. They

of 19

17:23 24/05/23

do not provide an exact picture of what is in the mind, because like all concept maps they are ‘dynamic not static’ (Jonassen et al. 1998), but they reveal a general overview of the extent and breadth or an individual’s knowledge. In Project REPRESENTATION these non-verbal concept maps proved to be very useful in revealing pupils’ representations of ICT.

The concept mapping tasks given to pupils in ImpacT2, as in the REPRESENTATION Project, are specifically for the purpose of gaining access to pupils’ representations of their knowledge and understanding. We want to know the extent and sophistication of pupils’ secondary artifacts of ICT, because effective use of ICT for learning depends upon the purposeful actions and imaginings that use these secondary artifacts as mediating tools. This makes these tasks rather different from the concept mapping described by some others. Andersson and Ditson (1999) describe concept mapping as a process that enables teachers and students to ‘negotiate meaning’, building directly on Novak’s work in which he puts forward concept mapping as a tool to support learning. Buzan’s work, on the other hand, suggests mind mapping as a metacognitive tool that assists learners in externalising their own learning to make it accessible for reflection. In ImpacT2 we are asking pupils to produce concept maps of their understanding of ICT so that we can (a) gain an understanding of their existing knowledge and feed this back to teachers and policy-makers; (b) predict from our analysis of their concept maps which pupils are likely to make significant learning gains, using ICT tools, which may not be identified from traditional testing methods; and (c) track the development of their understanding of ICT over time, so that we can discern trends in the growth of their understanding and relate these to what we know of their experience with ICT during the life of the project.

The concept mapping task was administered for the first time in June-July 2000 and will be repeated by the same pupils after a year. We will thus be able to compare the concept maps of three cohorts of pupils from 20 schools as they move from year 6 to 7, year 8 to 9, and year 10 to 11. The purpose of repeating the task is not to provide an exact measurement of knowledge gains, but to collect snapshots of developments in pupils’ thinking. In their original work, Novak and Gowin (1984, pp. 94-95) looked at the validity of conventional multiple choice tests by carrying out post-test interviews with children and asking them to explain their reasons for selecting answers. They found that the tests were not giving valid results because the children’s reasons for ticking items bore little resemblance to the expectations of the adults who had constructed the tests. Concept mapping was developed as a result of this work, in order to gain better insights into children’s thinking. Teachers found it particularly useful to ask children to produce concept maps on a topic, prior to beginning to teach it, because in this way they were able to understand the wide range of existing knowledge among the children and think of ways to introduce the topic that would best build upon this existing knowledge. ImpacT2 concept maps will, in like manner, inform our understanding of the kind of curriculum and teaching that pupils need to make best use of ICT for learning. For example, pupils who already have sophisticated mental representations of ICT would be likely to find the current National Curriculum for ICT dull.

of 19

17:23 24/05/23

The concept mapping tasksThe concept mapping tasks were administered to pupils by the teacher researchers in each school. They were introduced to concept mapping at the two day residential conference which provided their induction to ImpacT2. The introduction consisted of the teacher-researchers and the university-based researchers all producing concept maps of our mental representations of computers in our world. In a single one and a half hour session there was time for an introduction, supported by a written handout, the production of our concept maps and some final discussion of how the task might be received by pupils. The maps were collected for analysis. The conference took place just before the Easter vacation and teacher-researchers received the concept mapping instruments for the different age groups and instructions/ support materials when they returned to school in early May.

The task for year 5 pupils was preceded by some instructions about concept mapping (which they had already practised with their teachers in advance) and was then presented by the teachers using the following script:

Now here is exactly what I want you to do. We want to know your ideas about Computers in Today's World. So that will be the title of your Mind Map. When we were your age computers were hardly spoken about. Today it is very different. We want you to think about your world and all the types of computers within it. Where are computers placed at home … at school … in the outside world … or in the work place? Are they connected in any way? Think of all the people who use them. And why they use them. What would you say was the most simple computer system, and the most complex computer system you can think of? Take a minute or two to think before you start drawing. (Suggest that children might close their eyes and put their heads down on their folded arms for this thinking time.)

The script of the preliminary instructions also included the following:

This is not a test. We want to know what you think, not what you have learnt at school, or what the person sitting next to you thinks. Please draw what is in your own mind. You can draw things that are imaginative and fun as well as things you know. Don’t forget, the idea is to draw quickly to communicate your ideas, and not to worry about making the drawings perfect.

The tasks for year 8 and year 10 pupils was very similar, except that year 10 pupils were given the option of producing text-based maps: ‘For the ‘nodes’ of your concept map you can either draw an object or write a word in a box to represent an object’. Year 10 pupils, like the teachers, were also given an information sheet giving a brief explanation of the psychological theory of learning that underpins concept mapping. We felt this was important in gaining their interest in the task and would be likely to improve the quality of the data.

A number of methodological issues arose in relation to the conduct of the concept mapping task. All of them related to more general issues relating to the role of the

of 19

17:23 24/05/23

teachers-researchers in ImpacT2, in particular the limitations to their autonomy resulting from the evaluation methods and the role of the university-based researchers in managing the power relations between themselves and the teachers. These issues will be dealt with more fully in the next section. In relation to concept mapping we want to discuss three issues here:

First, it was necessary to give teachers a script and ask them to administer the task formally, reading the script word for word. This was important in making sure that all the pupils did the same task, so that differences in the contents of the map or the way in which it was drawn could be ascribed to differences in the pupils’ representations rather than resulting from ‘leads’ given to them on the day. Teachers accepted this readily and the preparation at the conference appeared to have been adequate because there is no evidence from the concept maps that the procedure was not followed.

Second, there was some discussion about whether or not older pupils should be asked to draw pictures as opposed to producing concept maps made up of textual labels. The evidence from the REPRESENTATION Project had suggested that ten year old pupils very much enjoyed drawing image-based concept maps. The accompanying written task showed no evidence of pupils being inhibited by being asked to draw, whereas there was a small number of cases in which drawing had allowed pupils with poor literacy skills to present their ideas much more fully. We wanted to replicate this sense of fun and enjoyment, particularly as imaginative ideas which might not be factually accurate would enable us to understand the pupils’ understanding of the potential role of ICT in the future. This had to be balanced against the view of some teacher-researchers in secondary schools who felt that such a task would be too childish for pupils of aged 13 or 15. In the end, a compromise was reached and the 10 year old and 13 year old pupils were asked to produce image-based maps while the 15 year olds were asked to produce text-based maps, with the option of drawing. Preliminary analysis of the products has shown that the image-based concept mapping task provides a much clearer picture of secondary representations of ICT, since ICT consists of concrete objects being used by people in actual locations, such as banks, schools, or private houses, that can easily be represented in iconic drawings. Text-based maps contain abstract concepts that provide additional affective information (e.g. anxiety) but transform the task into something completely different. It will, therefore, not be posible to draw any comparisons between the year 10 maps and those produced by pupils in years 5 and 8. An interesting outcome of this whole debate about image-based versus text-based concept maps has been to clarify our thinking about the implications of these two different forms of communication. The assumption of teachers, who are presumably representative of the majority of educated adults, appears to be that text-based communication is more serious, more adult, more informative and more subtle; whereas, the image-based concept maps produced by pupils in both years 5 and 8 display all of those qualities. In our current society, with its enormous emphasis upon communication through images in a range of media, and the new mix of text and both moving and still images in web-sites on the Internet, young people are not being inducted into the same assumptions that privilege the written word. We would certainly have simplified our data analysis and might have reaped other benefits in terms of focus and clarity if we had asked pupils of all ages, as well as the

of 19

17:23 24/05/23

teachers, to produce image-based concept maps.

Third, there was the issue of whether or not teachers should be asked to produce concept maps as well as pupils, and if so whether they should be asked to produce image-based or text-based maps. The REPRESENTATION concept mapping data suggested that 10 year old children have a much richer and deeper conceptual understanding of ICT than either the researchers or the teachers had expected. This suggested a likely mismatch in teachers’ expectations and pupils’ potential performance in using ICT tools. It also gave rise to speculation about whether or not teachers had representations of ICT that were as rich and deep as the children’s. What was also clear, however, was that teachers would be influenced in the kind of concept maps they produced if they administered the task to pupils before doing it themselves. This was one of the reasons why we asked all the teacher-researchers to produce concept maps of their own at the conference, as well as producing them ourselves. We have also asked them to invite 4 or 5 other teachers who work with the ImpacT2 pupils in their schools to do a teachers’ concept mapping task to provide us with additional data. As could have been expected, the ImpacT2 teacher-researchers, who all have responsibility of some kind for ICT in their schools, produced very detailed, complex concept maps. In most cases these were text-based maps as we were reluctant to ask teachers to draw. These data from the teacher-researchers and their colleagues, about their own secondary artifacts of ICT, will be an important component in our analysis of the context for pupils’ learning with ICT provided by the schools.

Analysis of the concept mapping tasksIn our analysis of the concept maps we have drawn upon the methods of phenomenographic research. This work, which originated in Sweden under the leadership of Ference Marton (1994), started out as an investigation of why some people learn more easily than others. Phenomenographic researchers have predominantly used interviewing as a means of understanding the learning process of the individual. They have favoured very open-ended interviews in which the interviewer attempts not to structure the interviewee’s thinking in any way. In this way, it is possible to re-construct the learning process by tracking how concepts are established and how one builds upon another. The analysis of the ImpacT2 concept maps has been designed to enable us to make a preliminary analysis of the conceptual patterns and linkages of pupils’ representations of ICT, by recording their ‘spheres of thinking’ and ‘zones of use’. We expect to gather some significant data on the way these develop for the entire cohort over a year. This initial analysis of the concept maps will also enable us to identify pupils for follow-up interviews, whose current understanding of ICT makes it likely that they will make significant gains in ‘strand two’ learning. Phenomenographic researchers normally work in pairs to carry out the analysis according to an agreed system and then compare results to establish inter-rater reliability. Marton suggests that ‘there should be a reasonable degree of agreement between two independent and competent researchers’ and suggests that ‘reasonable degree’ is reached ‘where the two researchers agree in at least 2/3 of the cases when comparing their judgements and where they reach atreement in 2/3 of the remaining cases after discussion’ (Marton 1994, p. 5). There is not an attempt to make the scoring process absolutely precise, because the value lies in broad comparisons across the

of 19

17:23 24/05/23

scores of many individuals and the general understanding that emerges from qualitative assessment of the maps.

The analysis of the concept mapping data posed another set of decisions. With somewhere between 1500 and 2000 maps to analyse from June 2000 alone, we needed a simple means of obtaining quantitative data that can be analysed with other data such as SATs results, home ownership of computers and socio-economic status (gained from post code data). Further qualitative analysis will then be possible on a small sample of maps selected to represent different kinds of responses. The decision on which to select will be taken when the analysis is complete. Interviews will be conducted with these pupils to enable full phenomenographic analysis of their mental representations of ICT.

The analysis will record for each concept map: The number of nodes (images or textual labels) The number of links The connectivity of the map calculated as the number of links divided by the number

of nodes. The number of ‘spheres of thinking’ (these might include: games, information (e.g.

the web), communication (e.g. email, chat), music. The number of ‘zones of use’ (these might include: home, school, workplace,

banking, transport)

Two methodological issues arose in relation to the analysis of the concept maps.

a) We discussed how we might be able to record the complexity of the map. One suggestion involved categorisation of the maps, e.g. as ‘linear’, ‘single node’, ‘multiple node’, ‘spaghetti’. However, it was not clear what such a crude categorisation would tell us about the individual’s thinking. This approach, which is commonly used by researchers exploring the use of concept maps as a tool for learning, is only applicable when a map has been constructed more carefully, and has been part of the process of developing ideas, rather than, as in ImpacT2, a quick means of brainstorming ideas and their connections. We also discussed how we might be able to record the ‘depth’ of the map. This refers to the number of linkages between a key concept (central node) and other concepts, which can be thought of as representing layers of thinking. In practice, all but the most simple maps had multiple links that made it impossible to decide where to begin or end in counting a linked sequence of images. In the end it was decided to substitute for both complexity and depth the simple calculation of ‘connectivity’.

b) We also discussed the extent to which pupils would perform differentially depending on their previous experience of concept mapping or their confidence and ease in drawing images. After careful discussion, this was disregarded in the analysis because the outcomes will be aggregated to establish broad trends rather than precise measurements. There will be no attempt to use the concept mapping task to assess individuals in any way that would influence their educational opportunities; indeed the data will be kept confidential, identified only by a

of 19

17:23 24/05/23

unique code. We will, however, get some indications of the extent to which the data is influenced by these factors through the small sample of in-depth interviews.

Working with teacher researchers

As a government commissioned and funded evaluation ImpacT2 is bound by the DfEE’s directive, that is to investigate the impact of networked technologies on attainment. The project is also subject to the more detailed interpretation of key issues and methodologies set out in the original bid. Two subsequently commissioned papers, which were separate studies with separate contracts, impose further obligations. Preliminary papers 1 and 2 provide clearer detail of the project design through their focus on challenging methodological issues. As indicated above, ImpacT2 has two distinct strands. Strand 1 uses statistical analysis of GCSE and SATs results correlated with PIPS and YELLIS information and the concept mapping tasks. Although it is envisaged that this may reveal interesting patterns and relationships we are uncertain of the degree to which direct causality will be proven. Strand 2 entails more innovative research techniques. Through pupil and teacher hypotheses it aims to discover more about networked technologies and learning in and out of school. This research design was subsequently subject to consultation at an ‘expert seminar’ to which specialists from schools, local education authorities, universities and government departments were invited. In agreeing to take part in ImpacT2, schools consented to government requirements and the methodological design of the research. Those opting into the evaluation are bound by the project contract and its inherent opportunities and constraints. Thus this study is one in which the role of government agencies has been substantial in the shared process of setting and reviewing the nature of the research.

Consequently by the time of our initial meeting with our 60 ImpacT2 teachers at a two-day residential conference the project design was broadly fixed. Nevertheless, there remained scope for precisely when and how particular methods might be employed and finer detail about the sorts of questions we might ask. In addition to information giving, the elicitation of preliminary data and modelling of/training in data collection methods, a main aim of induction was therefore consultation. One of the surprising outcomes of the conference and subsequent interactions with our teachers has been a leaning towards clearly defined prescription rather than more open-ended opportunities. Feedback resulting from teacher/teacher interactions at the conference emerged through informal discussion. There was clear acknowledgement of the need for a unified and consistent approach to running such a geographically and empirically extensive project. Nevertheless, some teachers were anxious to know exactly what to do and the ‘right’ way to do it. The two-day (rather than one-day) conference thus enabled a process of teacher reflection, discussion and feedback and time for us, as the university-based team, to agree on solutions and to respond.At the conference we had already planned to consult teachers on the content and layout of the pupil instruments developed to date. These Strand 2 materials seek to explore if, what, how and why learning might be supported, enhanced and extended through networked technologies. For example, the Logs are designed to track ‘typical’ ICT use in and out of school over a one-week period; the Diaries (subsequently named the Special

of 19

17:23 24/05/23

Reports) seek reflection on and analysis of an occasion when use of ICT has been particularly helpful in schoolwork. Group discussion helped teachers to become familiar and engage with some of the associated issues as well as occasion to contribute to the design of these resources. We were able to negotiate changes and to encourage a sense of teacher participation and ownership. Our joint interest in teaching and learning using ICT also provided common ground and enabled teacher and university-based researchers to consider issues from each other’s perspectives.As a result of team discussion, we also promised to provide a more structured timetable of teacher researcher tasks which would set out key topics and focused questions and suggest methods for acquiring particular data. These have subsequently been located on a timeline (for a six month period initially) in such a way as to enable teachers to forward plan but also to allow flexibility for us to include additional investigations into interesting or problematic areas arising from preliminary data analysis. Our planning also allows space for teachers to devise and implement their own investigations in their own schools. We have also attempted to order the suggested mini-research projects carefully to ensure variety of content, method and level of demand. Monthly reports have now become the means of communicating the findings from these small research projects to the link researchers. Limited to 200 words, we (the university-based team) have engaged in much discussion about the balance between demand on teachers’ time and the need for depth and detail. We have also discouraged unexpected use of the ImpacT2 listserve for posting monthly reports for two reasons. Firstly, models were influencing many teachers and, in some cases, less than perfect examples were being used as prototypes. Secondly, although we were already aware that findings within individual schools might change ways of thinking and working, this sharing of findings between schools might also influence practice. In this way the measurement of impact could in itself have an impact. We have also needed to reach agreement about reporting requirements: what ‘compulsory’ means and how we should deal with non-submissions. We implemented a tracking system early on as a monitoring device and have since decided on standard email messages to supplement the more personal (and time consuming) individual approach.However, what really intrigued teachers at the conference was our project director’s introduction of the notion of knowledge transformation. This injected something really quite distinctive. Learning is at the heart of what teachers are intensely interested in. Furthermore, we, as academics, had no quick or easy answers. This brought us together in mutual enquiry. A challenging concept which engaged us all, we were united in a shared quest for what knowledge transformation might be and we speculated together on what it might look like in the classroom. Our teachers were keen to explore subject-specific examples. We reflected on a situation where on-line use for a very limited time span might have far-reaching consequences. An idea communicated through email might lead to investigations using a range of resources away from the computer. The presentation of synthesised information in a different genre could result in new knowledge and understandings. Knowledge transformation has continued to be an important focus of our research and has since become an ImpacT2 buzzword.This process of feedback and response helped to facilitate the building of team spirit where, as partners, we were working with mutual respect and increasing understanding of each other’s roles, pressures and needs.

of 19

17:23 24/05/23

Working with pupil researchers One way in which ImpacT2 very significantlly extends the action research paradigm with which we started this paper is that pupils will also be working as researchers. Children and students have been extremely keen to volunteer and they are excited at the prospect of embarking on their work. On the one hand they will be informants and materials such as the Internet questionnaire will try to get at if, how and why children and teenagers are using on-line resources. They will also try to get at knowledge transformation from pupils’ perspectives. These materials are quite challenging and will entail deep thought and detailed responses. We need to get beyond the surface level to reflection on the impacts of new technologies on learning in its broadest sense. We have put a strong emphasis on the fact that these instruments are not tests and there are no right and wrong answers. They will not be marked and we are not looking for correct spellings. We want to investigate what young people are actually doing, discovering and thinking not what they think we want or what they think will please their teachers. This has had implications for layout, a more informal yet ‘official’ look being preferred through consultation with pupils in the piloting process (and not always in agreement with teacher preferences). Language use has also been modified to encourage broader consideration of school and home use so that terms like ‘learning’ have been avoided as this tends to be perceived by pupils as relating to schoolwork only. Teacher researcher guidance emphasises the need for a delicate balance between teacher support and influencing the nature and content of outcomes. Description, analysis and critical reflection on particular experiences of new technologies will be exciting data if we are able to reach it.However, ImpacT2 pupils are also researchers. One of the main benefits of this is that pupils may divulge particular information to peers which they might be unwilling to offer to a teacher or an outsider, or in materials teachers might see. Thus pupils are in a unique position to elicit data it might otherwise be difficult or even impossible to get at. In one aspect of the research process, for example, they work as interviewers, interpreters and report writers. The broad focus of their research is clearly defined through teacher researcher briefing and support materials which explain procedures for interviewing and report writing. Beyond that, the pupils themselves will decide on precise questions and the balance and direction of their interview. In considering how interactions should be recorded we decided that note taking would be too demanding and would reduce the pace and inhibit the flow of the interview. We therefore opted for use of a cassette recorder.Decisions also had to be made about the nature of report writing. Although, at the consultation stage, some teachers preferred that pupils should put their findings in their own words we have persisted with the need for verbatim quotations, as this promises rich data. Pupil researchers will also extract three significant issues from their own perspective. This models accepted research practice. Most important, however, we hope to enable pupils to find their own voice and to provide some insight into child and youth culture.The development and implementation of the pupil materials have also prompted team discussion on ethical issues. We have had concerns about the need for confidentiality, a matter also emerging at the piloting stage (particularly with secondary pupils). These young people need to state their views without fear. We do not wish to put pupils or teachers in a difficult position, nor would we wish an atmosphere of distrust to develop

of 19

17:23 24/05/23

between any of ImpacT2’s participants. Our solution has been to provide pre-paid envelopes or direct emailing to link researchers so that pupils have some degree of choice about who has access to their responses. Furthermore, it is important that pupils are clear about what the data is for, who will see it and how it will be used. Both pupil informants and researchers will therefore be briefed by their teacher researcher about the broad aims of ImpacT2 and the specific objectives of particular aspects individuals are involved in. Pupils will also be provided with guidance materials.

ConclusionHow then does ImpacT2 fit into the original paradigm of action research? Firsly, the existence of the strand one work sets it in some respects well outside that tradition. Even as far as the strand two aspects go, it is certainly some way from the original paradigm. We might summarise ImpacT2 in total as a project characterised by:The use of qualitative and quantitative methods of data collection and analysisControl of the research being shared between teachers, researchers and funders Researching both standard and non-standard forms of pupil learning and teacher activityThe research so far (in the two preliminary studies) being conceptualised and presented in terms of theory rather than practice.

Nevertheless fundamental to ImpacT2 is our belief in partnership as a means of acquiring and processing relevant data. This collaborative approach is based on the principle that each co-worker brings particular expertise to the work. Teacher, pupil and university researchers are all able to contribute in very specific ways which are suited to their respective roles, knowledge and experience. Each partner’s contribution, grounded in diversity of expertise, strength and skill, is complementary to and intrinsically dependent on that of the others. We recognise that in practice, thought and action are complicated by different agendas, epistemologies, values and beliefs. The assumption of a new role in a familiar context may give a sense of empowerment but individuals are also subject to the evaluation’s focus and methodology as well as contextual opportunities and constraints. Nevertheless, our aspiration is to create a strong sense of interdependence and mutual respect in ImpacT2 in which each partner is valued for equally worthy but different contributions to the research process. In that respect, despite all the new features and emphases, the project arguably represents a continuation of the spirit and values that generated the original educational action research tradition.

ReferencesAusubel, D. P. and Robinson, F. G. (1969). School Learning: an Introduction to Educational Psychology. London and New York, Holt, Rinehart and Winston.

Brown, J. S., Collins, A., et al. (1989). “Situated Cognition and the Culture of Learning.” Educational Researcher 32(Jan-Feb): 32-42.

of 19

17:23 24/05/23

Buzan, T. (1993). The Mind Map Book: Radiant thinking - the major evolution in human thought. London, BBC Books.

Crawford, K., Neve, L., Pearson, M. and Somekh, B. (1999). Creative Tensions and disrupted routines: the impact of the internet on primary children's understanding of the world. British Educational Research Association, Brighton.

Dawes L. Teaching Talking. Wegerif R. and Scrimshaw P. (eds.) (1997) Computers and Talk in the Primary Classroom. Clevedon, Multilingual Matters Ltd.

Elliot J. (1988) Educational Research and Insider-Outsider relations Qualitative Studies in Education 1, 2. 155-166.

Engeström, Y. (1999). Innovative learning in work teams: Analyzing cycles of knowledge creation in practice. Perspectives on Activity Theory. Y. Engeström, R. Miettinen and R.-L. Punamaki. Cambridge, New York and Melbourne, Cambridge University Press.

Hogg, K., Johnston, I., and Crawford, K. (July 1997). Using Concept maps to investigate students' understanding of physical optics. Science Education Conference, Perth WA.

Jonassen, D. H., Carr, C., and Yueh, H-P. (1998). “Concept mapping as cognitive learning and assessment tools.” Journal of Interactive Learning Research 8: 289-308.

Lave, J. and Wenger, E. (1991). Situated Learning: Legitimate peripheral participation. Cambridge, New York and Melbourne, Cambridge University Press.

Marton, F. (1994). Phenomenography. The International Encyclopedia of Education. T. Husen and T. N. Plostlethwaite. Oxford, Pergamon. 8: 4424-4429.

McFarlane, A., Harrison, C., Somekh, B., Scrimshaw, P., Harrison, A., and Lewin, C. (2000). Establishing the Relationship between Networked Technology and Attainment, ImpacT2 Preliminary Report 1, University of Nottingham.Novak, J. D. and Gowin, D. B. (1984). Learning how to Learn. New York, Cambridge University Press.

Pearson, M. and Somekh, B. (2000). Concept mapping as a research tool: a study of primary children's representations of ICT. British Educational Research Association, Cardiff.

Representation (1998). Work Programme, Annex 1 - Project Programme. European Union Project ref MM 1045. http://hermes.iacm.forth.gr

Salomon, G. (1993). No distribution without individuals' cognition: a dynamic interactional view. Distribute Cognitions: psychological and educational considerations.

of 19

17:23 24/05/23

G. Salomon. Cambridge, New York and Melbourne, Cambridge University Press: 111-138.

Scrimshaw P. (ed.) (1993) Language Classrooms and Computers. London: Routledge

Scrimshaw P. and Perkins G. Tinker Town: talking together. Wegerif R. and Scrimshaw P. (eds.) (1997) Computers and Talk in the Primary Classroom. Clevedon, Multilingual Matters Ltd.

Scrimshaw P., Lewin C., Mercer N. and Wegerif R. Combining qualitative and quantitative methodologies to evaluate the effect of low cost word processors on literacy. Paper presented at BERA, September 9th, 2000.

Lewin C., Scrimshaw P., Mercer N. and Rupert Wegerif R. (2000) [KS1 report- XXfull title to come].

Somekh B. (1994) Inhabiting each other’s castles: towards knowledge and mutual growth through collaboration Educational Action Research Journal, Vol 2 No 3

Stenhouse L. (1975) Introduction to Curriculum Research and Development London, Heinnemann Educational Books.

Wartofsky, M. (1979). Models: Representation and Scientific Understanding. Dordrecht, Reidel.

Watson M. Improving group work at computers. Wegerif R. and Scrimshaw P. (eds.) (1997) Computers and Talk in the Primary Classroom. Clevedon, Multilingual Matters Ltd.

Wegerif R. and Scrimshaw P. (eds.) (1997) Computers and Talk in the Primary Classroom. Clevedon, Multilingual Matters Ltd.

Wenger, E. (1998). Communities of Practice: learning, meaning and identity. Cambridge UK, New York and Melbourne, Cambridge University Press.

Wertsch, J. V. (1998). Mind as Action. New York, Oxford, Oxford University Press.

of 19

17:23 24/05/23

APPENDIX 1

Socially Constructed Integrated Model of Learning (SCIM-L). Solid arrows indicate probably causal relationships, broken arrows denote potential relationships.

of 19

Out-of-school learning Overlap School-based learning

Traditionally defined ICT skills

Improved school-related knowledge and understanding

Improved school attainment

Learner-defined curriculum and culture

Traditionally defined school curriculum and learning culture

Learner’s personal representation of the task, and available resources

Teacher/ curriculum-specified tasks for the learner

Homework task

Learner activity, using ICT skills

Potential benefits for the learner:- knowledge and skills

(beyond those validated or approved by school)

- autonomy and confidence in learning

- extended awareness of the social benefits of the Internet

Improved knowledge, understanding and skills, of personal and social benefit

17:23 24/05/23

APPENDIX 2

Roles and responsibilities of school-based colleagues

Teacher researchersEach of the 60 schools in the sample nominated one named teacher researcher responsible for coordinating the project. Their responsibilities include:Contribution to project design (consultation at the two-day residential conference).Development of project materials (critical evaluation at the two-day residential conference).Dissemination of project information (to senior managers, teachers, pupils, governors and parents).Regular communication through email, closed email listserve and restricted website (laptop provided).Monthly (electronic) reports.Organisation of link researcher visits.Training and management of pupil researchers.Data collection (information, delegation, elicitation).

Pupil researchers

A small number of pupils in each school have volunteered to work as pupil researchers. Their responsibilities include: Attendance at short training sessions in preparation for their research, additional

support materials also being provided. Informant completion of logs, special reports and questionnaires (data available for

pupil investigations within the school). Researcher interviewing of peers, interpretation and report writing.

of 19

17:23 24/05/23

APPENDIX 3

Teacher researcher activities over the first term

In chronological order over the first term these include: Project dissemination (to senior managers, teachers, pupils, governors and parents). First monthly report (description of project dissemination and contextual issues). Selection of the statistical cohort (approximately 30 pupils: one class of Year 5/6

pupils in each primary school and one tutor group of Year 8/9 and Year 10/11 pupils in each secondary school).

Recruitment of pupil researchers. Second monthly report (progress report). Administration of concept mapping (under test conditions, instructions provided in

script form). Administration pupil questionnaire (providing support where necessary, particularly

for younger informants). Organisation of and involvement in the link researcher visit (negotiation with

colleagues, provision of information, opportunities to observe modelling of techniques by link researchers).

Third monthly report (analytical reflection on issues arising from concept mapping).

The next stages will entail more autonomous research, for example: Analytical reflection on teaching and learning (especially knowledge

transformation). Elicitation of information from colleagues (indirectly through questionnaires and

directly through informal discussion, interviewing and focus groups). Briefing and advice to pupil researchers (need for teacher researcher

understanding of ImpacT2’s aims and methods). Organisation and management of pupil researchers (guidance materials provided).

We will continue to support our teachers through regular email communications and our telephone hotline.

of 19

BERA Representations Paper · Web viewPreliminary analysis of the products has shown that the...

Documents

Transcript of BERA Representations Paper · Web viewPreliminary analysis of the products has shown that the...