1

Investigating the Design and Use of a New Form of Screen-less Digital Technology in Relation to Young Children’s Play and Communication Practices Dylan Yamada-Rice, School of Education, Sheffield University, UK

Project outline

This STSM project relates directly to COST Action IS1410 that is interested in the changing ‘array of digital, interactive, converged and personalised devices’ and to ‘examine how young children’s literacy development is being shaped by changes brought about by the digitisation of communication’ (http://digilitey.eu). In relation to this, the STSM reported on here had the following three objectives:

1. To undertake knowledge exchange (KE) with a EU industry host who are developing an emergent form of technology that connects online and offline domains within a toy.

2. To conduct small-scale research on the kinds of play and communication practices children develop in the use of these toys.

3. To contribute to the development of theoretical tools to understand the communication practices that occur across digital and non-digital domains in the absence of screens.

The technology at the heart of the study was a new type of interactive, digital toy known as Avaikai (Figure 1) being developed by a German company called Vai Kai (www.vaikai.com) who hosted the STSM.

Figure 1 Avakai

The Avakai are a wooden toy that is technology-enhanced to allow communication across a set of twin dolls using movement that produces sounds. Traditional and digital communication practices have a history of being positioned in opposition to one another. However, more recent research continues to look at the crossover between online and offline communication practices as well as their contemporary connection to play in young children’s lives. Increasingly, technology is being created that takes account of the crossover in these two

2

domains, such as augmented reality apps, in which literacy and play practices combine. However, it is the norm for such technology to foreground screens but what if screens become increasingly removed in future digital devices? Development of screen-less digital devices for the promotion of play and communication is an entirely new area and thus the STSM sought to look at its development by the designers and use of the dolls by a group of young children. Specifically to investigate screen-less digital devices in relation to how traditional communication practices and play may or may not change when young children use Avakai. This was considered to be particularly important considering the concerns some adults have about perceived negative effects of screens in young children’s lives.

The host-company Vai Kai believes that a different type of play and communication can emerge when the screen is removed, and the digital remains taking on another function. Just before announcement of the STSM funding stream I had a chance meeting with Vai Kai who mentioned that parents who had had the opportunity to see Avakai asked them about the purpose of the digital layer in the toy’s design. Why not just leave the Avakai dolls as a wooden ‘unconnected’ toy? When the funding stream was announced I saw this as an opportunity to look into how young children use connected and unconnected versions of the Avakai dolls and think about the shift in digital communication practices away from screens. At the same time to use a KE strand to learn more about the design and how this relates to children’s use. As a result the mission was split into two stages. The first stage was KE looking at the design of the Avakai between the industry host and myself an academic interested in young children’s digital play and multimodal communication practices. The second stage observed how a group of young children used the Avakai when they were technological enhanced compared to when they were not.

This report is split into two key sections. The first part entitled ‘Play, Communication and Design’ reports on three key themes from my field notes during the course of being hosted by Vai Kai and having the opportunity to learn about the design of Avakai. The second part reports on three key themes from the second stage of the STSM that looked at young children’s use of the Avakai.

Part 1: Play, Communication and Design

This part reports on key themes that emerged from Stage 1 of the STSM, which was based on knowledge exchange (KE) between the designers of the Avakai, Matas Petrikas and Justyna Zubrycka and myself. This section sought to add to understanding of contemporary communication practices by seeking links beyond education and social sciences to consider how multimodal communication practices are being used and designed for young children in the digital toy industry. This builds on work by academics such as Rowsell (2013) who considered the role of multiple modes of communication within a range of industry practices. In relation to this, this part of the study started with the following questions:

1. How was the design of Avakai conceived and developed? 2. What were the designer’s intentions for children’s play and communication practices? 3. How did these intentions manifest in the design of the toy?

Answers were sought through a series of conversations with the Avakai

3

designers. These were captured in a combination of email exchanges and written notes taken during face-to-face conversations. Details of these conversations were considered in relation to other notes made on a project blog (https://screenlessdigitalplay.wordpress.com) during the course of the STSM. Three key themes emerged from this part of the study and are discussed under separate subheadings next. Design Theme 1: The Changing Aesthetics of Digital Play and Communication Kress (2003) writes that emerging communication practices are the result of ‘simultaneous social, economic, communicational and technological changes’ (p.9), which are connected to patterns formed by cultures in relation to their individual histories. Given this I decided to start the STSM project with a visit to the Berlin Computersiele Museum with the view to thinking about the history of digital play and communication practices. On entry to the museum there was a temporary exhibition called Pixelkunst. This was art made from computer keyboards (Figure 2). Looking at the pictures I thought about how largely redundant keyboards have become to a lot of communication practices connected to digital play, where touch-sensitive screens or hand-held controllers have replaced keys. Simultaneously I reflected on how communication practices might change further if screens become redundant as in the case of the Avakai toy that was at the centre of this STSM.

Figure 2 Pixelhunst Art

In my work I have always been interested in how communication practices relate to multiple modes, and therefore I have come to largely try to avoid use of the term literacy that refers in its most direct definition to reading and writing alphabetic scripts. In relation to this the museum mentioned the connection between computer games and art which can also be seen as a communication practice that extends beyond reading and writing:

“The oldest connection between games and art can probably be traced back to the early days of dramatic theatre. During the 1960s, new

4

playful elements were integrated into art, especially within the context of the Fluxus movement. The photo [Figure 3] shows a public concert performance in Toronto, during which sounds and projected images were controlled directly by two artists, Duchamp and Cage playing a game of chess.”

(Computerspiele Museum wall writing)

Figure 3 Reunion (1968) Marcel Duchamp, John Cage, David Tudor, Lowell Cross

The Reunion (1968) reminded me that play and communication is often a performance and also made me think about the importance of aesthetics in digital game design and communication. For example, how different generations of computer gamers grow up using different kinds of game aesthetics. In relation to this, two of the games I liked best in the museum were Asteroids (Figure 4) and Pac-Man (Figure 5) that directly relate to my age:

Figure 4: Asteroids

5

Figure 5: Pac-Man

My interest in these two games tied with another piece of wall writing from the museum:

“The pleasure of playing games is closely related to the desire to transcend and forget present reality in order to enter an artificial, man-made world. Images have always been the most potent instrument for achieving this goal.”

(Computerspiele Museum Wall Writing)

In an article (under review) that I have written with Richard Finn, we discuss visual modality, that is the term given to the extent to which images connect to reality or not (Kress & Van Leeuwen, 2006), and its connection to emotions and perceived realness. In doing so, we consider the extent to which particular media are linked to specific cultural histories to symbolise reality and in this way link to the way in which they allow viewers to ‘transcend and forget present reality’. In that article Finn and I describe how common understandings of the visual mode, modality and reality that have derived from seminal texts on semiotics such as Kress and Van Leeuween (2006) are not the same for all ages, subcultures or geographic locations. Thus high-modality imagery that can be seen in the latest computer game technology does not provide me the opportunity to transcend and forget present reality in the way that the low-modality aesthetics used in Asteroids and Pac-Man do. I can relate this also to digital hardware and the appeal of the keys in the earlier artwork Pixelhurst that might similarly be uninteresting to younger viewers who have grown up with technology that foregrounds touch-screens. In relation to these ideas on design, aesthetics and cultural histories I was interested from the outset in the decision of the designers to make Avakai screen-less and from wood. For example, did the designers see this material as appealing to a wide audience? The documentary ‘In the best possible Taste’ by the artist Grayson Perry about social class and aesthetic tastes also provides fascinating insight into the subject:

“Do you think that has implications for taste and maybe increases snobbery towards the remaining working class people’s tastes? I think there will always be this barrier where there are people who are looking for rules. A lot of the lower middle classes still need reassurance and clear rules, which they find in brands and in definite trends because they perhaps don’t have the confidence to go on their own intuition and try something else out. So there’s always going to be a large proportion of the population that have what they think is a very

6

clear idea about what is good taste. But of course the good taste is just an illusion; it’s just that they’re obeying the rules of their tribe. Do you think the gap between the lower middle class and the upper middle class is essentially as big a gap as between different classes in terms of taste? The conclusion that I came to is that the real ‘Berlin Wall’ of the taste landscape is between the lower middle and upper middle class. It’s made out of education, a kind of knowingness, an understanding, a bit of confidence, and ease with the whole thing. It’s almost as if the moment you understand it you realise ‘does it actually matter?’ It’s about awareness as well. If you’re insecure about the track you’re on, if you don’t know what the landscape is either side of it, you’re very desperate to stay on the track, but if you’re confident you stride off into the undergrowth and say ‘it’s just as nice out here’. A lot of people probably think they’re displaying individuality when they demonstrate their taste but do you think they’re fooling themselves? I think there is an element of that. I think often when we think we’re at our most individual we’re most vulnerable to influence, and perhaps the hard-wiring of our upbringing comes into play; the material culture that one imbibed with one’s mother’s milk, that’s the default setting on your taste, and often people don’t even realise that’s happening, when they make microscopic decisions all the time about what clothes to put on and how to decorate their houses.”

(Taken from a Q and A with Grayson Perry that can be found online here).

This similarly connects with ideas by Van Leeuwen (2013) that shows how toy design, in the case of his work Mecanno and Lego, are reflections of wider aesthetic changes and social practices. Again this was reflected in the Computersiele Museum that mapped computer game and interior design of the same period, as can be seen in the images below (Figures 6 and 7):

Figure 6 Living Room late 1970s and the game Pong (1972)

“The first generation of home video games were not sold as children’s toys but geared at adults or marketed as family entertainment… Accordingly the design of these console tried to comply with the aesthetics of the environment e.g., by using wood veneer”.

(Computersiele Museum Wall Writing)

7

Figure 7 Children’s room 1980s and Supermario (1985)

“Nintendo turned them [computergames] into children’s toys by devising products that directly addressed them as their target audience. This move was supported by a demographic change. An increasing number of TV sets came to be placed in children’s rooms”.

Kress’ (2003) ideas of how communication patterns reflect wider changes in society and van Leeuwen’s which suggest the same in relation to toy design were also evident in the conversations I had with Matas Petrikas, one of the Ava Kai designers. Previously Matas worked for Soundcloud, a digital platform for sharing sounds. After many years of working in this digital field Matas said that he became interested in the physical again and its connection to the digital. He noticed that a movement was beginning to take place back to foregrounding the material. Also that advances in digital technology was making many things possible in the physical, such as through 3D printing and laser cutters. This leads into the second design theme that considers the crossover between physical and digital materialites.

Design Theme 2: Design that Crosses Physical and Digital Materialities

While in Berlin on the KE strand of the STSM I also went to the Museum for Communication. I was intrigued by a comment on the museum’s website which read that the museum:

“…sees itself as providing a venue for encounters, exchanges and entertainment – in brief, for communication. It focuses on the changing approaches to signs, codes and media, and the effects these have on public and private life.”

The word ‘entertainment’ stood out within that description. Everyday communication practices are very often entertaining or tied with playful encounters but when communication and literacy practices are framed in formal educational contexts such descriptions rarely come to mind.

The museum trip was interesting in other ways too, for example part of the Avakai design is about allowing children to communicate at a distance across a set of twin dolls, and many of the exhibits reminded me of the long history communication technologies have of being concerned with this. In particular, I was struck by the design of an early radio postcard that used simple circuit board

8

technology to record sound, which the receiver could listen to by plugging in headphones (Figure 8):

Figure 8 Radio-Postcard

In ways such as the radio postcard, communication technologies seem to have consistently tried to experiment with multiple modes of communication. This example also tied with ideas derived from my trip to the Computersiele Museum where I began thinking about how the very nature of trying to communicate in multiple modes provides the need to combine both physical and digital platforms. Indeed this is reflected throughout computer game history where physical artefacts have repeatedly been added to enhance play and communication practices on the digital platform. Figure 9 showing the game Odyssey provides an example of this:

Figure 9: Odyssey (1972) Magnavox Wall writing in the museum described the game as follows:

“The odyssey console…could only display rectangular forms. In spite of this restriction, Magnavox supplied 12 game variations and some additional parts like boards and tokens or coloured plastic overlays that could be attached to

9

the front of the screen. A light gun which allowed the user to shoot moving rectangles, was also offered in addition”.

(Computerspiele Museum Wall Writing)

In many ways this is similar to the design of the Radio Postcard that combined the material card with writing and technology to embed audio. In relation to this I began thinking about the materiality of changing literacy practices. What does it mean for the Avakai design to have a doll as a physical object but use digital means to convey a range of sounds through the doll’s movement? At the same time it is interesting to think about the points in time where changes in technologies have allowed communication practices to shift and change giving preference to different modes. Geo Panch’s (2007) map illustrates this well (Figure 10):

Figure 10 Geo Paunch’s map illustrating a brief history of communication practices

The concluding point of Geo Paunch’s communication timeline is signalled by smartphone technology, but my conversations with Matas, one of the Avakai’s designers informed me about the current shift away from using screens to a return to thinking about the materiality of communication practices. This illustrates how the timeline is already extending forward to incorporate screen-less technologies. How have screen-less technologies affected communication practices is the focus of the last section on design themes next.

Design Theme 3: Screen-less Digital Play and Communication

Following on from the last theme it is interesting to consider how communication practices might change if screens are removed. In relation to this I looked around for examples of screen-less digital play in the Computersiele Museum and found two examples:

10

Figure 11 Touch Me (1978) Atari

“The Sound in Touch-me constitutes an essential element of the game concept. It is the home adaptation of the slot-machine game from Atari (1974), in which the players have to remember and reconstruct a successively longer sequence of tones by pressing the Corresponding buttons” (Computerspiele Museum wall writing).

Figure 12 Postcard Chess

These two examples provide affordances of play and communication that are also evident in the Avakai design. Firstly the foregrounding of modes of communication that are tactile and use sound such as in Touch Me (Figure 11), secondly, the ability to play and communicate across a distance such as in postcard chess (Figure 12). Screens, as Kress (2003) states, foreground affordances of the visual mode. Thus when screens are removed other modal properties and their affordances come to the fore as in the two examples above. This is also the case with Avakai as can be seen from the description on Vai Kai’s website of the dolls’ properties, which include (1) reacting to expressive gestures with sounds (2) a heart beat that increases in strength when it comes into close contact with another Avakai, (3) Avaikai make a kiss sound when their mouths are touched (4) Bluetooth technology allows sets of twin Avakai to communicate across a distance of 50m or at a longer distance via an app.

11

These affordances of the Avakai connect to changes in communicative and play history but are also embedded in the personal background of the designers. When I spoke to Justyna Zubrycka, one of the designers of the Avakai I asked for some insight into her background in order to understand how the design for the toys had come about. Justyna has a background in design production and during her studies she was interested in sensory and musical approaches to communication. As a result she began to think about how meaning-making with particular modes are usually closely associated with specific media. Therefore she began experimenting with what would occur if she matched modes of communication with media that are not normally combined. This reminded me of Norman’s (1990) book ‘The Design of Everyday Things’, which describes how bad design mismatches perceived with actual design, for example, when the design of a door looks like you should push it but actually requires it to be pulled to open it. It also reminded me of the process of what Kress (2010) terms transduction, that is when messages in one mode are changed to be conveyed in another. This can be seen in Bezemer and Kress’ (2010) work which considers what is gained and lost in the process of transduction.

Further, processes of transduction are what I have found interesting about my involvement with students on Kevin Walker’s Information Experience Design (IED) programme at the Royal College of Art (RCA), where students often use modes not commonly associated with portraying certain information/messages. For example, one of the first student pieces I came across was ‘Change Ringing’ by Peter Shenai. The RCA IED website describes this pieces as:

‘Change Ringing is a collaborative artwork with composer Laurence Osborn. Six bells have been cast in bell bronze by Shenai. Their shapes and sounds derive from climate change data. This forms the basis for a large orchestral composition for the bells by Osborn’.

When I first saw the piece I was interested in how understanding is constrained or changed by the modes and media used to convey the message. Specifically how listening to climate change data instead of viewing it as bell graphs alters how the data is perceived, interacted with and therefore understood.

Similarly, the Avakai designer Justyna stated that her own design history has derived largely from her interest in the deaf percussionist Evelyn Glennie. Through exposure to this musician Justyna became interested in how people perceive sound through the body instead of the ears. These influences led her to create Morimo, a sensory instrument where sounds are received through tactile vibrations as the user lies face up on top of it. After making this instrument Justyna worked with children aged 3-6 and 6-10-years, allowing them to experience Morimo and understand that sound is also a tactile element. She was thinking about how certain communication practices are given greater preference than others. How education is predominately interested in communication that happens with the mind rather than the body.

The first part of this report has reflected on emerging themes to derive during the KE stage of the STSM that looked at the design of the Avakai. It has related these to a range of cultural practices in digital game deign and communication practices, as well as to the personal backgrounds of the designers. Part 2 of this report describe how the Avakai features and design outlined in Part 1 were received by a group of young children.

12

Part 2: Young Children’s Use of the Avakai

The second stage of the project sought to understand the similarities and differences in the way in which a group of young children used the Avakai when they were enhanced by technology compared to when they were not. It sought to answer research questions 5-7 of the study:

5. How does a group of young children play with the non-digitally enhanced Avakai? 6. How does a group of young children play with the digitally enhanced Avakai? 7. What are the similarities and difference between the two types of Avakai play in relation to alphabetic print and wider multimodal literacy practices?

Initially the intention was to record the play using GoPro cameras mounted on chest harnesses worn by the child-participants. However, this proved problematic because not all the children in the after school club setting where the study took place had received parental permission to take part. Further, although the intention was to separate children with and without permission this proved to be impossible due to limited space. Therefore filming was replaced by the use of detailed field notes that recorded children’s physical movements of the dolls when playing, alongside examples of what they were saying. This was supported by the use of a mobile phone to photograph examples of key play moments. The inability to use the video cameras planned for the study affected the multimodal means of analysis I had intended to apply to the data collected. This would have provided more detailed insight into the children’s actions, gestures, and language when playing with the Avakai, because as Kress and Van Leeuwen (1996) state ‘particular modes of communication should be seen in their environment, in the environment of all the other modes of communication which surround them’ (p. 33). Further, that ‘learning is realised through the interaction between visual, actional and linguistic communication and involves the transformation of information across different communicative systems (‘modes’)’ (Jewitt et al 2001, p.5). Instead the photographs were inserted into the field notes and thematic analysis was applied to these following guidelines laid out by Braun and Clarke (2006).

The child-participants were recruited via an early childhood after school club, attached to an infant school in an inner city area in the North of the UK. Ethical approval was gained from the University of Sheffield after which permission was sought first from the head teacher of the infant school where the after school club was based, then from the owner of the after school club. Once these sets of permission had been received parents were given an information sheet and a consent form to sign if they were happy for their child to take part. Finally, all children who had parental consent had the project explained to them and their verbal consent was gained. In addition, the children were observed for signs that they might want to leave the project. Given the nature of the after school set-up, where children moved between activities of their choice, such as small world play, arts, and free play with construction toys, children stopped playing with the Avakai when they felt like it and naturally moved on to other activities. The data were collected across four days. For the first two days Avakai dolls that were not technologically enhanced were placed in the after school sessions and observations were made of children's play and communications practices with the dolls. For the second two days the dolls were changed for two that were digitally

13

connected. Unfortunately soon into the first day of using the digitally enhanced dolls one of them short-circuited and broke, and because it was a prototype it was not possible to fix it before the end of the project. This meant that only one technology enhanced Avaikai could be used and the feature to communicate across the two dolls could not be studied. In spite of this the data recorded were used to compare the difference and similarities that exist across the two sets of the dolls, those that were digitally connected and the one that was not. In addition to the limitations described with the restricted use of the GoPro cameras, and the short-circuited Avakai, the project was also limited by the development stage of the toy designs which had not yet developed to take account of communication at a distance via an app which is a feature currently underdevelopment. This section introduces three key themes that emerged across the data sets and relates this back to the design of the Avakai discussed in Part 1 of this report. These are children’s customization of the doll, interest in emotional modes of communication and use of the compartment in the base of the Avakai. These are discussed under separate headings next. Child Use Theme 1: Children’s Customization of the Toy Design In the knowledge exchange part of the STSM, conversations with the designers showed how they had always been interested in creating a design that would use children’s imaginations in their play. Their first prototype was a toy that could be customized using a selection of different shaped wooden pegs (Figure 13):

Figure 13 Early Avakai Prototype

After building this prototype Vai Kai became interested in how technology could enhance the experience of the object. One of the designers, Justyne Zubrycke mentioned how she was influenced by Machiko Kasahara’s ideas on ‘Device Art’:

“More recently, interactive art has redefined forms of art and the role of artists. What we call device art is a form of media art that integrates art and technology as well as design, entertainment, and popular culture. Instead of regarding technology as a mere tool serving the art, as it is

14

commonly seen, we propose a model in which technology is at the core of artworks.”

(Kasahara, 2006, n.p)

Kasahara’s idea that technology should be aesthetically beautiful and concerned with the design of the object and the material also tied with ideas of Matas Petrikas, the other Avakai designer, who as stated in Part 1, became interested in how advances in technology was making it possible to focus on the material aspect of the digital. Watching videos on interaction design, Matas began to think that the future of the digital would be in its return to foregrounding the physical materials of the object, which embodies the technology. As a result he saw the future of toys as those that are “connected” to the digital and those that are “unconnected” but all revolving around physical objects in other words toys not screens. In this way, Matas saw the digital as being able to add layers to physical toys. Specifically, he described the digital as being able to add “magical/ special” properties to children’s narratives in their physical play. In relation to this and in developing their product design further Vai Kai decided to focus on character design. They believed this might make it easier for children to invent stories and play imaginatively with the toy when customising it. Vai Kai’s toy workshop contained many examples of early prototypes and character design. It was during this stage that they began to formulate the current Avakai character (Figure 14):

Figure 14 Avakai prototypes The images in Figure 14, including early Avakai show how from the start the design was kept very simple seemingly encouraging customization. Such ideas tie with those of McCloud (1993) who writes in relation to low-modality graphics that the more simple an image of the “human” form, the more able the viewer is to interpret it in a way that is meaningful for them. The child-participants in the project were not told of the intentions of the designers but customization of the Avakai was something that happened immediately, as is shown by the following extract from the data:

Researcher: “You have stuck feathers in its ears now” (Figure 15).

15

Figure 15 Feathers added to an Avakai

Child 1: “Yes it is a bird. He is actually a bird. He is a bird at his party. Look!” Child H: “I want to have a go at that.” Child1: “G have a look at this!”

The adding of the feathers transformed the Avakai into what Hughes (2002) describes as an imaginative state of play. In other examples from the data children wanted to draw on additional facial features, such as a mouth or added stickers to its face to see what it would look like it if it were a different colour. In this way the children’s use of the Avakai seemed to match the designers hopes:

“Avakai Twins were designed for free play, that is the most crucial for healthy development of a child. Many educational digital toys are very goal oriented, preprogrammed to teach specific skills and limited to certain rules. Whereas a child directed, open ended play enhance self motivation, creativity and skills building… Avakai twins have traditional form of wooden dolls connected with digital technology without adding any sort of interface, like screen, joystick or controller.”

(Justyna Zubrycka, Avakai Designer)

This kind of imaginative play encouraged a range of communication practices. For example, in relation to the extract outlined above this included, storytelling and the making of a birthday card for the Avakai to have at his party, using drawing, stickers and writing.

Child Use Theme 2: Aesthetic Simplicity and Emotion

In part the simplicity of the Avakai design and the desire to customise it also related to a specific emotional quality in the children’s play and communication practices. Earlier I described that in my work with Finn (under review) we have written about how the affordances of low-modality images allow for the communication of emotions in ways better connected than other modalities. In both the children’s use of the digitally enhanced and unenhanced Avaikai the children played with themes concerning emotions.

Firstly, the children were interested in the Avaikai’s lack of mouth and asked to be able to draw on their own happy, sad or angry mouths. Prior to this I had already asked Matas about the decision to only give the Avakai eyes and no other facial features to which he commented that it had not been an intentional decision. I

16

was still thinking about what this might mean for young children’s interactions with the dolls on my walk home from the meeting with Matas. Berlin, where the conversation with Matas took place, is famous for graffiti and the walk offered me the chance to look at all the faces I encountered painted or stuck to walls, and to think about how their facial features drew me to them [or not] and why (Figure 16). Also if I could judge the character’s emotional states based on their facial features.

Figure 16 Faces on the walls of Berlin Additionally, Vai Kai, the company designing the Avakai was interested in possible patterns of emotional play and communication with the dolls by young children. As has been stated above, the Avakai can communicate across pairs, through set movements with the dolls. Therefore Matas, Justyna and I had a conversation before the start of the project to think about investigating what patterns of communication naturally arise in children’s play when they are using narratives about emotions. Here are two examples: In the first example I suggested to the children that it was the Avakais’ birthday:

Researcher: “Will you play with them as if it is their birthday?” Child 1: [singing] “Happy birthday happy birthday”.

She makes the Avakai dance around Child 1: “Yay, yay yay it is my birthday”. Child 2: “It is my birthday.” Child 1: “It is my birthday too”. Child 2: “How old are you?” Child 1: 16

The two Avakai are turned to face each other. Child 2: “Oh I am only 4”. Child 1: to Researcher: “My brother is 16 so that is why I chose 16”. Child 2: to researcher: “My brother is 4”. Child 1 to Child 2 I got a sticker for my birthday. Do you want to share

mine? Child 2: “Oh yes. Happy birthday!”

17

Both children sing Happy Birthday and make their Avakai dance around.

Child 1: “I want a cake with candles because it’s my birthday too and I am 16 years old”. Child 2: Today is my birthday too. Pow!

Child 2 make her Avakai bump into child 1’s. Child 1 moves the Avakai around and around in circles she sings….

Child 1: “…I am 16. I am the eldest. I am the eldest”. Child 1 makes the Avaikai jump up and down on the table

Child 1: “I can’t stop making it jump up and down on the table. It is excited” Child 1 bends the Avakai forward and says: “I cant help blowing out my candles”.

On the second occasion I presented the scenario that the Avakai were sick:

Researcher: The Avakai is not feeling very well? The child immediately lies her Avakai down.

Child 1: “I’ve got a bad temperature.” The child rolls the Avakai off the side of the table.

Child 1: “I can’t help it I am going to roll off the side of my bed”. The child brings the Avakai back to the table.

Child 1: “Urgh I need to get back into my bed.” The child rolls the Avakai across the table.

Child 1: “Uh oh there is going to be a trouble. A wolf is going to come and eat us up. Help a wolf is going to come and eat us up. I looked out of the window and a wolf is going to come and eat us up. Hello run away as fast as you can. Even though you are not feeling well it is best to do it”.

The child makes the Avakai run across the table. Child 1: “Quick he is going to get you. Run as fast as you can.” The child makes the Avakai run all around the room, over the benches,

walls and tables. Child 1: “Oh phew I am at a safe cottage no one will find me here. Quick come to the cottage. Quick, quick faster, faster!”

The child makes her Avakai jump quickly up and down. Child 1: “I feel sick and I’ve got a headache”. The child lies the Avakai down. She rolls it around on the table. Child1 : “What can I do if I feel so sick? What can I do if I feel so sick? It is good that we are inside and everything is shut. The curtains are shut. The wolf wont get us here”.

Through examples such as these the children used both gestures and movement to naturally link to their spoken narratives when playing with the Avakai. For example, the Avakai were turned to face each other when in conversation, they were placed on their backs when tired or sick, and when they were happy, singing or dancing they were moved around in circles. In addition the children’s spoken narratives drew on their experiences from both everyday life and other stories such as in the case of the “big, bad wolf” storyline and details of the child’s brother’s age when pretending it was an Avakai’s birthday. These findings seem to suggest that the way in which young children naturally play to include movement and gesture alongside their narratives seems to tie with the design of the Avakai that also links these two modes [sound and movement] of communication in its design.

18

Finally, in addition to the above examples of children’s emotional narratives and play, the dolls also included a very tactile display of emotion in the embedded heartbeat of the Avakai, which is another design feature. Specifically, a vibrating heartbeat can be felt through the simple wooden shape of the Avakai when it is held. Earlier I wrote that one of the Avakai designers, Justyna Zubrycka, has a background in design production and that during her studies she was interested in sensory and musical approaches to communication. From this interest the heart beat design was included into the Avakai. The heartbeat seemed to be the technological detail that added that ‘special/magical’ power to children’s physical play that I described in relation to Matas earlier. In general, the children were fascinated by the heart not only did they want to feel it, but they also wanted to see it, peering in through the charger hole at the back of the Avakai to do so. One child was interested in trying to make the Avakai’s heart beat faster. He ran around and around the room with it, paused and felt the heart, declared the Avakai strong and kept on running. When the technology enhanced Avakai was placed on its side its heart beat changed, it fell asleep and started to snore. It took a few seconds for this element to work and children discussed amongst themselves how long it took them to sleep and what they did trying to do so. In other instances the Avakai were swung in the air which produces a “Whoop” sound, when this happened right away the children would feel for the heart when they caught it again. Finally, when the technology-enhanced Avakai was sick the children touched to see how its heart was beating. Child Use Theme 3: The Compartment As has already been described the Avakai designers felt that the technology should complement traditional play. From the very first prototype they allowed the toy to be opened up so that things could be stored inside (Figure 17):

Figure 17 An early Avaikai compartment

The compartment in the Avakai can currently only be used in the non-technology enhanced prototype as when the technology is installed it is currently kept in this space. As a result there was no way of getting insight into how the children might use this space with the final technology enhanced Avakai. However, the compartment featured in the children’s play of the non-technology enhanced Akavai in two ways.

19

Firstly, it became a storage space for collections of “stuff”. Children on the project put feathers, sequins, stickers and other things in the base. Sometimes they told other children and asked them to come find their Avakai and its hidden treasures. This practice of storing things in containers reminds me of everyday practices I have anecdotally witnessed in young children’s lives (Figure 18).

Figure 18 Childhood collections of “stuff” This provides another interesting point of contemplation on young children and their interest in materials, the physical and tactile nature of things. Also how materials and “stuff” can be appropriated in play. For example, stones have been reoccurring features of my child’s collections since he was very young. At times I would find pockets full of them before tipping his clothes in the washing. One day he said they were Star Wars rocks for a game he played with friends in the school playground. This reminded me of Karen Wohlwend’s (2009) work about how children substitute everyday items to represent digital technology when none is available for them to play with. Secondly, the compartment became an extension of the Avakai’s body. Imaginary strawberries were put inside when it was hungry, as well as a midnight feast on asleep over. Also as is shown in the following extract a brick was used to represent medicine and placed inside the compartment when the Avakai was sick:

C1: “What would you like to eat?” C2: “Get me some orange juice”. C1: “Orange juice?” C2: “Orange juice.” C1: “Orange juice there you go. What would you like to eat?” C2: “Strawberries.” C1 “Strawberries. There you go.” C2: “With one hundred spoons of sugar.”

Child 1 counts out imaginary sugar while counting to 100 C1: “There you go.” C2: “Thank you.”

Child 2 opens up the base of her Avakai and puts the imaginary strawberries inside.

C1: “Whenever you want to you could just open yourself [referring to the compartment in the base of the Avakai] and put some medicine in it”. Child 1 walks around the room. Then picks up a Lego brick and puts it

inside the base of her Avakai. She walks to the researcher C1 to researcher: “Guess what I’ve got inside here?”

She opens it up to show the researcher. C1: “That’s a medicine”. Researcher: “That’s a piece of Lego”.

20

C1: “And I am pretending it is medicine.” Child 1 opens up the Avakai to show Child 2.

C2: “It’s a brick.” C1: “No it is medicine”. C2: “Lets swap [bases of the Avakai] my Avakai needs medicine”.

The children swap the bases of their Avakai

Figure 18 Medicine in an Avakai’s compartment

Conclusion

This report has described the two section of the STSM, that of learning about the Avakai’s design in the initial knowledge exchange stage and secondly understanding how a group of young children used the toy. The knowledge exchange side of the project has implications for thinking about how toys are designed for young children and where children exist, if at all, in these design stages. Also how academic and those in industry designing and making products for young children can come together to combine knowledge and knowhow. The second stage of the project emphasises the multimodal nature of children’s play to include tactile, sound and movement alongside more commonly considered modes such as speech and writing. I believe that these initial findings in relation to modes of communication that are commonly less foregrounded in studies of children’s literacy and wider communication practices have potentially interesting implications. For example, Theme 1 concerning children’s use of the toy that connected to customisation of the design ties with ideas I have been working on recently with Jackie Marsh around maker spaces for young children. As a result of this developing work with Jackie Marsh while conducting the KE stage of the STSM I undertook visits to Fab Lab, a maker space in Berlin where users can learn how to use a range of digital and practical tools such as 3D printers, laser cutters, metal work, knitting and other textile machines. In addition I was introduced to Glück who provide technology and play workshops for young children in Berlin. Teaching skills such as making simple circuits, using electricity conducting paint and learning from the work of professional artists, designers and makers. These experiences, combined with the findings of this project, allow me to imagine a future where children are designing and customizing their own toys.

21

Now, as much as ever, I believe it is important to expose young children to a range of art and art practices. Specifically, showing children how creativity and the arts are part of wider communication practices matters for allowing children to understand past, current and future creative industries. Being new-ish to living in Sheffield, in the North of England I am often amazed at the attention that is still given to the Steel Industry. Sheffield’s history is fascinating but it is still foremost in its identity. As an outsider I find it somewhat strange that my 7 year-old son’s education includes so many reference to an industry he will not be able to join. At the same time Sheffield has a lot of new creative industries. I know this through my work with local digital games makers, but evidence of this rarely makes into any of the city museums or in my son’s classroom. As a result I hope this study provides some thought to other COST- action members about how technology is not only changing play and communication practices but also making it possible for children to be aware of and contribute to these changes.

References

Bezemer, J. and Kress, G. (2010) Changing text: a social semiotic analysis of textbooks. Designs for Learning, Vol.3, No.1-2, p. 10-29. Braun, V. & Clark, V. (2006) Using Thematic Analysis in Psychology, Qualitative Research in Psychology, Vol. 3, No.2, p.77-101. Geo Panch (2007) A Brief History of Modern Communication. Available as part of an online website Geo Paunch Design and Direction < http://www.geopanch.com/> [Accessed 01/04/16] Hughes, B. (2002). A Playworker’s Taxonomy of Play Types. 2nd edition, London: Playlink. Jewitt, C., Kress, G., Ogborn J. & Tsatsarelis, C., (2001) Exploring Learning Through Visual, Actional and Linguistic Communication: the multimodal environment of a science classroom. Educational Review, Vol. 53, No. 1. Kusahara Machiko (2006) Device Art: A New Form of Media Art from a Japanese Perspective. Intelligent Agent, Vol. 6, No. 2. Available online at <http://www.intelligentagent.com/archive/ia6_2_pacificrim_kusahara_deviceart.pdf> [Accessed 01/04/16] Kress, G. (2003) Literacy in the New Media Age. Oxfordshire & New York: Routledge. Kress, G. & van Leeuwen, T. (2006) Reading Images. The Grammar of Visual Design. Second Edition. London & New York: Routledge. Kress, G. & van Leeuwen, T. (1996) Reading Images. The Grammar of Visual Design. First Edition. London & New York: Routledge. McCloud, S. (1993) Understanding Comics. Northhampton, MA: Kitchen Sink Press. Norman, D. A. (1989) The Design of Everyday Things. New York: MIT Press. Rowsell, J. (2013) Working with Multimodality: Rethinking Literacy in a Digital

22

Age. London and New York: Routledge. Wohlwend, K. (2009) Early adopters: Playing new literacies and pretending new technologies in print-centric classrooms. Journal of Early Childhood Literacy, vol. 9 no. 2, p.117-140. Yamada-Rice, D, Finn, R. (under review) The visual portrayal of emotion: examples from Japanese semiotics and graphic novels about mental health. Visual Communication.