1

TINKERING WITH POCKET CODE, A SCRATCH-LIKE PROGRAMMING APP FOR YOUR SMARTPHONE

Wolfgang Slany

1)

Abstract Pocket Code allows you to create and execute Catrobat programs in a visual, "LEGO-style" programming environment. Pocket Code and Catrobat are inspired by MIT’s Scratch but independently developed and entirely running on smartphones and tablets – no Laptop or PC whatsoever is needed to create or execute Catrobat programs. With Pocket Code you can develop your own games, animations, or music videos, taking advantage of multi-touch, built-in sensors, and the full display resolution of your device. It also allows wirelessly controlling external hardware such as Arduino boards, Lego Mindstorms robots, or Parrot’s AR.Drone. Pocket Code allows you to download Catrobat programs that others have made and upload programs to the Web, all under free open source licenses to foster learning, sharing, and remixing. All major mobile platforms are supported. Google is promoting Pocket Code through their Google Play for Education platform, and Catrobat has been chosen as the official programming language for OLPC tablets. Pocket Code encourages tinkering and allows you to iteratively create your own apps using just one finger. A first app can be running in less than one minute. I will demo Pocket Code live during my presentation and report about our experiences in physics project at school. Keywords visual programming, kids, children, teenagers, smartphones, tablets, Arduino, robots, games, mobile app, remixing, schools, computer science education, STEM 1. About the Catrobat project Whether for good or not, most kids – starting from a very young age and everywhere on the globe – will soon have their own smartphones. Our non-profit free open source Catrobat project aims at unleashing their creativity by encouraging them to create their own apps and content. As of May 2014, more than 270 volunteers from all over the world have contributed to the project. Our Pocket Code and Pocket Paint apps are freely available on Google’s Android market. More of Catrobat’s apps and web services, for other smartphone platforms, for other age and interest groups, and with different functionality (e.g., oriented towards music creation), are currently in various alpha and beta stages. Our project thrives on user experience, with a large dedicated team uniquely focussing on usability and design issues, its members continuously testing our software with real kids and advanced recording and analysis systems. The results of these tests are immediately fed into the prioritized list of open issues and user stories. Another focus is software quality through, e.g., permanent code reviews, full automatic test coverage, permanent refactoring, etc. Catrobat has won the Austrian National Innovation Award in the category Multimedia in 2013 and has been chosen for the fourth time as a Google Summer of Code mentoring organization in 2014.

1 wolfgang.slany@catrobat.org – www.catrobat.og

2

2. Your first app in one minute – A few pictures will say more than 1000 words!

Figure 1: After installing Pocket Code, create a new empty program and name it “compass”.

Figure 2: Then create a new object by drawing a needle for the compass using our Pocket Paint app.

3

Figure 3: After cropping the image, name the new object “needle” and start to add a script to it. Note that tapping the “play” triangle button at the bottom right would already show a static, non-moving needle.

Figure 4: The needle of a compass at all times has to keep pointing in the direction of the magnetic north, thus we need a “Forever” loop. Note the similarity with Scratch’s blocks.

4

Figure 5: We then need a brick that points the needle in the correct direction.

Figure 6: Drag and drop the “Point in direction…” brick to insert it into the “Forever” loop. Using the formula

editor, replace the default value “90” with the value of the magnetic field sensor built into the phone.

5

Figure 7: Select the compass sensor value, confirm the formula, and press the ► play button to start your app.

Figure 8: Turn your phone: the needle keeps pointing to the north. Voilà, your first app! Exit your app by pressing your device’s back button, and press the back button again to return to Pocket Code’s main screen.

6

3. Share your program, or explore and remix programs that others have made

Figure 9: Upload your new program to www.pocketcode.org, or download programs that other users have contributed to play, and also to learn from them. All programs are available under free open source licenses,

and remixing projects of others is encouraged as it helps to quickly understand how others have created, e.g., a moving background as used in the “Air fight” program shown above among the most downloaded programs.

4. Tinkering with Pocket Code in a middle school physics course When Google decided to feature Pocket Code on its Google Play for Education platform, it sent a camera team to Austria and gave 30 Nexus 7 tablets to Graz Bilingual International School (GIBS) to create a promotion video2 with GIBS’s teachers and kids using Pocket Code. Since then members of the Catrobat team have continued to work together with GIBS to explore ways how to use Pocket Code in middle and high school curricula. Computer Science courses are compulsory in Austrian schools only in 9th grade (age 14–15), but 10 teachers of other courses (2×physics, psychology, art, biology, 3×music, German, math) were interested in using Pocket Code for projects relating to their own classes (age 12–17). Indeed, with Pocket Code it is easy to let students create games or animations that include spoken foreign languages (recording one’s voice from inside Pocket Code is straightforward), interactive geography quizzes with clickable maps, historical animated stories, animated mathematical proofs (see the Pythagorean Theorem program on the right of Figure 9), interactive art, and much more. One physics teacher immediately volunteered to use Pocket Code with her 8th graders (age 13–14), who just had learned about velocities, acceleration, and gravity in class. Together with the teacher, we decided to let her students develop their own games or animations that, as the sole condition, would have to include simulated gravity. None of the 24 students had any previous experience with

2 http://youtu.be/75i10o_uv0U

7

Scratch or any other kind of programming. Though all of them owned smartphones, for the physics project we had them using the Nexus 7 tablets that Google had donated to GIBS. Of course, some of the students continued exploring Pocket Code at home on their own devices. Instead of teaching the class all details about Pocket Code, I first showed them, projecting from my tablet’s screen to the classroom’s wall through a beamer, some apps that were created with Pocket Code. This included the “Gossip Girl” music animation and the “Air fight 0.5” game both listed under “Most Downloaded” in Figure 9, as well as a live demo of how to program a Parrot AR.Drone using a special version of Pocket Code3. We then gave the tablets to the students and I walked them through the programming of the compass (Figures 1 to 8), again projecting each step from my tablet to the wall. Three pedagogy students from Graz University of Technology were present and were additionally supporting students on a one to one basis. In a regular classroom setting, students from higher classes could do this mentoring. All students thus had successfully created their own first smartphone app, the compass. They had seen the drone programmed live to fly in one meter height in front of them while the video from the drone’s camera, with their faces on it, was streaming live to the projector through Pocket Code on the classroom’s wall using only five bricks of Catrobat code. They also had seen an amazing music video animation really made by a girl of their age, as well as a graphically appealing game with airplanes that could explode spectacularly when shot down. Seeing is believing, but creating a program oneself allows to really acquire computational thinking skills… The students were thrilled and hooked! Since the students had no previous knowledge of programming but we had only a very limited amount of time for the physics project with Pocket Code, we decided that I would jump-start the students on the physics project by programming a very simple game in a highly iterative way, together with them, while projecting my tablet’s screen to the wall. Students who wanted to follow their own ideas were free to do so. At any time, students could ask for help from the three university students who were also present. My first step was to have a green ball (with a red spot to indicate the direction of its movement) speed around on the screen while bouncing from the walls, without considering gravity yet (script of the ball in Figure 10) — this task is easy to explain and implement. The intention also was to quickly give students a feeling of successful achievement. Note that “Forever” loops have a built-in delay (as in MIT’s Scratch) so that games have the same speed on all devices. Instead of just presenting the solution, I always asked them to first try to find the next step themselves before showing them one possible way to achieve whatever was next, e.g., by asking them to find and add the right brick to move the ball. Drawing the ball (or something more creative…) and letting it move in various directions, with various speeds, with and without the ball bouncing from the walls, all kept us busy for some time. Once all students had this successfully running in various forms, I suggested adding a paddle at the bottom of the screen. The paddle should move according to the horizontal inclination of the tablet. This required experimenting: Where to initially place the paddle, which inclination sensor to use, how to make sure that the paddle would not leave the screen, and with or without a minus in front of the sensor value. Students who had already finished ahead of others were invited to, in the mean time, decorate their ball and/or paddle, or to add a background, either by drawing the background themselves, or by taking photos with the built-in camera and then editing them, or to continue otherwise exploring Pocket Code and Pocket Paint. Pocket Paint allows to make parts of images transparent as well as to zoom up to pixel level, thus allowing to easily create and edit in a detailed

3 See http://youtu.be/350-8Fqgb1c for a video of how to program the drone using an earlier version of Pocket Code. Note that the version of Pocket Code supporting Parrot’s AR.Drone is still in alpha stage.

8

way the looks of objects in Pocket Code. After a few minutes, all students had some kind of moving ball and a paddle controlled via the tablet’s inclination (see Figure 10 below).

Figure 10: First version of the ball and the paddle, without gravity yet.4 The next step was to let the ball react to when it was touching the paddle, at first by simply stopping the ball’s movement for some time using a “Wait 1 second” brick. Once this worked nicely, we added a small change of direction of the ball bouncing from the paddle, the change being proportionate to the current inclination of the tablet, i.e., to allow “cutting” the ball according to the movement of the paddle. All this took quite some time to get right, e.g., we had to decide whether to add or to subtract the inclination from the direction of the ball. However, after some tinkering the ball bounced from the paddle in a realistic way according to the “spin” resulting from cutting, so we removed the wait brick and finally could move on to adding gravity to the ball in the next step. First, we added a “speed” variable to be used in the move brick. Next, during each iteration of the “Forever” loop, we first memorized the current x and y coordinates by storing them in appropriately named variables, and then moved the ball in its current direction for “speed” steps, followed by a displacement of the ball on the y-axis by a “gravity” constant. I walked them through the computation of the new velocity vector by subtracting the memorized x and y coordinates from the new x and y coordinates of the ball, finally computing the new scalar speed by applying the Pythagorean Theorem to the x and y values from the thus updated velocity vector. I did this by

4 Note that, while the Nexus 7 tablets have a resolution of 1920×1200, the program shown in the screenshots was made on an HTC Sensation phone with a smaller resolution of 960×540. The coordinate numbers given in the screenshots correspond to this smaller resolution. A Catrobat program always keeps the coordinate system of the device on which it was originally created, notwithstanding of having been remixed or being played on a device with a different resolution.

9

drawing the vectors on the tablet using our Pocket Paint app. The students had just learned all of these calculations and concepts in their math and physics classes. The next step might have been quite a challenge, though: The students had not yet learned how to use trigonometric and inverse trigonometric functions. The latter normally would be used to get the necessary new angular direction from the updated velocity vector. This value is needed to set the new orientation of the ball so that it moves in the correct direction when the loop is repeated in the next iteration. Luckily, Catrobat (like Scratch) provides a brick that allows one object to point towards another object. The solution: Create a new “pointer” object (its look can be deleted once it has been created), always place it at a position so that its x coordinate is the sum of the new x coordinate value of the ball and the x component of the ball’s velocity (similar for the y coordinate), and then point the ball towards this “pointer” object. The pointer’s x position can be expressed as the sum of the previous x coordinate value of the ball plus two times the x component of the ball’s velocity (similar for y). And that’s it: The ball would then move in parabolic curves, correctly bouncing from all walls, as one would expect from a real physical ball! Parts of the final program5 can be seen in Figure 11. I explained these steps and calculations, always interactively exploring together with the students possible ways to approach the next goal, thereby always taking care not to lose any one of the students, and suggesting steps only when no student had an idea leading in the right direction.

Figure 11: Parts of the final program5 showing the calculations of the new speed value using the Pythagorean Theorem (“Set variable speed …” brick in the screenshot on the left) and the new direction

computed without any inverse trigonometric function from the new velocity vector including gravity. Students then had time for extending this basic program on their own, e.g., by showing increasing “life” points whenever the paddle touches the ball, by adding more balls by copying the ball object, by converting the game to landscape mode, or, alternatively, time for creating an entirely new

5 The complete Catrobat program is available under the name “gravity pong” at https://pocketcode.org/details/1655

10

game. At the end of the class, I was amazed to see, e.g., a very nice looking Badminton game based on our basic gravity ball program on one of the tablets. I had the strong impression that several of the students had not only clearly understood the underlying science, but also caught on with the idea of iteratively creating more and more complex games or animations. They were always tinkering with some new aspect or feature, one small step after the other, with immediate feedback on whether it would work the way they had imagined, sometimes trying different approaches, and always learning from each other as well as from exploring the programs available on Pocket Code’s sharing site. Whether this really was the case of course needs further, more systematic analysis. 5. Success factors of Pocket Code The following aspects contribute in a unique way (compared to similar programs) to sustainably motivating kids to create their own programs with Pocket Code:

Kids can create their own programs as well as run and remix programs made by others on their own private and personal smartphones, which they have readily available in their pockets, at any place and whenever they have time, independently of schools and parents.

Kids can easily show their own programs and those created by others to their friends, e.g., during school breaks, as well as family members, and share these programs with them.

Smartphones and tablets allow using various sensors (inclination, camera, etc) and effectors (sound, vibration, and flashlight) of the devices in one’s programs.

The multitouch nature of the smartphone and tablet displays makes it possible, e.g., to program a piano allowing to play chords with multiple keys at the same time.

The high resolutions of modern smartphones and tablets, the ease of taking snapshots from, e.g., arbitrary web pages, the high resolution photos that can immediately be taken with the built-in camera at any place, the possibility to precisely edit each pixel’s RGB+alpha values of the looks of objects with our Pocket Paint companion app, as well as the high speed of the built-in GPUs allows creating high quality, visually highly appealing programs.

The small size of smartphones and the built-in Bluetooth capability allows mounting them directly on the front of Lego Mindstorms robots. This makes these robots much more interesting through colorful and complex animation of their “face”, allows using the camera to locate and turn to human faces, sound and speech playback as well as speech recognition, and use of acceleration or self-localization sensors to make the robots really autonomous. And of course, Arduino boards and Parrot’s AR.Drone can be programmed as well…

The non-profit and multilingual character of our Catrobat project as well as the free open source character of programs contributed by kids themselves makes creating one’s own apps widely accessible to kids in their own language on a worldwide scale.

Our free open source approach and the high quality of our development process guarantee the long-term availability and future extendibility of our apps and services.

Many fascinating Catrobat subprojects that already are in an advanced development stages will soon allow, e.g., creating real Android apps6 or Android live wallpapers.

The platform agnostic approach of the Catrobat project will soon allow kids to use whatever smartphone or tablet operating systems they own, independently on which platform a Catrobat program originally was created.

Acknowledgements: See http://catrob.at/credits and http://developer.catrobat.org/special_thanks

6 The Tic-Tac-Toe-Master program https://pocketcode.org/details/817 created with Pocket Code, e.g, is available also as a regular Android app (http://catrob.at/gptttm downloaded 19,000 times from Google Play as of May 2014).