Ruf, A., Mühling, A., & Hubwieser, P. (2014). Scratch vs. Karel: impact on learning outcomes and motivation. In Proceedings of the 9th Workshop in Primary and Secondary Computing Education (pp. 50-59). New York: ACM.

Summary

Type: Empirical

Purpose: "This paper presents the results of an experiment regarding the e ects of using one of two di erent programming environments in secondary schools. Both \Scratch" and \Karel the Robot" have been successfully used in these settings previously. These two environments are also representative for two classes of programming environments for beginners. One is more graphically oriented and may therefore alleviate the steep learning curve of programming while the other is text-based and therefore more akin to\real"programming. Also, one places more emphasis on the visualization of program structure and the other emphasizes visualizing program flow" (p. 1).

Findings: "The abilities of the students were tested before and after the experiment as well as their intrinsic motivation and the perceived self-regulation. The results show, that the class using Scratch has higher intrinsic motivation and performs better, however the Karel class shows a higher identi ed [self]regulation" (p. 1). "The Scratch class has a signi cantly higher value on the sub scale of interest/enjoyment regarding their intrinsic motivation...Concerning the actual abilities, the picture is not as clear cut. While the Scratch class did perform better on the post-test as discussed in more detail below, there are no meaningful di erences to be found in the graded exam. The better performance in the post-test is especially interesting, since there is no signi cant di erence on the intrinsic motivation sub scale regarding the perceived competence. In other words, the students of the Scratch class did perform better on average than the students of the Karol class, but they did not perceive themselves as more competent...Concerning the self-regulation, there is only one sub scale that shows a signi cant di erence between the two classes. The identi ed regulation is higher for the Karol class. This is very interesting as it shows that the programming environment is perceived more relevant for \real-life" than Scratch { even though Scratch was more fun for the students!" (p. 7).

Recommendations: "there are many additional experiments that could provide further insight into the learning curve of programming. For example, it may be interesting to have two classes learn both Karol and Scratch but in opposite order. Maybe teaching both environments with their respective focus on program structure and program ow may provide a \best of both worlds" approach for the students... it seems favorable to choose an environment that visualizes program structure..." (p. 8).


Methodology

Sample Size: 56

Notes: 7th grade (students in Germany). Informatics is compulsory in German high schools and teachers have a set curriculum to follow. Students at a Bavarian Gymnasium (HS). 29 students in Scratch and 27 in Karol.