Grover, S., Cooper, S., & Pea, R. (2014). Assessing computational learning in K-12. In Proceedings of the 2014 conference on Innovation & Technology in Computer Science Education (pp. 57-62). New York: ACM.

Summary

Type: Empirical

Purpose: To present multiple forms of assessments for student learning of computational concepts used in a 6 week middle-school curriculum. Uses instruments developed in prior research. Reports on the results of two U.S. studies (under "Foundations for Advancing Computational Thinking" - FACT).

Findings: All students scored higher on post-test versus pre-test (regarding mastering Scratch concepts, deciphering the code, debugging). T-tests revealed that the score differences in Study 1 and Study 2 were significant (higher in Study 2). Girls performed significantly better than boys. No significant differences were found by age or grade. The authors also compared the students from the two U.S. studies (all 54 students) on several questions from the Israeli national exam (6 questions, with 22 sub-questions). The Israeli study had 4082 students (Bargury and colleagues, 2013). No significant difference was found, except on question 9 in which the U.S. students performed better. That questions was the only non-multiple choice one and required students to fill in 10 blanks in a Scratch script and involved the highest level of thinking (Evaluating & Creating in Bloom's taxonomy).

Recommendations: Multiple modes of assessment of computational learning in K-12 settings should be used.

Methodology

Sample Size: 54

Participant Type: Northern California middle school students (7th and 8th grade) were involved in two 6-week module-based studies. The FACT module aimed at building awareness of computing and engaging students in foundational concepts of algorithmic flow of control comprising sequence, looping constructs, and conditional logic. Scratch was used.

Notes: 26 students in Study 1 and 28 students in Study 2 (the U.S. studies). The module was taught face-to-face in a computer lab in Study 1 while it was taught online on the Stanford's OpenEdX online platform in Study 2. Lectures were in a short Khan academy style video format in Study 2. Some changes were made in Study 2 from Study 1, and in Study 2 more time was devoted to loops and variables, more projects with games and art, and there was a more formal final project requirement.