Design-Based Research
An important aspect of our work is documenting our game design process, with the goal of making our design choices explicit as well as making clear the steps that were taken in the development and testing of the game and our emerging understanding of how computer science concepts might be married with gameplay and story. We are following tenets of design-based research, a methodological approach “for understanding how, when, and why educational innovations work in practice” (The Design-Based Research Collective, 2003, p. 5). The goal of design-based research is not just to design and test educational interventions, but to add to a larger body of design knowledge (in this case, of educational game design) and contribute to contextualized theories of teaching and learning (ibid, p. 8).
Our team members met weekly or bi-weekly by conference call throughout the first year; most of these conversations were audio-recorded to document the team’s decision-making process. Meeting notes, multiple versions of written design documents, audio and video recordings of focus groups, prototype and pilot testing, as well as field notes, comprise additional data that will be used to create a “design narrative” that fully describes the rationale for and evolution of our games, as well as the context in which they were developed.
Design Issues
As we worked on our games, we grappled with several design issues of significance to the wider field of game-based learning and to CS learning in particular. We will be exploring these issues in academic papers for a wider audience; below is a brief description of several issues.
Issue #1: What CS concepts might be introduced to middle school students through games? In the rationale for our project, we noted that the current emphasis in CS games is on using programming as a means of introducing learners to computer science. This was certainly evident in the games we reviewed before and during this first project year. We intended to introduce computer science concepts in a manner more aligned with approaches commonly associated with teaching computational thinking, such as CS Unplugged. Computational thinking, however, has been defined in different ways, and in some cases so broadly that it loses a connection to computer science. There also is no general agreement on how to define a CS “concept.”
We began by drawing on the College Board’s (2014) Advanced Placement (AP) CS Principles curriculum framework, which consists of seven big ideas, each with a set of essential questions, enduring understandings, learning objectives, and knowledge statements. The curriculum also identifies six “computational thinking practices;” each learning objective reflects a computational thinking practice as well as one of the seven big ideas. This framework initially was useful, since in our games, we wished to marry conceptual understanding with practice, though many of the objectives were too broad or emphasized declarative knowledge. Furthermore, we needed to identify a limited set of concepts appropriate for informal learning settings. We focused on concepts that are fundamental for a person to “think like a computer scientist.” In our view, a computer scientist needs to be able to understand and to write an algorithm to solve the problem at hand, to represent data in a meaningful way, and to organize data to support efficient retrieval, and to abstract general solutions from specific case examples. Therefore the (broad) concepts we initially selected were (1) data representation, (2) algorithms, (3) data searching and sorting, and (4) abstraction.
In narrowing the games’ focus, we relied on the following criteria: (a) Is the concept at an appropriate level of complexity for this age group? (b) Is the concept general enough to be understood by a student with no prior computer science training? and (c) Does the concept have the potential to be explicated through a puzzletype game format? Ultimately we dropped abstraction, which proved to be too complex given our goals.
Issue #2: What are appropriate CS learning objectives for game-based learning? While game-based learning has at times been promoted as a way to make any sort of learning or content more appealing and effective, our team began with the assumption that games are not always the optimal approach to learning. Games and game mechanics need to be selected for their strengths in achieving particular outcomes. Furthermore, games by themselves are not necessarily complete learning experiences, but rather should be integrated with other strategies such as pre- and post-game discussion and reflection. These assumptions guided our process of identifying specific learning objectives in relation to each broad concept, as well as in developing appropriate game mechanics. We identified a broad learning objective or knowledge statement for each game; for example, the knowledge statement for Data Representation is “Data can be represented in many different ways, and still have the same meaning. The different representations are for different purposes (faster, shorter, more visual, more hidden, more secretive). Changing from one representation to another has to follow agreed translation rules.” Each objective or knowledge statement could be translated into a game mechanic that allows the player to have an experience that instantiates the concept. building on the work of PI Gee in particular, we also took the stance that all learning does not and should not take place through game play alone. We are creating supplemental material that informal educators can use to introduce the games, explain key terminology (e.g., encoding and decoding) and guide reflection on the game play experience. In doing so, however, we are continually confronting the question of whether these activities are too didactic and will detract from the game experience.
Issue #3: How might game play be meaningfully integrated with fictional context and story to support learning? This issue is at the heart of our study, and one that educational game designers have not adequately grappled with. We do not assume that all games need story, nor that all learning requires a narrative framing; however, in the case of abstract concepts, our hypothesis is that story or at least fictional context will enhance learning as well as engagement. While we cannot study the variation of the entire spectrum of narrative elements in this small scale project, we are carefully documenting our design decisions related to story and players’ responses to story elements. Developing stories and fictional settings for analog games poses challenges that differ somewhat from digital games. Constructing a “real-life” immersive world was not feasible; instead, we had to create fictional settings and storylines that would capture players’ imaginations with few props.
All of our stories built on previous work suggesting that girls are particularly motivated by STEM activities that engage them in helping others or contributing to the social good. For each concept and game, the team identified a central premise that was consistent with this theme, offered a means of instantiating the CS concept, and seemed to be emotionally appealing. A premise sets the time and place, the main characters and the player’s objective or problem to be addressed (Fullerton, 2014). The stories themselves take the form of linear narratives; that is, players encounter a fixed sequence of challenges, and while the specific outcome may vary (e.g., which team wins), the ultimate story resolution remains consistent. One issue that already is apparent is how such linear narratives might limit the replayability of the games, and thus their value for practice with new concepts. Another issue seems to be that the more engaging the game play, the less players seem interested in the story. We will be examining these and other issues as we use the games in informal science learning programs and collect data on player engagement and learning over the coming year.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8, 35-37. http://www.designbasedresearch.org/reppubs/DBRC2003.pdf