C-OLiVE: Impact of group interaction on learning in VEs
COLiVE stands for Collaborative Orchestrated Learning in Virtual Environments, and is a testbed for assessing the impact of (large) group collaboration on learning, mainly in museum spaces. We have designed collaborative learning activities for three people co-located in front of a large-scale display, who have to orchestrate their actions in order to maintain the operation of an olive oil production factory. The game is addressed to middle school students, as the main audience of informal learning settings. We have also integrated game elements to increase motivation and engagement, which are the secondary outcomes we are measuring besides learning. The basic motivation behind this research is to find out if groups of school age students visiting museums with similar installations could benefit from their simultaneous collaboration and acquire more knowledge, compared to being passive recipients of information, as is the case in most similar settings. Our hypothesis is that greater opportunities for social interactions and collaboration will improve game experience and facilitate learning.
We have conducted two studies already, using groups of three students in varying levels of interaction between each other and the game: Auto, where students watch someone else playing the game; 1P, where one player interacts directly with the game and they all suggests and negotiate plans of action; and 3P, where every students has direct agency in the game using a game controller (see a clip of this condition below). The first study was a controlled experiment with middle school students in schools and summer camps in the US (Virginia), from June 2013 to November 2013. We found that tripartite co-located collaboration was associated with improved learning benefits over passively watching someone play a game. The second study took place in a museum in Greece, where the passive (Auto) condition was replaced with a facilitator interacting with the game and prompting students to respond to questions. We found that facilitated students by an expert guide revealed greater learning gains as compared to students playing alone, with one or three simultaneous game controllers. We also compared these results with the previous controlled experiment and besides learning, measures of game experience and social presence where significantly higher in the interactive conditions. Drawing ideas from sociocultural and cognitive theories we interpreted the contradictory findings, identifying the impact of prior knowledge and culture on the game experience, and eventually learning, in these spaces.
Our next step was to investigate how involving a whole classroom with the gaming experience affects engagement, entertainment, and eventually learning. Thus, we have revised the game to include a whole classroom (up to 24 students) in the game play. Our main goal is to evaluate how different levels of audience involvement affect their game experience and learning benefit. For this reason we are comparing these variables among two conditions: Low Involvement (LI), where audience members are attending the game (performed by two of their peers) and have to provide responses in a timely manner, mainly for motivational purposes; and High Involvement (HI), where audience members are active participants in the game and need to either provide their assistance to players (by majority rule) or actually perform specific tasks as a group (by aggregating their input). We hypothesize that the HI group, by affording greater opportunities for interaction, will increase the sense of interconnectedness between players and audience (social presence), improve their game experience overall (i.e., immersion in the story, flow, competence, challenge), and eventually the learning outcome. We conducted this study in collaboration with the Montgomery County Public Schools (Virginia), during Fall 2016. We had 507 students complete all three stages of the experiment occurring on different days (i.e., intro & pre-test, game, post-test). Findings from this study indicated a higher retention of information over a period of two days for the students who were more involved in the game, compared to no difference when the post-test was taken the day after the game. Additionally, students with direct agency in the game (using XBox controllers) revealed greater learning gains and improved game experience compared to the audience (using iPads). Socioeconomic status, affecting academic performance, also impacted the capacity of students to successfully follow a less involved participation scenario.
Our final step would be to revise the game design once more, based on the most significant findings so far, in order to draw some design guidelines that can be used by other researchers and/or museologist in developing large group collabotive games for informal learning. In order to assess the ecological effectiveness of the proposed instructional approach for large groups, we will conduct an informal case study inside an actual museum. The museum will be in charge of recruiting the school groups for their own purposes, giving us permission to use some of their time and space for running the case study. Ideally, the subject of the exhibition will be pertinent to olive oil production; possible examples include food production, complex engineering processes, or Greek heritage. The three main goals of the case study are:
· Make the game part of a museum visit: gaining ecologically valid data was set as one of our main objectives since the beginning.
· Get exposure to large number of students: testing the game in a museum with large throughput will enable gathering more generalizable observations.
· Test the design guidelines: the observations gathered from the previous studies will be tested and refined in an ecological setting.
Proceedings of the 15th SIGCHI International Conference on Interaction Design and Children (IDC textquoteright16), ACM Press ACM Press, Manchester, United Kingdom, 2016, ISBN: 9781450343138.
@conference{241,
title = {Audience Involvement and Agency in Digital Games: Effects on Learning, Game Experience, and Social Presence},
author = {Panagiotis Apostolellis and Doug A Bowman},
url = {http://dl.acm.org/citation.cfm?doid=2930674http://dl.acm.org/citation.cfm?doid=2930674.2930700},
doi = {10.1145/293067410.1145/2930674.2930700},
isbn = {9781450343138},
year = {2016},
date = {2016-06-01},
booktitle = {Proceedings of the 15th SIGCHI International Conference on Interaction Design and Children (IDC textquoteright16)},
publisher = {ACM Press},
address = {Manchester, United Kingdom},
organization = {ACM Press},
abstract = {<p><span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12.8px;">One of the most popular audiences of museums is classroom-size groups of students, in the context of school field trips. However, students do not get adequate involvement during interactive group experiences, which might affect their impression and learning gained from the visit. In this paper, we present our findings from a recent study in middle schools, where 507 students were engaged with their class in a learning game about olive oil production. We had two players directly control the game and varied the level of involvement of the audience (the rest of the class), using iPads. We found that higher involvement in the game afforded greater retention of information after two days, while there was no difference after one day. Also, students with direct agency in the game revealed greater learning gains than the audience members. Results about the impact of socioeconomic status and social interactions on learning are reported, along with the most important design implications.</span></p>},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
<p><span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12.8px;">One of the most popular audiences of museums is classroom-size groups of students, in the context of school field trips. However, students do not get adequate involvement during interactive group experiences, which might affect their impression and learning gained from the visit. In this paper, we present our findings from a recent study in middle schools, where 507 students were engaged with their class in a learning game about olive oil production. We had two players directly control the game and varied the level of involvement of the audience (the rest of the class), using iPads. We found that higher involvement in the game afforded greater retention of information after two days, while there was no difference after one day. Also, students with direct agency in the game revealed greater learning gains than the audience members. Results about the impact of socioeconomic status and social interactions on learning are reported, along with the most important design implications.</span></p>
CHI Workshop: Involving the Crowd in Future Museum Experience Design at the ACM Conference on Human Factors in Computing Systems (CHItextquoteright16), San Jose, CA, 2016.
@conference{242,
title = {Supporting Social Engagement for Young Audiences with Serious Games and Virtual Environments in Museums},
author = {Panagiotis Apostolellis},
editor = {Doug A Bowman},
url = {https://museumsandcrowds.files.wordpress.com/2016/02/supporting-social-engagement-for-young-audiences_apostolellis.pdf},
year = {2016},
date = {2016-05-01},
booktitle = {CHI Workshop: Involving the Crowd in Future Museum Experience Design at the ACM Conference on Human Factors in Computing Systems (CHItextquoteright16)},
address = {San Jose, CA},
abstract = {<p>Considering the shift of museums towards digital experiences that can satiate the interests of their young audiences, we suggest an integrated schema for socially engaging large visitor groups. As a means to present our position we propose a framework for audience involvement with complex educational material, combining serious games and virtual environments along with a theory of contextual learning in museums. We describe the research methodology for validating our framework, including the description of a testbed application and results from existing studies with children in schools, summer camps, and a museum. Such findings serve both as evidence for the applicability of our position and as a guidepost for the direction we should move to foster richer social engagement of young crowds.</p>},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
<p>Considering the shift of museums towards digital experiences that can satiate the interests of their young audiences, we suggest an integrated schema for socially engaging large visitor groups. As a means to present our position we propose a framework for audience involvement with complex educational material, combining serious games and virtual environments along with a theory of contextual learning in museums. We describe the research methodology for validating our framework, including the description of a testbed application and results from existing studies with children in schools, summer camps, and a museum. Such findings serve both as evidence for the applicability of our position and as a guidepost for the direction we should move to foster richer social engagement of young crowds.</p>
@conference{235,
title = {Small Group Learning with Games in Museums: Effects of Interactivity as Mediated by Cultural Differences},
author = {Panagiotis Apostolellis and Doug A Bowman},
year = {2015},
date = {2015-06-01},
booktitle = {ACM Interaction Design and Children},
publisher = {ACM Press},
address = {Medford, MA},
organization = {ACM Press},
abstract = {<p>Museums are rich and complex learning experiences, using a variety of interactive approaches to engage their audiences. However, the largely unstructured nature of free-choice learning calls for alternative approaches that can effectively engage groups of school age students with diverse cultural backgrounds. In this paper, we present our findings from a recent study in a museum in Greece, where triads of stu-dents had to learn about olive oil production using a game enabling different levels of interactivity and collaboration. We found that facilitation by an expert guide led to greater learning gains as compared to students playing alone, with one or three simultaneous game controllers. We also com-pared these results with a previous controlled experiment conducted in the US with middle school students, using the same game but without the ecologically valid facilitation. Drawing ideas from sociocultural and cognitive theories we interpret the contradictory findings, identifying the impact of culture on their (social) interactions, their subjective game experience, and eventually learning, in these spaces.</p>},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
<p>Museums are rich and complex learning experiences, using a variety of interactive approaches to engage their audiences. However, the largely unstructured nature of free-choice learning calls for alternative approaches that can effectively engage groups of school age students with diverse cultural backgrounds. In this paper, we present our findings from a recent study in a museum in Greece, where triads of stu-dents had to learn about olive oil production using a game enabling different levels of interactivity and collaboration. We found that facilitation by an expert guide led to greater learning gains as compared to students playing alone, with one or three simultaneous game controllers. We also com-pared these results with a previous controlled experiment conducted in the US with middle school students, using the same game but without the ecologically valid facilitation. Drawing ideas from sociocultural and cognitive theories we interpret the contradictory findings, identifying the impact of culture on their (social) interactions, their subjective game experience, and eventually learning, in these spaces.</p>
@conference{205,
title = {Evaluating the Effects of Orchestrated, Game-Based Learning in Virtual Environments for Informal Education},
author = {Panagiotis Apostolellis and Doug A Bowman},
doi = {10.1145/2663806.2663821},
year = {2014},
date = {2014-11-01},
booktitle = {Advances in Computer Entertainment Technology (ACE)},
publisher = {ACM Press},
address = {Funchal, Madeira, Portugal},
organization = {ACM Press},
abstract = {<p>In informal learning spaces employing digital content, such as museums, visitors often do not get adequate exposure to content, or they passively receive instruction offered by a museum docent to the whole group. This research aims to identify which elements of co-located group collaboration, virtual environments, and serious games can be leveraged for an enhanced museum learning and entertaining experience. We developed C-OLiVE, an interactive virtual environment supporting tripartite group collaboration, which we used to explore our hypothesis that synchronous, co-located, group collaboration will afford greater learning compared to conventional approaches. In an empirical study, we found some evidence supporting this hypothesis, taking into consideration other factors such as game experience and social presence. Students participating in the three-player condition demonstrated a better understanding of the collaborative tasks compared to their single-player counterparts. We discuss these results and outline future studies using the same virtual environment. </p>},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
<p>In informal learning spaces employing digital content, such as museums, visitors often do not get adequate exposure to content, or they passively receive instruction offered by a museum docent to the whole group. This research aims to identify which elements of co-located group collaboration, virtual environments, and serious games can be leveraged for an enhanced museum learning and entertaining experience. We developed C-OLiVE, an interactive virtual environment supporting tripartite group collaboration, which we used to explore our hypothesis that synchronous, co-located, group collaboration will afford greater learning compared to conventional approaches. In an empirical study, we found some evidence supporting this hypothesis, taking into consideration other factors such as game experience and social presence. Students participating in the three-player condition demonstrated a better understanding of the collaborative tasks compared to their single-player counterparts. We discuss these results and outline future studies using the same virtual environment. </p>
@conference{232,
title = {C-OLiVE: Group co-located interaction in VEs for contextual learning},
author = {Panagiotis Apostolellis and Doug A Bowman},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6802085},
doi = {10.1109/VR.2014.6802085},
year = {2014},
date = {2014-04-01},
booktitle = {2014 IEEE Virtual Reality (VR)2014 IEEE Virtual Reality (VR)},
publisher = {IEEE},
address = {Minneapolis, MN, USA},
organization = {IEEE},
abstract = {<p><span style="color: rgb(51, 51, 51); font-family: sans-serif; font-size: 15px; line-height: 24px;">In informal learning spaces employing digital content, such as museums, visitors either do not get adequate exposure to content or get information through passive instruction offered by a museum docent to the whole group. This research aims to identify which elements of co-located group collaboration, virtual environments, and serious games can be leveraged for an enhanced learning experience. Our hypothesis is that synchronous, co-located, group collaboration will afford greater learning compared to conventional approaches. We developed C-OLiVE, an interactive virtual learning environment supporting tripartite group collaboration, which we will use as a testbed to respond to our research questions. In this paper, we discuss our proposed research, which involves exploring some benefits of the involved technologies and proposing a list of design guidelines for anyone interested to exploit them in developing virtual environments for informal learning spaces.</span></p>},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
<p><span style="color: rgb(51, 51, 51); font-family: sans-serif; font-size: 15px; line-height: 24px;">In informal learning spaces employing digital content, such as museums, visitors either do not get adequate exposure to content or get information through passive instruction offered by a museum docent to the whole group. This research aims to identify which elements of co-located group collaboration, virtual environments, and serious games can be leveraged for an enhanced learning experience. Our hypothesis is that synchronous, co-located, group collaboration will afford greater learning compared to conventional approaches. We developed C-OLiVE, an interactive virtual learning environment supporting tripartite group collaboration, which we will use as a testbed to respond to our research questions. In this paper, we discuss our proposed research, which involves exploring some benefits of the involved technologies and proposing a list of design guidelines for anyone interested to exploit them in developing virtual environments for informal learning spaces.</span></p>
In: Vermeeren, Arnold; Calvi, Licia; Sabiescu, Amalia (Ed.): Museum Experience Design: Springer Series on Cultural Computing, pp. 19 - 43, Springer International Publishing, Cham, 2018, ISSN: 2195-9056.
@inbook{273,
title = {Supporting Social Engagement for Young Audiences with Serious Games and Virtual Environments in Museums},
author = {Panagiotis Apostolellis and Doug A Bowman and Marjee Chmiel},
editor = {Arnold Vermeeren and Licia Calvi and Amalia Sabiescu},
url = {http://link.springer.com/10.1007/978-3-319-58550-5},
doi = {10.1007/978-3-319-58550-510.1007/978-3-319-58550-5_2},
issn = {2195-9056},
year = {2018},
date = {2018-01-01},
booktitle = {Museum Experience Design: Springer Series on Cultural Computing},
pages = {19 - 43},
publisher = {Springer International Publishing},
address = {Cham},
organization = {Springer International Publishing},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
