Assessing Online Collaborative Problem Solving Among School Children in Finland: A Case Study Using ATC21S TM in a National Context

Arto Kalevi Ahonen, Susan-Marie Harding

Abstract


Online collaborative problem-solving tasks were piloted in five Finnish schools participating in the Assessment and Teaching of Twenty-first Century Skills (ATC21STM) project. Process stream data from online tasks were captured from 480 Finnish students who explored dyad problem spaces. The log file data were explored to identify indicative behaviours of collaborative problem solving (CPS), which were coded into a series of dichotomous indicators. The Rasch simple logistic model was used to analyse student performance across the social and cognitive dimensions of CPS. Analysis of the Finnish data indicated that the construct can be interpreted using a two-dimensional model, matching the findings of the total cohort of 4,078 students. Further analysis of the data provided evidence that gender has no statistical impact on Finnish students’ CPS abilities in the social or cognitive dimensions. Analysis of the differences in student abilities across grades within and between schools suggested that upper-year students had stronger CPS skills.

 

 


Keywords


Assessment, collaboration, problem solving, school, students

Full Text:

PDF

References


Adams, R., Vista, A., Scoular, C., Awwal, N., Griffin, P., & Care, E. (2015). Framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 115–132). Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_2

Adams, R. J., Wilson, M., & Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.

Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries (OECD Education Working Paper No. 41). Paris: OECD. https://doi.org/10.1787/218525261154

Ahonen, A.K., & Kankaanranta, M. (2015). Introducing assessment tools for 21st century skills in Finland. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (Vol. 2). (pp. 213-226). Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_10

Ahonen, A.K., & Kinnunen, P.(2015). How do students value the importance of twenty-first century skills? Scandinavian Journal of Educational Research, 59(4), 395–412. https://doi.org/10.1080/00313831.2014.904423

Binkley M., Erstad, O., Herman J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66).Dordrecht: Springer. https://doi.org/10.1007/978-94-007-2324-5_2

Bock, R. D., & Lieberman, M. (1970). Fitting response model for dicotomously scored items. Psychometrica, 35, 179–197. https://doi.org/10.1007/bf02291262

Care, E., Griffin, P., Scoular, C., Awwal, N., & Zoanetti N. (2015). Collaborative problem solving tasks. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 85–104). Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_4

Chyung, S.Y. (2007). Age and gender differences in online behavior, self-efficacy, and academic performance. The Quarterly Review of Distance Education 8(3), 213–222.

Csapo, P., Ainley, J., Bennett, R. E., Latour, T., & Law, N. (2012). Technological issues for computer-based assessment. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 143–230). New York, NY: Springer. https://doi.org/10.1007/978-94-007-2324-5_4

Creemers, B., & Kyriakides, L. (2010). School factors explaining achievement on cognitive and affective outcomes: Establishing a dynamic model of educational effectiveness. Scandinavian Journal of Educational Research, 54(3), 263–294. https://doi.org/10.1080/00313831003764529

Gentry, M., Gable, R., & Rizza, M. G. (2002). Students’ perceptions of classroom activities. Are there grade-level and gender differences? Journal of Educational Psychology, 94(3), 539–544. https://doi.org/10.1037//0022-0663.94.3.539

Gordon, J., Halász, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., . . . Wiśniewski, J. (2009). Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education (CASE Network Report No. 87). Warsaw: CASE Centre for Social and Economic Research. https://doi.org/10.2139/ssrn.1517804

Griffin, P., Care, E., & McGaw, B. (2012). The changing role of education and schools. In P. Griffin, B. McGaw, & E. Care (Eds.) Assessment and teaching of 21st century skills (pp. 1–16).Dordrecht: Springer. https://doi.org/10.1007/978-94-007-2324-5_1

Griffn, P., Care, E., & Harding, S.M. (2015). Task characteristics and calibration. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 133–177). Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_7

Greiff, S., Niepel, C., & Wistenberg, S. (2015). 21st century skills: International advancements and recent developments. Thinking Skills and Creativity, 18, 1–3. https://doi.org/10.1016/j.tsc.2015.04.007

Harding, S.M.E., Griffin, P.E., Awwal, N., Alom, B.M.M., Scoular, C. (2017). Measuring collaborative problem solving in a mathematics context, AERA Open. https://doi.org/10.1177/2332858417728046

Harding, S.M.E., & Griffin, P.E. (2016). Rasch measurement of collaborative problem solving in an online environment. Journal of Applied Measurement, 17(1), 35–53.

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem-solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 37–56).Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_2

Hämäläinen, R, & Vähäsantanen, K. (2011) Theoretical and pedagogical perspectives on orchestrating creativity and collaborative learning. Educational Research Review, 6, 169–184. https://doi.org/10.1016/j.edurev.2011.08.001

Järvelä, S., & Hadwin, A. (2013). New frontiers: Regulating learning in CSCL. Educational Psychologist, 48(1), 25–39. https://dx.doi.org/10.1016/j.chb.2015.01.050

Kartovaara, E. (2009). Opetuksen järjestäjien ja rehtoreiden näkemyksiä ja kokemuksia perusopetuksen vuoden 2004 opetussuunnitelmauudistuksesta [Experiences from curriculum change in 2004]. Helsinki: Opetushallitus.

Krokfors, L., Kangas, M., Vitikka, E., & Mylläri, J. (2010). Näkökulmia tulevaisuuden koulupedagogiikkaan [Insights into future school pedagogy]. In R. Smeds, L. Krokfors, H. Ruokamo, & A. Staffans (Eds.), InnoSchool – välittävä koulu. Oppimisen verkostot, ympäristöt ja pedagogiikka [InnoSchool – caring school. Learning networks, learning environments and pedagogy] (pp. 51–85). Espoo: Aalto-yliopisto.

Lekholm, A. K. (2011). Effects of school characteristics on grades in compulsory school. Scandinavian Journal of Educational Research, 55(6), 587–608. https://doi.org/10.1080/00313831.2011.555923

Miller, M., & Hadwin, A. (2015) Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior, 52, 573–588. https://dx.doi.org/10.1016/j.chb.2015.01.050

Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177–196. https://doi.org/10.1007/bf02294457

Niemi, E. K. (2012). Opettajakysely. In E. K. Niemi (Ed.), Aihekokonaisuuksien tavoitteiden toteutumisen seuranta-arviointi 2010 [Evaluation of the fulfilment of cross-curricular themes 2010] Koulutuksen seurantaraportit 2012:1 (pp. 19–39). Helsinki: Opetushallitus.

OECD. (2016). PISA 2015 results (Vol. 1): Excellence and equity in education. Paris: OECD Publishing. http://dx.doi.org/10.1787/9789264266490-en

OECD. (2017). PISA 2015 Results (Volume V): Collaborative Problem Solving, PISA, OECD Publishing,Paris. http://dx.doi.org/10.1787/9789264285521-en

OECD. (2017). PISA 2015 collaborative problem solving framework. Retrieved from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf

Pepper, D. (2011). Assessing key competences across the curriculum and Europe. European Journal of Education, 46(3), 335–353. https://doi.org/10.1111/j.1465-3435.2011.01484.x

Rotherham, A. J., & Willingham, D. (2009). 21st century skills: The challenges ahead. Educational Leadership, 67(1), 16–21.

Silva, E. (2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 90(9), 630–634. https://doi.org/10.1177/003172170909000905

Skelton, C. (2001). Schooling the boys: Masculinities and primary education. Buckingham: Open University Press.

Strijbos, J.-W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies,4(1), 59–73 https://doi.org/10.1109/tlt.2010.37

Tang, M., & Neber, H. (2008). Motivation and self-regulated science learning in high-achieving students: Differences related to nation, gender, and grade-level. High Ability Studies, 19(2), 103–116. https://doi.org/10.1080/13598130802503959

Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A review of school climate research. Review of Educational Research, 83(3), 357–385. http://dx.doi.org/10.3102/0034654313483907

Vygotsky, L. S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.

Woods, K., Mountain, R., & Griffin P. (2015). Linking developmental progression to teaching. In P. Griffin, & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 267–292). New York, NY: Springer. https://doi.org/10.1007/978-94-017-9395-7_14

Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ACER ConQuest version 2.0: Generalised item response modelling software. Melbourne: ACER Press.

Zimmerman, B. J., & Martinez-Pons, M. (1992). Student differences in self-regulated learning: Relating grade, sex, and giftedness to self-efficacy and strategy use. Journal of Educational Psychology 82(1), 51–59. https://doi.org/10.1037//0022-0663.82.1.51


Refbacks

  • There are currently no refbacks.


e-ISSN: 1694-2116

p-ISSN: 1694-2493