Validation of an indicator model (INCODIES) for assessing student digital competence in basic education

  1. García-Valcárcel Muñoz-Repiso, Ana 1
  2. Casillas Martín, Sonia 1
  3. Basilotta Gómez-Pablos, Verónica Magdalena 2
  1. 1 Departamento Didáctica, Organización y Métodos de investigación, Universidad de Salamanca
  2. 2 Universidad a Distancia de Madrid
    info

    Universidad a Distancia de Madrid

    Madrid, España

    ROR https://ror.org/01r9skd65

Revista:
NAER: Journal of New Approaches in Educational Research

ISSN: 2254-7339

Ano de publicación: 2020

Título do exemplar: Intercultural education from Pre-School to Secondary education, new approaches and practices

Volume: 9

Número: 1

Páxinas: 110-125

Tipo: Artigo

DOI: 10.7821/NAER.2020.1.459 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

Outras publicacións en: NAER: Journal of New Approaches in Educational Research

Obxectivos de Desenvolvemento Sustentable

Resumo

Digital competence is a basic skill for citizens and should be systematically assessed, taking into account characteristics such as knowledge, skills and attitudes. However, measuring and identifying the digital competence of students is still a major challenge. To this end, the indicator model (INCODIES) has been developed and validated following the structure of the DIGCOMP model, which can be used to create assessment tests that measure student digital competence when they have completed their compulsory education. This article presents the results of a process of content validation carried out by 77 experts, in both the school and the university context. The analysis of the experts’ assessments and the level of agreement among those rating them were carried out following an original method which consists of determining a quantitative index for the validity of content called the Content Validity Ratio (CVR). The results obtained allowed us to confirm that the competences included in the reference framework DIGCOMP are correctly represented in the items formulated for the INCODIES model, which obtained high content validity. The model can serve as a structure and basis for the designing of specific evaluation tests on this topic.

€ Ver financiamento

Información de financiamento

Funded by: Ministry of Economy and Competitiveness, Spain Funder Identifier: http://dx.doi.org/10.13039/501100003329

Financiadores

Referencias bibliográficas

  • Aesaert, K, & Braak, J Van (2014). Exploring factors related to primary school pupils’ ICT self-efficacy: A multilevel approach. Computers in Human Behavior, 41, 327–341. https://doi.org/10.1016/j.chb.2014.10.006.
  • Aesaert, K, Voogt, J, Kuiper, E, & Braak, J Van (2017). Accuracy and bias of ICT self-efficacy: An empirical study into students' over- and underestimation of their ICT competences. Computers in Human Behavior, 75, 92–102. https://doi.org/10.1016/j.chb.2017.05.010 .
  • Ait, E, & González, J (2017). Instruments per a l’avaluació de la competència digital de l’alumnat de secundària. UTE. Revista de Ciències de l’Educació, 1, 21–34. http://dx.doi.org/10.17345/ute.2017.1.1747 .
  • Ala-Mutka, K (2011). Mapping Digital Competence: Towards a Conceptual Understanding (and others, Ed.). Luxemburgo: JRC-IPTS European Commission. Retrieved from http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4699 .
  • Almanasreh, E, Moles, R, & Chen, T (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066 .
  • Amaro, A C, Oliveira, L, & Veloso, A I (2017). Intergenerational and collaborative use of tablets: «in-medium» and «in-room» communication, learning and interaction. Observatorio (OBS), 11(1), 83–94. https://doi.org/10.15847/obsOBS1102017995.
  • Barroso, J, & Cabero, J (2010). La investigación educativa en TIC. Visiones prácticas (and others, Ed.). Madrid: Síntesis.
  • Calvani, A, Fini, A, Ranieri, M, & Picci, P (2012). Are young generations in secondary school digitally competent? A study on Italian teenagers. Computers & Education, 58(2), 797–807. https://doi.org/10.1016/j.compedu.2011.10.004.
  • Cameron, L, Wise, S L, & Lottridge, S M (2007). The development and validation of the information literacy test. College & Research Libraries, 68(3), 229–237. https://doi.org/10.5860/crl.68.3.229 .
  • Carrió, C, Soler, M, & Aymerich, M (2015). Análisis de la Validez de Contenido de un Cuestionario de Evaluación del Aprendizaje Basado en Problemas: Un Enfoque Cualitativo. Formación universitaria, 8(1), 13–22. https://dx.doi.org/10.4067/S0718-50062015000100003 .
  • Colás, P, Conde, J, & Reyes, S (2017). Competencias digitales del alumnado no universitario. Revista Latinoamericana de Tecnología Educativa, 16(1), 7–20. https://doi.org/10.17398/1695-288X.16.1.7.
  • Dardanou, M., & Kofoed, T. (2019). It is not only about the tools! Professional digital competence. In G. Colette, & I. Palaiologou (Eds.), Early learning in the digital age (pp. 61-76). London: SAGE Publications.
  • Ding, C, & Hershberger, S (2002). Assessing content validity and content equivalence using structural equation modeling. Multidisciplinary Journal, 9(2), 283–297. https://doi.org/10.1207/S15328007SEM0902_7 .
  • Dornaleteche, J, Buitrago, A, & Moreno, L (2015). Categorización, selección de ítems y aplicación del test de alfabetización digital online como indicador de la competencia mediática. Comunicar, 22(44), 177–185. https://doi.org/10.3916/C44-2015-19 .
  • Enochsson, A (2019). Teenage pupils’ searching for information on the Internet. Information Research, 24(1), Retrieved from http://www.informationr.net/ir/24-1/isic2018/isic1822.html .
  • Escobar-Pérez, J, & Cuervo-Martínez, A (2008). Validez de contenido y juicio de expertos: una aproximación a su utilización. Avances en Medición, 6, 27–36.
  • Esteve, F, Gisbert, G, & Lázaro, J L (2016). La competencia digital de los futuros docentes: ¿cómo se ven los actuales estudiantes de educación?. Perspectiva Educacional. Formación de Profesores, 55(2), 38–54. https://doi.org/10.4151/07189729 .
  • European Comission. (2007). Competencias clave para un aprendizaje a lo largo de la vida un marco de referencia europeo. Bélgica: Comunidades Europeas, Bélgica: Comunidades Europeas.
  • Fernández-Miravete, A D (2018). La competencia digital del alumnado de Educación Secundaria en el marco de un Proyecto educativo TIC (1:1). Edutec. Revista Electrónica de Tecnología Educativa, 63, 60–72. https://doi.org/10.21556/edutec.2018.63.1027 .
  • Ferrari, A (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe, Joint Research Centre, Institute for Prospective Technological Studies. Sevilla. http://dx.doi.org/10.2788/52966.
  • García, R, Duarte, A, & Guerra, S (2014). Propuesta de un instrumento de evaluación para medir el grado de competencia mediática en la etapa de la educación infantil. Pixel-Bit: Revista de medios y educación, 44, 81–96. https://doi.org/10.12795/pixelbit.2014.i44.06 .
  • García-Valcárcel, A (2016). Las competencias digitales en el ámbito educativo. Retrieved from http://hdl.handle.net/10366/130340.
  • García-Valcárcel, A, & Tejedor, F J (2017). Percepción de los estudiantes sobre el valor de las TIC en sus estrategias de aprendizaje y su relación con el rendimiento. Educación XXI, 20(2), 137–159. https://doi.org/10.5944/educxx1.19035.
  • Gisbert, M, Espuny, C, & González, J (2011). INCOTIC. Una herramienta para la @utoevaluación diagnóstica de la competencia digital en la universidad. Profesorado: Revista de curriculum y formación del profesorado, 15(1), 75–90.
  • Gisbert, M, & Esteve, F (2011). Digital learners: la competencia digital de los estudiantes universitarios. La cuestión universitaria, 7, 48–59.
  • González, J, Espuny, C, Cid, M J De, & Gisbert, M (2012). INCOTIC-ESO. Cómo autoevaluar y diagnosticar la competencia digital en la Escuela 2.0. Revista de Investigación Educativa, 30(2), 287–302. https://doi.org/10.6018/rie.30.2.117941 .
  • Hazar, E (2018). Digital competence in primary education: The case of Turkish language, mathematics and personal and social development courses. International Online Journal of Education and Teaching (IOJET), 5(2), 443–458.
  • Hernández, A, & Iglesias, A (2017). La importancia de las competencias digitales e informacionales para el desarrollo de una escuela intercultural. Interacçoes, 43, 205–232. https://doi.org/10.25755/int.12038 .
  • Hyrkäs, K, Appelqvist-Schmidlechner, K, & Oksa, L (2003). Validating an instrument for clinical supervision using an expert panel. International Journal of nursing studies, 40(6), 36–37. https://doi.org/10.1016/S0020-7489(03)00036-1 .
  • ISTE, (2016). ISTE Standards for students. Retrieved from http://www.iste.org/standards/ISTE-standards/standards-for-students.
  • Katz, I R (2007). Testing information literacy in digital environments: ETS's iSkills assessment. Information technology and Libraries, 26(3), 3–12. https://doi.org/10.6017/ital.v26i3.3271 .
  • Lawshe, C H (1975). A quantitative approach to content validity. Personnel psychology, 28(4), 563–575. http://dx.doi.org/10.1111/j.1744-6570.1975.tb01393.x .
  • Lee, L., Chen, D.T., Li, J.Y., & Lin, T.B. (2015). Understanding new media literacy: The development of a measuring instrument. Computers & Education, 85, 84–93. https://doi.org/10.1016/j.compedu.2015.02.006 .
  • Matamala, C. (2014). Factores predictivos de las competencias TIC en alumnos chilenos de secundaria. Revista iberoamericana de educación, 67(1), 121–136. https://doi.org/10.35362/rie671267 .
  • MEC-INTEF. (2017). Marco común de la Competencia Digital Docente. Retrieved from http://aprende.educalab.es/wp-content/uploads/2017/ 11/2017_1020_Marco-Com%C3%BAn-de-Competencia-Digital-Docente.pdf .
  • Oakleaf, M (2008). Dangers and Opportunities: A Conceptual Map of Information Literacy Assessment Approaches. Portal: Libraries and the Academy, 8(3), 233–253. https://doi.org/10.1353/pla.0.0011.
  • Pinto, M (2009). Design of the IL-HUMASS survey on information literacy in higher education: A self-assessment approach. Journal of Information Science, 36(1), 86–103. https://doi.org/10.1177/0165551509351198 .
  • Pinto, M, & Guerrero, D (2017). Cómo perciben las competencias informacionales los estudiantes universitarios españoles: un estudio de caso. Investigación bibliotecológica. Investigación bibliotecológica, 31(73), 213–236. http://dx.doi.org/10.22201/iibi.24488321xe.2017.73.57854 .
  • Quiroz, J S, Miranda, P, Gisbert, M, Morales, J, & Onetto, A (2016). Indicadores para evaluar la competencia digital docente en la formación inicial en el contexto Chileno-Uruguayo. RELATEC, 15(3), 55–68. https://doi.org/10.17398/1695-288X.15.3.55 .
  • Rodríguez, M J, Olmos, S, & Martínez, F (2012). Propiedades métricas y estructura dimensional de la adaptación española de una escala de evaluación de competencia informacional autopercibida (IL-HUMASS). Revista de Investigación Educativa, 30(2), 347–365. https://doi.org/10.6018/rie.30.2.120231 .
  • Scherer, R, Rohatgi, A, & Hatlevik, O (2017). Students' profiles of ICT use: Identification, determinants, and relations to achievement in a computer and information literacy test. Computers in Human Behavior, 70, 486–499. https://doi.org/10.1016/j.chb.2017.01.034.
  • Siddiq, F, Hatlevik, O E, Olsen, R V, Throndsen, I, & Scherer, R (2016). Taking a future perspective by learning from the past - A systematic review of assessment instruments that aim to measure primary and secondary school students' ICT literacy. Educational Research Review, 19, 58–84. https://doi.org/10.1016/j.edurev.2016.05.002.
  • Stopar, K, & Bartol, T (2019). Digital competences, computer skills and information literacy in secondary education: mapping and visualization of trends and concepts. Scientometrics, 118(2), 479–498. https://doi.org/10.1007/s11192-018-2990-5.
  • Tristán-López, A (2008). Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo. Avances en Medición, 6, 37–48.
  • Urrutia, M, Barrios, S, Gutiérrez, M, & Mayorga, M (2014). Optimal method for content validity. Revista cubana de educación médica superior. Revista cubana de educación médica superior, 3, 547–558.
  • Van Deursen, A., Helsper, E. J., & Eynon, R. (2016). Development and validation of the Internet Skills Scale (ISS). Information, Communication & Society, 19(6), 804–823. https://doi.org/10.1080/1369118X.2015.1078834 .
  • Van Deursen, A., & Van Diepen, S. (2013). Information and strategic Internet skills of secondary students: A performance test. Computers & Education, 63, 218–226. https://doi.org/10.1016/j.compedu.2012.12.007 .
  • Van Deursen, A., & Van Dijk, J. (2011). Internet skills and the digital divide. New media & Society, 13, 893–911. https://doi.org/10.1177/1461444810386774 .
  • Vuorikari, R, Punie, Y, Carretero, S, & Brande, L Van Den (2016). DigComp 2.0: The digital competence framework for citizens Scientific and Policy Report, Seville: Joint Research Centre of the European Commission.
Fonte de datos: Dialnet