Análisis de contenido en SPSS y KALPHAProcedimiento para un Análisis Cuantitativo Fiable con la Kappa de Cohen y el Alpha de Krippendorff

  1. Goyanes, Manuel 1
  2. Piñeiro-Naval, Valeriano 2
  1. 1 Universidad Carlos III de Madrid
    info

    Universidad Carlos III de Madrid

    Madrid, España

    ROR https://ror.org/03ths8210

  2. 2 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

Revista:
Estudios sobre el mensaje periodístico

ISSN: 1988-2696

Ano de publicación: 2024

Volume: 30

Páxinas: 125-142

Tipo: Artigo

DOI: 10.5209/ESMP.92732 DIALNET GOOGLE SCHOLAR lock_openAcceso aberto editor

Outras publicacións en: Estudios sobre el mensaje periodístico

Obxectivos de Desenvolvemento Sustentable

Resumo

During the last decades, content analysis has emerged as one of the main research techniques in communication and journalism studies. An extensive literature has described, in great detail, the procedures for its systematic and correct implementation, which has led to more empirically robust and precise research. This advance in the application of methodological protocols has not been transferred, in such a gradual manner, to the computations and statistical tests to establish reliability between coders in publications in Spanish, despite its central importance when it comes to offering reproducible scientific evidence. This has been due, among other reasons, to the need to calculate the indicators in statistical programs, which has hindered their widespread adoption. In this work we illustrate, using SPSS and the KALPHA macro, the computation of two statistical tests to measure the agreement between coders: Cohen’s Kappa (κ) and Krippendorff’s Alpha (a). The objective is, therefore, to offer and detail to the Hispanic academic community a protocol to generalize the use of computations that allow researchers without advanced statistical knowledge to report reliable and standard results in the field.

Referencias bibliográficas

  • Altman, D. G. (1999). Practical statistics for medical research. Chapman & Hall.
  • Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational linguistics, 34(4), 555-596. https://doi.org/10.1162/coli.07-034-R2
  • Bardin, L. (1986). Análisis de contenido. Akal.
  • Bennett, E. M., Alpert, R., & Goldstein, A.C. (1954). Communications through limited-response questioning. Public opinion quarterly, 18(3), 303-308. https://doi.org/10.1086/266520
  • Berelson, B. (1952). Content analysis in communication research. Free Press.
  • Berger, A. A. (2016). Media and communication research methods. An introduction to qualitative and quantitative approaches (4th Ed.). Sage.
  • Bos, W., & Tarnai, C. (1999). Content analysis in empirical social research. International journal of educational research, 31(8), 659-671. https://doi.org/10.1016/S0883-0355(99)00032-4
  • Caffarel-Serra, C., Ortega-Mohedano, F., & Gaitán-Moya, J. A. (2017). Investigación en comunicación en la universidad española en el periodo 2007-2014. Profesional de la información, 26(2), 218-227. https://doi.org/10.3145/epi.2017.mar.08
  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104
  • Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5), 378-382. https://doi.org/10.1037/h0031619
  • Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd Ed.). John Wiley & Sons.
  • Freelon, D. (2013). ReCal OIR: ordinal, interval, and ratio intercoder reliability as a web service. International journal of internet science, 8(1), 10-16.
  • Goyanes, M., Rodríguez-Gómez, E. F., & Rosique-Cedillo, G. (2018). Investigación en comunicación en revistas científicas en España (2005-2015): de disquisiciones teóricas a investigación basada en evidencias. Profesional de la Información, 27(6), 1281-1291. https://doi.org/10.3145/epi.2018.nov.11
  • Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77-89. https://doi.org/10.1080/19312450709336664
  • Igartua, J. J. (2006). Métodos cuantitativos de investigación en comunicación. Bosch.
  • Kolbe, R. H., & Burnett, M. S. (1991). Content-analysis research: an examination of applications with directives for improving research reliability and objectivity. Journal of consumer research, 18(2), 243-250. https://doi.org/10.1086/209256
  • Krippendorff, K. (2004a). Content analysis: an introduction to its methodology (2nd Ed.). Sage.
  • Krippendorff, K. (2004b). Reliability in content analysis. Some common misconceptions and recommendations. Human communication research, 30(3), 411-433. https://doi.org/10.1111/j.1468-2958.2004.tb00738.x
  • Krippendorff, K. (2011). Agreement and information in the reliability of coding. Communication methods and measures, 5(2), 93-112. http://dx.doi.org/10.1080/19312458.2011.568376
  • Lacy, S., Watson, B. R., Riffe, D., & Lovejoy, J. (2015). Issues and best practices in content analysis. Journalism & mass communication quarterly, 92(4), 791-811. https://doi.org/10.1177/1077699015607338
  • Lombard, M., Snyder-Duch, J.,, & Campanella, C. (2002). Content analysis in mass communication. Assessment and reporting of intercoder reliability. Human communication research, 28(4), 587-604. https://doi.org/10.1111/j.1468-2958.2002.tb00826.x
  • Lovejoy, J., Watson, B. R., Lacy, S. & Riffe, D. (2014). Assessing the reporting of reliability in published content analyses: 1985-2010. Communication methods and measures, 8(3), 207-221. https://doi.org/10.1080/19312458.2014.937528
  • Lovejoy, J., Watson, B. R., Lacy, S., & Riffe, D. (2016). Three decades of reliability in communication content analyses: reporting of reliability statistics and coefficient levels in three top journals. Journalism & mass communication quarterly, 93(4), 1135-1159. https://doi.org/10.1177/1077699016644558
  • Manterola, C., Grande, L., Otzen, T., García, N., Salazar, P., & Quiroz, G. (2018). Confiabilidad, precisión o reproducibilidad de las mediciones. Métodos de valoración, utilidad y aplicaciones en la práctica clínica. Revista chilena de infectología, 35(6), 680-688. https://dx.doi.org/10.4067/S0716-10182018000600680
  • Neuendorf, K. A. (2002). The content analysis guidebook. Sage.
  • Neuendorf, K. A. & Kumar, A. (2016). Content analysis. En G. Mazzoleni (ed.), The International Encyclopedia of Political Communication. Wiley. https://doi.org/10.1002/9781118541555.wbiepc065
  • Piñeiro-Naval, V. (2020). The content analysis methodology. Uses and applications in communication research on Spanish-speaking countries. Communication & society, 33(3), 1-15. http://dx.doi.org/10.15581/003.33.3.1-15
  • Piñeiro-Naval, V., & Morais, R. (2019). Estudio de la producción académica sobre comunicación en España e Hispanoamérica. Comunicar, 27(61), 113-123. http://dx.doi.org/10.3916/C61-2019-10
  • Piñuel, J. L. (2002). Epistemología, metodología y técnicas del análisis de contenido. Estudios de sociolingüística, 3(1), 1-42.
  • Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of applied communication research, 27(3), 258-284. https://doi.org/10.1080/00909889909365539
  • Riffe, D., & Freitag, A. (1997). A content analysis of content analyses: twenty-five years of journalism quarterly. Journalism & mass communication quarterly, 74(3), 515-524. https://doi.org/10.1177/107769909707400306
  • Riffe, D., Lacy, S., Watson, B. R., & Fico, F. (2019). Analyzing Media Messages. Using Quantitative Content Analysis in Research (4th Ed.). Routledge.
  • Scott, W. A. (1955). Reliability of content analysis: the case of nominal scale coding. Public opinion quarterly, 19(3), 321-325. https://doi.org/10.1086/266577
  • Segado-Boj, F., Piñeiro-Naval, V., & Antona-Jimeno, T. (2023). Spanish research on communication in WoS: thematic, methodological, and intellectual comparison between SSCI and ESCI. Profesional de la información, 32(3), e320309. https://doi.org/10.3145/epi.2023.may.09
  • Singletary, M. W. (1993). Mass communication research: contemporary methods and applications. Addison-Wesley.
  • Stemler, S. E. (2015). Content analysis. En: Robert A. Scott; Marlis C. Buchmann (eds.), Emerging trends in the social and behavioral sciences. Wiley. https://doi.org/10.1002/9781118900772.etrds0053
  • Walter, N., Cody, M. J., & Ball-Rokeach, S. J. (2018). The ebb and flow of communication research: seven decades of publication trends and research priorities. Journal of communication, 68(2), 424-440. https://doi.org/10.1093/joc/jqx015
  • Wimmer, R. D., & Dominick, J. R. (2011). Mass media research: an introduction (9th Ed.). Wadsworth.
Fonte de datos: Dialnet