Development and Psychometric Evaluation of Faculty Evaluation for Online Teaching (FEOT)

Gil Soriano

doi:10.58870/berj.v6i1.28

Authors

Gil Soriano

Keywords:

Faculty evaluation, online teaching, psychometric properties, reliability, validity

Abstract

With the shift of the instruction from the traditional Face-to-face instruction to online, there is a need to develop a valid and reliable instrument that is responsive to the current demands of instruction. Hence, the objective of the study was to develop and evaluate the psychometric properties of the Faculty Evaluation for Online Teaching (FEOT) among University Students. A descriptive methodological study was used and a convenience sample of 2985 students were employed. The 20-item FEOT confirmed the four-factor loadings following the domains in the Denison Framework for Teaching. The factor loadings of the items were between 0.619 to 0.791 while the CFA model revealed a x2/df = 2.35, root mean square error of approximation= 0.071, comparative fit index= 0.962, goodness of fit index=0.957, Tucker-Lewis index= 0.956, incremental fit index= 0.962, and standard root mean square residual= 0.023. The overall Cronbach's alpha of the instrument was 0.923 while the sub-domains have Cronbach’s alpha of 0.906 for Planning and Preparation, 0.942 for Classroom Environment, 0.929 for Instruction and 0.921 for Professional Responsibilities indicating high reliability and the item-total correlations ranges from 0.764 to 0.868. At the item level, 60.60% of the ratings had a CVI of greater than 0.78. The FEOT was shown to be valid and reliable in assessing the competencies of faculty members in an online environment.

References

Akram, M. & Zepeda, S. (2015). Development and Validation of a Teacher Self- assessment Instrument. Journal of Research and Reflections in Education, 9(2), 134-148.

Berk, R. A. (2013). Face-to-face versus online course evaluations: A “consumer's guide” to seven strategies. Journal of Online Teaching and Learning, 9(1), 140-148.

Burns, N. & Grove, S.K. (2009). The practice of nursing research appraisal, synthesis, and generation of evidence (6th ed.). Saunders

Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86(2), 127–137.

Costello, A. B., & Osborne, J. W. (2003). Exploring best practices in Factor Analysis: Four mistakes applied researchers make. Paper presented at the Paper presented at the annual meeting of the American Educational Research Association.

Creasman, P. A. (2012). Considerations in online course design. The IDEA Center. http://www.theideacenter.org/ sites/default/files/idea_paper_52.pdf

Danielson, C. (2007). Enhancing Professional Practice: A Framework for Teaching. Association for Supervision and Curriculum Development (ASCD).

Eskey, M. T., & Schulte, M. (2012). Comparing attitudes of online instructors and online college students: Quantitative results for training, evaluation and administration. Online Journal of Distance Learning Administration, 15(5).

Ferketich, S. (1991). Focus on psychometrics. Aspects of item analysis. Research in Nursing & Health, 14(2), 165–168.

Fleiss, J. (1981). Statistical methods for rates and proportions. John Wiley& Sons

Gordon, G. & Partington, P. (1993). Quality in Higher Education: Overview and Update. USDU Briefing Paper Three. Sheffield: Universities Staff Development Unit

Gözüm, S., Tuzcu, A., & Kirca, N. (2016). Validity and Reliability of the Turkish Version of the Nurse Cultural Competence Scale. Journal of Transcultural Nursing, 27(5), 487-495

Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g

Hathorn, L., & Hathorn, J. (2010). Evaluation of online course websites: Is teaching online a tug-of-war? Journal of Educational Computing Research, 42(2), 197-217 https://doi.org/10.2190/EC.42.2.d

Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). The Guilford Press.

Lee, C. J. (2007). Academic help seeking: Theory and strategies for nursing faculty. Journal of Nursing Education, 46(10), 468-475. https://doi.org/10.3928/01484834-20071001-07

Li, H.C. & Lopez, V. (2007). Development and validation of a short form of the Chinese Version of the State Anxiety Scale for Children. International Journal of Nursing Studies, 44(4): 566-73

Madernach, B. J., Donnelli, E., Dailey, A., Schulte, M. (2005). A Faculty Evaluation Model for Online Instructors: Mentoring and Evaluation in the Online Classroom. Online Journal of Distance Learning Administration, 8(3). https://www.learntechlib.org/p/193203/

Marzano, R. J., Frontier, T., & Livingston, D. (2011). Effective supervision: Supporting the art and science of teaching. Association for Supervision and Curriculum Development (ASCD).

Ngoma, S. (2011). Improving teacher effectiveness: an examination of a pay for performance plan for boosting student academic achievement in Charlotte-Mecklenburg Schools. http://files.eric.ed.gov/fulltext/ED518815.pdf.

Oducado, R. M. & Soriano, G. (2021). Shifting the Education Paradigm amid the COVID-19 Pandemic: Nursing Students’ Attitude to ELearning. Africa Journal of Nursing and Midwifery, 23(1), 1-14. https://doi.org/10.25159/2520-5293/8090

Polit, D & Beck, C. (2006). The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Research in Nursing & Health. 29, 489-497. https://doi.org/10.1002/nur.20147

Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30, 459-467. https://doi.org/10.1002/nur.20199

Rothman, T., Romeo, L., Brennan, M., & Mitchell, D. (2011). Criteria for assessing student satisfaction with online courses. International Journal for e-Learning Security, 1(1-2), 27-32.

Schulte, M. (2009). Efficient evaluation of online course facilitation: The “quick check” policy measure. The Journal of Continuing Higher Education, 57, 110-116

Soriano, G. & Aquino, M, G. (2017). Characteristics of a Good Clinical Teacher as Perceived by Nursing Students and Faculty Members in a Philippine University College of Nursing. International Journal of Nursing Science, 7(4), 96-101

Soriano, G. P., & Calong Calong, K. (2019). Construct validity and reliability of caring nurse patient interaction scale-nurse among Filipino nurses. International Journal of Caring Sciences, 12(2), 1017-1022.

Stronge, J. H. (2010). Teacher effectiveness = student achievement: What the research says. Eye On Education

Tabachnick, B.G. & Fidell, L S. (2007). Using Multivariate Statistics (5th ed.) Pearson Education. Inc.

Tobin, T. J. (2004). Best practices for administrative evaluation of online faculty. Online Journal of Distance Learning Administration, 7(2).

Walsh, D. & Maffei, M. (1994). Never in a class by themselves: an examination of behaviors affecting the student-professor relationship. Journal on Excellence in College Teaching, 5, 23-49.

Waltz, C. Strickland, O. & Lenz, E. (2005). Measurement in nursing and health research (3rd ed.). Springer Publishing Co.