What does Item Analysis Tell Us? Factors affecting the reliability of Multiple Choice Questions (MCQs)

Abstract

Introduction: Item analysis is a very important part of the examination cycle, as it can be used to judge the difficulty and discrimination of test items as well as their reliability.

Objective: The objective of this study is to determine the relation between reliability and other components of the item analysis of Multiple Choice Question (MCQs).  

Methodology: The item analysis of 38 modules conducted at the Faculty of Medicine-Jazan University in the academic year 2010-2011 were further analyzed descriptively and correlation were made between the reliability co-efficient (KR-20) and other elements of the item analysis such as the difficulty and discrimination indices

Results: All the items have five options, the percentage of nonfunctioning options is between 27.6% and 70% across items. The facility index is < 35% is found in 18.3% of the items and facility index of > 90% is a maximum of 75%. All exams have items with a discrimination index <19% .

There is a positive correlation among the KR-20, number of items and the percentage of items with discrimination index >60% and negative correlation with percentage of the nonfunctioning options, percentage of items with facility index <35%, percentage of items with facility index >90% and percentage of items with discrimination index <19%.

Further analysis of the modules in each phase (Introductory, organ system and clinical) and qualitative analysis of the items are needed.