Analysis of Multiple-Choice Questions (MCQs): Item and Test Statistics from the 2nd Year Nursing Qualifying Exam in a University in Cavite, Philippines

Abstract

Multiple Choice Questions (MCQs) is used extensively as a test format in nursing education.
However, making MCQs still remains a challenge to educators. To avoid issues about its
quality, this should undergo item analysis. Thus, the study evaluated item and test quality
using difficulty index (DIF) and discrimination indices (DI), with distractor efficiency (DE);
determined the reliability using Kuder-Richardson 20 coefficients (KR20); and identified
which valid measure was developed. The study was conducted among 41 level two nursing
students in the College of Nursing. The qualifying examination comprised of 194 MCQs.
Data were entered in Microsoft Excel 2010 and SPSS22 and were analyzed. According to
DIF, out of 194 items, 115 (59.3%) had right difficulty and 79 (40.7%) were difficult.
Regarding DI, 17 (8.8%) MCQs were considered very good items to discriminate the low and
high performer students. While 21 (10.8%), 32 (16.5%), 24 (12.4%), and 100 (51.5%)
demonstrated good, fair quality, potentially poor, and potentially very poor items,
respectively. On the other hand, the number of items that had 100% distractor effectiveness is
57 (29.4%), as 65 (33.5%), 49 (25.3%), and 23 (11.9%) revealed 66.6%, 33.3% and 0%,
respectively. The reliability of the test using KR20 is 0.9, suggesting that the test is highly
reliable with considered good internal consistency. After careful analysis of each item, 55
(28.35%) items were retained without revisions. Further, the stem of the 24 (12.37%) items,
the distractors of the 66 (34.02%) items and both the stem and distractors of 46 (23.71%)
items were modified, and 3 (1.55%) items were removed. The researcher recommends doing
an analysis between upper and lower scorers and its relationship to DE. For future study, it
will be beneficial to explore other factors like student’s ability, quality of instructions, and
number of students in relation to quality of MCQs.