Análise de uma prova de ciências humanas utilizando Teoria de Resposta ao Item

Abstract

This study evaluates the quality of items and the overall structure of a Humanities simulation test through Classical Test Theory (CTT) and Item Response Theory (IRT). The test consisted of 30 items from public and private institutions and was answered by 664 participants. Only responses from individuals who completed all items were analyzed. In the CTT analysis, a Cronbach’s alpha coefficient of 0.744 was obtained, indicating good internal consistency of the test; the analysis also identified 7 items with pointbiserial correlations below 0.30. In the IRT analysis, 1-, 2-, and 3-parameter logistic models (1PL, 2PL, and 3PL) were fitted, and the unidimensional 3-parameter model showed the best fit. Model comparison was conducted using the likelihood ratio test. The fit quality was evaluated with the M2 index, which demonstrated a good fit, while the RMSEA2, TLI, and CFI indices also indicated good model adequacy. In the analysis of the IRT parameters, two items showed negative discrimination, indicating problems. Upon investigation, one item was found to have an incorrect answer key, and the other was poorly constructed, leading inattentive readers to select an incorrect answer. Several items contributed almost no information about the measured skill, as for many, the maximum information was close to zero. The test provided more information for low skill levels, with most estimated examinee abilities being moderate to high.