Professor puts exams to the test

Imagine putting a test through its paces. That’s exactly what an FIU researcher and his colleagues have done to create a tool that gauges whether an exam is properly designed.

“We wanted to see what happens as a result of instruction,” said Kyle Perkins, a professor of applied linguistics and teaching English to speakers of other languages in the School of Education and Human Development. “After a lesson, you hope that some test items become easier and some do. You also see that some questions that are difficult in a pre-test remain difficult in the post-test.”

In the era of high-stakes testing and teacher retention where pay is often tied to student performance on exams, this tool could have far-reaching implications on teaching, said Perkins, who co-authored the study, published recently in the Journal of Applied Measurement.

Perkins and his co-authors gave college students a pre-test, attend multiple lectures and complete reading and homework assignments. Afterward, the students were asked to retake the original test.

Perkins and his team dissected the exams. They grouped questions by themes and then compared the difficulty of each question to the competency of each student in the class to see whether students improved their scores and to determine whether Instruction during the lectures confused students’ perception of test questions.

The result found the pattern they were ultimately looking for – students indeed improved as they found some questions easier to answer on the post-test, but by and large, students were still vexed by a few challenging questions, Perkins said.

In addition, Perkins and the team confirmed that students perceived the themes in the pre-test and post-test. The verdict? This was a fair test for the students.

“If you found a shift from pre-test to post-test, it shows there are problems with the questions,” Perkins said. “Someone needs to go back and examine the test because there’s something that’s just not right.”

Educators from various disciplines could evaluate their exams using this method at the end of a course to make adjustments prior to the start of a new semester.