The 7 questions you need to ask your psychometric assessment vendor

Alva’s assessment tools use IRT logic. What IS this, you ask?

What is IRT?

 IRT means Item Response Theory; in its simplest form, IRT is a form of logic that allows a psychometric tool to adapt to the candidate. Based on how you answered the previous question (item), the tool will select the next question that makes the most sense (that will provide as much relevant information as possible).

This allows every candidate to only answer questions that are necessary, making the assessment shorter (and leaving candidates happier). It also enables updating the tools on a regular and frequent basis, since single items can easily be altered, added or removed.

What is CTT?

CTT, or Classical Test Theory, is used by the majority of test providers. It means that every candidate receives the same questions, irrespective of their answers. It’s less agile and takes a long longer to complete. And if you want to update the items, you need to update the entire list of questions. And frankly, that is often not done frequent enough.

Questions to ask your assessment provider

The next time you’re looking at which logic your test provider uses, we’ve got some simple questions for you to ask:

1. What logic do you use, IRT or CTT?
2. Why have you made that decision?
3. How does using this logic affect the reliability of your assessments?
4. Have you assessed how that decision affects the validity of your tools ?
5. Does your use of logic affect the length of your tools?
6. How often do you review and retire items in your logic assessments?
7. How are those decisions made?
   
   

Want even *more* info on how psychometric assessment *should* work? Sure you do! 👇