Is your choice of psychometric test creating noise - and wasting your time?

What is noise?

So what was all that irritation? It was noise. It’s the confusion that we find when we are trying to concentrate on one thing, but there are too many things competing for our attention and distracting us from the task at hand - and risks clouding our judgement, leading to bad decision making.

In statistics, we call this “noise”. “Noisy data” is data that’s rendered meaningless because there are too many variables at play.

Audio engineers coined the term, “Signal to Noise Ratio. It’s really helpful here because it reminds us that there will always be noise when we’re working on a question. What we need is the right data to be pointing us towards the signal, aka ‘who is the best candidate for the job?

Are all psychometric tests as good as each other?

Not all psychometric tests are created equally - not all tests are even psychometrically validated. Sure, there are tests based on what you favourite colour is (red, yes, thanks for asking), but that doesn’t mean that they have any scientific validity to be measuring something as important as personality or decide someone’s trajectory.

They’re super fun when you’re bored on a train and doing a magazine personality quiz as your coffee goes cold, but potentially less helpful when you’re trying to decide who is going to make your interview shortlist.

So if we’re going to use psychometric tests, how can we know that what we’re using is actually refining the lens that we need - and not creating more noise?

When can statistical noise be a problem in hiring?

When a team creates (or “builds”) a test, they also need to build in the scoring scale - so, if a total score possible is 60, and you have 20 questions with 3 possible answers, each answer will score 1, 2 or 3.

Let’s think about bulk hiring drives? Candidates can work through 3 rounds of assessments, 2 interviews and 1 work test during an assessment day. 

But how can we know if each test is measuring anything new, or if they’re simply being asked the same questions three times over? Are these just overlapping methods?

More than wasting the time of candidates and recruiters, the additional data that is created by repeated tests simply adds to the ‘noise’ that the recruiter has to sift through. The repetition can give the sheen of uncovering insights; however, humans are complex and not every nuance can be measured (which is great!).

Decades of research into what predicts performance at work is General Mental Ability + personality. Adding additional scales and questions at an early stage simply adds to the potential for noise, and your judgement becoming clouded.

Refining your recruitment lens

But say you’re super busy and you’re hiring tech support and data leads. Many test providers will put another lens on the data, with the aim of making each data set stand alone.

However, to build a psychometric test system that is true over your entire hiring ecosystem, one consistent scale helps to allow a clear picture of the talent pool as a whole - who you are attracting and why. By having a clear view of the whole pool, recruiters are able to understand their own metrics, rather than have to learn a new set of metrics that shift and bend with every position.

Can noise ever be useful?

Sure. Everything in moderation, right?

If we think of ‘noise’ as ‘extra information’ rather than simply ‘the wrong information’, we’re thinking differently - as they say, a weed is just a plant growing somewhere the gardener doesn’t want it.

Additionally, what we think of as ‘noise’ can help us to amplify the right signals to find out who the best person is for our vacancy. In statistical terms we call these covariates - that is, variables (questions or measurable things such as educational qualifications) that are distinct from, but linked to the overall question of ‘who should we hire? So educational qualification should not decide who we hire; however it can help recruiters to know if someone has an appropriate level of qualifications for a hire.

So, when education, years of experience are all held constant, we are able to look at what the contributions of our assessments are when predicting job performance or success.

Noise can provide clarity in the dark

If that signal is always there, pinging away amongst random irregular noise, then it can let us know what the signal of interest is! When you’re interested in understanding the data and what it can tell you, looking at statistics becomes the creative enterprise that all data nerds love to hide away!

When we think about noise in hiring, we can think of it as someone’s bio-data - their name, ethnicity (which can be assumed or inferred from a person’s name, work and/or educational history). We can think of it as their years of experience and their background.

This information is super-useful in hiring but only when we get to the point of interviewing, when we’re really thinking about the person who we want on the other end of the zoom or the end of the corridor. That’s when noise can be helpful.

Understand noise to find clarity in hiring

The subject of noise is huge, but essential to consider when you’re building a resilient and future proofed recruitment process. Because when we’re hiring new talent, we need to make sure that we’re using the correct lens that can help us see what we need to know - a candidate’s soft skills, personality and potential - and not muddy the waters with unnecessary information.

By being more aware of how and when noise affects our decision making we are clearing the path for a more exciting and clear path to brilliant teams.