How to assess applicants

1^st September 2020 by Joseph Dudley

An assessment process with objective criteria and carefully considered questions can save you time in reviewing applications, improve your interviews, and minimise bias.

Recognise your bias

One of the trickiest things about bias is that most of the time we don’t even notice ourselves being biased. Very few people identify as being racist or sexist. We are most of us proud to say that we accept people no matter their gender, sexuality, ethnicity and so on, and diversity statements at the bottom of many job adverts proclaim the same.

The inconvenient truth is that it doesn’t matter how many times we say this, it still rarely corresponds with reality. We all have unconscious bias, and by its very nature it is hard to detect and eliminate. The kind of prejudice that most of us have internalised isn’t thinking that certain ethnic groups are less intelligent, it’s much much more subtle than that. It’s defaulting to male pronouns when describing an imaginary person. Most of the time you don’t even notice you’re doing it, and most of the time the impact is pretty minor, but when it comes to hiring it can change the face of entire industries.

Bias is virtually impossible to eliminate, but it can be minimised. The starting point is to recognise it’s there and that even if it’s unconscious we have a duty to address it.

Ensure there’s diversity in your hiring team

Each person brings their own biases to the table, so hiring decisions should be made by as diverse a group as possible to help minimise this. This might be diversity of gender, age, or background.

Before you begin the assessment process, assemble your hiring team. Everyone should have visibility over the process, even if they’re not involved in every stage or every decision.

Have objective criteria for assessment

Start by drawing up a set of objective criteria based on the person specification in your advert. Decide which of these criteria are assessable in the application, and which need to wait until interview. You may also want to weight different criteria according to how important they are to the role.

Start with eligibility criteria that are very easy for anyone to quickly assess. These can be yes/no questions, or ones with very simple answers. For example, if you need an expert in programming in Python, then you can very quickly eliminate any application that doesn’t mention Python.

For more detailed criteria, create a scoring system. What are the characteristics of a good answer, a mediocre one, and a poor one? For example, CAST, a technology charity, has the following criteria in their recruitment guide:

Each candidate’s ability to meet the requirements of the job, as stated in the person specification, should be assessed using a simple scoring system:

• No demonstration of skill/ experience/ qualification = 0
• Mentions the skill/ experience/ qualification = 1
• Mentions and brief example of skill/ experience/ qualification = 2
• In depth example and detail of skill/ experience/ qualification and relates their example back to the Job Description = 3

CAST Handbook 1

You’ll want at least two people to review each application or each criterion independently, and then to total or average their scores. Identify which criteria need special expertise to assess, and try to leave these until last to make the best use of your team’s time.

A trick teachers use for marking exams which can help you assess lots of applications quickly, is to assess all applications against one criterion at a time rather than assessing one application at a time against all the criteria. The advantage here is that you can look at everyone’s answers to one question at the same time and get a really good picture of what a good and a bad answer looks like. You also don’t have to keep flicking back to check the criteria for each question.

Applications

Assess blind

You don't need to know a candidate's name or gender or any other personal attribute to know if they are a good fit for a role, so don't look at that information until you have to. When you first review applications, we recommend removing candidates’ personal details so that you are only looking at the quality of their application. That said, there is some interesting debate about anonymisation as a tool for recruitment; Hanna Naima McCloskey, CEO of equity and inclusion consultancy Fearless Futures, asks why we ‘need to erase certain people’s identities to enable their employment?’.

A study by Ryerson University and the University of Toronto found that job seekers with ‘Asian’ sounding names were 28% less likely to be invited to interview than those with ‘White’ sounding names 2. The National Bureau of Economic Research (NBER) found a similar result, with job seekers who had ‘African American’ sounding names receiving 50% fewer invitations to interview 3. Employers who described themselves as being an “Equal Opportunity Employer” in their job advert discriminated just as much as others.

NBER also found that using screens to hide the identity of musicians during auditions increased female musicians’ probability of advancing to the next round by 50% 4. Biases can have huge impact.

You may also want to hide university names. The Social Mobility and Child Poverty Commission found that working-class graduates are ‘locked out’ of top careers 5. They often come from disadvantaged backgrounds, which means they do less well in school, get into lower-ranked universities, and are then overlooked in favour of candidates from the Russell Group. A less prestigious university does not mean a less capable applicant.

Don’t look at social media profiles

Social media profiles offer a tempting look into who a candidate is beyond what they have stated in their application. Don’t give into this temptation. A candidate’s private life should be private, and you risk introducing significant and potentially illegal bias into your recruitment process 6.

Don’t use let algorithms make decisions for you

AI and machine learning algorithms are the buzzwords of the moment, and whilst it’s true that they can be very useful for certain tasks, you should be extremely wary of using them for recruitment:

As with many other things in this new industry, that sounds good until you think about it; then it becomes replete with problems. Given the best performers of the past, the algorithm will almost certainly include white and male as key variables. If it’s restricted from using that category, it will come up with attributes associated with being a white male, such as playing rugby.

Dane E. Holmes, Goldman Sachs Global Head of Human Capital Management 7

You should also bear in mind that GDPR gives people the right not to be subject to solely automated decisions, including in the context of recruitment 8.

Interviews

Though we can assess CVs and application forms without knowing anything about the applicant, this is harder to do at interview.

Entirely text-based interviews do exist, and some companies have found success with them. These can be realtime like a traditional interview, or asynchronous like an application form. The major benefits are lower cost for both sides, and less chance of unconscious bias influencing the process 9.

However, there is a reason that even in our digitally connected world, video calls are often preferred to voice-only ones, and we still opt to meet our colleagues in person. There is a human element that purely text-based communication can’t replace. How important this is to your organisation will depend on your specific circumstances.

At the moment, the majority of organisations still conduct interviews in-person, by video, or over the phone, and in these circumstances we can’t avoid knowing a candidate’s name, seeing their face, and hearing their voice. All of these things inevitably create impressions which are the basis for bias and discrimination. This bias is virtually impossible to eliminate, but it can be minimised.

Have objective criteria for assessment

Whichever medium you choose for your interviews, they should be standardised as much as possible. Just as for assessing applications, you need a list of questions and a scoring system with a set of objective criteria based on the person specification in your advert.

Objectivity extends to culture fit questions too. These are questions intended to give you a feel of how this candidate would get along with your team. They are valuable, but they are also a potential minefield of unconscious bias because we're hardwired to show favouritism to people who remind us of ourselves 10. Try to make these questions objective too. Make them about the qualities you want in your team rather than whether or not a candidate has the same interests as you.

Each interviewer should have a scoring sheet to complete and take notes on. You might also want to record the interview so that you can review it again afterwards. An important thing to bear in mind when making notes is that these are usually covered by GDPR 11. That means that a candidate can request a copy of the notes made about their application. You should keep your notes for a minimum of six months as this is the time limit in which discrimination claims can be raised.

The idea of having your notes reviewed may sound a little daunting, but provided you’re being objective in your assessments, you have nothing to worry about. When recruiters know they may have to justify their reasons for hiring or rejecting someone they are less likely to be biased 12.

Ask everyone the same questions

Make sure you’re asking all interviewees the same initial questions, and that they all have the same amount of time for their interview. This doesn’t mean you can’t ask candidates specific follow-ups or questions about their own employment history and experience, but the content of each interview should be substantially the same.

Design technical questions and tasks carefully

If you are asking technical questions, try to ensure that these questions are comparable to the kinds of challenges the candidate would face in the role. The tech industry in particular is replete with ‘whiteboarding’ – asking candidates to write out algorithms on a whiteboard – something they would rarely, if ever, have to do in their actual jobs.

As an alternative, or in addition to, technical questions in the interview, you can give candidates a task to complete at home. This allows you to keep the interview short and focused, and gives your candidates time to demonstrate their ability in a less pressured environment. As with the questions, this task should mirror as closely as possible the work expected in the role.

Make sure what you’re asking for is reasonable and can be completed in no more than an hour. If you need candidates to complete a more complex task that is going to take a significant chunk of their time, then you should offer to pay them for this.

Don’t make it an interrogation

There are very few people who enjoy being interviewed for a job. It’s usually a pretty stressful experience as a candidate, trying to make a good impression and give the best possible answers, while avoiding tripping over your own feet. Ensuring your candidates are comfortable can help them perform better and show you what they’re really capable of.

In a typical interview, we start by asking a candidate about their background, skills, and experience, and then end by asking if they have any questions for us. Basecamp, a leading provider of project management software, suggests ‘flipping the script’ and beginning with the candidate’s questions rather than our own. They argue that it allows the candidate to relax and get the conversation going before moving on to trickier questions, allowing them to put their best foot forward. I think this way of doing things has a lot to recommend it, but if you do choose to go down this route, make sure you’ve told the candidate well in advance so they have time to prepare some questions.

There are some interview guides which warn against building any kind of rapport with a candidate because “you risk hiring a person that you liked, and not the one that knows how to do the job”, but I think this goes too far the other way. We are not mere functional machines assessable purely on how well we can perform our duties.

Our success as a team depends not just on our ability to fulfil our individual roles, but also on how we interact with each other. The word ‘teamwork’ does not mean the work of a team but the relationships inside that team. This can and should be reflected in the interview process. There is a balance to be had between leaving our biases at the door and accepting that personality necessarily plays a role in how teams operate and recruit. Getting this balance right is the distinction between hiring someone who is like you and hiring someone who you can work with.

Basecamp sums up the kernel of a good interview in this message to their interviewees:

This is a collaborative chat — we want you to succeed, and it’s the start of working together. That might last for an hour or 5 years, or the rest of your career, but it starts here.

Basecamp interview email 13