How to Prepare for an Automation Testing Interview

I was recently asked to help a non-technical Test Manager to draw up a job specification for an Automation Test Engineer with at least one year’s test automation experience. Not too difficult a task as I knew the client fairly well and understood the sort of person they would need but...

The conversation turned to how would this non-technical Test Manager know if the answers given in an interview were: 

• A: right and...
• B: a reflection of the candidate’s true test automation ability

I came up with an approach that I think will work well. More about that in a moment… but first, I will explain about the problem of interview questions.

I was talking to Ben, a Team Leader of a highly skilled development team. He was interviewing for a Technical Tester. Ben knew his stuff and could spot a bluff a mile off but he told me of a phenomenon he had not experienced before.

He had a raft of applicants, short-listed them and set about interviewing. The answers to Ben’s technical questions from the early candidates were very poor and this was reflected in their performance with the technical test.

The first-round candidates were rejected and a week later, he held a second round. This time, the answers to the technical questions were spot on but alarmingly similar, almost word-for-word repeated sentences from different candidates. Again, the technical performance was poor.

One of the candidates explained that he was part of a tester’s interview group, where members share the questions asked at an interview. The group then researches what the answer should have been and post the job details, questions asked and model answers.

Whilst it’s wonderful to see people working together as a community and problem-solving, the purpose of the test may be to evaluate whether someone already has an understanding in this area. So, problem-solving as a group can invalidate the test. That means as interviewers, we need to think outside the box and come up with tasks that cannot be prepared for in this way.

This aside, the problem with non-technical interviewers using a crib sheet of questions and answers is that they may not know enough to understand if an answer given is a perfectly valid answer. Even though it might be different to the crib sheet answer or wrong; and even though it might sound a bit like the crib sheet answer. In these circumstances, a simple test automation exercise that is simple enough for the interviewer to understand the results but complicated enough to test the automation skills of the candidate could well provide the key to finding the right candidate.

Back to my client. She was looking for a test automation resource, with at least one year of experience. The question she asked was, “how will I know if this person is good enough if I don’t have any experience of test automation?” So in order to address that concern, I devised a short exercise to run during the interview.

Here, it is:

When inviting the candidate for interview, include the following text:

There will be a technical test as part of the interview.

Please bring a laptop pre-installed with the tools you would need to create and execute a web-based automated test script. Please note that this is a mandatory part of the interview process, those unable to meet this requirement will not be considered for the role.

Give this worksheet to the candidate on the day of the interview and allow 45 minutes for the completion of the test:

Please write an automated test for this scenario.

Go to www.nfocus.co.uk - select the “Test Automation” link on that page, confirm by code, that the resulting page has the text 'Test Automation Solutions Award winning Consultancy', then navigate back to the homepage and take a screenshot.

Things to note:

How quickly do they complete the task?

15 mins – above average

30 mins – average

45 mins below average

More than 45 mins – not a suitable candidate

Do they run the test manually before coding?

How do they present the results of the testing (particularly the evidence of the text)?

How confident do they look approaching the exercise

What questions do they ask, if any?

The thinking behind this exercise was:

• that the process being tested needed to be simple enough for the interviewer to easily know if the test followed the right route and produced the right results.
• that the exercise gave a clear signs to the interviewer of the level of skill being displayed, regardless of the interviewer's technical ability.
• that it allowed the interviewer to observe any behavior displayed by the candidate, (panic, confusion, confidence etc.) that may indicate the technical and social competencies the candidate has.