How to create an effective Product Manager hiring challenge

If you've ever had to hire even a single person, you know how much of a mine-field it can be. In my experience, hiring excellent Product Managers is even more tricky than other roles because there's no traditional 3rd level course of education that covers the topic concisely. You can't just look for a candidate with that such-and-such degree. Even the title "Product Manager" can mean such vastly different things in different companies, so you can't even fully trust that when you see it on a CV.

So what do you do? Well, first off, follow these steps on how to hire excellent Product Managers. For the sake of this post, let's assume you've confirmed the candidate is confident, can communicate, has the right experience and technical ability and you're aligned in your goals.

The next step, is to confirm they can actually do Product Management, and the most effective way to determine that, is with a challenge.

The Product Manager Challenge - what not to do

When I first started hiring Product Managers, I created a challenge where it would outline a typical problem that would arise in the organisation, and let the candidate go off for a week or more and come up with a proposed solution, along with some sample user stories and a reasonably solid idea of a flow of work.

This wasn't optimal, because it could take a candidate more than a week to complete, and often they'd obsess over details to the point where they might even not want to complete it for fear of it not being perfect. All the while during that week, the good candidates are being poached by competing companies hiring too.

The ideal Product Manager challenge

What you're really testing for with the challenge is the candidates instinctual response to a product problem or opportunity. What are the immediate questions they would ask, and what is the most logical, simplest solution that they can come up with that addresses it.

Creating the challenge

Create a one-page (often half a page) document that explains that they are the PM for team X, and that team's primary KPI is Y. Your team thinks that feature Z is the best feature to do next. (explain to the candidate that they can assume this feature is the next priority, bypassing all the questions about how they'd priorities the feature etc).

Outline the deliverables. They can be anything. It could be a Google doc, a Miro board, or something else entirely.

Format

Ask the candidate to schedule two consecutive hours of a given day.
Just before the first hour, you send them the challenge
For the first hour, they look at the challenge, generate a proposed solution, and create any content they feel is appropriate
For the second hour, it's a call with you and anyone else involved in the hiring process. Let the candidate bring you through their response, and ask probing questions appropriately.
Given the time pressure, the candidate will have made a lot of assumptions. That's fine. Ask them to explain their assumptions. Make them defend their assumptions. They won't always get it right, but a robust defence is essential nonetheless.

This challenge format is great because

It's a lot quicker than some alternatives.
It means the candidate isn't going off for a week or more, spending hours and hours on a challenge.
It means the candidate is less likely to get excessive help from someone else
You're testing their instinctual reaction under pressure, which is a more representative real-world test (at least if your company is anything but a mature, slow moving behemoth)

What do you think?

I'm pretty sure we haven't perfected the challenge just yet, I'd love to hear what you'd change about the above approach, or if it's entirely wrong in your opinion!