How to test Product Managers for technical ability

Chris Dermody -

I don't care what you say, a Product Manager who is not technically competent will mean your team is slower. In any kind of tech company, as a Product Manager if you don't understand some rudimentary technical concepts, you will be at a severe disadvantage.

Think of the following scenario. Your Support team report that there's a problem for a customer, where it's not clear if what's required is a bug fix, or an improvement. The Product Manager who is technically competent can open up the browser console, inspect the API calls being made, check the responding JSON for the appropriate property that's needed, and confirm if it's there or not. They now immediately know if there's a bug, and if there is, is the fix going to be frontend or backend.

If the Product Manager is not technical, an alternative flow of work might be

  1. Product Manager creates a spike ticket based on the support ticket.
  2. Dev team (eventually) gets to it. but it's a frontend dev who picks it up. They have to stop doing something else (context switching is bad).
  3. Later that day it feeds back to the PM that the problem is backend.
  4. A backend dev (eventually) picks it up when they're finished a different task and addresses the problem.

In my opinion, the first scenario is by far the most optimal. In the time it takes to debug the problem and understand where it lies, the non-technical PM would still be creating a ticket.

Other benefits of being a technical Product Manager

  • They can understand database structure and run complex SQL queries in a tool like metabase to dig into data and quickly, not having to wait for a data analyst or similar (which may not even exist?)
  • They understand what services are available in the world via API. They know that Google provides address lookup and geocoding. They know that IPinfo can give us an approximation of where in the world a user is, etc.

How to test a Product manager's technical ability

If you read this blog you'll know I'm a fan of tests. They're the quickest way to discover lots of things, and when hiring, we want to do ourselves and the candidate the good deed of not wasting anyone's time.

Set up a call with the candidate. It shouldn't take more than 30 minutes. Ask them these questions (or adapt them to your company as appropriate).

  1. Show me how you'd test a website's speed.
    With this question, I would hope that the candidate could open up the browser console, open the network tab, and enable throttling. Bonus points if they run a lighthouse audit.
  2. Can you inspect an API response and check the JSON that's returned?
    Here, again I'd expect them to be able to open the browser console and check a network request. Ideally, provide them with a web page that makes an API request that's easily inspectable
  3. How would you check how a website behaves on a mobile device?
    Guess what, browser console again. They should be able to toggle on the device toolbar and select different devices to view the page on
  4. Can they explain what a feature flag is?
    A candidate who is unfamiliar with this term is not up to speed on the latest methodologies of building software
  5. How would they mock up a proposed user flow?
    If they open up powerpoint, I'd start to worry. They should be at least somewhat familiar with tools like Balsamiq, Figma or similar
  6. Can they clear their browser's cookies
    Ok we're back to the developer console with this one, but if they can't even do this, what are they even doing on the internet, honestly.

Note, that if they candidate can't answer every question, that wouldn't necessarily rule them out. But, if they can't answer any, I would seriously question how effectively they can operate within a development team.