Product management for engineers · 2. Discovery: talking to users without leading them

Product management for engineers · Chapter 2 of 10 · Product · new chapter every Wednesday morning

By the end of this chapter: Run a user conversation that produces evidence instead of agreement.

The problem

You are a senior backend engineer at a fast‑growing SaaS startup. The next sprint is due in two weeks and the team has a feature – “custom report scheduling” – that the sales team has promised to a key prospect. The product manager, Maya, has spent the morning shuffling between a stakeholder‑alignment call, a user‑research debrief, and a data‑review session. By 11 am she emails the engineering channel: “We need to push the UI changes to the API tomorrow – the client is asking for a demo on Friday.” At the same time, a separate Slack thread shows her updating the roadmap to move the “export to CSV” work to Q3.

You have already started writing the API endpoint, but the design rests on an assumption Maya took from a user interview earlier that day: “Users want to set up recurring reports because they forget to run them manually.” When you ask Maya for the raw interview notes, she sends a three‑sentence summary that reads like a sales pitch, not a transcript. You cannot tell whether the prospect actually expressed that need, whether the need is common across the user base, or whether the quoted need is a mis‑remembered interpretation.

The cost of proceeding on that assumption is immediate – you will spend a week building an API that may never be used, delaying other work and risking the demo that the prospect expects. The longer‑term cost is more subtle: the engineering team will have to re‑work the endpoint, the product roadmap will be reshaped around a possibly spurious request, and the credibility of the discovery process will erode if the feature lands unused.

You need a reliable way to talk to users that yields evidence – concrete statements, frequencies, and pain points – rather than agreement that merely mirrors what the PM thinks the user wants.

Before you start

One‑off verification command

Run the following to ensure the recording pipeline works:

ffmpeg -f avfoundation -i ":0" -t 5 -y test.wav && ffprobe -i test.wav -show_entries format=duration -v quiet -of csv=p=0

Expected output: 5.000000. If you see an error, reinstall ffmpeg before proceeding.

How do I avoid leading the user?

1. What does “leading” actually look like in a live interview?

A leading question nudges the interviewee toward a particular answer. Examples:

• “You said you often forget to run reports, right?”
• “Would it be helpful if we added a ‘schedule once a month’ button?”

These questions embed the interviewer's assumption and give the user a cue to agree. The evidence you collect becomes agreement rather than truth.

2. Which question types produce raw evidence?

3. How do I keep the interview focused on evidence?

• Use a script – a checklist of question types (see Appendix B).
• Record, don’t rely on memory – the audio file is the source of truth.
• Ask for concrete numbers – “How many reports do you run in a typical month?”
• Avoid “why” after a statement – instead ask “What happened when you tried X?”

How do I structure the interview to surface evidence?

1. What preparation steps guarantee a clean session?

2. What does the interview flow look like in practice?

1. Opening (5 min) – thank the user, confirm consent, explain that you are learning about how they work, not what they think they should want.
2. Contextual walk‑through (15 min) – use the “Contextual” question type to let the user narrate a recent reporting task.
3. Pain extraction (10 min) – ask “What frustrations did you encounter?” and probe for specific moments (e.g., “When you clicked Export, what happened?”).
4. Frequency check (5 min) – ask concrete numbers about report generation.
5. Closing (5 min) – ask for any “wish‑list” items, but frame them as “If you could redesign one step, what would it be?” and note that this is future‑looking data, not current pain.

3. How do I capture the evidence in a reusable format?

After the call:

1. Export the audio – ffmpeg -i interview.wav -ac 1 -ar 16000 interview.flac.
2. Transcribe – use a speech‑to‑text service (e.g., OpenAI Whisper) and save the raw transcript as interview-<date>.md.
3. Highlight evidence – in the markdown, prefix each evidence line with > **EVIDENCE:** and include the timestamp, e.g., > **EVIDENCE (00:12:34):** “I run the monthly sales report three times a week.”
4. Summarise with counts – at the bottom of the file, add a table of frequencies (e.g., “Report runs per week – 3 users said 3+, 2 users said 1‑2”).

The transcript becomes the single source of truth for any downstream discussion.

Build it

Below is a concrete, end‑to‑end workflow you can run today. It assumes you have the prerequisites from the Before you start section.

1. Create the consent file

   cat > consent.txt <<'EOF'
   Subject: Interview Consent
   Date: $(date +%F)
   I, ______________________, consent to being recorded for product research purposes.
   Signature: ______________________
   EOF

Why: Legal and ethical baseline.

Confirm: Open consent.txt and verify the placeholders are present.

1. Send the consent request

   echo "Please sign the attached consent and reply." | mail -s "Interview consent" -A consent.txt user@example.com

Why: Ensures you have documented permission before recording.

Confirm: Check your sent mail folder; you should see the email with attachment.

1. Schedule the interview

   echo "Meeting: Reporting workflow – 45min\nWhen: $(date -d '+2 days' +%F'T'15:00)" >> meetings.ics

Why: Calendar invites provide a shared reference point.

Confirm: Open meetings.ics in a calendar app; the event should appear with the correct time.

1. Test the recording setup (15 min before the call)

   ffmpeg -f avfoundation -i ":0" -t 5 -y test.wav && ffprobe -i test.wav -show_entries format=duration -v quiet -of csv=p=0

Why: Guarantees you will capture the interview.

Confirm: Output 5.000000. If not, troubleshoot microphone drivers.

1. Run the interview – start recording at the exact start time:

   ffmpeg -f avfoundation -i ":0" -ac 1 -ar 16000 -y interview-$(date +%F).wav

Why: Captures a lossless mono track at 16 kHz, optimal for transcription.

Confirm: When the interview ends, stop ffmpeg with q. Verify file size > 500 KB.

1. Transcribe – using Whisper (install openai-whisper version 2023‑09‑01):

   whisper interview-$(date +%F).wav --model base.en --output_format md --output_dir transcripts

Why: Generates a searchable markdown file.

Confirm: Open transcripts/interview-<date>.md; the first line should be a timestamped transcript.

1. Highlight evidence – open the markdown in Obsidian, search for verbs like “always”, “never”, “every”. For each, add the > **EVIDENCE:** prefix and timestamp.
   
   Why: Makes evidence discoverable for stakeholders.
   
   Confirm: Use Obsidian’s search (> **EVIDENCE:**) to see all highlighted lines.

2. Create a summary table – at the end of the file, add:
   

   | Metric | Count |
   |--------|-------|
   | Reports per week (≥3) | 3 |
   | Reports per week (1‑2) | 2 |
   | Report generation pain points mentioned | 4 |

Why: Provides quantitative backing for decisions.

Confirm: Table renders correctly in preview mode.

You now have a reproducible artefact that contains raw evidence, not a paraphrased agreement. Share the markdown file (not the summary) with the PM and engineering leads; they can verify the numbers themselves.

When this breaks

What it costs

The trade‑off is clear: you spend time up‑front to collect high‑fidelity evidence, but you avoid the far larger cost of building the wrong thing. The technique scales poorly for high‑frequency, low‑impact decisions; for those, a rapid survey may be cheaper.

In the interview

Typical interview prompt

“Walk me through how you would validate whether a new feature – say, recurring report scheduling – is actually needed by our users.”

Weak answer

“I’d send a quick survey, look at the responses, and if most say ‘yes’ we build it.”

Why it’s weak: It glosses over the need for unbiased data, assumes survey responses are reliable, and ignores the importance of evidential artefacts.

Strong answer

“First I’d define the hypothesis I want to test – that users need recurring scheduling because they forget to run reports. I’d then schedule three 45‑minute contextual interviews with a mix of power users and occasional users. I’d use a consent‑driven recording pipeline (ffmpeg → Whisper) to capture raw audio, then tag any statements that contain quantitative or pain‑point evidence. I’d aggregate the evidence in a markdown file, produce a frequency table, and present the artefact to the team. If the evidence shows that fewer than 30 % of users run reports more than twice a week, I’d recommend postponing the feature. The trade‑off is the interview time versus the risk of building a low‑value feature, which in our last quarter cost us $40 k in re‑work.”

Senior vs junior expectations

Junior: Should be able to describe the interview flow, identify leading questions, and produce a basic transcript.

Senior: Must also articulate the sampling strategy, quantify the evidence, and discuss how the artefact integrates with backlog grooming.

Director/Engineering‑manager: Looks for the governance aspect – consent, GDPR compliance, and how the evidence feeds into portfolio‑level decision making.

Follow‑up that separates experience from theory

“Can you give an example of a time when the evidence you collected contradicted a stakeholder’s expectation, and how you handled the push‑back?”

A candidate who has actually run the workflow will describe the specific evidence tag, the stakeholder’s reaction, and the concrete decision that resulted (e.g., postponing the feature, re‑prioritising the roadmap).

Your tasks

1. Run a single interview with a colleague who uses the current reporting UI.
   
   Done looks like: a signed consent file, a 5‑minute interview-*.wav, a markdown transcript with at least three > **EVIDENCE:** lines, and a one‑row frequency table.

2. Audit an existing interview (if you have one from a previous sprint). Identify any leading questions, add the missing > **EVIDENCE:** tags, and produce a short summary of the evidence.
   
   Done looks like: a revised markdown file with all evidence highlighted and a note explaining the changes you made.

3. Create a sampling plan for a new feature (e.g., “bulk export”). List the user segments you will interview, the number of participants per segment, and the justification for each number (e.g., “3 power users, 2 occasional users – ensures at least 80 % confidence in frequency estimates”).
   
   Done looks like: a one‑page document with a table of segments, counts, and confidence rationale.

4. Simulate a disagreement: share your evidence markdown with a teammate who believes the feature is essential. Ask them to critique the evidence. Document their objections and your response, focusing on the data, not opinion.
   
   Done looks like: a short email thread or meeting notes that reference specific evidence timestamps and show how you defended the conclusion.

5. Scale the pipeline – automate the transcription step with a simple script that watches a recordings/ folder and runs Whisper on any new file, outputting markdown to evidence/.
   
   Done looks like: a bash or Python script, a test run that produces a markdown file, and a log confirming the automation worked.

Your tasks this week

Do the exercises above before the next chapter. Reading a tutorial and doing
one are different activities and only one of them changes what you can build.

Stuck on any of them? Say so — describe what you tried and what happened:
tell me where you got stuck. I read every one, and the questions
that come back more than twice get answered in the next chapter.

Product management for engineers

Chapter 2 of 10. New chapter every Wednesday morning.
Next: Writing a problem statement an engineer can build from.

· The full syllabus and every chapter so far
· Subscribers also get the condensed notes for this chapter, the running
recap of everything the series has covered, and the extended guidance:
subscribe

Written by Amit Chakraborty — founding engineer and senior architect: React Native, AI and RAG systems, production architecture. Portfolio · LinkedIn · GitHub.

Need this built, reviewed or taught to your team? Get in touch or email amit@devamit.co.in. Available for senior and founding engineering roles, consulting and training, remote worldwide.