Scaling product sense: merging quant and qual

"Your gut is the world's most sophisticated machine learning model ever created." - David Lieb, General Partner at YC

The $2M Dashboard That Nobody Used

Three months into my role at a healthcare analytics company, our CEO pulled me aside with a problem. We'd just spent $2M building what our data team called "the most comprehensive patient experience dashboard in the industry." It had every metric you could imagine: bed utilization rates, patient throughput times, staffing efficiency scores, patient response scores, and predictive models for everything from readmissions to length of stay.

Hospital executives loved the demos. The visualizations were beautiful. The data was "accurate" - the feed from the EHR was clean.

Nobody used it.

"What are we missing?" he asked. I thought I knew the answer: more data, better visualizations, maybe some AI-powered insights. I was wrong.

The breakthrough came during a casual conversation with Dr. Martinez, a nursing supervisor at one of our pilot hospitals. She mentioned something that never showed up in our metrics: "I wish I could just know when Mrs. Chen in room 302 is getting anxious about her discharge. That's when problems happen."

Our dashboard could predict average length of stay across thousands of patients, but it couldn't tell Dr. Martinez what she actually needed to know about the one patient causing her stress at 2 PM on a Tuesday.

That's when I realized the fundamental problem: we'd built a quantitative masterpiece that ignored qualitative reality.

Why Data PMs Are Caught in the Quant vs. Qual Trap

Data product managers face a unique challenge that traditional software PMs don't encounter. We live in the intersection of two worlds that often feel incompatible:

The Quant World: Dashboards, metrics, A/B tests, and statistical significance. Great for answering "what" questions, terrible for "why" and "what should we do about it?"

The Qual World: User interviews, support tickets, anecdotes, and edge cases. Rich with context and emotion, but difficult to scale and prone to bias.

Most data teams pick a side. The analytically-minded gravitate toward dashboards and metrics. The user-focused lean into research and interviews. Both approaches miss the magic that happens when you combine them.

As Sachin Rekhi points out in his work on product intuition: "Market research and customer surveys can become proxies for customers. Good inventors and designers deeply understand their customer... They study and understand many anecdotes rather than the averages you find on surveys."

But here's the problem: understanding "many anecdotes" doesn't scale. Traditional user research is painfully manual, slow to generate insights, and expensive to maintain. Meanwhile, dashboards give us averages that miss the nuanced stories where real opportunities hide.

Product Intuition(Sense) as Your Internal ML Model

The best data product managers I know have something that's hard to define but easy to recognize: product sense. As Jules Walter defines it in Lenny's Newsletter,

💡

Product sense is the skill of consistently being able to craft products (or make changes to existing products) that have the intended impact on their users.

They can look at a metric anomaly and immediately know which customers to call. They spot opportunities in customer complaints that others dismiss as edge cases. They make bets that seem risky but consistently pay off.

This isn't magic—it's pattern recognition built through exposure to both quantitative signals and qualitative stories. Product sense relies on both empathy to discover meaningful user needs and creativity to come up with solutions that effectively address those needs.

David Lieb's quote about gut instincts being "the world's most sophisticated machine learning model" isn't just clever—it's a profound insight into how product intuition actually develops. Like any ML model, building strong product sense requires:

High-Quality Training Data: Direct, unfiltered customer feedback connected to product outcomes, not summaries or averages.

Lots of Iterations: Fast feedback loops where you can see the immediate impact of product decisions on real user behavior.

Pattern Recognition: Exposure to enough examples that you start seeing connections between seemingly unrelated signals.

The challenge? Most data product teams never get enough "training data" to build strong intuition. They're too removed from customers, too focused on metrics, or too buried in feature delivery to develop the pattern recognition that separates good PMs from great ones.

The AI Shift: From Process to Intuition

Here's what's changing: AI is automating much of the mechanical work that used to define good product management. Data analysis that took hours now happens in minutes. Competitive research that required days can be done with a few prompts. A/B test results that needed statistical expertise are now interpreted automatically.

As Brian Balfour argues in his recent work on AI-native product teams, this automation creates a fundamental shift in what makes product managers valuable. Process skills are becoming commoditized. The new differentiator is intuition—the ability to synthesize disparate signals into confident product bets.

But here's the opportunity: the same AI tools that are commoditizing process can also help us build intuition faster than ever before.

Feedback Rivers: Merging Quant and Qual in Real Time

The concept of "feedback rivers" isn't new—Sachin Rekhi pioneered the approach while building LinkedIn Sales Navigator. The idea is simple: create continuous streams of unfiltered customer feedback flowing directly into teams' existing workflows.

But AI has transformed what's possible with feedback rivers. Instead of choosing between overwhelming fire hoses of information or sterile dashboard summaries, we can now create intelligent, personalized streams that surface both quantitative signals and qualitative context.

This approach mirrors what Jules Walter calls "observing people interacting with products"—one of the key practices for building product sense. The difference is that AI now lets us do this observation at scale, systematically, rather than just in occasional user research sessions.

Here's how it works in practice:

Step 1: Set Up Your Feedback Infrastructure

Connect multiple data sources into a unified stream:

Support tickets and chat logs
User interview transcripts
Sales call recordings (with permission)
Product usage analytics
NPS surveys and feedback forms
Social media mentions and reviews

Step 2: Intelligent Routing and Personalization

Use AI to automatically:

Tag feedback by product area, feature, or user segment
Route relevant feedback to the right team members
Surface anomalies and patterns that need attention
Connect quantitative spikes or drops to qualitative explanations

Step 3: Active Exploration

When you have a hypothesis or idea, use AI to instantly:

Pull all relevant feedback and data
Identify customers to interview
Quantify potential impact
Generate follow-up questions

Step 4: Close the Loop

Track how feedback translates to product decisions:

Connect insights to shipped features
Measure before/after impact
Build institutional knowledge that survives employee turnover

A Real Example: Intuition in Action

Let me share a concrete example of how this approach changed everything for our team.

We were building a patient medication adherence tool for a large health system. Our dashboard showed that medication compliance was dropping among diabetes patients, particularly in the 55-75 age group. The metric was clear, but the "why" was missing.

Traditional approach would have been to schedule focus groups, design surveys, or dig through support tickets manually—all taking weeks to organize and execute.

Instead, our AI-powered feedback system immediately surfaced relevant signals:

12 support tickets mentioning "confusing medication reminders"
3 sales calls where nurses mentioned "patients don't understand the app"
Usage data showing 73% of users abandoned the medication logging flow at step 3
Interview transcript from one month prior: "My mom gets overwhelmed by all the buttons"

Within 30 minutes, we had a hypothesis: the medication logging interface was too complex for our core demographic. We called three patients from the feedback system and confirmed the hunch. Two weeks later, we shipped a simplified interface.

Result: 34% improvement in medication logging completion rates and a 28% increase in overall adherence scores.

The breakthrough wasn't the individual pieces of data—it was the ability to connect quantitative signals (dropping compliance) with qualitative context (interface confusion) in real time. This example illustrates several key insights about user behavior that experienced PMs learn to recognize: people are time-crunched and distracted when using products, they drop out as soon as they feel confused, and too much information at once creates overwhelm.

Common Pitfalls and How to Avoid Them

After implementing feedback river approaches across multiple data product teams, I've seen the same mistakes repeatedly:

Drowning in Data

The Problem: Teams set up comprehensive feedback systems but get overwhelmed by volume.

The Solution: Start narrow. Pick one product area and one type of feedback. Build your pattern recognition gradually.

Over-Relying on AI Summaries

The Problem: AI-generated summaries lose nuance and emotional context that's crucial for understanding customer pain.

The Solution: Always dig into raw feedback. Use AI to surface relevant examples, but read the actual customer words.

Ignoring Edge Cases

The Problem: Focusing only on majority feedback and missing outliers where the best opportunities often hide.

The Solution: Specifically look for anomalies and edge cases. The patient who uses your tool in an unexpected way might reveal your next major feature.

Feedback Without Action

The Problem: Collecting lots of feedback but not connecting it to product decisions and outcomes.

The Solution: Create clear processes for translating insights into roadmap changes. Track which feedback led to which features.

Building a Culture of Merged Intelligence

The technical setup is only half the battle. Building intuition requires cultural changes that support both quantitative rigor and qualitative empathy:

In team meetings, lead with customer anecdotes that provide context for metrics. Instead of "engagement dropped 15%," try "engagement dropped 15%, and here's what Sarah told us about why she stopped using the feature."

Create Feedback Rituals

Build regular touchpoints where teams discuss both quantitative trends and qualitative insights:

Weekly "signal review" meetings combining metrics and customer feedback
Monthly deep dives into edge cases and outliers
Quarterly retrospectives on how feedback influenced product decisions

Democratize Customer Access

Don't gate customer interactions behind specialized roles. Make it easy for engineers, designers, and data scientists to access customer feedback and participate in user research.

Document Pattern Recognition

Create shared repositories where teams can document the connections they've found between metrics and customer behavior. This institutional knowledge becomes invaluable as teams grow and change.

The New PM Workflow

Here's what the day-to-day workflow looks like for data PMs who have successfully merged quant and qual:

Morning Routine: Check your personalized feedback stream alongside your metrics dashboard. Look for connections between quantitative anomalies and qualitative signals.

Hypothesis Formation: When metrics raise questions, immediately query your feedback system for relevant customer context. Use AI to surface related conversations and behavioral patterns.

Decision Making: Before committing to product changes, validate hypotheses with a quick outreach to customers identified through your feedback system.

Impact Measurement: After shipping features, track both quantitative metrics and qualitative feedback. Connect outcomes back to the original insights that drove decisions.

Knowledge Sharing: Document successful connections between signals and outcomes. Share patterns with your team and contribute to institutional learning.

As you implement this workflow, you'll likely notice improvements in your product sense: you'll start anticipating non-obvious user problems, develop higher-quality hypotheses about product experiences, and contribute more unique insights to your team. Most importantly, you'll be right more often about what impact changes will have on both metrics and user behavior.

The Future of Data Product Intuition

As AI continues to evolve, I believe we'll see an even tighter integration between quantitative and qualitative signals. The best data product teams will be those who:

Build AI systems that enhance human pattern recognition rather than replacing it
Create feedback loops that connect customer insights directly to product outcomes
Develop institutional knowledge that transcends individual team members
Balance statistical rigor with empathetic understanding of user needs

The goal isn't to choose between data and stories—it's to build systems where both inform each other in real time.

Getting Started: Your 30-Day Action Plan

Ready to start building your own feedback river and scaling product intuition? Here's a practical 30-day plan:

Week 1: Audit Your Current State

Map all your existing feedback sources (support, sales, research, analytics)
Identify gaps where quantitative signals lack qualitative context
Choose one product area to focus on initially

Week 2: Set Up Basic Infrastructure

Connect 2-3 feedback sources into a unified stream (Slack channel, shared dashboard, etc.)
Start routing relevant feedback to appropriate team members
Begin tagging feedback by theme and product area

Week 3: Active Exploration

Pick one metric anomaly from the past month
Use your feedback system to find related customer conversations
Interview 2-3 customers to validate or challenge your hypotheses

Week 4: Close the Loop

Implement one small change based on your merged insights
Measure both quantitative and qualitative impact
Document what you learned and share with your team

The Bottom Line

In the genAI era, the competitive advantage for data product managers won't come from better processes or more sophisticated dashboards. It will come from the ability to synthesize quantitative signals with qualitative insights and build strong product sense.

The teams that master this synthesis—that can spot opportunities in the intersection of metrics and customer stories—will build products that don't just work, but that users actually love.

Your gut instinct is indeed the world's most sophisticated machine learning model. But like any ML system, it's only as good as the training data you feed it. Start building your feedback river, start training your intuition, and start making product bets that combine the rigor of data with the wisdom of real customer stories.

The future belongs to data product managers who can think like algorithms and feel like humans.

References

What's your experience with merging quantitative and qualitative insights? I'd love to hear your stories and lessons learned. Connect with me on LinkedIn or drop a comment below.