Skip to content
Lenny's 🏦 Vault

Shared Framework Needed For Human-AI Relationships

by Mike Krieger on June 5, 2025

The evolution of AI is reshaping product development, with Anthropic's experience showing how teams must adapt when AI becomes a core collaborator rather than just a tool.

How AI Changes Product Development Workflows

  • 90% of code at Anthropic is now AI-generated (Claude Code team uses Claude to build Claude Code)

  • This shift creates new bottlenecks in the development process:

    • Merge queues become overwhelmed with AI-generated pull requests
    • Over 70% of pull requests are now AI-generated
    • Traditional code review processes become impractical at this scale

  • Teams must evolve their development practices:

    • Moving from line-by-line code reviews to higher-level acceptance testing
    • Using AI to review AI-generated code
    • Focusing human attention on system coherence rather than implementation details

New Product Development Bottlenecks in the AI Era

  • The bottlenecks shift from implementation to:

    1. Upstream activities: Deciding what to build and aligning teams
    2. Strategy formulation: Determining where to compete and how to win
    3. User comprehension: Making AI capabilities understandable to users
    4. Integration management: Ensuring systems work coherently together

  • AI accelerates implementation but doesn't solve:

    • Building something coherent that users understand
    • Creating effective launch moments
    • Enabling discovery and adoption
    • Learning from user feedback

Where Product Teams Add Value in the AI Era

  1. Making AI comprehensible to users

    • The gap between AI-adept users and average users remains huge
    • Product teams must bridge this gap through intuitive interfaces and workflows

  2. Strategy development

    • Determining where to play and how to win
    • Understanding which customer segments to target and their specific needs
    • Identifying the right positioning against competitors

  3. Opening people's eyes to possibilities

    • Showing users what's possible with AI capabilities
    • Addressing the "overhang" between what models can do and how they're being used

  4. Deep market understanding

    • Building domain expertise in specific industries
    • Creating solutions tailored to specialized workflows
    • Developing go-to-market strategies for specific customer types

Measuring Success in AI Products

  • Traditional engagement metrics can be misleading for AI products

  • Time spent may not be the right metric when depth matters more than frequency

  • Better signals of success:

    • Time saved for users
    • Enabling work that wasn't possible before
    • User testimonials about increased creativity and productivity
    • Creating space in users' lives for other activities

  • Focus on whether the product is genuinely serving people rather than convincing yourself through metrics that it is

View full episode