Ship AI Products Fast to Learn What to Polish

Nick Turley, head of ChatGPT at OpenAI, shares a fundamentally different approach to product development in AI - where shipping quickly and learning from real usage is more important than polish or perfection.

Core Principles of AI Product Development

Ship First, Polish Later

• "This is a pattern with AI: you won't know what to polish until after you ship"
• "You're gonna be polishing the wrong things in the space... you won't know what to polish until after you ship"
• "My dream is that we ship daily or even hourly like in software land"
• The team went from decision to ship ChatGPT to public launch in just 10 days

Maximize Acceleration

• "Is it maximally accelerated?" - a core question used to challenge assumptions about timelines
• "I just really wanna jump to the punchline of 'why can't we do this now?'"
• This question helps cut through blockers and distinguish critical path items from non-essential work
• The team created a Slack emoji for this principle to reinforce the culture of speed

Empiricism Over Planning

• "You really have to ship to understand what is even possible and what people want"
• "The only way to find out what people like and what's valuable is to bring it into the external world"
• "You need real failure cases to make these things better... the only way you get that is by shipping"
• "The benchmarks are increasingly saturated so really you need real world scenarios"

Treat the Model as the Product

• "There really is no distinction between the model and the product - the model is the product"
• Must iterate on the model like a product by analyzing what people are trying to do
• Systematically improve on the use cases people care about through data science and user feedback
• "We were able to make iterative improvements to it just like software"

Balance Speed with Responsibility

• Apply rigorous process for safety and frontier models
• "We're very deliberate on that where your process is a tool"
• "You have to separate out the product development velocity which has to be super high from... frontier models [where] there actually needs to be a rigorous process"
• "For things like frontier models there actually needs to be a rigorous process where you red team, you work on the system card, you get external input"

Implementation Tactics

Prioritize Learning Over Perfection

• "Shipping is just kinda one point on the journey towards awesomeness"
• "You should pick that point intentionally... it doesn't have to be the end of your iteration at all"
• "Once you know what people are doing, there's no excuse to not polish your product"
• "In a world where you don't know yet, you might get very distracted"

Embrace Imperfect Solutions

• The model chooser UI was criticized but shipped anyway to get feedback
• "Is it better to wait until you got a polished product or to ship out something raw even if it makes less sense and start learning?"
• "We should ship. It's embarrassing, but that's strictly less bad than not getting the feedback"

Run Lean Teams

• "We've always had this tendency to run relatively lean"
• Take inspiration from WhatsApp - "a very small team running a very global scope product"
• Treat hiring "more like executive recruiting and less like just pure pipelined recruiting"
• Focus on "barrels" (people who can make things happen) rather than just adding "ammunition"

Discover Through Data and User Feedback

• Built a data science team before the product team to understand usage patterns
• Use conversation classifiers to understand what people are doing without manual review
• Study viral TikTok posts and comments to discover emergent use cases
• "There's these crazy TikTok posts that go viral and they have like 2,000 use cases in the comments"

First Principles Thinking

• "Approaching each scenario from scratch is so important in this space"
• "There is no analogy for what we're building... you can't copy an existing thing"
• "You can learn from everywhere but you have to do it from scratch"
• Question standard practices like "you have to have a product manager and an engineering manager and a designer"