Community-Rooted Expert Network Boosts Retention

Handshake AI's rapid growth from zero to $100M+ ARR in under a year demonstrates how existing platforms can leverage proprietary networks to create value in the AI data ecosystem.

The shift in AI data labeling has moved from generalists to experts, creating an opportunity for platforms with access to specialized talent. Handshake capitalized on this by leveraging their existing network of 18 million professionals, including 500,000 PhDs and 3 million master's students.

The Evolution of AI Model Training

• AI model development has two primary phases:
  • Pre-training: Using existing internet data (text, videos, books)
  • Post-training: Targeted improvement in specific capability areas
• Most gains now come from post-training as models have absorbed most available internet data
• Post-training requires high-quality expert data across specialized domains:
  • Identifying model weaknesses in reasoning steps
  • Providing ground truth answers in complex domains
  • Creating step-by-step reasoning processes
  • Developing trajectories (complete workflows with screen/mouse tracking)

Strategic Advantages in AI Data Supply

• Access to audience is the only true moat in human data

  • Competitors spend millions on recruitment and ads to find experts
  • Handshake's decade of trust with 18M users provides zero CAC advantage
  • High conversion rates due to existing brand affinity

• Expert-first platform design creates retention advantages

  • Traditional platforms treat experts like low-cost labor
  • Community-rooted experience with cohort-based training
  • Higher payment rates ($100-200/hour for specialized expertise)
  • Clear progression path from internal projects to customer projects

• Quality assurance creates defensibility

  • Internal post-training team validates data quality
  • Approximating gain from each data unit
  • Building proprietary data packages that can be sold to multiple labs

Building a Business Within a Business

• Complete separation is critical

  • Separate engineering, design, operations, and finance teams
  • Different physical workspace
  • Separate all-hands meetings and onboarding processes
  • Different compensation structures tied to new business metrics

• Founder-led execution

  • CEO dedicating 80%+ of time to the new initiative
  • Hiring entrepreneurial talent comfortable with ambiguity
  • Setting clear expectations about pace and work requirements
  • "Leave nothing to chance" mentality to capitalize on the opportunity

• Metrics-driven from day one

  • Weekly/monthly/quarterly operating cadence
  • Data-oriented decision making
  • Clear KPIs and tracking

• Selective talent reallocation

  • Moving top performers from core business to new initiative
  • Being upfront about different expectations and workload
  • Creating ownership through equity and recognition

Future of AI Data Collection

• Data types will continue to evolve beyond text:

  • Multimodal data (audio, video, images)
  • Tool use and trajectories
  • Scientific workflows and specialized domain knowledge
  • Professional workflows across industries

• The need for human experts will persist for years:

  • Models still struggle with complex reasoning
  • Domain expertise remains critical for identifying model weaknesses
  • Human judgment needed for non-verifiable domains