AI Product Requirement Discovery and PRD Writing

In the previous two chapters, we established AI product thinking and design methodology. This chapter delves into the most crucial technical collaboration aspect for an AI Product Manager—understanding the ML Pipeline, collaborating effectively with Machine Learning Engineers, managing AI project timelines filled with uncertainty, and formulating data strategies and making Build vs Buy decisions.

Understanding the ML Pipeline: A Product Manager's Perspective

You don't need to write model code, but you must understand the complete workflow of a machine learning project. This enables you to make the right product decisions at the right milestones, rather than discovering the direction is wrong late in the project.

The Six Stages of an ML Pipeline

**Stage 1: Problem Definition**. Translating a business problem into a machine learning problem. For example, the business problem is 'reduce customer service ticket handling time,' which translates to the ML problem 'train a model to automatically classify tickets and generate draft replies.' The Product Manager's role is most critical at this stage—you need to ensure the team is solving the *right* problem.

**Stage 2: Data Collection & Processing**. Collecting training data, cleaning noise, handling missing values, performing feature engineering. This stage typically consumes 60-70% of the entire ML project's time. The Product Manager needs to focus on: Where does the data come from? What is the data quality? Is manual labeling required? What are the labeling costs and timelines?

**Stage 3: Model Training**. Selecting algorithms, designing model architecture, training, and hyperparameter tuning. If using a large language model API, this stage mainly involves Prompt design and parameter tuning. The Product Manager needs to focus on: Which model to choose (trade-off between cost and performance)? How much time and resources are needed for training?

**Stage 4: Model Evaluation**. Evaluating model performance on a test set. The Product Manager *must* participate in this stage—because you need to judge whether the model's performance meets product requirements. Is 85% accuracy sufficient? What types of errors are unacceptable?

**Stage 5: Deployment**. Deploying the model to a production environment. The Product Manager needs to focus on: Does the inference latency meet user experience requirements? How to conduct a phased rollout? How to design a rollback plan?

**Stage 6: Monitoring & Iteration**. Continuously monitoring the model's performance on real-world data, collecting feedback, and iterating for improvement. The Product Manager needs to focus on: Which metrics need monitoring? What conditions trigger a model update? How can user feedback be translated into model improvement directions?

Prompt Engineering for PMs: The Product Manager's Prompting Skill

In the era of large language models, Prompt Engineering is the most practical technical skill for an AI Product Manager. You can use prompts to rapidly prototype product ideas, validating feasibility without waiting for engineers to write code.

Prompt Techniques PMs Need to Master

**Role Assignment**: Give the AI a clear role definition, e.g., 'You are a professional legal advisor helping users interpret contract clauses.' Role assignment can significantly improve the AI's output quality in specific domains.

**Few-shot Examples**: Provide 2-3 input-output examples to help the AI understand your expected format and quality standard. This is particularly useful when prototyping AI features—you can use examples to define 'what constitutes good output.'

**Output Format Control**: Explicitly specify the desired output format (JSON, Markdown table, bulleted list, etc.). This is crucial in AI products—you need the AI's output to be parsable and displayable by programs.

**System Prompt Design**: Design System Prompts for AI features within the product, defining the AI's behavioral guidelines, capability scope, and output specifications. A good System Prompt is the core competitive advantage of many AI products.

Using Prompts for Product Prototyping

Before writing a PRD, first use prompts in ChatGPT or Claude to simulate the AI feature you want to build. For example, if you want to create an 'AI Contract Review Tool,' simulate it with a prompt: input a contract text and ask the AI to identify risk points. This way, you can validate in 30 minutes whether the AI can achieve your envisioned effect—100 times faster than waiting for engineering development.

Model Evaluation Metrics: Key Metrics Product Managers Must Know

You don't need to derive formulas, but you must understand the meaning and business significance of these metrics to communicate effectively with engineers.

Core Metrics for Classification Tasks

**Accuracy**: The proportion of correct predictions. However, accuracy can be misleading—on an imbalanced dataset (e.g., 99% normal emails + 1% spam), a model that labels all emails as normal also has 99% accuracy, but is useless.

**Precision**: Among the samples the model predicts as positive, the proportion that are truly positive. High precision means 'if the AI says yes, it's almost certainly yes'—suitable for scenarios with high false positive costs (e.g., content moderation: mistakenly deleting normal content angers users).

**Recall**: Among all samples that are truly positive, the proportion correctly identified by the model. High recall means 'most of what should be found is found'—suitable for scenarios with high false negative costs (e.g., medical diagnosis: missing a patient can be life-threatening).

**F1 Score**: The harmonic mean of precision and recall, providing a balanced measure of model performance. When unsure whether to prioritize precision or recall, F1 is a balanced metric.

Evaluation Metrics for Generation Tasks

**BLEU/ROUGE Scores**: Measure the similarity between generated text and reference text. Commonly used for translation and summarization tasks. However, such automatic evaluation metrics have limitations—a high BLEU score doesn't necessarily mean user satisfaction.

**Human Evaluation**: Have human evaluators score AI outputs (e.g., 1-5) or conduct A/B blind tests (comparing which of two model outputs is better). Although costly, this remains the most reliable evaluation method for generation tasks.

**User Behavior Metrics**: Adoption rate (proportion of users directly using the AI output), edit rate (proportion and extent of users modifying the AI output), regeneration rate (proportion of users requesting regeneration due to dissatisfaction). These behavioral data points from real users are the most valuable evaluation signals.

The Art of Collaborating with ML Engineers

The collaboration between an AI Product Manager and ML Engineers differs significantly from that between a traditional PM and developers. Understanding these differences can greatly enhance collaboration efficiency.

Three Core Principles of Communication

**Specify 'What,' Not 'How'**. The Product Manager should say, 'I need a feature that can extract action items from meeting recordings, with accuracy no less than 85% and latency under 30 seconds,' not 'Use GPT-4 with RAG to implement this.' The choice of technical solution should be left to the engineers.

**Speak with Data, Not Intuition**. When discussing model performance, don't say, 'It doesn't feel good enough.' Prepare specific bad cases (instances where the AI made errors), analyzing error patterns and frequency. Engineers need concrete data to locate and fix issues.

**Understand Non-Deterministic Timelines**. Time estimation for AI projects is more uncertain than for traditional software development. When an engineer says, 'Roughly 2-4 weeks,' that range might be realistic—because model tuning can sometimes be solved in a day, sometimes requiring repeated attempts with multiple approaches. Product Managers need to build buffer time into project plans.

Tools and Methods for Effective Communication

**Establish a Shared Evaluation Benchmark**: Define an evaluation dataset and metrics together with engineers. This becomes your common language for communication. After each model update, both sides review the same set of data and metrics for discussion.

**Regular Model Review Meetings**: Conduct a model review with engineers every 1-2 weeks to examine the latest evaluation results, discuss bad cases, and align on the next optimization direction. This is more efficient than asking for daily progress updates.

**Maintain a Bad Case Repository**: Continuously collect cases where the AI made errors, categorizing and labeling error types. This repository serves as both a data source for engineers to improve the model and a basis for product decisions.

Managing AI Project Timelines

Time management for AI projects is one of the biggest challenges for an AI Product Manager. Traditional software projects are relatively predictable—once requirements are clear, experienced engineers can estimate timelines fairly accurately. AI projects are filled with uncertainty—you don't know how many approaches need to be tried to reach the target accuracy.

Project Management Strategies to Handle Uncertainty

**Time-boxed Iterations**: Don't set goal-driven timelines like 'achieve 95% accuracy.' Instead, set 'spend 2 weeks maximizing accuracy improvements, then make decisions based on the results.' This time-boxing strategy prevents projects from extending indefinitely.

**Phased Decision Points**: Set clear decision milestones within the project. For example: Week 2 - Evaluate baseline performance → If below 70%, consider changing the technical approach or adjusting product strategy; If above 80%, continue optimizing details; If above 90%, prepare for launch.

**Parallel Solution Exploration**: In the early stages of a project, have engineers explore 2-3 technical approaches in parallel (e.g., trying different models, different prompting strategies simultaneously), rather than sequentially trying them one by one. Although the short-term investment is higher, it helps find the optimal solution faster.

Data Strategy

Data is the fuel for AI products. AI Product Managers need to consider data strategy from the product design phase, not wait until model training is needed to discover 'we have no data.'

Four Pathways for Data Acquisition

**User-Generated Data**: Data naturally produced through product usage. This is the most valuable data source as it directly reflects real-world usage scenarios. Product design must consider how to effectively collect this data.

**Public Datasets**: Standard datasets released by academia and industry. Suitable as training data during the project initiation phase, but they often have gaps with your specific scenario.

**Manual Labeling**: Hiring labelers to annotate data. High cost but quality is controllable. The Product Manager needs to create labeling guidelines—this essentially defines 'what constitutes good AI output.'

**Synthetic Data**: Using AI to generate training data. For example, using GPT-4 to generate large volumes of simulated dialogues to train a customer service bot. Low cost but quality needs verification.

Data Privacy & Compliance

AI products handle large amounts of user data; privacy compliance is a red line Product Managers must focus on. China's Personal Information Protection Law (PIPL), the EU's GDPR, and industry-specific regulations (e.g., finance, healthcare) impose strict requirements on data usage. Product Managers need to specify in the PRD: What user data is collected? For what purpose is the data used? Is user informed consent required? Is data storage and transmission encrypted? What is the data retention period?

Build vs Buy: Build In-House or Purchase?

A frequent decision for AI Product Managers: use an in-house model or a third-party API? Both options have pros and cons.

Using Third-Party APIs (e.g., OpenAI, Anthropic, Baidu ERNIE Bot)

**Advantages**: Fast development (can launch in days), low maintenance cost (model updates handled by vendor), continuous quality improvement. **Disadvantages**: Data needs to be sent to a third party (privacy risk), potentially high long-term cost (pay-per-token), limited customization, vendor dependency (risk of API changes or price hikes).

Building In-House Models (Fine-tuning open-source models or training from scratch)

**Advantages**: Complete data control (suitable for high-privacy scenarios), deep customization (suitable for vertical domains), potentially lower long-term cost (at large scale). **Disadvantages**: Long development cycles (months), requires an ML engineering team, high operational costs, model quality may lag behind top commercial models.

Decision Framework

Recommended decision path: First use third-party APIs to quickly validate product ideas (MVP stage) → If product validation is successful and usage grows → Evaluate the need to migrate to an in-house model (considering cost, privacy, customization needs). Most AI products in their early stages should choose third-party APIs—speed and flexibility are more important than a perfect technical solution.

Having mastered AI technical understanding and team collaboration skills, the next chapter will take us into practical application—completing a full AI product MVP project from 0 to 1, applying all the knowledge from the first three chapters.

Freelancing sounds wonderful—no boss, no commute, flexible hours, unlimited income potential. But is the reality truly like that? Before you decide to embark on this path, let's objectively analyze the advantages and challenges of freelancing.

Advantages of Freelancing

Time Freedom

You decide when to work. Night owls can code at 2 AM, early birds can start at 5 AM and finish by 3 PM. No 9-to-5, no need to ask for leave.

Location Freedom

With a computer and internet, anywhere in the world can be your office. Many freelancers choose to live in cities with a lower cost of living while earning income from clients in first-tier cities or even overseas.

High Income Ceiling

No fixed salary cap. Your income depends on your skills, efficiency, and client volume. Excellent freelancers can earn far more than their counterparts in full-time positions. The emergence of AI tools further amplifies this advantage—one person + AI can accomplish the workload of a past team.

Rapid Skill Growth

Freelancers need to master professional skills, communication skills, business skills, and self-management simultaneously. Although exhausting at first, the growth rate far exceeds that of being a cog in a large corporation.

Challenges of Freelancing

Income Instability

This is the biggest challenge. No fixed salary means you might earn 100,000 one month and zero the next. You need to build a savings buffer (recommended at least 6 months of living expenses) and learn to manage irregular cash flow.

Social Security and Benefits

No company handles your social insurance and housing fund (五险一金); you need to arrange social security, medical insurance, and pensions yourself. This requires special attention in China—many cities link home purchase and residency qualification rights to the number of years of social security contributions.

Loneliness

Working alone long-term can lead to social isolation. No colleagues, no water cooler chats, no team atmosphere. You need to proactively build a social network—join freelancer communities, work from cafes regularly, attend industry events.

High Self-Discipline Requirement

No one supervises you; you are your own boss and your own employee. Procrastination, distraction, blurred lines between work and life—these are real challenges.

How is AI Changing Freelancing?

The emergence of AI tools has significantly lowered the barrier to entry for freelancing while increasing efficiency. Tasks that one person couldn't handle before can now be done by one person + AI. For example: using Claude for copywriting, Midjourney for design, Cursor for coding—you can simultaneously offer writing, design, and development services.

But AI also brings intensified competition: a lower barrier means more people entering the field. You need to excel in both efficiency and quality to stand out in the competition.

Are You Suited for Freelancing?

Answer the following questions. If most answers are 'Yes,' you might be suited for freelancing: Can you accept income instability? Do you have sufficient self-discipline? Do you have at least one marketable professional skill? Do you have savings covering over 6 months of living expenses? Can you tolerate working alone for long periods?

If you decide to try freelancing, the next chapter will detail how to get started on platforms like Upwork and Fiverr.