Demystifying the Agile Product Spec
Great Product Specs drive great products — even in Agile
You already know what a Product Spec is.
It defines the problem to solve, who has this problem, why it is important to solve, and what to build to solve it.
Yet many complain that Product Specs are too ‘bulky’ for Agile. That no one reads or follows them. These same startups sacrifice forethought with experimentation, and will struggle sooner or later with quality, accountability, scale, and speed.
True, specs have evolved to support Agile’s emphasis of communication over documentation, shifting from lengthy documents to crisp ‘briefs’ that enable teams to start building sooner and more iteratively so as to maximize learning. Yet even in Agile, getting ‘briefs’ right is critical.
There are many ‘how to write product specifications’ articles out there — this article boils it down to ‘minimum viable spec’ that focuses on building a high quality feature without sacrificing speed.
I’ll use the Golden Gate Bridge as an example because who doesn’t like the Golden Gate Bridge. It doesn’t matter what product you are building, from a button to a bridge — its success or failure hinges on a Great Product Spec.
Importance of a great Product Spec
It defines why working on this feature is important — this gives purpose and drive to the engineering team building it
It is a contract between the different Product teams, such as Product Management, Design, and Engineering — this ensures that there is clear handoff and accountability
It is a defines the hypothesis and how it will be validated — this holds the Product Manager accountable for the hypothesis made and measurements to validate
It de-risks projects with forward thinking all the while limiting scope — this ensures go to market version is the minimum viable product
Pre-requisites to a great Product Spec
Understand the problem you are trying to solve, deeply — like if it was your own problem
Define which customer segment you are trying to target. Speak for them.
Identify the metrics you are trying to move and how you plan on A/B testing to measure impact
Work backwards and write out the press release. Write the spec so as to deliver on that vision
Finalize the designs and user experience with the design team. Align on the design / UX MVP vs. the nice to haves (at many companies this will be a major undertaking requiring close collaboration between Design and Product)
Get required sign off from other stakeholders (legal, marketing, communication, compliance, etc.) — and keep them involved
Guiding principles of a Product Spec
Always focus on the user. User centricity is core to Agile and ensures you are building a feature that adds value to your users.
Define the what, not the how. Leave the how to the Engineering team — that is their job, not yours.
Be succinct in your writing. Impress with accuracy not volume. Don’t overwhelm with detail, but detail enough to reduce risk.
Clearly define what is in scope vs. not in scope. Focus on the ‘minimum viable product’ — the smallest scope that you can go to market with and test your hypothesis and focus on the MVP.
Teamwork ensures quality — get input from other Product Managers, designers, compliance, legal and engineers. Input from strong engineering managers is crucial to help identify areas that need more detail.
Pre launch is as important as post launch — define the test cases that need to pass as well as the controls / metrics that need to be monitored to ensure the product is working as intended.
User story: What feature do you want to build and for whom?
Business value: Why build this feature?
Acceptance Criteria: How is this feature meant to work?
Business Metrics: How do I know this feature is having an impact?
Data Model: What data should this feature generate?
Controls: How do I know this feature works after launch?
Testing: How do I ensure all of the above works before launch?
Content of a great Product Spec
Section 1: User Story
The User Story defines the feature you are trying to build, the problem you are trying to solve, and who you are trying to solve it for. It boils down to a simple sentence:
“As a ____, I want to ____ so that ____”
A user story has three parts:
Persona: what customer segment are you solving for?
Action: what action do you want to enable?
Outcome: what is the outcome of enabling this action?
GGB: As a commuter from Marin, I want to drive from Marin to SF in less than 1h, so that I can live in Marin and work in SF.
Section 2: Business Value
The Product Spec should clearly define what business value this feature will bring to the organization.
The business value of a feature, along with the effort estimate to build the feature, drive prioritization of what gets built.
The Product Spec should also explain how impacting this business value relates to the overall product/business strategy. This is key for initiatives that might not have immediate business value but unlock significant value or impact for the company.
Business value boils down to the value this feature will bring to the shareholders of the company. Some customer centric companies will focus purely on customer value. Business value is often measured as a dollar figure such as revenue increase or cost reduction — this helps normalize different ideas in order to more easily prioritize.
GGB: Business value is to increase growth rate of Marin county, which was low given that Marin was only accessible by Ferry. Toll revenue was a means to an end to finance the project.
Section 3: Acceptance Criteria
The clearer your acceptance criteria, the more likely the engineer builds the right feature, correctly. Acceptance criteria are the conditions that the product must satisfy to be accepted by a user, customer, or in the case of system level functionality, the consuming system. They are the written instructions for how the software should behave. acceptance criteria include the feature details such as design wireframes, copy, flows, etc.
Acceptance criteria can be broken down in the following way:
What actions should occur in the use case.
What should be valid before starting the actions
What are the outcomes of each action or set of actions (workflow), and what is the outcome after all is done
What are the error cases that can happen for each action, and what is the expected outcome if it occurs
GGB: Design and wireframes are part of the acceptance criteria. Another component could be around capacity in different weather patterns: “it must support a flow of up to 40,000 cars / hour on a day with wind up to 20mph”, and should detail what happens if more than 40,000 cars show up or if the wind is higher.
Section 4: Business Metrics
In comparison to business value which is a measure of the feature’s impact, the business metric is a measure of the feature’s performance. Business metrics show how we plan to measure any metric that ensures the feature is working correctly and having an impact.
The Product Spec should outline, for each business metric:
The Business Metric
How to measure it
Its baseline value
If it is a metric you want to move: Its target increase once the product launches — this should be framed as a hypothesis
If it is a metric you want to monitor: Its threshold after which you know the product is not working (or being used) as intended
Clearly defining the business metrics will ensure that the engineering teams generate the data needed to measure them. Companies have used analytics tools such as Mixpanel and Amplitude to facilitate the implementation and measurement.
Experimental design (A/B testing) is critical to ensure that measurements are unbiased. At the end of the day, a Product Manager’s job is to identify the correct metrics for his/her business, and make the right hypothesis on features/products that will move these metrics.
This is not to be confused with System Metrics. System metrics define how to monitor system health. The definition of system metrics and their associated thresholds is owned by the engineering team who will be ‘on call’ for any alerts based on these thresholds. Alerts based on system metrics should be actionable for the engineer on call. These are to be codified in the Tech Spec that is paired with the Product Spec.
GGB: To measure increase in growth rate in Marin County from the bridge, business metrics for the Golden Gate Bridge could include the number of people living in Marin, and the number of people driving across the bridge in the morning compared to using the Ferry. This is valid input into a multi regression model to determine statistically significant correlation (but not causation).
Section 5: Data Model
The Data Model defines the analytical data that is to be accessible once this feature launches. Analytical data (data extracted from operational systems into a data warehouse for consumption) is not to be confused with operational data (data stored in operational databases, defined by engineering).
This ensures all consumers of data are satisfied with their data needs — this includes the Product Manager but also business units such as Operations, Data Science, Growth, Finance, Compliance, etc. The Data Model should also satisfy the requirements to measure business and operational metrics of the feature.
The Data Model includes:
Data Definition: Name of the field and how it is defined
Data Values: Expected values in the field
Data Validations: Rule sets that must hold true for each field and between fields
GGB: Generate data on the number of cars that pass, the time each entered the bridge and the time each exited the bridge. This way, one would know how many cars are on the bridge at the same time (relating to operational controls), and how many cars are serviced by the bridge (relating to the business metrics).
Section 6: Controls
Controls help determine if the product is working as intended after launch. Controls are a set of rules that the product should not break. These rules should be defined for each of the acceptance criteria in the Product Spec. That way, if a control is triggered, you will then know that your product does not meet the acceptance criteria.
Controls are a set of rules that must hold true. This rule set can change over time
Controls should be operationally independent to decrease risks in single point of failure.
Controls can be detective or preventative. Detective controls are passive and alert when an action happens that should not have. Preventative controls prevent the action from happening.
Since you cannot add a control for every piece of the feature, your must prioritize (like everything a PM does) the controls that check outcomes which are to be prevented at all costs.
GGB: One control could be a pressure control system in the pillars to alert if there is too much weight on the bridge. This control could be preventative if it closes the entry of the bridge when pressure is past a certain threshold. Don’t put this system on the same power grid in case there is an outage.
Section 7: Testing
There are many types of tests for software development. The Product Manager is accountable for defining the functional tests that will ensure each piece of the Product Spec is properly implemented, from the acceptance criteria, to the Data Model, Business Metrics, and Controls.
A good test suite covers all potential inputs and circumstances and defines the required output. It should be a playbook for anyone to test a product before it goes live.
Agile advises the engineering team to implement these tests in an automated fashion to remove the need for manual testing. Some companies feel the need to hire a QA team to support testing — this may shift responsibility of quality away from engineering which could have adverse consequences. Regardless, User Acceptance Testing by the Product Manager is always encouraged before go live to ensure the product meets the acceptance criteria.
GGC: Test that if winds are up to 60mph the the bridge will be at 80% tension capacity. The control for tension should fire. The data model should read the correct max tension for that day, etc.