What is Product Development?

Product Management Guide

Product Development is a complex topic that has evolved over the past couple of decades.  In this guide, we’ll review those processes, the most common standards used today, and the roles and responsibilities of Product and the rest of the team.  With those concepts in place, we’ll walk through the discovery, development, and deployment steps of development a new feature, to give a sense of how it all comes together.    Let’s get started!

Development Process

Software development processes has evolved over the past 50 years, in part due to realizations of what works best for developing software, and because the delivery model has shifted from products on shelves, to online delivery, that enable a more incremental approach. 

In the 1960s, Waterfall methodology was the dominant approach.  There was a series of distinct “stage gated” steps and one must be completed before proceeding to the next.  First requirements needed to be completed, then design, then development, validation, and finally release. The approach resembles an assembly line that can maximize the efficiency of resources when developing a known commodity.  Unfortunately, there’s also a downside – because of the stage-gated approach, practitioners and project managers would try to minimize having to go backwards in those steps, leading to perfectionism and trying to account for every detail before proceeding to the next stage. This inevitably slowed the whole process down and increased cost for getting anything completed. 

In the 1990s, a number of contrarian processes like Rapid Application Development (RAD), and Extreme Programming (XP) began to emerge, which proposed another approach to development that was more lightweight and iterative in nature.  And in 2001, a number of software engineering leaders met on a retreat to establish the Manifesto for Agile Software Development.  This was the beginning of Agile Development, which put people over process, working software over documentation, and responding to change over simply following a plan.   In the subsequent years, Scrum became the most common Agile Project Management process, and teams begin working in 2-week increments, and learning how to develop requirements and design, in parallel to development and release efforts. A decade later in the 2010s, Lean product development rose to popularity and built on top of Agile.  Whereas Agile is about developing increments of software and delivering them quickly, Lean provides a methodology for efficiently achieving product-market fit.  Lean introduced the idea of a Minimal Viable Product and espouses the goal of getting to product-market fit as quickly and efficiently as possible, with minimal waste.  This is a natural complement from the Product perspective, to Agile in the development perspective. The rapid build>measure>learn loop provides a recipe for discovering and building value through an Agile development pipeline.

Today, there are any number of combinations of these processes in practice across the industry.  For the most part the industry as shifted toward Agile thinking, though many larger organizations will still practice some hybrid of Agile and Waterfall (aka “Wagile”).  There are even frameworks for formalizing that hybrid like the Scaled Agile Framework (SAFe). The truth is, there is no perfect process, and each one of them have their strengths and weaknesses – it’s just a question of what we want to optimize for.  Since most organizations do recognize some flavor of Agile/Scrum as the basis for their product development, we’ll use that as a point of reference for the rest of this discussion.

Today, there are any number of combinations of these processes in practice across the industry.  For the most part the industry as shifted toward Agile thinking, though many larger organizations will still practice some hybrid of Agile and Waterfall (aka “Wagile”).  There are even frameworks for formalizing that hybrid like the Scaled Agile Framework (SAFe). The truth is, there is no perfect process, and each one of them has their strengths and weaknesses – it’s just a question of what we want to optimize for.  Since most organizations do recognize some flavor of Agile/Scrum as the basis for their product development, we’ll use that as a point of reference for the rest of this discussion.

Roles in Agile/Scrum

There are three roles to consider in Agile/Scrum: the Product Owner, ScrumMaster, and the team.   The Product Owner “owns” the backlog, providing requirements and priority to the team.

The Product Owner

The Product Owner works with one foot outside of the team, understanding needs, opportunities, and priorities, and then communicating that back to the team.  In the context of Product Development, the Product Owner is often a member of the Product Management team and thus spending a good amount of time working with customers and business stakeholders to understand all aspects of the functional outcome that must be achieved with the solution the team is working on.  This is often expressed as a collection of requirements, stated in the form of User Stories, and accompanied by a set of Acceptance Criteria that must be met by the solution, which is designed by the team.  

Put another way, the Product Owner will represent the problem, and the cross-functional team, will design the solution.   In this context, it is sometimes said that Product works in the “problem space” whereas the team (Engineering, UX, Data, etc) work together in the solution space.   

The Product Owner is responsible for determining the “what & why”.  The team, led by the ScrumMaster, owners Delivery and determines how to solve the problem and how long that should take.  Release/launch plans are determined together by the Product Owner and ScrumMaster, based on the desires of Product and the capabilities of the team.

ScrumMaster

The ScrumMaster is the administrator, leader, and protector of the team. They ensure the team follows best practices, updating tickets, ensures the team is working efficiently toward priorities, and not committing to too much.

The ScrumMaster can be anyone on the team, though it is often an Engineering team lead.  Or, when the team is large and truly cross-functional with a lot of non-engineering capabilities represented, there may be a Program Manager in that case who plays this role and facilitates the team, and ensures Scrum best practices. A Program Manager is a process and delivery efficiency expert, who may play this role for multiple teams within a focus area, keeping the teams coordinated, aligned, and running smoothly.

Team

The team is comprised of a group (ideally 5-8) of cross-functional members that span all of the capabilities necessary to complete the requirements from Product.  For example, if the team is building a SaaS product, you’d ideally have WebApp developers, mobile app developers, UX, and QA on one team. In the case of a larger team working on a more sophisticated application, you might have multiple scrum teams each working on specific features of the application – these teams are sometimes referred to as feature teams, and similarly you would have the capabilities represented that are required for that feature.  If the team in this case was working on the search & personalization algorithms then you might have a data scientist on the team and API developers rather than WebApp and UX.

Responsibility of Roles

The strength of Agile development is flexibility and agility but this can also sometimes be its curse.  Everyone interprets roles and responsibilities a little bit differently which can lead to confusion, gaps where things aren’t accounted for, and overlaps that can lead to toe stepping and politics.  It can be really helpful to discuss expectations about roles and responsibilities as a team from the outset, to avoid such issues and make sure expectations are aligned.

The DACI model is a useful framework for aligning those expectations.  DACI is acronym and stands for Driver, Approver, Contributor, Informed.  The goal with this framework is to identify key activities of the team, and then map the responsibility for each of these activities, for each of the roles.   

For example, the Product Owner is Driving requirements definition, though the team may contribute insights to Product where it is helpful.   There shouldn’t be anyone on the team who needs to Approve the requirements, though it’s possible a Business or Product leader outside of the Scrum team may practice some level of oversight.  And, the ScrumMaster would be kept informed of the requirements as they enter the Scrum team, so they can manage the team’s activities against them, accordingly. 

Here is an example table that maps the common activities and responsibilities as reference.

Creating a Feature

Now that we have established processes, roles, and responsibilities, let’s walk through the process of creating a new feature to see how it all comes together.  Assuming we have already defined the feature requirements (User Stories), then we’re ready to engage the team on the 3 high-level stages of feature development – Discover, Develop, and Diagnose.   Let’s discuss each of these 3 D’s:

Discovery

The product requirements that we wrote for this feature, in the form of a User Story, describe the problem we need to solve, and include acceptance criteria that indicate the bounds of an acceptable solution.  We intentionally take this approach and avoid prescribing a solution in the requirements, in order to facilitate solution design by the team – that process begins with learning more about the context of the problem, in order to design an effective solution. 

In the case of a workflow feature, Product may partner with the UX designer to conduct user research, observe the difficulty, and validate prototypes of a possible solution. Or in the case of a search algorithm, Product may partner with the Data Scientist on the team to review data and better understand the insufficiency of the algorithm.  In both cases the approach is similar – Product partners with the capability expert on the team to do discovery and better understand the problem, handing off to the team member(s) to design the solution, and approving the design once completed.

Development

Once Discovery is complete, the User Story is taken to the next Backlog Grooming session where the team will break down the solution into sub tasks that must be done in order to build that solution.  The Product Owner supports this process by answering questions and providing clarity where it is needed, and working with the team to determine the release plan for the feature (timeframe, A/B testing, etc).  From there, the Product Owner becomes more passive until we’re nearing time for feature deployment. 

The primary responsibility of the Product Owner during the Development stage, is Discovery on the next feature, so that it is ready for Development when the team is done developing this feature. In Agile/Scrum process, Discovery can be thought of as a track of work that is happens in parallel to Development – a concept sometimes referred to as Dual Track Scrum.

A word of caution here – it can be to dismiss yourself form the day to day activities of your team during development; more than one Product Owner has succumbed to day-to-day need for administration and coordination by the team. Don’t do it!  These are the responsibilities of the ScrumMaster and spending your limited time here may seem like the right thing to do, but it will abdicate your core Product responsibility and earn you a reputation of being a Project Manager, not Product.  Remember, your core responsibility is to ensure your features are useful, usable, and valuable to the customer. 

Deploy

Finally, your feature is ready to go live and it is time for another flurry of Product-related activity.  The Product Owner will need to let affected Stakeholders know, coordinate with support staff so they can support the new feature, work with the analytics team to ensure the feature is properly instrumented and appropriate dashboards are available. If a limited rollout or A/B testing is planned, that will also need to be setup. 

Whenever possible, features should be deployed as a limited availability test, so impact can be measured and evaluated before full availability of the feature. When the feature is not performing optimally, or in the case of an MVP release, this can lead to a sub-stream of additional work in the coming sprints that we’ll balance with other feature development, in order to ensure we perfect the feature before calling it ‘done’.  Remember that as Product, we measure success by the outcome of our work against our objectives & KPIs, not merely the delivery of a feature.  

Conclusion

Agile/Scrum has become the most popular process for product feature development. The role of the Product Owner in Agile/Scrum is often played by a member of the Product Management team who works closely with the product development team(s) to discover, develop, deliver, and optimize features they create. In this context, the Product Owner always plays an active role in discovery and delivery of new features, always with an eye on what’s useful, usable, and valuable to the customer.   Beyond the initial stages of market analysis and product planning, Product Development is where the magic happens and new product features are brought to life.