Doug Hieber, Senior Software Architect
July 26, 2015
In Eric Ries’ book “The Lean Startup”, he describes a product development approach that drives innovation and success through continuous cycles of “Build – Measure – Learn”. How does it work? The author of this week’s blog post explains…
In Eric Ries’ ground breaking book “The Lean Startup”, he describes a product development approach that drives innovation and success through continuous cycles of “Build – Measure – Learn”. The focus is on making startups Lean, but its Lean Development principles add value beyond that.
The Lean Development approach has short development cycles whose focus is on discovery and adaptation to harness human creativity more effectively. Its goal is not just to create a product faster or have more frequent releases, but to use a scientific, systematic approach that ensures the right product is created. By avoiding the wasted time and energy of a large, long term business plan, it allows the business to get moving faster and to evaluate its assumptions quicker. That last part is key. By evaluating assumptions in fast, iterative loops, the business is able to test their vision early and often. And by doing so with a minimum of investment, it can cost-effectively change those assumptions or even pivot in an entirely new direction.
So how does it work?
The Lean approach starts with a Minimum Viable Product, which means you build just the bare minimum needed for the product to work, no more. Once deployed, the success of those features are measured through a set of predetermined metrics. For example, a web site might measure utilization rates of a new feature or the signup rate. Through these measurements, we gain what Eric Ries calls Validated Learning, where we learn by trying out an idea, generating metrics to measure progress toward a goal, and then analyzing the metrics to learn what is working and what isn’t. And by “working”, I mean it is advancing the business toward its goals. A feature that is bug free but does not achieve the intended business goal does not add value to the business.
It is important to note that the metrics need to be actionable so that they can lead to informed business decisions and subsequent action. They need to reflect the key business drivers and inform decision makers on the progress, or lack of it, toward the business goals and vision. This is in stark contrast to “vanity metrics” designed to present the best picture possible. In short, actionable metrics help uncover the truth, whereas vanity metrics just help us feel good.
The Lean approach is a perfect match for web startups and certain types of new product development. It enables a discovery process to help the business target the right features, or maybe even to pivot the business goals in a new direction. A famous example is Groupon, which started as a failed activism platform called The Point, until its Validated Learning pivoted it into the coupons space and a billion dollar success.
Is Lean a good match for the assessment industry?
This needs careful consideration to make sure the unique needs of the assessment industry are met. High stakes exams cannot afford to roll out new products in the form of new item types or new exam structures that are not thoroughly vetted. Further, the typical distribution channel to test centers does not provide the super quick feedback loop needed to drive the Build – Measure – Learn process. The candidate experience, exam reliability and certification program reputation are not to be toyed with lightly. As such, the Build – Measure – Learn process is not appropriate for the assessment areas that demand a more systematic and comprehensive approach.
But the Lean approach can still add value to assessment companies, even high stakes certification programs. You just need to apply it to the appropriate environment. New item types can be initially developed and field tested as a Minimum Viable Product and then improved through successive Build – Measure – Learn cycles. Item authoring systems can be enhanced in the same way with deployments going to a UAT environment with UAT user feedback being used to validate features, direct enhancement growth and avoid waste (features that add little value). Only when the new authoring feature or item type is stabilized would it be released into production.
And assessment companies also have non-mission critical system, like a public facing website. New features here can start small with a Minimum Viable Product, then have utilization rates measured to gauge marketplace interest.
The benefit of starting with a Minimum Viable Product is huge. It allows new features to grow in unexpected ways as users directly interact with the product. It also avoids the waste of expensive, often elaborate, features that rarely get used. By starting with a Minimum Viable Product and measuring its usage through predetermined metrics, that rarely used feature is identified as such early on. This ensures its enhancements are de-prioritized and that it never grows to be the elaborate waste of time and resources that would occur in the bad old days when large groups of SMEs would get together for a requirements brain storming session.
As the diagram above shows, any effort spent on a product destined to fail, or at least not meet its business goals, is a type of waste. The sooner issues are identified, the sooner the business can adapt.
For example, a company may have an idea to host live web events. It starts with a Minimum Viable Product of a simple interface listing a few webinars and expects it will drive more users to the company’s website. Unfortunately, the metrics show that there is only a small increase in web traffic. But they also learn from the metrics that webinars about career counseling is much more successful than any other webinar they post. And that viewers of these webinars had a high rate of clicking through to view the company’s other products. Since the live web events feature started small, the company still has resources available to adapt to this Validated Learning. As a result, enhancements to the Live Web Events are put on hold to pivot the business vision in the direction of a suite of career counseling tools. Again, the company should start the implementation of this new vision small, then measure and learn. But this time, they may find there is strong interest and invest more Build – Measure – Learn cycles in this area of the website. Which is not to say live web events should not be pursued again in the future. It just means that the company learned career counseling added more business value by drawing more people to the company than live web events. This makes the company stronger and more successful.
The Agile-ites among you are probably thinking this all sounds very familiar. For example, these cycles sound a lot like Scrum sprints. And you are right. Lean is a natural complement to Agile. It benefits from Agile’s short development cycles, continuous deployment of potentially shippable products and flexible approach to requirements. But it brings in an additional element of learning, adapting, adjusting and innovating. Its goal is not just the development of a product, but to validate if the product should be built at all, and if not, help the business learn how to adjust their business vision.
Eric Ries’ book covers many other concepts to enable innovation and is filled with real world examples. He provides detailed methods for gathering and analyzing metrics. There are a “number of counter intuitive practices that shorten product development cycles, measure actual progress without resorting to vanity metrics, and help us learn what customers really want”. He provides techniques for getting to the root cause of a problem through exploration of the underlying cause and effect relationships. Further, he brings a scientific approach to testing business assumptions.
I found it thought-provoking, informative and enlightening. It truly challenged my base of conventional wisdom and changed my way of thinking about requirements and product development. I suggest you give it a read.
Maybe you too will go Lean!
REFERENCES:
The Lean Startup: http://theleanstartup.com/
MVP: http://en.wikipedia.org/wiki/Minimum_viable_product
Validated Learning: http://en.wikipedia.org/wiki/Validated_learning
Doug Hieber, Senior Software Architect
July 26, 2015
In Eric Ries’ book “The Lean Startup”, he describes a product development approach that drives innovation and success through continuous cycles of “Build – Measure – Learn”. How does it work? The author of this week’s blog post explains…
In Eric Ries’ ground breaking book “The Lean Startup”, he describes a product development approach that drives innovation and success through continuous cycles of “Build – Measure – Learn”. The focus is on making startups Lean, but its Lean Development principles add value beyond that.
The Lean Development approach has short development cycles whose focus is on discovery and adaptation to harness human creativity more effectively. Its goal is not just to create a product faster or have more frequent releases, but to use a scientific, systematic approach that ensures the right product is created. By avoiding the wasted time and energy of a large, long term business plan, it allows the business to get moving faster and to evaluate its assumptions quicker. That last part is key. By evaluating assumptions in fast, iterative loops, the business is able to test their vision early and often. And by doing so with a minimum of investment, it can cost-effectively change those assumptions or even pivot in an entirely new direction.
So how does it work?
The Lean approach starts with a Minimum Viable Product, which means you build just the bare minimum needed for the product to work, no more. Once deployed, the success of those features are measured through a set of predetermined metrics. For example, a web site might measure utilization rates of a new feature or the signup rate. Through these measurements, we gain what Eric Ries calls Validated Learning, where we learn by trying out an idea, generating metrics to measure progress toward a goal, and then analyzing the metrics to learn what is working and what isn’t. And by “working”, I mean it is advancing the business toward its goals. A feature that is bug free but does not achieve the intended business goal does not add value to the business.
It is important to note that the metrics need to be actionable so that they can lead to informed business decisions and subsequent action. They need to reflect the key business drivers and inform decision makers on the progress, or lack of it, toward the business goals and vision. This is in stark contrast to “vanity metrics” designed to present the best picture possible. In short, actionable metrics help uncover the truth, whereas vanity metrics just help us feel good.
The Lean approach is a perfect match for web startups and certain types of new product development. It enables a discovery process to help the business target the right features, or maybe even to pivot the business goals in a new direction. A famous example is Groupon, which started as a failed activism platform called The Point, until its Validated Learning pivoted it into the coupons space and a billion dollar success.
Is Lean a good match for the assessment industry?
This needs careful consideration to make sure the unique needs of the assessment industry are met. High stakes exams cannot afford to roll out new products in the form of new item types or new exam structures that are not thoroughly vetted. Further, the typical distribution channel to test centers does not provide the super quick feedback loop needed to drive the Build – Measure – Learn process. The candidate experience, exam reliability and certification program reputation are not to be toyed with lightly. As such, the Build – Measure – Learn process is not appropriate for the assessment areas that demand a more systematic and comprehensive approach.
But the Lean approach can still add value to assessment companies, even high stakes certification programs. You just need to apply it to the appropriate environment. New item types can be initially developed and field tested as a Minimum Viable Product and then improved through successive Build – Measure – Learn cycles. Item authoring systems can be enhanced in the same way with deployments going to a UAT environment with UAT user feedback being used to validate features, direct enhancement growth and avoid waste (features that add little value). Only when the new authoring feature or item type is stabilized would it be released into production.
And assessment companies also have non-mission critical system, like a public facing website. New features here can start small with a Minimum Viable Product, then have utilization rates measured to gauge marketplace interest.
The benefit of starting with a Minimum Viable Product is huge. It allows new features to grow in unexpected ways as users directly interact with the product. It also avoids the waste of expensive, often elaborate, features that rarely get used. By starting with a Minimum Viable Product and measuring its usage through predetermined metrics, that rarely used feature is identified as such early on. This ensures its enhancements are de-prioritized and that it never grows to be the elaborate waste of time and resources that would occur in the bad old days when large groups of SMEs would get together for a requirements brain storming session.
As the diagram above shows, any effort spent on a product destined to fail, or at least not meet its business goals, is a type of waste. The sooner issues are identified, the sooner the business can adapt.
For example, a company may have an idea to host live web events. It starts with a Minimum Viable Product of a simple interface listing a few webinars and expects it will drive more users to the company’s website. Unfortunately, the metrics show that there is only a small increase in web traffic. But they also learn from the metrics that webinars about career counseling is much more successful than any other webinar they post. And that viewers of these webinars had a high rate of clicking through to view the company’s other products. Since the live web events feature started small, the company still has resources available to adapt to this Validated Learning. As a result, enhancements to the Live Web Events are put on hold to pivot the business vision in the direction of a suite of career counseling tools. Again, the company should start the implementation of this new vision small, then measure and learn. But this time, they may find there is strong interest and invest more Build – Measure – Learn cycles in this area of the website. Which is not to say live web events should not be pursued again in the future. It just means that the company learned career counseling added more business value by drawing more people to the company than live web events. This makes the company stronger and more successful.
The Agile-ites among you are probably thinking this all sounds very familiar. For example, these cycles sound a lot like Scrum sprints. And you are right. Lean is a natural complement to Agile. It benefits from Agile’s short development cycles, continuous deployment of potentially shippable products and flexible approach to requirements. But it brings in an additional element of learning, adapting, adjusting and innovating. Its goal is not just the development of a product, but to validate if the product should be built at all, and if not, help the business learn how to adjust their business vision.
Eric Ries’ book covers many other concepts to enable innovation and is filled with real world examples. He provides detailed methods for gathering and analyzing metrics. There are a “number of counter intuitive practices that shorten product development cycles, measure actual progress without resorting to vanity metrics, and help us learn what customers really want”. He provides techniques for getting to the root cause of a problem through exploration of the underlying cause and effect relationships. Further, he brings a scientific approach to testing business assumptions.
I found it thought-provoking, informative and enlightening. It truly challenged my base of conventional wisdom and changed my way of thinking about requirements and product development. I suggest you give it a read.
Maybe you too will go Lean!
REFERENCES:
The Lean Startup: http://theleanstartup.com/
MVP: http://en.wikipedia.org/wiki/Minimum_viable_product
Validated Learning: http://en.wikipedia.org/wiki/Validated_learning