The Lean Startup – Book Notes
Reading time: ~25 min.
Many management ideas come from manufacturing. The author’s Lean Startup ideas resulted partly from the lean thinking used in manufacturing by Toyota. So, lean thinking applied to startup processes.
A startup can be defined as an institution designed to create new products or services under conditions of extreme uncertainty. The goal of a startup is to figure out the right thing to build as quickly as possible. A startup can be defined as an institution designed to create new products or services under conditions of extreme uncertainty. The goal of a startup is to figure out the right thing to build as quickly as possible. One of the ways to achieve that is to iterate quickly. For example by making constant tiny adjustments instead of creating a plan and following it to the finest detail. The products a startup builds are, in essence, tests, whose point is to learn about the customer and learn how to build a sustainable business in that field.
Some believe that a company’s only long term path to success is to be constantly innovating. Long gone are the times when they could milk some innovation for many years and rest.
This chapter [which i found to be one of the most important] defines the view that learning is the most important measure of progress of a startup. The author also defends that the traditional ways of analysis such as doing a business plan and doing forecasts don’t work in a startup environment because the uncertainty is so high.
The learning process must be validated — it must be demonstrated that learning, and therefore progress, is indeed occurring. This can be done, e.g., by showing by means of real unambiguous data that the team has found important truths. For example, somehow we got the idea that doing X is something clients would like (this is our guess). Then we need to test that. If we do X and only X, and suddenly the number of buying customers increases, then our hypothesis about X is indeed true.
The author now tells the story of one of his startups: He says how they had this idea. They thought people would behave like this and that. They made predictions. When they launched it failed, no one wanted it. When they interviewed potential clients and explained them how they should behave they looked bewildered. Basically they had wrongly assumed a lot of things about the clients and about what they wanted. The interviews ended up showing them a new direction that clients actually wanted. So despite using lean thinking techniques to eliminate waste he had created the biggest waste — a product his customers didn’t want. Yet he tried to console himself by saying that it was a valuable learning process. However, couldn’t it have been much faster? How much of the effort and time really contributed to learning? It is here where he reaches the conclusion that the best way to measure progress of a startup is by the amount of learning about what customers want. Whatever effort is not spent on learning what clients want is a waste of time and energy. And this learning need to be validated, that is, we need to verify what we think we learned is really true. So we only really know we learned something when we apply it, and we see the customers talk with their wallets positively. So, in a startup, productivity is better thought of as the amount of learning per amount of effort. Try to learn the most but while doing the least.
We should prefer to getting real customer data by launching an early unfinished version of the business and doing real time tests to asking questions by surveys and building one big launch. We should learn from what is working today in real life conditions than from trying to guess what might work tomorrow based on assumptions and surveys.
So this way of setting up hypothesis and testing them by means of verifiable real-world data is a part of the scientific method in action. Yes, we should view this as science, we should be that rigorous. [This is very similar to the way Google, the company, thinks] There are two main hypothesis that we should test — the value hypothesis and the growth hypothesis. The 1st is is about whether the business gives value to the customer and can be tested by e.g. retention rate (how many people come back for more). The 2nd is about whether it has traction for growth and can be tested by its virality (we’ll deal about this later on).
In the very beginning of a startup we should focus on early adopters, those that need the product the most and that more easily forgive problems.
4 questions that we should ask ourselves before starting:
1) do customers have that problem?
2) if they do, and if there was a solution, would they buy it?
3) would they buy it from you?
4) can you build that solution?
Identify elements of the plan that are assumptions and figure out ways to test them. Figure out what you need to learn, figure out how to measure things so that you’re getting validated learning, and then build the product that allows you to do all this.
Start testing the assumptions soon, with a minimum viable product/service (MVP/MVS), basically an alpha version. Make it highly targeted to the customers your aiming (early adopters specially).
Startup loop, genchi genbutsu and customer profile
We should have our startups constantly making the following loop — »» ideas »» build »» product »» measure »» data »» learn »» ideas »» and when building a startup we should focus our energies on minimizing the time and effort per lap.
Genchi Genbutsu — means “go and see” in Japanese and it is a key principle of Toyota’s Lean Production System. It means that to truly understand something we need to go to and physically see it for ourselves. It is unacceptable to take anything for granted or rely on the reports of others.
Design and the customer archetype — We need to contact early with the customers in order to know them. To know in broad strokes who they are and what their problems are — their profile. Yet, the customer profile is only an hypothesis until we validate it through real life data and see that we can serve this customer in a sustainable way.
The Minimum Viable Product
Too little analysis generally leads to failure — for example not doing basic research to check at least on a basic level if there really is a problem to solve, or if anyone else has solved it. But too much is also dangerous — example 1) people do extensive market research but then later could reach the conclusion that it was based on wrong facts; example 2) it can lead to paralysis by analysis (which is the state of over-thinking a situation so that a decision or action is never taken. So the best way to go about launching a product is the MVP. The author gives the example of Groupon and how it started by being a company with a totally different objective but by seeing what customers wanted changed to dealing with coupons entirely. And also how they first started by sending the coupons by email one by one in a word press made page. All very basic but enough to see if the new idea had feet to walk. The MVP is meant to go through the loop mentioned before as quickly as possible. Note that it is not the fastest product possible, but the one that allows the planned learning to occur the quickest. (by planned learning I mean that we build the easiest MVP we can build that allows us to test the idea or learn about something we want to learn.) remember how Gmail stayed in beta for so long? It was very simple but had the most important core elements that made it the best. YouTube the same. It was buggy, ugly, etc. But it’s main functionality worked the best.
Sometimes putting out an unfinished product calls the attention of early adopters that even prefer it that way, as they can feel as they can be part of the development. Some case studies are then presented.
Dropbox distinguished itself from the others because it worked perfectly. But the development team would not be able to show that by doing a mvp. So they did a video showing how it would be (a 3 minute video demonstration) catering to the early adopters (tech people) and overnight the sign ups on the website grew by 1500%.
The Concierge MVP
The concierge MVP is basically a temporary solution for a product, that is used for example:
- Find out which solution customers prefer without investing too many resources developing all the alternative solutions;
- As a transition solution that works when the customer numbers are lower, that is simpler and less expensive;
- To work as crutches until the company has reached maturity.
To show how the concierge MVP works the author gave an example of an online service that shows people a shopping list and its total price for some suggested recipes using available ingredients in nearby stores based on the subscriber’s tastes and location. The CEO started with one customer and doing it by person. That is, he would go personally give the customer the list, and did it with just one store. Then as he learned more and improved the service, more customers started appearing and he started automating certain aspects culminating in a fully automated service. This strategy shows that the growth model doesn’t work unless the business strategy is changed. For example a single shop might result in profit, but opening more shops would mean that the uniqueness and value proposition of that shop changes (imagine a shop that specializes in something that is catered to a specific tourist attraction in that place and only that place. If that shop were to open somewhere else to sell other tourist attractions then it’s value proposition would have to change from specific to the original tourist attraction to a more general statement about tourist attractions).
Some guys had a vision: that there should be a way to ask an engine a very specific question and it would answer (not like Google where it just shows links and not actual answers). What they did was very smart and allowed them to be purchased for $50 million by Google: they launched several products each that solved that a problem or variations of it in different ways, in order to know which one people liked the most. But to accomplish that the product was kinda fake behind the scenes. Parts of some products were humans that did things that were supposed to be an algorithm doing. This way they saved a lot of time in developing complicated algorithms before even knowing if people would want the final result. Basically it is a situation in between the demo video and doing the complete product.
The Problem of quality in MVPs
If we don’t know who the customer is and what he wants we don’t know what quality is. The author mentions how in the product they were developing they avoided creating 3D characters and used a teleportation without sound or visual effects to solve the problem of the characters moving around. Turns out it was the most liked feature even above others that had taken them much more time to develop. Customers don’t care about how much time something takes to develop, in the end they only care about their needs being met. So the conclusion is that something they thought was low quality was actually higher quality.
So, consider this simple rule: remove any feature, process or effort that does not contribute directly to the learning you seek.
Some reluctances for creating and delivering an MVP:
- Decreasing the headstart to the competition and allowing them to steal the idea earlier — One of the biggest fears of people releasing a MVP is the fear that their idea is stolen by a bigger company that will then outperform them. Sooner or later a startup will face competition from fast followers. The only way to win is to learn, that is, go through the the learn loop, faster than anyone else. It is the only way to avoid being out-executed by someone else;
- Ruining brand reputation because of releasing an initial low quality product — Startups are really obscure when they start, so it’s OK. You can use the low number of costumers to experiment and then when the product has proved itself with real customers you launch a bigger marketing campaign. What larger companies can do to avoid shaming their brand is launching their product under a different brand. But then again what is better: be seen as the company who spent months getting ready for a full body dive only to see the water was too cold and all that time was wasted or the company that since it doesn’t know how hot is the water, only dips the feet as a test to see if it should go deeper or change place?
Are you making your product better? How do you know? And remember that better, is subjective, better for the customers not for you.
The author gives 3 steps to guide a startup’s strategy:
- Test the most important assumptions by launching a MVP. This will bring to the table real data so that we can assess where we really are, how customers behave right now. This is called the baseline;
- Then we try to steer the company’s baseline values to the ideal values e.g. by trying to saturate the local market, by improving the product, and so on;
- Assess how fast the business values are going to the projected ones, and if they are going too slow or not at all, rethink that maybe it’s time to change strategy. If the values with which we measure the performance of our business don’t improve towards what is ideal, then we’re not making progress.
So far we’ve talked about 1). Now let’s move on to 2).
One of the most important analysis tools for a startup is a cohort analysis. A cohort analysis is when we study the behavior of separate groups of customers that were grouped based on a certain common characteristic (a cohort is a group of people who share a common characteristic over a certain period of time). For example to study those that access our website we can see how many people came back, how many of those registered, of those how many logged in, of those how many bought the product, etc. Flows are sequences of customer behaviors that are needed in order for the product or service to be sold (for example in an online store one needs to go the website, maybe login, they add a product to the cart, then check out, input data, etc. this is the flow of actions). But the point the author is making is that analysis cohort is much more useful than analyzing gross (i.e.total) numbers. He also mentions the importance of split tests (also called A/B tests) , where 2 different versions of the same product are launched, and then data is analysed. This is a great way of knowing which variables are responsible for what. Also it often uncovers surprising results, namely that many features or things that engineers or those that are developing the product think is an improvement have no impact in customer behavior. Even though it takes a bit more effort two launch 2 versions (or simply keep the old one and test it against a new one) it saves a long time in the long run by preventing you from working on things that don’t affect customers, that is, with split testing you learn better what customers really value.
What constitutes a good metric?
- It must be able to show inequivocally cause and effect. It must show which variable was responsible for which effect;
- Easily understandable (in simple language) so that people from different backgrounds can give their inputs.
Like the Japanese manufacturing technique, this development technique consists of only doing things when necessary, just in time. In the case presented by the author he suggested 4 stages that an activity goes through (backlog >> in development >> developed >> being validated) with a limit of 4 activities simultaneously in each stage. So for example if there are 4 developed products but that aren’t being validated, no more can developed (otherwise when the development ended they could end up passing the limit of 4 developed products).
Do we need a major change?
For example, after a year of a perfect loop of ideas >> testing >>learning >>… That is A/B testing, validated learning, and continuous implementations of those learnings with little improvements it might be time to ask ourselves the question if a major change in the product is needed. When your startup doesn’t grow exponentially or even just to close to the ideal values you’re stuck in what they in Silicon Valley call the land of the living dead: your startup makes just enough to stay alive but not much else. You might feel like success is just around the corner. Also the more you invested in it both in energy time and money, the more difficult it may be to let it go. This is why it is so important to learn fast and learn early and as cheaply as possible. This major change might mean you create a similar startup where you still apply things you learned but where you’re going to test other assumptions that you couldn’t on your previous startup because it wasn’t how it worked. For example if your business was about selling lamps, now it could be about selling parts of lamps, or designing lamps. What you might discover is that suddenly your return on investment skyrocketed. The general premises from the previous startup up had shown how far they could go and it wasn’t that much. Maybe there are other premises out there that can go much further.
Sometimes the same startup has to go through many major changes (what the author calls pivot). However all those major changes might not necessarily bring cash but actually burn it as they require extreme energy to be developed. There is a term that is used to mean how much time a startup has, it’s runaway (like the path of the airplane before it lifts), so in the end it either lifts or crashes. It a startup is burning 100k/month and has 1M in the bank it’s runaway is 10 months. But the author suggests that a better way to define it is in terms of how many major changes can it take. Which means that, again, the faster you do each change the better.
One of the greatest concerns of entrepreneurs that wonder if they should do a major change is that the startup hasn’t yet had time to prove itself. This is even more visible when the brand is well known and the people behind it have very public exposure (thus they feel shamed). So do things undercover to mitigate that.
Some important concepts:
- The cost of acquiring a new customer — customer acquisition cost;
- Distribution channels — the mechanism by which a company delivers a product to its customers. A direct distribution channel would be one where the manufacturer sells directly to the consumer. An indirect one could pass from the manufacturer to the importer, to the distributor, to the wholesaler and to the retailer;
- Overhead (also called operating expenses) — which are those that are necessary for the business to run, kind of like how much gas your car consumes just by being turned on);
Another important thing to do: to develop an archetype of your customer (trying to predict how he is, how he behaves).
Going back to the major change topic. The author defines some types of pivots :
- Zoom in — a single feature of the product ends up becoming the whole product (e.g. company that sold clothes with artistic drawings pivoting to only sell the drawings);
- Zoom out — the whole product becomes a single feature (e.g. a shop that only sold shoes pivoting to sell all kinds of footwear);
- Customer segment — the product is solving a problem but for another customer segment than originally predicted so the company pivots it’s main customer segment;
- Customer need — the company might see that the problem they’re trying to solve is not really a problem but they may discover a completely new need to solve;
- Business architecture — businesses can be high margin low volume, low margin high volume and somewhere in between. The pivot is when it changes from one to the other. For example if a company sees that what they originally had created to sell to other companies might be much more interesting to sell to the mass market;
- Value capture — changes in the revenue model. For example from fremium to subscription based;
- Growth strategy — will be talked about later;
- Distribution channel — the Internet is a great example of a technology that forced many companies to change their distribution channels;
- Technology — when the company decides to test another technology to achieve the same solution.
A pivot can be understood as a strategic hypothesis that will require a new MVP. They are a permanent fact of life for growing businesses.
After a small introduction now we’re talking about lean manufacturing and how many lessons from it can be applied to startups, with the author focusing on the power of small batches. An example is given that it is faster to seal and attach stamps to envelopes one by one (batch size of 1) than doing each operation for all the envelops (batch size = all the envelopes). Fun fact: many of these lean thinking concepts were developed at Toyota which at the time couldn’t compete with American car companies that produced very large quantities for a big market. Toyota by implementing this lean thinking into their manufacturing not only achieved a higher flexibility in the types of products they could make but they also could compete in mass manufacturing. In 2008 they became the biggest automaker.
Small batches allow problems to be corrected earlier and more cheaply. Again this is a good analogy for the cycle that we talked about earlier. So let’s say we just started producing a part. It is much better to find a defect or a problem in that first small batch and do so early than only to discover it after producing 100’s of it. The same with a startup that is creating a product. The problem could be that customers don’t care about the product. Or that a pivot is needed. An example of a company that despite having internal lean thinking still results in the old mass manufacturing ideas is apple. They release one flagship a year, with about 1500 innovations. Their managers think about what customers might want, and they work on them for the whole year and release them all at the same time.
The extreme of small batches is continuous development, for example, in development where the changes that are made are continuously updated online. This method of development may clash with hypotesis testing because if too many new variables are introduced simultaneously then we may not be able which are responsible for a certain effect.
This small batch size in manufacturing allows greater flexibility in changing products in the production line and in making them customizable, even more so if SMED is applied. The author points out some reasons why production has been changing from big to small batches: things getting computerized, 3D printing over injection molding, and lean thinking.
He then gives the example of a company that specializes in rapid manufacturing of everything, SGW Designworks, who work by doing 3D modeling and talking with the customer very early, then quickly proceeding to a prototype. Iterations are obtained until the client is satisfied and voilá, they soon arrive at the final product that the clien wants. Another example is the innovative school of one, where instead of the normal method of mass teaching, things go much much differently and in ways where the teacher gets feedback of the methods he’s employed much sooner.
The author then goes deeper into concepts of manufacturing and delves into the concept of small batches further also explaining that it reduces work in progress (WIP) explains the pull system. In startups it may be a bit hard to identify what is the WIP because it deals with intangibles. It’s things that haven’t gone live (e. g. website version, patches,…)
He says something that I liked — “Startups don’t starve, they drown.” — meaning that in startups there are always too many ideas going around on how to improve a product, but most only help marginally, they are but optimizations. One should keep focused on the big experiments that lead to validated learning.
Back to the methods of growing. (as a side note I don’t understand the focus with growing, why does a company need to be constantly growing? Why is the concept of ‘staying where it is’ referred to with pejorative terms such as ‘stagnating’?)
1) Growing based on long term customers
E.g. cellphone service providers, gyms, subscription based… For this type of growth it is important to know the business’ s customer retention rate and leaving rate (also called churn rate). If more people stay than leave (for a given period of time) then it had growth. Now now that a company can have many new customers, but if they don’t come back to buy stuff more times it’s going to have a problem. This is why it’s important to do cohort analysis and not just look at certain statistics and say “we’re growing!”. In this case a number of new users that is increasing can mean a bad thing if they don’t come back or buy stuff. To solve this we can incentive customers to come back, e.g. with random sales and price fluctuations, so that they must come often to find the best bargain.
Another useful concept is customer lifetime value (CLV), basically how much customers are expected to pay in total from when they pay the first time to the last. Another one is cost per acquisition (CPA), that is, how much does it cost us to acquire a new customer. For example if we’re paying 100€/month on ads and we know that those ads get in 100 customers, our CPA is 1€. If CLV > CPA then we’re growing baby!
2) Viral Growth
This growth depends on word of mouth to spread itself. Virus coefficient represents how viral the growth is, or how many new costumers will use the product as a consequence of each new customer that signs up. So if on average a new customer brings another, then v=1. If 10 customers bring a new one, then v=0.1. Above v=1 (each new customer brings more than one) the growth is exponential. From what I understood most viral products rely on indirect sources of revenue, like advertising, instead of charging the customer directly. E.g. Facebook, blogs. This is a reason why these should be free, because they will increase v. These startups should focus on increasing v as much as possible because the effects have increasingly high returns (how to do it?) (how to know whether our company will grow viraly?)
When a startup needs to pay to acquire new customers. Once again we study the concept of CLV and CPA. If we have two companies, one makes 1€ per customer that signs up and the other 100 000€, but the first only spent 0.80€ in advertising through adwords to get him, while the other spent 120 000€ in R&D and engineering stuff then the 1st will grow faster.
The MVP should contain no additional features other than those absolutely necessary to early adopters/any other work that is not necessary for learning is a waste.
There’s technique called the 5 why’s, where when you encounter a problem you ask why 5 times in order to know the root cause. Generally the first answers are but symptoms and the real cause lays deep beneath them. According to the author that root cause is generally human-related. The lesser a problem the less resources we should invest in it. This is because when creating a MVP and trying to learn full speed sometimes quality is lost and problems can arise, which in turn slows down the learning process. So there should be a balance between putting out new things to learn and correcting problems so that we can keep putting those things out rapidly.
An example is given of a company that did yearly release cycles of their product. After a while it started to show severe problems because customer feedback came in too late in the development process to change the product significantly before its release. So they reduced the time of the cycle development-release, that is, they started putting out smaller batches and started having better reviews. (This way of incremental development was heavily criticized by Theil) The author also mentions the problem of people having muscle memory. So people used to working in big batches find it hard to adapt, so it’s better to start from the beginning working in small batches.
Other concepts mentioned in the final chapters:
- Venture-backed startup — are those funded by venture capital, which is money provided to seed/early-stage, emerging and emerging growth companies (so it’s a fancy way of saying money given to startups);
- Bootstraped startup — are those that launch and are built with little or no funding.
- The Toyota Way: 14 Management Principles from the World’s Greatest Manufacturer
- The Lean Startup website;
- The Lean Startup: How Constant Innovation Creates Radically Successful Businesses
There are lots of legitimate ideas that anyone could and should use. The idea of an MVP, A/B testing, build-measure-learn feedback loops are all simple things that most people should be aware of. Have you built any startup using these principles? How have they worked for you?