When I had a motorcycle, I also had, inevitably, a theory about motorcycle safety. The theory went like this: Motorcycle’s are about as safe as cars. On the plus side, you sit higher than in a car seat, so you can see better. Also a motorcycle is smaller, more maneuverable, and quicker than a car, all features that give the alert cyclist a better chance to maneuver out of sticky situations and avoid accidents. On the minus side, you’re not protected on a bike, so if you do have an accident, you’re likely to be much worse off than the driver of a car.
This theory served its purpose admirably. I liked riding motorcycles, and needed a reason to believe I wasn’t putting myself or my family in harm’s way. At the same time, I wanted to be realistic, which required some acknowledgement of danger.
Adducing evidence for this theory was easy. Good view? Quick? Maneuverable? Check. Check. Check. I (thankfully) never tested the part about the consequences of an accident at speed, but that stood to reason anyway. It never occurred to me, or if it did, I suppressed the thought, that I could test my theory in some other way, like by checking motorcycle accident statistics.
Startups operate on theories too. Actual businesses may or may not operate on a theoretical basis. It’s possible to have a business that simply does x and customers spend money for it, with virtually no broader view. Most of the time though, and all the time with startups, companies operate on the basis of theories. “We believe (because of X, Y, or Z) that if we do A, customers will do B.
Startups easily fall into the trap of testing the validity of the hypotheses in their business model canvas the same way I tested the validity of my theory about motorcycle safety. They take the factors that led them to the theory in the first place, and look for further evidence to support them. That evidence is almost certainly available – if it wasn’t, the entrepreneur probably wouldn’t have come up with it in the first place. Sadly, it’s not evidence that the theory is true.
This is a reason why Flashpoint asks entrepreneurs to do three things that aren’t intuitive or easy.
- Distinguish between having a theory about an aspect of your business and having a hypothesis about it. A theory describes the mechanism of how a process or transaction important to the business works (e.g. I can get people to buy my service by advertising in bus shelters.) A hypothesis is part of a plan you use to test whether the theory is correct.
- Validating your theory involves not only coming up with a good hypothesis, but also setting validation conditions before the test, rather than trying to make sense of what happens afterwards. For example, “I will put ads in three bus shelters. Each one will show a special URL with a counter. If at least ten people per week come to our site through that URL and purchase a product, I’ll judge the theory to be correct.”
- Seek to disconfirm rather than confirm the theory. In my motorcycle theory, “good view,” “quickness,” and “maneuverability” are hypothesized to be evidence that I’d be less likely to get into an accident. But I could confirm that motorcycles met those criteria all day long, and be no closer to actually knowing whether that was true. If instead I’d been (like my mother) equipped with a theory that motorcycles are VERY DANGEROUS because they tend to skid out on wet roads, I could have found plenty of evidence for that too, and been no closer to valid knowledge on the subject.
A much better approach, if my theory is that motorcycles are less accident prone than cars, is to look for evidence that that isn’t true. Evidence that I’m wrong (like,say,data showing more accidents per miles driven) forces me to ask “is this piece of evidence an anomaly, or should I trust it, reject my theory, and come up with a different one?” If I go in the other direction and look for confirming evidence, than whenever I have it, I have to ask “is this piece of evidence enough to confirm my theory, or do I need to keep testing and find more?” Generally, coming up with anomalies will poke holes in your theory and lead you to abandon it, a lot faster than coming up with confirmations will prove your theory true. In practice, entrepreneurs rarely even stop to think whether the confirming evidence is enough to rely on – they believe their theory to begin with, and even minimal evidence is plenty.
The ubiquitous lean entrepreneurship catchphrase, “get out of the building and test” is a great start. But following that advice doesn’t make your business valid, any more than getting accepted into medical school makes you a doctor.
A very perceptive, connected, media savvy, friend just wrote some nonsense to me about the terrible events in Boston over the last few days. He wrote “ who would have thought that the blow-back from Chechnya would come to America in the form of an attack on the Boston Marathon?”
Chechnya? I’d be extremely surprised. There’s a well studied
cognitive illusion called WYSIWTI (what you see is what there is)
that’s been playing throughout this sad event. When the bombs went
off, I heard media speculation that it was tax day, so it must be a
right wing anti tax nut, and that the bomb was placed on the ground
where it would cause maximum damage to legs (and at the Boston
Marathon) , so it must have been a disgruntled runner, and that it was
probable Somalia’s Al Shabab, trying to blow up Kenyan runners to maximum
The way the illusion works is this: we’re able, maybe evolved, to
form conclusions from whatever evidence is available, and we’re good
at it. We have much poorer capacity to assess the value of the
evidence we’re using and adjust how certain we are about the
conclusion. So for example, if I see someone make a dumb move on the
road, I’ll decide he’s a bad driver. An analyst at an insurance company can take
that person’s whole driving record, instrument the car, and analyze
the data about how fast he goes, how hard he hits the breaks, crunch all the numbers, look at comparables, and
decide he’s a bad driver too.
When you test my certainty about my conclusion, and the analyst’s
certainty about her conclusion, you’ll find very little difference.
As I post, we’re a moment later in the story. Two brothers of Chechen origin are alledged to have committed the crimes. One is dead, and the other now on the run. But we’re in the next phase of making the same mistakes.
The information about about Dzhokar Tsarnaev is very poor and scattered. His Facebook page mentions Islam, his family is from a war-torn part of the former Soviet Union, his uncle was quoted as saying he’s a “loser.” These are scraps of information, and anything we might conclude from them is barely more likely to be true than something we just make up out of whole cloth.
But that’s not how our minds work. The moment we form a conclusion, even on very shaky grounds, it becomes a firmly held belief.
WYSIWTI explains why it’s so painful for entrepreneurs to do the hundreds of interviews that are necessary for finding a valid business model. They’ve come to the idea that customers have problem X, and they can solve it with product Y, usually based on tiny scraps of evidence. But having reached the conclusion, the shaky ground faded from view, and asking more questions of more people began to seem like a waste of time.
Here’s a comment to an entrepreneur. He was getting signups but not good retention, and was worried that the process of testing new features to fix the problem would take forever. Thought we’d share it:
Well, retention is the right question – a lot of startups appear to go wrong by focusing on acquisition instead of retention. That doesn’t seem to work because you can only get retention through a fit between the customer’s improvement goal and your product, while you can get acquisition a number of other ways, such as a write-up in Techcrunch, that give you misleading data about what’s working.
I’ve avoided reading anything by Robert Caro for years now. He’s the guy who wrote a Pulitzer Prize -winning book about Robert Moses (published in 1974) and since then, has been writing a multi-volume biography of Lyndon Johnson. Thirty years writing about Johnson? I assumed Caro was just crazy and obsessive. History will probably thank him for his painstaking research and everything, but who really wants to read painstaking detail about Johnson’s 1948 senatorial campaign?
But the latest volume was excerpted in the New Yorker last year, and my wife and I have a habit of bringing stacks of old New Yorkers on vacation to read and get rid of them, so I finally read some Caro. It was a transformative experience, a red-pill experience.
The result of Caro’s craziness and obsession is that he figures out the truth and writes it. Reading his history is not only mesmerizing; it casts a harsh light on most other history and biography. After you read Caro, you read something else that’s ostensibly true and realize that it was mostly made up.
Try it yourself: here’s Caro, writing about Johnson in a rented convertible trailing the Kennedy motorcade down Elm Street in Dallas:
“There was a sharp, cracking sound. It “startled” him, Lyndon Johnson later said; it sounded like a “report or explosion.”…Rufus Youngblood, the Secret Service agent in Johnson’s car, didn’t know what it was, but he saw “not normal” movement in the Presidential car ahead – President Kennedy seemed to be tilting toward his left.…Whirling in his seat, Youngblood shouted – in a “voice I had never heard him ever use,” Lady Bird recalled – “Get down! Get down!” and, grabbing Johnson’s right shoulder, yanked him roughly down toward the floor in the center of the car, as he almost leaped over the front seat, and threw his body over the Vice-President, shouting again, “Get down! Get down!” By the time the next two sharp reports had cracked out – it was a matter of only eight seconds, but everyone knew what they were now – Lyndon Johnson was down on the floor of the back seat of the car. The loud, sharp sound, and the hand suddenly grabbing his shoulder and pulling him down: now he was on the floor, his face on the floor, with the weight of a big man lying on top of him, pressing him down – Lyndon Johnson would never forget “his knees in my back and his elbows in my back.”
Reading that, you know in your bones that it’s a true account of what was happening at that moment in that car fifty years ago. Caro has interviewed everyone he could, watched the tapes, reconstructed the events, tested his reconstruction, re-interviewed, and written an account that includes what he knows for a fact and leaves out what he doesn’t know. It’s not omniscient – we don’t know what Ladybird was thinking, or whether Lee Harvey Oswald was considering a shot at LBJ. And there’s a tremendous amount of selectivity (most accounts of those few seconds, of course, focus on Kennedy’s car). So, partial, yes; selective, yes, but still, you read it and know it’s true.
Contrast that with a story that made the front page of the New York Times this week, headlined “Payment for Act of Kindness: 2 Days in Car Trunk at Age 89.” It’s a moving and powerful story. A woman named Margaret E. Smith offered a ride to two teenage girls, who locked her in the trunk. Here are some excerpts:
“Two girls, 15, and 14, appeared at the window, calling her “Miss” and offering to pay for a ride to the other side of town. Her inclination was to say no, but her strong belief in offering kindness to strangers won out.”
“The Buick roared away with its frail owner curled up in the hold’s casketlike darkness. She was tossed about like forgotten luggage with every bump and turn.”
It’s a powerful story. Margaret Smith’s strength and resilience is commendable, even inspirational. But the details ring false. They’re just props. Her strong belief in offering kindness to strangers wining out, the “casket-like darkness,” being tossed about like forgotten luggage. Did the reporter investigate whether there was any evidence about Margarets’ beliefs, or press her on how she really made the decision about letting the girls into her car? Did he crawl into the trunk to see how dark it really was? If you’re in the trunk of a Buick, how recklessly does the driver have to go before you’re really tossed around? Caro would have figured out all those things, and he would write it in a way you could trust.
The world is a lot less coherent than we think. In a coherent world, Youngblood would have leaped to the back seat of the car, not “almost leaped.” Johnson would remember “his knees in my back” but not his elbows, because someone’s knees and elbows in someone else’s back is an awkward, slightly incoherent picture.
The job of a startup is searching for the truth. Like Caro, you don’t need omniscient, unselective truth (which is a good thing, because you’ll never find it). But you do need to leave in what happened, and what people really said, and leave out the props that make sense of it all. When you remember what a customer said or how they acted, in a way that conforms with and supports your theory, then you’ve robbed the interaction of exactly the information you need to understand what they need.
Met two different flavors of entrepreneurs last Tuesday, and wondering whether and how they might correlate with success. The first was a phenomenon; one of those guys who, at least by appearances, “looked at the world sideways and realized it was his oyster.” He took apart vacuum cleaners and televisions when he was eight, learned to code at twelve, started his first company in college and sold it shortly after, currently running two pretty hot startups at once. The prototypical story was when he hacked into some company’s system to make it do what he wanted, got banned for it, but later went into business with the CEO of that company, who appreciate his chutzpah and skills, and besides, had made another fortune by appropriating the hack.
I’m not calling this person Allan Grant, because the details are wrong, but at a guess, details about stories like this are usually/always wrong. The important part isn’t about exploits anyway, but about an outlook. Not-Allan ‘s outlook is that you never know what will work, and can’t go about learning or teaching it scientifically. What you can do is come up with an idea, use it to make something happen (cheap and fast) that gets some result, and then start tinkering (adding and subtracting partners and features and customer types and value propositions.)
If you’re smart and quick in your tinkering, taking lots of steps, and taking the next one based on educated guesses about what happened when you took the last one, then you can do something cool: You can grow appendages and lop off pieces of the idea until it morphs into a real business. This works best under two conditions: First, the idea has to be in the realm of something you know about, ideally a need you have, so that it’s somewhere in the ballpark of being a good idea and also so that when you tinker and get a reaction, you can make sense of what just happened. Second, and for the same reason, it has to be crisp enough so that failures and successes can be traceable back to something.
I met entrepreneur number two driving my Uber back to the airport. His name was Luka, and he was from Lagos, Nigeria. In addition to driving a town car, Luka had started several businesses, all back in Lagos. He had a fish farm going, a cab business, a business exporting fancy cars from the U.S. to Nigeria, a software VAR with several small Nigerian government contracts, and a business he was trying to get going importing containers and equipment for liquefied natural gas. He ran this empire from his phone in the town car and by going back to Nigeria a couple times a year.
Luka didn’t have any domain skills for any of his businesses, except the very important one of understanding how people do things in Lagos. He explained it to me this way: “In Nigeria, you might see a sign on the street that says ‘don’t park at this curb.’ An American might think the meaning of the sign is ‘no one is allowed to park at this curb.’”
Both Luka and not-Allan are running multiple businesses at once, though they both complain about lack of hours in a day. Both are focused on and have a hard time with raising capital – sometimes it falls into place easily and sometimes it’s a bitch. Both are smart. Both are opportunistic. Both have the ability to create reality distortion fields around their opportunities and prospects. There’s probably no difference about scaling prospects either – Luka’s LNG equipment company could conceivably scale as big, or bigger, than not-Allan’s online startups.
The main difference is that not-Allan is doing what we’re pleased to call “innovating,” while Luka is just hustling. So what, really, does that difference amount to?
Where people often go wrong is thinking of innovation as a clever new tech thing that can find itself a market. A better bet is to start with the customers. But beginning there starts with a puzzle: You can look at your customers two ways: On one hand, they’re alive and doing whatever they’re doing, so the world and the world’s economy is making available to them everything they need to live and do. On this hand, it’s a closed system, and you can only create a business by stealing a piece of your customer’s attention and purse from some other provider. On the other hand, in the foreseeable future, all your potential customers will die, so the world doesn’t provide the thing they want most of all. And working down from there, everyone is in endless pain and there are endless gaps to be filled by endless new products and services. In this scenario, innovation doesn’t need to squeeze into a closed system, it fills in empty holes.
But whether you think of innovation as displacement or as filling empty holes, it always means new approaches to solving problems or addressing improvement goals. The problems and goals themselves – hunger, status, sex, health, shelter, etcetera, never change much. But the value of different approaches to solving those problems or meeting those goals not only change, but are extremely volatile and are hugely sensitive to context. The telegraph machine that so magnificently addressed long-distance communications problems 170 years ago wouldn’t register as a solution to anything today. The smart phone, which wouldn’t have registered as a solution to anything a decade ago, has become the indispensable communications solution for many people today. Big innovations like these move markets, but so do small feature changes. A new generation game console, or this year’s car, can pull a big, valuable market out from under last year’s product.
This volatility in the relative value of solutions means that small product changes are capable of producing big market outcomes. So improvements in service, price, features, etcetera, are at the line of scrimmage for big market players. Lots of R&D and marketing money and talent goes into incremental change both because that’s what big companies know how to do, and because the cost can be justified by the potential ROI.
With the bulk of the money, talent, and resources struggling over small changes at the scrimmage line, real innovation is more like an end run. Startups are little and quick and don’t have the advantage of helmets or an offensive line. Their only chance is to make a move that takes them to where the defense isn’t – an innovation.
In the Silicon Valley, angel-and-VC world of scalable startups, innovation is a must – but not in Nigeria. Even there, Luka would fail in the oil business, or the road building business, and he probably has little chance as a VAR selling software to government offices. Where you have big players and entrenched interests and a lot at stake over small product changes, hustle alone rarely wins. But in liquefied natural gas, which is brand new to Nigeria, or in importing fancy cars, like to emerging cities where no fancy cars have been sold before, he can just hustle and not worry about innovation.
Back in Tech Crunch America, we’ve got a mythology going about startup innovators. We love the end run, the company that can suddenly materialize on an unexpected point in the field of solutions to a problem. But there’s nothing intrinsic that makes that approach smarter or more likely to succeed – it’s a question of context.
Less like this
And more like this:
In retrospect, creating a company looks like instantiating an idea. So it’s easy and seductive to imagine that a “metrics based” approach to startups involves breaking the business down into components and measuring the creation of each component.
Unfortunately, though, you can’t build companies.in retrospect. Instead, you need to identify gaps (needs, holes, problems, etc.) in the existing economic and psychological world, and in the process, discover what your company needs to be to fit into those gaps. This has to happen simultaneously in lots of dimensions: the dimension of what you can provide, the dimension of how customers become aware of you, the dimensions of who you need to work with, what you need to outsource vs doing it yourself, how customers want to relate to you, etcetera. Each dimension impacts the others, so it’s common, for example, that a new thing you learn with respect to a way you have to be with a customer invalidates something you thought you knew about the way you could work with a partner.
As a consequence, metrics that track what you’ve built can lead you into a false sense of security. Because tomorrow, you may discover that that thing you built (a prototype, a relationship with one type of customer, a plugin for one platform) is no longer relevant to the business model that can work as a whole.
The trick is to track velocity of learning.
Nassim Nicholas Taleb is idiosyncratic, smug, and self indulgent; but you can’t deny he comes up with some gems. The Black Swan was one; Antifragile is another.
“Antifragile” is a neologism he intends as the third vertex in a kind of triad: when you expose things (people, systems, companies) to random attacks and shocks, then “fragile” things break down, “robust” things remain intact, and “antifragile” things get stronger. The idea is sort of like “adaptability,” but more positive.
This is a clarifying concept for startups. One way to think is that startups ought to be as robust as possible. The best ones, according to this theory, have strong backing, so they don’t run out of money if there’s a glitch, they have experienced entrepreneurs, who know, at least in general, what they’re in for, and can plan for it. And they have multiple options, so if one market or technology bet doesn’t work out, they can find another.
In contrast, an antifragile startup embraces the iterations and pivots, rejections and delays, personnel melt-downs and product failures, and above all, the lethal indifference to your product, that adds up to life in the startup lane. You embrace all this misery not because you need to be tough and what doesn’t kill you makes you stronger. You embrace it because you’re lost without it – according to Taleb, what doesn’t at least try to kill you makes you more fragile.
So which would you prefer to put your money or your life into? A robust startup or an antifragile one?
From my point of view, that depends on how early it is. A really early startup can’t be robust, so trying to be (or worse, trying to look) robust is a bad path. The things that make a startup seem robust, like super experienced and talented entrepreneurs or great IP, are in fact liabilities when they’re part of a startup without a discovered market. These are the sorts of things that entail commitments. (We’ve got great IP – let’s exploit it in our product; our CEO is great at managing or getting people to make things or selling or raising money – let’s hire people and make and sell stuff and raise money.) In this situation, the company suffers twice – it suffers from the illusion of being robust, when it’s actually just a vortex of liabilities. And worse, it suffers from the illusion that it ought to be robust and resistant to shocks.
That’s a death sentence. Remember that the only thing a startup necessarily does is spend money. A robust startup is one that keeps its form and goals intact, despite the blows from the market. Do that for a while, and the money will be gone. It’s no accident that Steve Blank’s proverbial definition of a startup starts with the words “a temporary organization.”
On the other hand, transforming a startup into a company is the process of building something organizationally robust – something that can consistently do real jobs for customers despite forces that knock it off course. How do you get there without building in robustness? A startup’s job is to drag facts into the building, as the saying goes. But a business model canvas full of validated hypotheses still isn’t a business. There’s no neat endpoint of the startup process, where a company just flips a switch, from fully validated business model to smoothly humming business.
Maybe a baseball metaphor works here. A startup is like a new team showing up at spring training. Its job is to get knocked into shape to play the game, so being antifragile is the key to success. Once the season starts, they don’t stop practicing, and lessons learned in games constantly feed back into practices, but winning requires robustness. How robust is your business? How antifragile?