An “idea maze” is a map of all the key decisions and tradeoffs that startups in a given space need to make:
A good founder is capable of anticipating which turns lead to treasure and which lead to certain death. A bad founder is just running to the entrance of (say) the “movies/music/filesharing/P2P” maze or the “photosharing” maze without any sense for the history of the industry, the players in the maze, the casualties of the past, and the technologies that are likely to move walls and change assumptions.
- Balaji Srinivasan, “Market Research, Wireframing and Design”
I thought it would be interesting to show an example of an idea maze for an area that I’m interested in: AI startups. Here’s a sketch of the maze. I explain each step in detail below.
“MVP with 80–90% accuracy.” The old saying in the machine learning community is that “machine learning is really good at partially solving just about any problem.” For most problems, it’s relatively easy to build a model that is accurate 80–90% of the time. After that, the returns on time, money, brainpower, data etc. rapidly diminish. As a rule of thumb, you’ll spend a few months getting to 80% and something between a few years and eternity getting the last 20%. (Incidentally, this is why when you see partial demos like Watson and self-driving cars, the demo itself doesn’t tell you much — what you need to see is how they handle the 10–20% of “edge cases” — the dog jumping out in front of the car in unusual lighting conditions, etc).
At this point in the maze you have a choice. You can either 1) try to get the accuracy up to near 100%, or 2) build a product that is useful even though it is only partially accurate. You do this by building what I like to call a “fault tolerant UX.”
“Create a fault tolerant UX.” Good examples of fault-tolerant UXs are iOS autocorrect and Google search’s “did you mean X?” feature. You could also argue Google search itself is a fault tolerant UX: showing 10 links instead of going straight to the top result lets the human override the machine when the machine gets the ranking wrong. Building a fault tolerant UX isn’t capitulation, but it does mean a very different set of product requirements. (In particular, latency is very important when you want the human and machine to work together—this generally affects your technical architecture).
Ok so let’s suppose you decide to go for 100% accuracy. How do you get there? You won’t get the 10–20% through algorithms. You’ll only get there with lots more data for training your models. Data is the key to AI because 1) it’s the missing ingredient — we have great algorithms and virtually endless computational resources now, and 2) it’s the proprietary ingredient—algorithms are mostly a shared resource created by the research community. Public data sets, on the other hands, are generally not very good. The good data sets either don’t exist or are privately owned.
“Narrow the domain.” The amount of data you need is relative to the breadth of the problem you are trying to solve. So before you start collecting data you might want to narrow your domain. Instead of trying to build a virtual bot that can do anything (which would basically mean passing the Turing Test—good luck with that), build a bot that can just help someone with scheduling meetings. Instead of building a cloud service that predicts anything, build one that can predict when a transaction is fraudulent. Etc.
“Narrow domain even more.” After you are done narrowing the domain, try narrowing it even more! Even if your goal is to build X, sometimes building an MVP that is part of X is the best way to eventually get to X. My advice would be to keep narrowing your domain until you can’t narrow it anymore without making the product so narrow that no one wants to use it. You can always expand the scope later.
“How do you get the data?” Broadly speaking, there are two ways: build it yourself or crowdsource it. A good analogy here is Google Maps vs Waze. Google employs thousands of people driving around to map out roads, buildings, and traffic. Waze figured out how to get millions of people to do that for them. To do what Google does, you need far more capital (hundreds of millions, if not billions of dollars) than is generally available to pre-launch startups.
Startups are left with two choices to get the data. 1) Try to mine it from publicly available sources. 2) Try to crowdsource it.
The most common example of 1) is crawling the web, or big websites like Wikipedia. You could argue this is what the original Google search did by using links as ranking signals. Many startups have tried mining Wikipedia, an approach that hasn’t led to much success, as far as I know.
The most viable approach for startups is crowdsourcing the data. This boils down to designing a service that provides the right incentives for users to give data back to the system to make it better. Building a crowdsourced product is its own topic (which is why that part of the idea maze points to another, nested idea maze), but I’ll give an example of one approach to doing this, which was tried by company called Wit.ai that we invested in last year. Wit’s idea was to provide a service for developers for doing speech-to-text and natural language processing. The v1.0 system gave the right answer most but not all of the time. But it also provided a dashboard and API where developers could correct errors to improve their results. For developers using the free version of the service, the training they performed would get fed back to make the overall system smarter. Facebook acquired Wit so their future will unfold now as part of a larger company. The approach they took was very clever and could apply to many other AI domains.
This is a rough sketch of how I see the AI startup idea maze. A few caveats: 1) I could very well be mistaken or have overlooked other paths through the maze — idea mazes are meant to aid discussion, not serve as gospel, and 2) As Balaji says, new technological developments can “move walls and change assumptions.” Look out especially for new infrastructure technologies (internet, smartphones, cloud computing, bitcoin, etc) that can unlock new pathways in many different idea mazes, even ones that at first seem unrelated.