Inferring intent on mobile devices

2011-04-24

[Google CEO Eric] Schmidt said that while the Google Instant predictive search technology helps shave an average of 2 seconds off users’ queries, the next step is “autonomous search.” This means Google will conduct searches for users without them having to manually conduct searches. As an example, Schmidt said he could be walking down the streets of San Francisco and receive information about the places around him on his mobile phone without having to click any buttons. “Think of it as a serendipity engine,” Schmidt said. “Think of it as a new way of thinking about traditional text search where you don’t even have to type.”  - eWeek

When users type phrases into Google, they are searching, but also expressing intent. To create the “serendipity engine” that Eric Schmidt envisions would require a system that infers users’ intentions.

Here are some of the input signals a mobile device could use to infer intent.

Context

Location: It is helpful to break location down into layers, from the most concrete to the most abstract:

  1. lat / long – raw GPS coordinates
  2. venue – mapping of lat / long coordinates to a venue.
  3. venue relationship to user – is the user at home, at a friend’s house, at work, in her home city etc.
  4. user movement – locations the user has visited recently.
  5. inferred user activity – if the user is at work during a weekday, she is more likely in the midst of work. If she is walking around a shopping district on a Sunday away from her home city, she is more likely to want to buy something. If she is outside, close to home, and going to multiple locations, she is more likely to be running erands.

Weather: during inclement weather user is less likely to want to move far and more likely to prefer indoor activities.

Time of day & date: around mealtimes the user is more likely to be considering what to eat. On weekends the user is more likely to be doing non-work activities. Outside at night, the user is more likely to be looking for bar/club/movie etc.  Time of days also lets you know what venues are open & closed.

News events near the user: they are at the pro sporting event, an accident happened nearby, etc.

Things around the user: knowing not just venues, but activities (soccer game), inventories (Madden 2011 is in stock at BestBuy across the street), events (concert you might like is nearby), etc.

These are just a few of the contextual signals that could be included as input signals.

Taste

The more you know about users’ tastes, the better you can infer their intent. It is silly to suggest a great Sushi restaurant to someone who dislikes Sushi. At Hunch we model taste with a giant matrix. One axis is every known user (the system is agnostic about which ID system – it could be Facebook, Twitter, a mobile device, etc), the other axis is things, defined very broadly: product, person, place, activity, tag etc.  In the cells of the matrix are either the known or predicted affinity between the person and thing.  (Hunch’s matrix currently has about 500M people, 700M items, and 50B known affinity points).

Past expressed intent

- App actions:  e.g. user just opened Yelp, so is probably looking for a place to go.

- Past search actions: user’s recent (desktop & mobile) web searches could be indications of later intent.

- Past “saved for later” actions:  user explicitly saved something for later e.g. using Foursquare’s “to do” functionality.

Behavior of other people

- Friends:  The fact that a user’s friends are all gathered nearby might make her want to join them.

- Tastemates: That someone with similar tastes just performed some actions suggests the user is more likely to want to perform the same actions.

- Crowds: The user might prefer to go toward or avoid crowds, depending on mood and taste.

How should an algorithm weight all these signals? It is difficult to imagine this being done effectively anyway except empirically through a feedback loop. So the system suggests some intent, the user gives feedback, and then the system learns by adjusting signal weightings and gets smarter.  With a machine learning system like this it is usually impossible to get to 100% accuracy, so the system would need a “fault tolerant” UI.  For example, pushing suggestions through modal dialogs could get very annoying without 100% accuracy, whereas making suggestions when the user opens an application or through subtle push alerts could be non-annoying and useful.

Next post: There are two kinds of people in the world
Previous post: Financing risk

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at https://a16z.com/investments/. Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see https://a16z.com/disclosures for additional important information.