The next step in this evolution is to create web services that are both federated (APIs) and extensible (Apps).

In my ideal world, the social graph would not be controlled by a private company. That said, Facebook, to its credit, has aggressively promoted a fairly open API through Facebook Connect. Facebook has also been a leader in promoting Apps. For Facebook, creating extensible, federated services would mean providing a framework for Facebook Connect Apps – apps that extend Facebook functionality but reside on non-Facebook.com websites.

Consider the following scenario.  Imagine that in the future a geolocation data/algorithm provider like SimpleGeo takes Facebook Places check-in data and, using algorithms and non-Facebook data, produces new data sets, for example: map directions, venue recommendations, and location-based coupons. The combination of Facebook’s data (social graph and check-ins) and SimpleGeo data/algorithms would create much more advanced feature possibilities than either service acting alone.

With today’s APIs, if, say, Gowalla wanted to integrate Facebook plus SimpleGeo into their app*, they would basically have 3 choices:

1) Embed Facebook widgets in Gowalla.  These are simple iframes (effectively separate little websites) that don’t interact with SimpleGeo.  Gowalla would just have to sit and wait and hope that Facebook decided to bake in SimpleGeo-like functionality.

2) Pre-import SimpleGeo data. This significantly limits the size and dynamism of the SimpleGeo data sets and doesn’t incorporate SimpleGeo algorithms, thus severely limiting functionality.

3) Host an instance of SimpleGeo’s servers internally.  This requires heavy technical integration, undermining the main benefit of APIs.

In a world of extensible APIs (or “API Apps”), Gowalla could instead send Facebook data back to SimpleGeo.  The data flow would look something like this:

(Note how there are three parties involved – @peretti calls this a “data threesome”). This configuration is much simpler to integrate – and potentially much more powerful and dynamic – than the other configurations listed above.  You could implement this today, but it would create user experience challenges.  For example, Gowalla would be sending Facebook data to a 3rd party (step 3), which might (depending on the data sent) require explicit user opt-in. Things become more onerous if SimpleGeo wanted to share its own user data with Gowalla. That would require an additional oAuth to SimpleGeo (authorizing step 4).

Allowing websites to be federated and extensible will open up a whole new wave of innovation.  Ideally some spec like oAuth could include the multiple authorizations in a single authorization screen.  Facebook could also do this by allowing 3rd parties to be part of the Facebook Connect authorization process.  Inasmuch as Facebook’s seems to be trying to embed their social graph as deeply as possible into the core experiences of other websites, allowing extensible APIs would seem to be a smart move.

* I have no connection to any of these companies (Facebook, Gowalla, SimpleGeo) and have no knowledge of their product plans beyond their public websites.  I am imagining functionality that Gowalla and SimpleGeo might include someday but for all I know they have no interest in these features – I just picked them somewhat arbitrarily as examples.

A graph consists of a set of nodes connected by edges. The original internet graph is the web itself, where webpages are nodes and links are edges. In social graphs, the nodes are people and the edges friendship. Edges are what mathematicians call relations. Two important properties that relations can either have or not have are symmetry (if A ~ B then B ~ A) and transitivity (if A ~ B and B ~ C then A ~ C).

Facebook’s social graph is symmetric (if I am friends with you then you are friends with me) but not transitive (I can be friends with you without being friends with your friend).  You could say friendship is probabilistically transitive in the sense that I am more likely to like someone who is a friend’s friend then I am a user chosen at random. This is basis of Facebook’s friend recommendations.

Twitter’s graph is probably best thought of as an interest graph. One of Twitter’s central innovations was to discard symmetry: you can follow someone without them following you. This allowed Twitter to evolve into an extremely useful publishing platform, replacing RSS for many people. The Twitter graph isn’t transitive but one of its most powerful uses is retweeting, which gives the Twitter graph what might be called curated transitivity.

Graphs can be implicitly or explicitly created by users. Facebook and Twitter’s graphs were explicitly created by users (although Twitter’s Suggested User List made much of the graph de facto implicit). Google Buzz attempted to create a social graph implicitly from users’ emailing patterns, which didn’t seem to work very well.

Over the next few years we’ll see the rising importance of other types of graphs. Some examples:

Taste: At Hunch we’ve created what we call the taste graph. We created this implicitly from questions answered by users and other data sources. Our thesis is that for many activities – for example deciding what movie to see or blouse to buy – it’s more useful to have the neighbors on your graph be people with similar tastes versus people who are your friends.

Financial Trust: Social payment startups like Square and Venmo are creating financial graphs – the nodes are people and institutions and the relations are financial trust. These graphs are useful for preventing fraud, streamlining transactions, and lowering the barrier to accepting non-cash payments.

Endorsement: An endorsement graph is one in which people endorse institutions, products, services or other people for a particular skill or activity. LinkedIn created a successful professional graph and a less successful endorsement graph. Facebook seems to be trying to layer an endorsement graph on its social graph with its Like feature. A general endorsement graph could be useful for purchasing decisions and hence highly monetizable.

Local: Location-based startups like Foursquare let users create social graphs (which might evolve into better social graphs than what Facebook has since users seem to be more selective friending people in local apps). But probably more interesting are the people and venue graphs created by the check-in patterns. These local graphs could be useful for, among other things, recommendations, coupons, and advertising.

Besides creating graphs, Facebook and Twitter (via Facebook Connect and OAuth) created identity systems that are extremely useful for the creation of 3rd party graphs. I expect we’ll look back on the next few years as the golden age of graph innovation.

Ask anyone who ran or invested in a mobile app startup pre-iPhone (I invested in one myself). Since the carriers had all the power, getting any distribution (which usually meant getting on the handset “deck”) meant doing a business development deal with the carriers. Business development in this case meant finding the right people at those companies, sending them iPods, taking them to baseball games, and basically figuring out ways to convince them to work with you instead of the 5,000 other people sending them iPods and baseball tickets.  The basis of competition was salesmanship and capital, not innovation or quality.

The carriers had so much power because consumers made their purchasing decisions by choosing a carrier first and a handset second. Post-iPhone, tens of millions of people started choosing handsets over carriers. People like me suffer through AT&T’s poor service and aggressive pricing because I love the iPhone so much.

I’ve talked to a number of mobile app startups lately who say their former contacts at the carriers are shell shocked: no one is knocking on their doors anymore. I guess they have to buy their own iPods and baseball tickets now.

Yes, Apple has rejected some apps for seemingly arbtrary or selfish reasons and imposed aggressive controls on developers. But the iPhone also paved the way for Android and a new wave of handset development. The people griping about Apple’s “closed system” are generally people who are new to the industry and didn’t realize how bad it was before.

When legendary VC John Doerr quit Amazon’s board a few months ago, savvy industry observers like TechCrunch speculated that Google might begin directly competing with Amazon:

[Google] competes with Amazon in a number of areas, particularly web services and big data. And down the road, Google may compete directly in other ways as well. Froogle was a flop, but don’t think Google doesn’t want a bigger chunk of ecommerce revenue from people who begin their product searches on their search engine.*

In fact, Google and Amazon’s are already direct competitors in their core businesses. Like Amazon, Google makes the vast majority of its revenue from users who are looking to make an online purchase. Other query types – searches related to news, blog posts, funny videos, etc. – are mostly a loss leaders for Google.

The key risk for Google is that they are heavily dependent on online purchasing being a two-stage process:  the user does a search on Google, and then clicks on an ad to buy something on another site. As long as the e-commerce world is sufficiently fragmented, users will prefer an intermediary like Google to help them find the right product or merchant. But as Amazon increasingly dominates the e-commerce market, this fragmentation could go away along with users’ need for an intermediary.**

Moreover, Google’s algorithmic results for product searches are generally poor. (Try using Google to decide what dishwasher to buy). These poor results might actually lead to short term revenue increases since the sponsored links are superior to the unsponsored ones.  But long term if Google continues producing poor product search results and Amazon continues consolidating the e-commerce market, Google’s core business is at serious risk.

—

* Froogle (and Google Products) have been unsuccessful most likely because Google has had no incentive to make them better: they make plenty of money on these queries already on a CPC basis, and would likely make less if they moved to a CPA model.

** Most Amazon Prime customers probably already do skip Google and go directly to Amazon.  I know I do.

When users are actively looking to purchase something, they typically go to search engines or e-commerce sites. Through advertising or direct sales, these sites harvest intent. Google and Amazon are the biggest financial beneficiaries of intent harvesting.

When the user is not actively looking to buy something, the goal of an online ad is to generate intent. The intent generation market is still fairly fragmented and will grow rapidly over the next few years as brand advertising increasingly moves online. P&G – which alone spends almost $4B/year on brand advertising – needs to convince the next generation of consumers that Crest is better than Colgate. This is why Google paid such a premium for Doubleclick, Yahoo for Right Media, and Microsoft for aQuantive (MS’s biggest acquisition ever).

In 2003, Google introduced AdSense, a program to syndicate their intent harvesting text ads beyond Google’s main property Google.com.  The playbook they followed was: use their popular website to build a critical mass of advertisers; then use that critical mass to run an off-property network that offers the highest payouts to publishers. AdSense became so dominant that competitors like Yahoo quit the syndicated ad business altogether. Today, Google has such a powerful position that they don’t disclose percentage revenue splits to publishers and extract the vast majority of the profits.

It is widely believed that Facebook will soon follow the AdSense playbook by introducing an off-property ad network. They’ll try to use their strong base of advertisers to dominate intent generating ads the way AdSense dominated intent harvesting ads.

But to win the intent generation ad battle, data is as important as a critical mass of advertisers. For intent harvesting, users simply type what they are looking for into a search box. For intent generating ads, you need to use data to make inferences about what might influence the user.

This is what the introduction of the Facebook Like button is all about.  Intent generating ads – which mostly means displays ads – have notoriously low click through rates (well below 1%). Attempts to improve these numbers through demographics have basically failed. Many startups are having success using social data to target ads today. But the holy grail for targeting intent generating ads is taste data – which basically means what the user likes. Knowing, for example, that a user liked Avatar is an incredibly useful datapoint for targeting an Avatar 2 ad.

Publishers who adopt Facebook’s Like feature may get more traffic and perhaps a better user experience as a result.  But they should hope the intent generation ad market doesn’t end up like the intent harvesting ad market – with one dominant player commanding the lion’s share of the profits.

* Most text ads are about intent harvesting and most display ads are about intent generation, but they are not coreferential distinctions. For example, with techniques like “search retargeting” (you do a Google search for washing machines and the later on another site see a display ad for washing machines), sometimes intent harvesting is delivered through display ads.

It is in Facebook’s economic interest to extract most of Zynga’s profits, leaving them just enough to keep investing in games and advertising. Last year’s reduced notification change seemed like one move in this direction as it forced game makers to buy more ads instead of getting traffic organically. This probably hurt Zynga’s profitability but also helped them fend off less well-capitalized rivals. Facebook could also hold up Zynga by entering the games business itself, but this seemed unlikely since thus far Facebook has kept its features limited to things that are “utility like.”

The way Facebook now seems to be holding up Zynga – requiring Zynga to use their payments system –  is particularly clever.  First, payments are still very much a “utility like” feature, and arguably one that benefits the platform, so it doesn’t come across as flagrant hold up. It is also clever because – assuming Facebook has insight into Zynga’s profitability – Facebook can charge whatever percentage gets them an optimal share of Zynga’s profits.

The risk for Zynga is obvious — if they don’t diversify their traffic sources very soon, they are left with a choice between losing profits and losing their entire business.  But there is a risk for Facebook as well. If buyers of traffic (e.g. app makers) fear future hold up, they are less likely to make investments in the platform. The biggest mistake platforms make isn’t charging fees (Facebook) or competing with complements (Twitter), it’s being inconsistent.  Apple also charges 30% fees but they’ve been mostly consistent about it. App makers feel comfortable investing in the Apple platform and even having most of their business depend on them in a way they don’t on Facebook or Twitter.

(Eisenmann acknlowledges the iPhone isn’t fully open to the end user – in the US you need to use AT&T, etc.  I would argue the iPhone is semi-open to the app developer and mobile app development was effectively closed prior to the iPhone. But the main point here is that platforms can be open & closed in many different ways, at different levels, etc.)

The next important distinction is whose interest you are considering when asking what and when to open or close things.  I think there are at least 3 interesting perspectives:

The company: Lots of people have written about this topic (Clay Christensen, Joel Spolsky, more Eisenmann here).   In a nutshell, there are times when a company, acting solely in its self-interest, should close things and other times they should open things.  As a rule of thumb, a company should close their core assets and open/commoditize complementary assets. Google’s search engine is their core asset and therefore Google should want to keep it closed, whereas the operating system is a complement that they should commoditize (my full analysis of what Google should want to own vs commoditize is here). Facebook’s social graph is their core asset so it’s optimal to close it and not interoperate with other graphs, whereas marking up web pages to be more social-network friendly (open graph protocol) is complementary hence optimal for FB to open.  (With respect to social graphs interoperating (e.g. Open Social), it’s generally in the interest of smaller graphs to interoperate and larger ones not to – the same is true of IM networks).  Note that I think there is absolutely nothing wrong with Google and Facebook or any other company keeping closed or trying to open things according to their own best interests.

The industry: When I say “what is good for the industry” I mean what ultimately creates the most aggregate industry-wide shareholder value.  I assume (hope?) this also yields the maximum innovation.  As an active tech entrepreneur and investor I think my personal interests and the tech industry’s interests are mostly aligned (hence you could argue I’m talking my book).  Unfortunately it’s much easier to study open vs. closed strategies at the level of the firm than at the level of an industry, because there are far more “split test” cases to study.  What would the world be like if email (SMTP) were controlled by a single company?  I would tend to think a far less innovative and wealthy one. There are a number of multibillion dollar industries built on email: email clients, webmail systems, email marketing, anti-spam, etc.  The downside of openness is that it’s very hard to upgrade SMTP since you need to get so many parties to agree and coordinate.  So, for example, it has taken forever to add basic anti-spam authentication features to SMTP.  Twitter on the other hand can unilaterally add useful new things like their recent annotations feature.

Here’s what Professor Eisenmann said when I asked him to summarize the state of economic thinking on the topic:

With respect to your question about the impact of open vs closed on the economy, the hard-core economists cited in my book chapter have a lot to say, but it all boils down to “it depends.” Closed platform provides more incentive for innovation because platform owner can collect and redistribute more rent and can ensure that there’s a manageable level of competition in any given application category. Open platform harnesses strong network effects, attracting more application developers, and  thus stimulates lots of competition. There’s some interesting recent work that suggests that markets may evolve in directions that favor the presence of one strong closed player plus one strong open player (consider: Windows + Linux; iPhone + Android). In this scenario, society/economy gets best of both approaches.

Society:  I tend to think what is good for the tech industry is generally good for society.  But others certainly have different views.  Advocates of openness are often accused of being socialist hippies.  Maybe some are.  I am not.  I care about the tech industry.  I think it’s reasonable to question whether moves by large industry players are good or bad for the industry.  Unfortunately most of the debate I’ve seen so far seems driven by ideology and name calling.

Twitter’s moves this week were particular interesting.  A lot of third-party developers were unhappy. I think this is mainly a result of Twitter having sent mixed signals over the past few years. Twitter’s move into complementary areas was entirely predictable – it happens with every platform provider. The real problem is that somehow Twitter had convinced the world they were going to “let a thousand flowers bloom” – as if they were a non-profit out to save the world, or that they would invent some fantastic new business model that didn’t encroach on third-party developers. This week Twitter finally started acting like what it is: a well-financed company run by smart capitalists.

This mixed signaling has been exacerbated by the fact that Twitter has yet to figure out a business model (they sold data to Microsoft & Google but this is likely just one-time R&D purchases). Maybe Twitter thinks they know what their business model is and maybe they’ll even announce it soon. But whatever they think or announce will only truly be their business model when and if it delivers on their multi-billion dollar aspirations. It will likely be at least a year or two before that happens.

Normally, when third parties try to predict whether their products will be subsumed by a platform, the question boils down to whether their products will be strategic to the platform. When the platform has an established business model, this analysis is fairly straightforward (for example, here is my strategic analysis of Google’s platform).  If you make games for the iPhone, you are pretty certain Apple will take their 30% cut and leave you alone. Similarly, if you are a content website relying on SEO and Google Adsense you can be pretty confident Google will leave you alone. Until Twitter has a successful business model, they can’t have a consistent strategy and third parties should expect erratic behavior and even complete and sudden shifts in strategy.

So what might Twitter’s business model eventually be?  I expect that Twitter search will monetize poorly because most searches on Twitter don’t have purchasing intent.  Twitter’s move into mobile clients and hints about a more engaging website suggest they may be trying to mimic Facebook’s display ad model. (Facebook’s ad growth is being driven largely by companies like Zynga who are in turn monetizing users with social games and virtual goods.  Hence it’s no surprise that a Twitter investor is suggesting that developers create social games instead of “filling holes” with URL shorteners etc.) Facebook’s model depends on owning “eyeballs,” which is entirely contradictory to the pure API model Twitter has promoted thus far.  So if Twitter continues in this direction expect a lot of angst among third-party developers.

Hopefully Twitter “fills holes” through acquisitions instead of internal development. Twitter was a hugely clever invention and has grown its user base at a staggering rate, but on the product development front has been underwhelming.  Buying Tweetie seemed to be a tacit acknowledgement of this weakness and an attempt to rectify it. Acquisitions also have the benefit of sending a positive signal to developers since least some of them are embraced and not just replaced.

What’s Facebook doing during all of this?  Last year, Facebook seemed to be frantically copying Twitter – defaulting a lot of information to public, creating a canonical namespace, etc. Now that Twitter seems to be mimicing Facebook, Facebook’s best move is probably just to sit back and watch the Twitter ecosystem fight amongst itself.  As Facebooker Ivan Kirigin tweeted yesterday: “I suppose when your competition is making huge mistakes, you should just stfu.”

Disclosure: As with everything I write, I have a ton of conflicts of interest, some of which are listed here.

The paradox, it seems is this: in a pay-per-click driven world, site visitors who want to stay on your site — due to it having the once-much-lauded quality of “stickiness” — are worth much less than those who want to flee your site because it’s clearly not valuable, and hence will click through to somewhere else.

Facebook recently became the most visited site on the web. Yet their revenues are rumored to around $1B – about 1/30 of what Google’s revenues will be this year. Google has the perfect revenue-generating combination:  people come to the site often, leave quickly, and often have purchasing intent. Facebook has tons of visitors but they generally come to socialize, not to buy things, and they rarely click on ads that take them to other sites. Facebook is like a Starbucks where everyone hangs out for hours but almost never buys anything.

The revenue gap between sites like Facebook and Google should narrow over time.  Cost-per-click search ads are extremely good at harvesting intent, but bad at generating intent.  The vast majority of money spent on intent-generating advertising — brand advertising — still happens offline. Eventually this money will have to go where people spend time, which is increasingly online, at sites like Facebook. Somehow Coke, Tide, Nike, Budweiser etc. will have to convince the next generation to buy their mostly commodity products. Expect the online Starbucks of the future to have a lot more – and more effective – ads.

The reality is that news is a lousy business. Period. Even Google doesn’t make money on it. For example, here are Google’s search results for the phrase “afghanistan war”:

Notice there aren’t any ads on the page. This is because ads for “afghanistan war” generate such low revenues per query that Google doesn’t think it’s worth hurting the user experience with a cluttered page. Google can afford to do this on news queries (along with many other categories of queries) because their real business is selling ads on queries where the user likely has purchasing intent. Big money-making categories include travel, consumer electronics and malpractice lawyers. News queries are loss leaders.

It’s an historical accident that hard news categories like international and investigative reporting were part of profitable businesses. The internet upended this model by 1) providing a new delivery method for classified ads (mainly Craigslist), 2) increasing the supply of newspapers from 1-2 per location to thousands per location, thereby driving the willingness-to-pay for news dramatically down, and 3) unbundling news categories, making cross subsidization increasingly hard.

The internet exposed hard news for what it is: a lousy standalone business. Google arguably contributed to this in many indirect ways, including by helping users find substitute news sources. But the idea that Google takes profits directly from newspapers is simply misinformed.