Instrumenting the offline world

In the last decade there have been major advances in storing, analyzing, and acting upon extremely large data sets. Data sets that were previously left dormant are now being put to (mostly) constructive use. But the vast majority of information in the world isn’t available for analysis because it isn’t being electronically collected.

This is changing rapidly as new data collection mechanisms are implemented – what engineers refer to as instrumentation. Common examples of instrumentation include thermometers, public safety cameras, and heart rate monitors.

Smart phones are one obvious new source of potential instrumentation. A person’s location, activities, audio and visual environment – and probably many more things that haven’t been thought of yet – can now be monitored. This of course raises privacy issues. Hopefully these privacy issues will be solved by requiring explicit user opt-in. If so, this will require creating incentives for people to do so.

Foursquare instruments location in an opt-in way through the check in. The incentives are social and game-like, but the data produced could be useful for many more “serious” purposes. Fitbit instruments a person’s health-related activity. The immediate incentive is to measure and improve your own health, but the aggregate data could be analyzed by medical researchers to benefit others.

In manufacturing, there has been a lot of interesting innovation around monitoring machinery, for example by using loosely joined, inexpensive mesh networks. In homes, protocols like ZigBee allow devices to communicate which allows, for example, automation of tedious tasks and improved energy efficiency.

In the next decade, there will be a massive amount of innovation and opportunity around the big data stack. Instrumentation will be the foundational layer of that stack.

Pingback: Tweets that mention cdixon.org – chris dixon's blog / Instrumentation -- Topsy.com
http://twitter.com/sungism sungism

Hi, Chris.
I totally agree with what you’re saying. I believe that the most impactful effect of digital innovation will be the collection of information that wasn’t possible before as well as its utilization. And I believe that the pinnacle of this innovation will be figuring out the patterns and rules in the implicit and ‘soft’ information such as one’s tastes, feelings, etc. I have utmost passion for this area and I believe Hunch is a service that’s also targeting this area and therefore I always had big admiration for you and Caterina.
I am a Korean entrepreneur and an entrepreneur who wants to do business in much bigger market esp. the States and I’m currently applying for HBS since I view this as one option of realizing my ‘want’. I hope to meet you in person oneday and I also hope to have good discussion on the web.
Thank you so much for being you and being here.

http://www.cdixon.org chris dixon

Thanks!

http://technbiz.blogspot.com paramendra

We are instrumenting ourselves! I like that image. What if all the photos everyone takes were to end up in some kind of an augmented reality pit for people who might not be traveling but wish they were?
http://technbiz.blogspot.com paramendra

We are instrumenting ourselves! I like that image. What if all the photos everyone takes were to end up in some kind of an augmented reality pit for people who might not be traveling but wish they were?
Pingback: Cleantech Insights » Blog Archive » Cleantech’s Creative, Data Renaissance
http://twitter.com/aortenzi Anthony Ortenzi

It seems like there’s a business opportunity in providing home and enterprise data aggregation services which allow for authorization policies to access that data. Otherwise, each of these instrumentation opportunities will end up as proprietary communications back to “home base” at a service provider in whose interest it is to maintain control of that data, as opposed to providing the most value-addition. Open standards for instrumentation integrated into devices, including encryption, authorization, and the ability to choose service providers or roll-your-own analysis solutions, choosing perhaps to sell the data to the highest bidder.

http://www.cdixon.org chris dixon

Totally agree that data being in silos is a huge issue. It is an issue with
many existing data sources as well. Standards for sharing data in ways that
protect users’ privacy are extremely important and hopefully part of a next
wave of innovation.

http://twitter.com/DreMoran Andres Moran

There’s been a proliferation of sensors and apps that monitor physical activity. We’re beginning to aggregate these services, and encouraging users to opt-in by giving them real-world prizes for doing so. http://www.earndit.com
http://www.twitter.com/famolari famolari

Totally agree. In many ways, this information will become the heartbeat of our physical world. Instrumentation of this sort is a major focus in automotive telematics with the DOT supporting a number trials under the Vehicle Information Integration (VII) program. Also lots of activity in infrastructure health monitoring for bridges, roadways and railways, not to mention health, energy, retail and warehouse applications.

While firms like IA have built themselves to serve the big data opportunity, I think other smart firms will emerge to serve the tremendous long term opportunity in instrumentation, sensors, M2M and the like that will be responsible for generating and propagating so much of this data.
http://www.rowanprice.com rowanprice

Chris, I think you have your finger on something big that’s just below the collective radar, and you’re right that its implications are exciting. One piece of this that particularly excites me — and I’m sure you’re aware of it — is the microprocessor-maker movement, around Arduino in particular, where $40 gets you a durable, simple, computer, loaded with an open source operating system. $3 more gets you an instrument like a temperature sensor that you can affix to it. Add in a few lines of C code and, presto, homemade thermostat that is just an ethernet cable’s distance from the Internet… only a matter of time before Yahoo weather no relies on airports to report your “local” weather?

40 years ago, our geekier dads built radios for fun that weren’t that far behind what you could buy at Sears, but the Digital Era has spawned a massive gap between the enthusiast and the corporation. If instrumentation narrows that gap that’s something we might keep our eyes on.
Anonymous

I too am excited by the proliferation of this new data, although part of me worries it could end up going too far.

As you said, the power of experimentation needed to figure out how to make this data useful is just getting started, and the possibilities are huge. I’d bet a computer who could effectively process crowd-sourced medical data could make better recommendations that any one doctor with only a finite amount of experience.

However, as collection of personal data becomes more ubiquitous, I’m sure there will be bp-level “oil spills” where vast amounts of personal information is leaked. I’m not sure I care about my gender and interests being shared, but when your info is linked back to your name there is certainly potential for damage. We can’t know what will happen at this point, but I could see a need for some kind of big regulation at some point down the road.
Florian Feder

See also the Economist’s special report on smart systems for a more detailed analysis of this topic: http://www.economist.com/node/17388368
Usama

So much information and knowledge is exchange in this century, its amazing.
http://twitter.com/getgingerd gingerd

Good post Chris. We use mobile phone data for social health analytics. Kind of like fitbit, but without the hardware. Learn more at gingerd.com.

I’d be delighted to talk to you / folks at Founder’s Collective about this, since we’re in the middle of closing a seed round. I don’t see a public facing email, but you can reach us at contact [at] gingerd.com. Thanks.

Chris Dixon