tech

Software eats software development

Software is eating the world, and doing so using smaller and smaller teams. WhatsApp was able to disrupt the global SMS industry with only a few dozen engineers. Small teams can have a big impact because software development (and deployment) has improved dramatically over the past decade. Some improvements include:

  • Infrastructure. Deploying a commercial website ten years ago required significant upfront capital. Now you can spin up virtual servers in minutes. Upfront costs are close to zero and ongoing costs are orders of magnitude lower than before.
  • Services. Startups created simple APIs that abstract away complex back ends. Examples: Stripe (payments), Twilio (communications), Firebase (databases), Sift Science (fraud).
  • Open Source. Open source dominates every level of the software stack, including operating systems (Linux), databases (MySql), web servers (Apache), and programming languages (Python, Ruby). These are not only free but generally also far higher quality than their commercial counterparts.
  • Programming languages. Developers have steadily marched upwards from Assembly to C to Java to, today, scripting languages like Ruby and Python. Moore’s Law gave us excess computing resources. We spent it making developers more effective.
  • Special-purpose tools for non-programmers. These tools let non-programmers create software in certain pre-defined categories, thereby lowering costs and reducing the demand for developers. Examples: Shopify (e-commerce), WordPress (blogging), and Weebly (small business websites).
  • General-purpose tools for non-programmers. In the pre-Internet era, tools like Hypercard and Visual Basic allowed hundreds of millions of semi-technical people to become software developers. Since then, there hasn’t been much work in these areas, but from what I’ve seen that might change soon. By allowing more people to program, these tools act as a force multiplier for the software industry.

In all likelihood, the demand for software development will continue to dramatically outpace the supply. If so, “software eats software development” will be an exciting area going forward, with lots of valuable startups created in the process.

Categories: tech

31 replies »

  1. Another interesting trend is the rise of computer science education. While early. Coursera, Khan Academy, Codecademy, and others will give birth to more engineers, which in turn further spins the flywheel.

  2. I couldn’t agree more. I think the Do-It-Yourself solutions available to businesses will lead the charge. No marketing or sales executive wants to spend time interfacing with a developer to get what they need. And we will see a lot of this on the enterprise mobile enablement front.

  3. How about smartphone marketplaces like Google Play and Apple’s App Store … in my view that’s the biggest reason a company like WhatsApp was able to grow as exponentially fast as they did.

  4. HyperCard remains, for me, such a fascinating technological oxbow lake. Like: they captured something special with that software, but then… It just got marginalized. I’d love to see some of those ideas return.

  5. > Programming languages. Developers have steadily marched upwards from Assembly to C

    There was a lot between assembly and C, enough so that C was a big step backwards in the theme of this thread!

    > If so, “software eats software development” will be an exciting area going forward, with lots of valuable startups created in the process.

    Don’t bet on that. (1) Just as a start, very likely you’d need yet another programming language, and really much better ideas for those have long been pursued, sometimes quite seriously with big bucks by some of the best people, but so far never seen. Actually Knuth’s tools for ‘literate programming’ as illustrated in the published documentation of the source code for his TeX is a nice step forward but never caught on. I’d want a language with sufficiently restricted syntax and semantics that some good ‘static analysis’ could give some really good information and, also, admit some useful automatic transformations. Too much to hope for. (2) A big deal for a language is not so much the language itself but the libraries. E.g., there is Microsoft’s .NET Framework; so far I’ve worked through 5000+ Web pages of documentation of the .NET Framework and suspect that the total is 20,000+ pages. Could soak up a big chunk of startup equity funding just writing the Web pages, f’getting about the code. (3) Microsoft’s Visual Studio is such a pain I won’t use it for free. (4) Tools for programmers are not nearly a new field; it will be tough to do better work than the people who did Lisp (before 1960 but still used by HN?) and many more to Ada (a huge effort), Smalltalk, C++ (a train wreck for serious work), Java, C#, etc.

    Easy to do something cosmetically better, but really better, enough really to improve programming and build a valuable company? Not so easy. Maybe GitHub is an example of a recent, useful software development tool. But, more generally, to heck with the promise of the ‘sector’ and, instead, just look for and evaluate individual projects that are promising. If someone has such a project, then fine, but I wouldn’t hold my breath waiting for one.

    Since you did Hunch, you might want to check the A16Z e-mail

    Date: Fri, 07 Mar 2014 20:52:08 -0500

  6. Hypercard was among the inspirations for Silk.co (where I work). The whole idea of melding something like Weebly and WP with the power of a DIY database is, I think, where we’ll see a lot of innovation going forward. First gen for non-coders was really about publishing. Next gen is about manipulating / controlling data and building things that look more like true applications. I’d add Kimono Labs with their visual APIs and the ability to create relatively simple but highly dynamic apps on phones with no tech skills at all to the mix.

  7. The internet just keeps on booming! Anyone who is willing to work their butt off can carve out a living (and a good one even) these days … they just have to direct themselves to use the almost free and free resources listed in this post!

  8. No, we spend excess computer power on making slower and slower software that runs no faster now than it did 15 years ago.

  9. …it’d almost make you wonder, if we can do so much with so little, why do we suddenly need enormous companies running huge amounts of infrastructure and where have we seen that particular pattern before?

  10. This is a fascinating area and one in which I have had more than a passing interest for several years.

    I’m currently working at a startup where the aim is to replace the whole process of building business applications, allowing business users themselves to build what they need when they need it, without teams of developers, etc.

    It sounds “too good to be true” to many, especially developers, but there are test sites running an early alpha version getting great value from it. One site even built an insurance underwriting and claims management system from scratch (hundreds of screens), with no developers, just the underwriters and their staff creating what they needed. This system is processing hundreds of millions of dollars of business every year and was designed, built and supported by the business users, with no involvement from IT.

    Most encouraging of all, real-world experience demonstrates over and over again that, with the right tools, anyone can build business applications and they can do so quicker than they could document their requirements in the traditional way.

    So, “Software eats software development” as we move the process of creating complex business applications into the 21st Century.

    We live in interesting times.

  11. Analysis and design are still the most complicated parts of software development. These tools are really targeted toward simple implementations.

  12. You’re connecting some interesting dots in a strange way.

    How much does open source dominate because of its quality, versus simply being free, combined with the programmer’s natural aversion to paying money for tools? Programmers (including me) flocked to Linux even when it was terrible. It’s still significantly worse than the commercial alternatives in many areas, like power management and usability. I love it dearly but I don’t pretend that’s because of its “quality”.

    I definitely wouldn’t hold up MySql and Apache as examples of software “far higher quality than their commercial counterparts”. I’m not sure what the commercial counterparts of Python and Ruby would even be — maybe commercial Smalltalk and Lisp compilers, which are still ahead of Python and Ruby in many areas. How many people using Ruby today use it instead of Smalltalk because they evaluated the syntax, semantics, features, and compilers for both languages, versus simply observing that one was free and the other (at least with any of the really good compilers) was not?

    When it comes to C compilers, Clang is making great improvements these days, but prior to that, Intel’s and Microsoft’s proprietary C compilers beat the pants off GCC. (GCC is free but the source code is so convoluted it’s actually easier to debug a proprietary compiler.) We’re lucky Apple was nice enough to open-source their LLVM work, or we’d be living in a world where the 3 best C compilers were all proprietary.

    We’re not getting to smaller teams because of higher level languages. We’re getting *back* to smaller teams because companies are finally discovering that Fred Brooks was right back in the 1960’s, and software teams don’t scale like factory labor. (How many people wrote Unix? C? Mac 1.0? Netscape? Not many! And it’s not because they used higher level languages than Ruby and Python.)

    In fact, most of these improvements you’re seeing are not new advances, but re-discovery of principles from the 1950’s, 60’s, and 70’s. That’s not “software eating software development”, but more like progressing out of 1990’s/2000’s “pop culture” (to use Alan Kay’s term) and into the first steps of making a real discipline out of computer science, by learning from the past. At least, I sincerely hope that’s what it is. In the unfortunate case where people are rediscovering old principles by chance, we’re in for a few more decades of stagnation.

  13. I think there will be many cases where a problem has been solved over an over again. In this situation, a widget will likely be available that you can drag an drop into a platform like WordPress.

    In other cases, a problem can be solved with a very simple data model that can be represented by one of these drag and drop (non-tech friendly) databases.

    However, there will still be many problems that require more advanced design skills (some things aren’t obvious) and implementation tools that use some flavor of coding. For this type of problem, we’re just spinning our own wheels thinking that we’ll be able to dumb it down enough to have simple (non-tech friendly) drag and drop tools to solve the problem.

    Keep in mind what’s happening to many of the bitcoin cheerleaders: they’re just staring to rediscover through first principles why financial regulation is necessary.

  14. Have you see Node-Red ? Not sure it is fully in the non-programmer camp, but I guess it tries to be.

  15. @Bubba Punk what they are trying to do is re-invent the wheel, but not based on trust, but based on math. They are definitely not done building. I’m neutral, I applied them for trying. I don’t know if they’ll be able to do it.

  16. Tools for non-programmers seems like a huge opportunity going forward.

    For example, front end for beautiful websites are a click and $20/month away, yet a great many people understand spreadsheets and what they’d like the back end of a site to do (“read from this cell, then write to this cell over here…”), and how they’d like the front end to respond, but have no idea how to translate that into code.

    Perhaps allowing a user to upload or build a spreadsheet in a familiar way, then helping connect that with their front end in a simple way (ITTT.com’s user-friendly approach is terrific, IMHO) is a place to start.

    Anything that started really gathering steam would likely need true professionals to rebuild everything from scratch, but that’s a great problem to have (and one that will easily attract funding).

  17. > Since then, there hasn’t been much work in these areas, but from what I’ve seen that might change soon
    What have you seen?

  18. hello sir, i want to learn software designing.. so please tell me how could i start software designing.
    . please can you help?

  19. Wonderful beat ! I wish to apprentice while you amend
    your site, how could i subscribe for a blog site?
    The account aided me a acceptable deal. I had been a little bit afquainted of this your broadcast provided bright clear idea

  20. This is the right webpage for anybody who really wants to
    understand this topic. You know a whole lot its almost hard to argue with you (not that I actually
    would want to…HaHa). You certainly put a brand new spin on a subject that’s been written about for decades.
    Wonderful stuff, just excellent!

  21. With havin so much content do you ever run into any issues of plagorism or copyright violation? My
    site has a lot of unique content I’ve either written myself or outsourced but it
    seems a lot of it is popping it up all over the web without my
    authorization. Do you know any techniques to help reduce content from
    being stolen? I’d definitely appreciate it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s