Do You Learn More at a Startup?

I’ve had The Debate many times with people at very different stages of their career – whether to go to a startup, or to work at an established company.  One of the classic arguments for the startup is that you learn more than you would inside the belly of the beast at a large company.

Why it’s True

One of the distinctive things about life at a startup is that everything happens at a hyper-accelerated rate.  Which means that for a given amount of time, you will generally experience much more of the lifecycle of a product, a business, and a company.

I experienced this really vividly when I did my second startup, a  I left Microsoft for the opposite coast to co-found the company.  In two years, we grew it from a few people to over 100, built a massively scalable server infrastructure from scratch and shipped it in six months, became the 50th most active site on the Web, went on a road show and took the company public, lived the exhilaration of flying high, got caught up in the crash and watched our stock go into the tank, had it come back to a more reasonable level, and merged the company with another.  Then my previous partner convinced me to come back to Microsoft to do an internal startup .. and I ran into people who were still working on exactly the same product cycle they had been doing when I left (!).  I felt like a traveller who has gone out into the world, had exotic adventures, and feels utterly changed by them, only to come back and encounter the polite incomprehension of the folks who stayed at home muddling along just as usual.

Another thing is that you typically get involved in a much broader range of activity.  At a big company, division of labor exists (must exist!) at an extreme.  There are hundreds of finance people at Microsoft who are extremely expert at what they do, so your involvement in that discipline even at senior levels of business ownership is very limited.  You consume their work, but that’s very, very different from actually doing it.  Similarly for legal, HR, recruiting, sales, lab management, datacenter design, office facilities, networking infrastructure, ad infinitum.

At a startup, there aren’t any specialists in most areas, so you have to jump in and do them yourself.  You get exposed to many aspects of the business that you wouldn’t otherwise know anything about.  If you like a holistic understanding of what’s happening, you love that.  If you want to focus deep in an area, it can drive you nuts.

But .. It’s Not That Simple

That’s the “pro” argument, but there’s another side of the coin that I think is often glossed over by the advocates.

Because things are moving fast and there aren’t a lot of “experts” around, you usually won’t get trained with any kind of deliberation.  Big companies are very uneven about how thoughtfully they develop their people, so it’s by no means assured that you will get a better experience, but hopefully you will.  I think one of the best way to learn, especially early in your career, is to “apprentice” with a more experienced and expert person.  Ideally, they are a great coach who will push you with challenging work, will evaluate it deeply and give detailed feedback, and they will be there to help when (and only when!) appropriate.  I think you are more likely to get that experience at an established company ..  but lousy managers abound everywhere, so you’ll have to be lucky or smart to find a good one.

It’s rare to get the opportunity to learn big and complex things systematically.  There are areas of expertise that are deep, hard, and take time to absorb.  Things like operating system and database kernels, distributed system design, compiler optimization, and machine learning, are systematic bodies of knowledge that call for the accumulation of knowledge and wisdom over many years to become a true expert.  In startups, you are scrambling like hell and need to get something up that works, so it’s hard to create something that is carefully and thoughtfully designed for the long term.  There are wonderful counter-examples of well-architected systems built by startups, and many pieces of crap built by big teams at established companies, so this is not some universal law.  But, in my experience, you are more likely to get a chance to master those kinds of areas at a big company that has the resources to invest in thoughtful architecture and quantities of deeply trained people available to work on it.

Running a business at scale is different than running a small one.  You won’t learn how to operate at scale at most startups.  Managing teams of hundreds of developers, keeping hundreds or thousands of sales people productive, coordinating hundreds of subsidiaries around the world – these are very difficult things to do well, and you won’t learn about them at a small company.

How it Nets Out

So will you learn more at a startup?  It depends on what you want to learn.  If you want to experience the whole business from customer experience to support to revenue, choose a startup every time.  If you want to move fast and see a lot of things quickly, ditto.

But, if you want to go really deep and immerse yourself in something complex, or you want to train yourself in your craft (whatever it is – systems programming, project management, finance), or you want to learn how to operate at high scale, you might find that you will do better at a larger and more established company.

What’s my approach?  Do both.  I have had by far the most fun at the three startups I’ve done, but I’ve learned powerful lessons at large companies that serve me well in everything I take on, with teams large and small.  And if you are at a startup, and it’s successful, then it’s nice to know that you have experience operating at larger scale – you won’t have to learn every lesson on the fly, when it’s life or death for the company that you do it right.

Rise of Data Science

Radically accelerated by the advent of cloud computing and devices, a role has begun to develop that will flourish in the coming years, and I am convinced that it will have a major impact on our lives.

New technologies often usher in new disciplines; they typically begin as a chaotic area of focus, with all sorts of people falling into them from different backgrounds.  Over time, they take on structure, books are written, educational and training programs develop, and they turn into a mature discipline.  That’s what happened when the Web was created – building a web site requires a mix of skills that draw from what had been quite separate worlds of activity: art and visual design, image processing, and programming (among others).

The same arc happened a few decades earlier when programming was invented – it drew from fields like mathematics, engineering, and linguistics.  It attracted people from those fields and many others (including more than a few high school students who were supposed to be doing something else!).

This new field hasn’t been officially named yet, but one of the terms that people are using for it is “data science”.  I’ve been diving into it pretty deeply for our startup, and some remarkably interesting work has been happening over the past several years.

What Does a Data Scientist Do, Anyway?

As you would expect from an emerging discipline, people don’t agree yet on exactly what it is all about.  But the fundamental idea is that enormous bodies of data are being gathered through software, and somebody has to make sense of them.  The analysis can influence decisions that people make (“hey, this version of our web service gets 15% more people to sign up for an account than the other one”) and decisions that software makes (while browsing items on Amazon, the web site will tell you that people who bought this product also were interested in …).

A data scientist is somebody who figures out what data to gather, how to analyze it, and what to do with the results of that analysis.  The discipline combines ideas from areas like statistics, machine learning, mathematics, databases, and psychology.

What’s it good for?  Well, here are just a few ways it is being used today:

  • The magical ability of Google search to find what you need from a couple of words and no other hints.  Compare that experience to what you typically get from software – you usually have to tell applications in painfully explicit detail exactly what you want, in very tightly scripted sequences of commands, and it can be extremely frustrating if the programmers haven’t anticipated what you want.  With Google, you type just about anything into the search box, and with incredibly high probability, it will give you a useful set of answers.
  • The ability to recommend things that are likely to interest you.  Amazon is very, very good at helping you find a book you want on any subject under the sun, through a combination of search and recommendations.  Netflix has gotten to the point where 75% of the shows that people watch on their streaming service come from a recommendation
  • Web sites present users with multiple versions of their product simultaneously, watch how users react, and pick the best one.  Large web companies are running dozens or hundreds of these A/B tests simultaneously and are updating their product daily based on the result.  I used to ship large packaged software products to enterprises, and we would conduct a manual poll of our users years after we shipped to try to figure out whether they used the product and what they did with it – the results were very spotty, very late, and highly inaccurate.  It’s like trying to drive by covering the windows of your car with black paint and having somebody write you an occasional letter about where you are and the condition of the road.

Those are just a few examples – almost every Web-based company depends on data science as its lifeblood to make its product come alive for users and to run its business internally.

The Future

What’s being done today with data science, while impressive, just scratches the surface.  The current economic models have only begun to evolve.  And many parts of our lives remain deeply inefficient and filled with friction:

  • Transit is very wasteful – guessing about traffic patterns, individual drivers maneuvering 3000 pound chunks of metal with dubious competence.
  • Integration of medical carediagnosis, and monitoring our bodies, remains technologically primitive.
  • Energy use is highly inefficient, partly because we have little idea how to optimize or the implications of our decisions.
  • Education hasn’t improved much in the last few hundred years, when President Garfield said that the ideal college was a famous teacher (Mark Hopkins) at one end of a log and a student at the other.  It’s arguably the most important competency of a successful nation in the modern age, and our system (in the United States at least) is hardly flourishing.

Along with much of the economy, these areas are ripe to be transformed, and I am convinced that data scientists will be at the heart of that transformation (for good and for ill!).  If you’d like to learn more:

It’s a discipline that I think anyone involved in technology should understand at some basic level.  Pretty much whatever you do these days, there are probably large quantities of data being generated around it that can be mined for insight.  You want to leverage this power, to make your own decisions and to create a great experience for people using your software.  It’s going to continue to transform the world over the coming years .. and maybe you can become a real-life Hari Seldon.