Rise of Data Science

August 2, 2012 by Leave a Comment

Radically accelerated by the advent of cloud computing and devices, a role has begun to develop that will flourish in the coming years, and I am convinced that it will have a major impact on our lives.

New technologies often usher in new disciplines; they typically begin as a chaotic area of focus, with all sorts of people falling into them from different backgrounds.  Over time, they take on structure, books are written, educational and training programs develop, and they turn into a mature discipline.  That’s what happened when the Web was created – building a web site requires a mix of skills that draw from what had been quite separate worlds of activity: art and visual design, image processing, and programming (among others).

The same arc happened a few decades earlier when programming was invented – it drew from fields like mathematics, engineering, and linguistics.  It attracted people from those fields and many others (including more than a few high school students who were supposed to be doing something else!).

This new field hasn’t been officially named yet, but one of the terms that people are using for it is “data science”.  I’ve been diving into it pretty deeply for our startup, and some remarkably interesting work has been happening over the past several years.

What Does a Data Scientist Do, Anyway?

As you would expect from an emerging discipline, people don’t agree yet on exactly what it is all about.  But the fundamental idea is that enormous bodies of data are being gathered through software, and somebody has to make sense of them.  The analysis can influence decisions that people make (“hey, this version of our web service gets 15% more people to sign up for an account than the other one”) and decisions that software makes (while browsing items on Amazon, the web site will tell you that people who bought this product also were interested in …).

A data scientist is somebody who figures out what data to gather, how to analyze it, and what to do with the results of that analysis.  The discipline combines ideas from areas like statistics, machine learning, mathematics, databases, and psychology.

What’s it good for?  Well, here are just a few ways it is being used today:

  • The magical ability of Google search to find what you need from a couple of words and no other hints.  Compare that experience to what you typically get from software – you usually have to tell applications in painfully explicit detail exactly what you want, in very tightly scripted sequences of commands, and it can be extremely frustrating if the programmers haven’t anticipated what you want.  With Google, you type just about anything into the search box, and with incredibly high probability, it will give you a useful set of answers.
  • The ability to recommend things that are likely to interest you.  Amazon is very, very good at helping you find a book you want on any subject under the sun, through a combination of search and recommendations.  Netflix has gotten to the point where 75% of the shows that people watch on their streaming service come from a recommendation
  • Web sites present users with multiple versions of their product simultaneously, watch how users react, and pick the best one.  Large web companies are running dozens or hundreds of these A/B tests simultaneously and are updating their product daily based on the result.  I used to ship large packaged software products to enterprises, and we would conduct a manual poll of our users years after we shipped to try to figure out whether they used the product and what they did with it – the results were very spotty, very late, and highly inaccurate.  It’s like trying to drive by covering the windows of your car with black paint and having somebody write you an occasional letter about where you are and the condition of the road.

Those are just a few examples – almost every Web-based company depends on data science as its lifeblood to make its product come alive for users and to run its business internally.

The Future

What’s being done today with data science, while impressive, just scratches the surface.  The current economic models have only begun to evolve.  And many parts of our lives remain deeply inefficient and filled with friction:

  • Transit is very wasteful – guessing about traffic patterns, individual drivers maneuvering 3000 pound chunks of metal with dubious competence.
  • Integration of medical care, diagnosis, and monitoring our bodies, remains technologically primitive.
  • Energy use is highly inefficient, partly because we have little idea how to optimize or the implications of our decisions.
  • Education hasn’t improved much in the last few hundred years, when President Garfield said that the ideal college was a famous teacher (Mark Hopkins) at one end of a log and a student at the other.  It’s arguably the most important competency of a successful nation in the modern age, and our system (in the United States at least) is hardly flourishing.

Along with much of the economy, these areas are ripe to be transformed, and I am convinced that data scientists will be at the heart of that transformation (for good and for ill!).  If you’d like to learn more:

It’s a discipline that I think anyone involved in technology should understand at some basic level.  Pretty much whatever you do these days, there are probably large quantities of data being generated around it that can be mined for insight.  You want to leverage this power, to make your own decisions and to create a great experience for people using your software.  It’s going to continue to transform the world over the coming years .. and maybe you can become a real-life Hari Seldon.

Filed Under: 1x Tagged With: , , ,

The Rip Van Winkle Question

July 18, 2012 by 2 Comments

Several years ago, I become responsible for a reasonably large business.  As you’d expect, the team regularly reviewed progress using a series of reports full of numbers.  Page after page of them, with thousands of numbers, analyzing performance by region, by pricing level, by licensing model, by customer type – you name it.

To an expert, these reports were filled with wonderful nuggets of insight.  Wow, what happened to sales in Germany last quarter – why did they tank even though the competitor’s results were strong?  Clearly, the Japanese sub is the only one leveraging the price increase – their average revenue per unit is spiking while everyone else is just plodding along.  And so on.

To somebody who was not expert (i.e. me), it was just a wall of numbers that didn’t convey much of anything.  To get a sense, check out a report like this one.  If you are an experienced investor, or used to reading accounting statements, you can glance over it and almost instantly you know a lot of interesting things about Google as a business.  If you aren’t, your eyes probably glazed over and you are hoping there won’t be a pop quiz at the bottom of this blog post.

In my case, I didn’t even know what half of the numbers on the business reports were about.  What the heck was the “PSP attainment vs. seasonality adjusted target”?  Did it matter that we seemed to be below what we had originally expected?  But I had to get smart quickly – these reports were the lifeblood of the business.  It’s like a medical chart to a doctor; they can spot patterns that reveal what is happening to their patients.  I had just become the doctor for this business, and I needed to know if the patient was suffering from any serious illnesses, so I could do something about them pronto.

So How to Start?

The best technique I’ve found is what I call the “Rip Van Winkle” question.  If you aren’t familiar with the short story, Rip was a man who fell asleep for twenty years, and found that the world had changed dramatically when he woke up.

What I did was to take every important angle on the business and find somebody who was really smart about it.  Then I sat down with them and asked the key question: “if you fell asleep for a year or two, when you woke up, what are the first things you would look at on this report to understand how the business is doing?”  Over and over again, I got amazing insights by asking this.  “Well, the first thing I’d do is look at new sales in the enterprise segment to make sure we are getting growth instead of just milking the installed base.  Then I’d divide that by the number of units for a quick check that our price was holding up and we’re not jacking up sales with deep discounting.  Then ….”

I did this walk-through with around 30 people, for a total of some sixty hours of discussions.  Finance people told me how they analyze the finance numbers.  Customer service showed me how they track and assess problems and customer satisfaction.  Sales managers talked about the pipeline and performance and hiring.  Often, different people would take me through the same report, and come at it from radically different directions.  I took copious notes, but I always asked for the top 2-4 things they would look at first.  I would highlight and number the places that held the answer.

Vital Signs

It turns out that for just about any report, even if it has hundreds or thousands of numbers on it, there are a handful that really tell the crucial story.  The rest of them might be useful to support the story or diagnose a problem, but you mustn’t get distracted.  In medicine they call them your vital signs – tell me your pulse and whether your eyes dilate and a couple of other things that can be measured by an EMT in seconds, and I will know if you are basically ok or deeply traumatized.  I may not know if you had a stroke or a concussion, but I’ll have a good basic sense of how you are doing.

This technique hinges, of course, on finding insightful people with an intuitive mastery of the numbers.  I could never predict who it would be from the org chart – they might be high up or buried deep.  But the people who worked in that area almost always knew whom I should talk to.  Ask around!  Once I found the right people, they were usually happy to share some wisdom with an interested and enthusiastic listener.  Buying them lunch never hurt, either.

By the end of those sixty hours, I was pretty darn good at diagnosing the business from the numbers, because I had learned from such a wide range of experts.  The process also turned out to be a useful diagnostic tool in its own right.  If I couldn’t find anyone in an area with great insights to share, chances were pretty good this was a side of the business that wasn’t being managed very well.

What I’ve learned by doing this exercise many times is that project reporting is almost always far too detailed – it’s like the old story about writing a shorter letter if you had more time.  It’s very hard to distill a lot of complexity into a tight report that shows only the key things – that means you have to figure out what those key things are (and have confidence that you didn’t miss anything vital!) – so most people cop out and throw in the kitchen sink.  As you are ramping up, think about how to cut way down on the amount of information being reported.  More is definitely not better, when it comes to metrics.  Einstein’s famous dictum applies perfectly here – “make things as simple as possible, but not simpler”.

The next time you have to get smart about a report full of numbers, give the Rip Van Winkle technique a try, and see if it works as well for you as it has for me.

Filed Under: 1000x, 100x, 10x, 1x, Rhythms of Business Tagged With: , ,

Manage Your Firedrill Capacity

June 29, 2012 by Leave a Comment

Ah, the firedrill – that urgent project that your team has to scramble to get done by the deadline.  Everywhere I have been, they are an inescapable fact of life.  “We just got a key customer meeting and we’ve got to have that demo ready!”  “There is a VP review of the project next week and we have to nail it!”  “I need this analysis done tonight for the board meeting – they said we have to cover this compete angle!”  “Our customer’s site is down and we have to get them back up ASAP – drop everything!”

Some of this is inevitable – things come up, so you have to rise to the occasion and get it done.  But some teams seem to exist in perpetual firedrill – life is just an endless succession of crises and it feels like a treadmill; you never get the chance to move forward on the really important projects that could be game changers, because you are running as fast as you can to handle the constant stream of do-it-now projects that bombard you.

How much you can control this problem depends on the role.  If your job is to fight fires (literal or metaphorical), then maybe the firedrill is what you do.  But if your work is more project-oriented, my guess is that a lot of them could have been anticipated and avoided.  That’s been my experience.  Test it for yourself – keep track of the firedrills you are pulled into for a couple of weeks, and look over the list.  How many of them really came out of the blue and could not have been avoided?

The Magic Wand: Designing Good Systems

Systems and processes can make life miserable and are one of the things that people complain most about.  They can get rigid, bureaucratic, and generally suck the spirit and energy out of life.  BUT … they are the critical tool that can eliminate firedrills, if they are done right.  What I’ve found is that you must design them thoughtfully, hone them until they cause the minimum of friction, and throw them out when they are no longer adding value. You probably will benefit from a system when something is predictable, repeated, and complex.

  • Predictable: hey, it’s a new year and we have to get our plan landed – wow, who could have seen that coming?  Excuse me, but it’s been on the Gregorian calendar for around 430 years, so this really shouldn’t catch you by surprise.
  • Repeated: if you do something once, then designing a system may or may not pay off.  If you do it over and over again, you don’t want to lose the hard-won lessons of the past.  Figure out how to do it well and bake it into a system so that you don’t have to think about it.  We’d never be able to function as human beings if our bodies didn’t do this constantly – we couldn’t walk, talk, eat, read, or write.  We spend much of our early childhood evolving our neural systems to master the key activities needed for life – think about how helpless an infant is.  Reinventing the wheel every time you do something keeps your team in perpetual infancy.
  • Complex: if it’s trivial, you may (may!) not need to remember how to do it.  But as things get complex, you are wasting enormous amounts of time and operating very inefficiently if you don’t capture that knowledge into a system.

Firedrills are a great indicator that you need to do something – jotting down a line or two about each one will take you less than a minute a day and looking the list over will help you diagnose places where you have a missing system or a failing one.

Don’t Waste Your Firedrill Capacity

Every team has some capacity to absorb firedrills.  That capacity increases as people are more committed to the mission.  It increases with confidence in the team leadership – people figure that if the leaders say something is important, it probably is.  But if you burn through that capacity too often, or you waste it on things that are obviously just screw-ups in planning that could have been prevented, you’ll pay the price.  People will get grumpy, cynical, burned out .. and eventually will leave the team.

So manage that capacity.  Fill the tank by fostering trust and commitment.  Avoid burning it up on dumb things that you could have avoided with some thoughtful planning.  Come clean when you screwed up and the team has to pay for your mistake.  And then when you really do need them to commit heart and soul to pulling off something heroic, the team will be right there with you, ready to dig deep and gut it out.

Filed Under: 1000x, 100x, 10x, Rhythms of Business Tagged With:

Rubber Gloves, Not White Gloves

June 14, 2012 by Leave a Comment

I’ve always loved to be involved in making things happen, on the ground.  The detail of execution, the individual decisions that mount up to determine success or failure – I like to plunge my hands into the engine grease.  Hence, rubber gloves, not white gloves.

That enthusiasm hasn’t always been embraced by the environment I’m in – places where the focus was on Big Ideas™, not the minutia of execution.  In academia, research labs, and staff or management roles, I was often supposed to step back, see the bigger picture, and identify the key trends and leverage points.  I love to do that, but I find that big ideas often become irrelevant if you are disconnected from the details.  In studying many projects that both succeeded and failed, my conviction has only deepened that the combination is where the magic lies.

Combining Vision and Execution

A great example is Brunelleschi, the architect who designed the beautiful Duomo – the dome that sits atop the famous cathedral that dominates the skyline of Florence.

It took 140 years to complete the building; the dome had remained a puzzle for decades because nobody could figure out how to build it.  The solution called for rediscovering ideas that had been forgotten since the days of the Roman Empire, and new inventions that had never been tried before.

Brunelleschi solved the problem in 1419 and got the commission, dedicating most of his adult life to constructing the dome and other parts of the cathedral.  The dome took more than four million bricks (37,000 tons of material!), and with the lantern on top stood 375 feet high.  He got a couple of the first patents ever issued to protect his ideas for a new kind of hoisting machine and a river transport vessel.  Brunelleschi was intimately involved in every step of the construction, laid some of the bricks himself, and immersed himself in every detail (including the work schedule and diet of the people doing the construction).  If you are interested, check out the Wikipedia entry; Ross King did a great job telling the story in more detail in the book, Brunelleschi’s Dome.

Another interesting case study was the race to reach the South Pole first in 1911.   Two expeditions were involved.  One party, led by Robert Scott, was less focused on the details of planning and preparation.  The other, led by Roald Amundsen, was a great example of deep focus on every detail.  Amundsen insisted that every member of the expedition be an expert skier who had been practicing the sport since they were children.  Scott’s team mostly couldn’t ski.  Amundsen prepared navigation sheets so that they wouldn’t have to make complex calculations when they were exhausted.  Four out of five members of his final team were experienced navigators.  Scott had one, and used a technique that required more calculation.  Amundsen relied heavily on sled dogs and figured out how to keep them properly fed and cared for.  Scott’s team didn’t want to rely as much on dogs so they were dependent on horses, which turned out to be poorly adapted for the conditions.

Both teams were staffed with brave and seasoned explorers, but the results starkly illustrate the cost of poor focus on execution basics.  Amundsen got to the pole first and returned successfully without losing a man.  Scott, and the four companions who made it to the pole with him, all died.

Right Vision, Bad Execution

There are many, many examples where a compelling vision but poor execution leads to disaster.  It often happens even if the vision is a great and achievable one.

I thought it was very interesting to learn about the history of the Panama Canal in the excellent book by David McCullough called Path Between The Seas.  Basically, there was a competition to come up with the design for a canal that would connect the Atlantic and Pacific Ocean and avoid the laborious and dangerous trip around South America into the treacherous waters of Drake’s Passage.  A Frenchman, Ferdinand de Lesseps, had been the leading spirit behind the Suez Canal and came up with a hastily slapped together plan that chose a sub-optimal path and paid little heed to the enormous engineering challenges.  An American team very carefully and thoughtfully developed an alternative based on extensive surveys.  Lesseps, who was a charismatic, larger than life figure with little engineering knowledge, carried the day anyway and inspired investors to finance the effort.

An engineering group estimated it would cost $214 million; Lesseps cut that to $120 million with no real justification.  After eight years, 22,000 people had died (mostly of malaria and yellow fever – nobody realized that they were spread by mosquitos) and $235 million dollars had been spent, but the project was only partially done with the hardest issues unresolved.  The company went bankrupt.   The project was eventually taken over by the American government, after a fifteen year delay, and it took another $375 million and nine years to complete (but came in under the budget estimate!).  I think it’s a powerful example of a great vision, uncoupled to strong execution, leading to horrible results.  Then that same vision, carried forward in more pragmatic and expert hands, created a vital global resource.

Vision is Crucial .. but Must Not Become Decoupled From Reality

I’m a great believer in the importance of vision and strategy.  But strategy has to be deeply, deeply informed by all the infinite detail of execution.  And in looking at brilliantly successful strategies in technology, I’ve found that they are often emergent properties.  They were glimpsed, and then refined, by someone steeped in a deep understanding of their markets and the technology forces transforming them.  A kernel of insight evolves into a huge success – it doesn’t leap forth from the mind fully formed, like Athena from the forehead of Zeus.   Having a big and bold goal is fantastic .. but strategy is shaped, and success or failure is determined, by the execution.

Great execution is hard and requires endless focus and obsessiveness to achieve.  I’ve mostly worked on large software projects, and when you are building something that has millions of lines of code in it, the team must work together extraordinary well.  Such projects call for Herculean efforts and tiny bits of friction can lead to huge problems.  The code, the test suites, the shiproom triage meeting, the bug count, the milestone definition .. those are the clay and brick from which the great cathedrals are forged.  Lose your connection to the grit and the glue at your peril.

That’s why I love being in a startup .. we have a bold vision for what we hope to achieve, but our days are full of designing and building the product with endless and loving focus on every detail.  Hopefully we are achieving a combination of vision and execution that will let us have the impact we aspire to make in the lives of our users.

Filed Under: 1000x, 100x, 10x, 1x Tagged With: ,

As the leader of the forces opposing evil in the universe .. you’re fired!

June 7, 2012 by Leave a Comment

Epic tales are a lot of fun to read – the struggle of good and evil, the climactic moment of truth when the future of humanity or the world hangs in the balance.  They also can provide good insight on managing complex projects .. or rather, how not to.  There are some great lessons to be learned from the bungling incompetence of fictional heroes (and yes, this point of view can make you a real buzzkill, so you might want to keep these thoughts to yourself!).

Risk Mitigation and Contingency Planning

The worst sins of omission are basic risk analysis, mitigation, and contingency planning.  Very brittle plans are made, with no effort to figure out a Plan B if something goes wrong.  That makes for great drama, but it’s lousy planning.

Let’s start with Lord of the Rings.  It’s a great story that I have loved since I discovered The Hobbit as a ten year old – it has edge of your seat excitement with a richly detailed universe as the backdrop.  But come on .. what kind of a grab-ass plan was that for saving the world from evil?  We’ve got a group of clueless hobbits wandering to Bree with the Nazgûl charging around and almost catching them. The hobbits are supposed to hook up with Aragorn, but they don’t even know what he looks like – would it have killed Gandalf to give them a description?  How about an escort?  The great civilizations of Middle Earth are facing Armageddon, and they can’t scare up a couple of people to help out?

On Star Trek they are always beaming the ship’s top officers onto a potentially hostile and unknown planet, leaving the Enterprise with mostly junior people to run it.  What’s up with that?  They probably shouldn’t be sending most of the senior officers into harms way in the first place.  And, a Constitution class starship is a massive investment and a jewel in the crown of the Federation .. how come they don’t have enough seasoned officers on board to be fully covered even if four or five of the most senior ones insist on wandering into danger all the time?

In the wonderful fantasy series “The Dark is Rising”, they almost lose the ancient artifacts that determine the victory of good or evil because .. somebody left a note with a family and they happened to forget to deliver it.  Really?  Come on!  If that’s the best you can do, time for a new project manager who has a clue.  To drive a complex and crucial initiative, think about the aspects of your plan that are fragile and could easily go wrong, and build in defense in depth.

The Dark is Rising heroes not only made one dumb and potentially fatal arrangement, they keep doing it.  Don’t be like them.  If you see a particular breakdown, think about the underlying causes and address them.  Don’t settle for the obvious explanation – dig deeper.  Say you are running an online service and it went down .. why did it happen?  Well, there was a bug in the code.

  • Sure, but why did the bug slip through?  Do you need better tests?  More tests?  More realistic load testing?
  • Why did it happen in the first place?  Was there some communication breakdown?  Is the architecture of the system too baroque?  Are there missing levels of abstraction between system components?
  • Why was it hard to find and fix?  Do you need better diagnostics?  Better logging?  Better monitoring?
  • Why did it affect so many people?  Could you have a more loosely connected system?  Could components be more resilient when others fail, and degrade the user experience more gracefully?
  • Why were you down so long?  Does it take too long to deploy a fix?  Too much time to restart components that depend on it?

In running a project, you often have to make bets and take gambles – that’s part of the game.  However, you should think about the key bets you are making, and what will happen if your bet is wrong.  How quickly can you tell that you made a mistake (and be fairly sure about it)?  What will you do to reverse course or mitigate the failure?  Are you keeping anything in reserve so that you have some resources you can apply to help rescue the situation?  If you are making an unrecoverable bet, are you clear about that and about the due diligence you need to do up front?

Defining Roles

One of the common sources of confusion and inefficiency in a team is not knowing what role everyone is supposed to play.  Often, you can muddle along until something really important comes up, and then under stress the team works very poorly to resolve the issue.

Think about Boromir and Aragorn – after Gandalf fell into the cavern fighting the Balrog, they hadn’t resolved who was left in charge of the group.  Boromir deeply disagreed with the strategy Aragorn laid out.  Frodo decided he didn’t want to be with any of them any longer.  Since he was the ring bearer, ultimately it was his decision .. but nobody had given much thought to it.   It’s critical to figure out how the most important decisions are going to get made, and it’s a lot easier to do that before you are in the middle of a stressful situation with emotions running high (though hopefully you won’t be getting attacked by the Uruk-hai).

Thinking Out of the Box

It’s easy to get trapped into conventional thinking.  We’re all prone to unconscious assumptions – how things are supposed to be done, constraints we think we have to live with.  For example, Gandalf is close friends with the eagles, who can .. fly.  While carrying riders, and even outmaneuvering the fell beasts that the Nazgûl are riding later in the story.  So why is the Fellowship slogging their way through the mines of Moria and playing tag with terrifying ancient spiders, when they could get to Orodruin in a couple of hours?  Maybe with some Legolas-class archers along to provide suppressing fire in case anybody tries to interfere?  The whole thing could have been wrapped up and the hobbits tucked cozily back in their beds after a nice end of the day snack, before Sauron had a clue.  This idea is hilariously developed on “How it Should Have Ended”.

In life, it’s impossible to identify and question all your unconscious assumptions .. but you can tease out the most important ones.  Ask yourself what you are assuming, and whether you have the evidence to back it up.  Ask “what do I have to believe?” in order to justify a proposed course of action, and see if you can get some kind of backup that those things are really true.  Or a way to notice that they aren’t, so if you are on a delusional path, you’ll figure it out as quickly as possible.

Learn From Everything

I have found that there are great lessons to be learned about accomplishing your goals from every life experience.  Learn on the job, by all means, but I try to make everything grist for the mill – books, stories, movies, tales from history .. it all provides insight and inspiration to help you pick the goals that matter to you and to find ways to achieve them.  And before you decide that “actually, hope is the plan”, remember that in real life, you aren’t the main character.  There is no author who will turn your heedless folly into an exciting story of success against all odds.

Filed Under: 100x, 10x, 1x Tagged With: ,

Eyes of a Stranger

June 1, 2012 by Leave a Comment

I’m always looking for best practices to adapt and adopt, and I got an idea that I really like from a mentor.  It is a way to combat the complacency that sets in as we settle into any role – that tendency to become accustomed to the way things are, even when they are pretty screwed up.  “Well, of course you stand on one foot and tug on your left ear with your right hand .. that’s just how things are done here.”  With fresh eyes, we might ask “but, umm, isn’t that kind of stupid?”  And, “about the fire burning over there .. maybe throwing some water on it would be good?”

So periodically – maybe once a quarter, or once a year, the idea is to do an exercise I like to call “eyes of a stranger”.  Pretend that you just got your job and are figuring things out – you are in your “first 30 days” and are coming up to speed on the important things you need to focus on.  What is a top priority issue or opportunity that you would see and decide that you absolutely have to pursue?  What inefficiency would you discover that would just bug you until you got it fixed?  What joy-killer is afflicting the team (or you) that needs to get taken care of?

There’s a scene from a book that got stuck in my mind; it’s in Kon Tiki, the really fun story about an anthropologist who gathers a set of kindred spirits to prove that it is possible to sail a raft from the Peruvian coast to the Polynesian islands, using only the technology available in ancient Peru.  At the end of the book, they have crashed on a reef and the raft is smashed, they are pinned down and waves are pounding over them, and one of the people clinging to the remains of the boat says calmly “This won’t do.”  I try to apply that same calm but determined spirit to situations at work that feel desperate.  As you look at your job and the environment around you, what “won’t do” that you’ve gotten used to and have been letting slide?

I’ve found that you probably want to come out of the exercise with a very short list of things you are going to pursue more aggressively than you have been – one is good, three is probably an absolute max.  If you come up with a longer list, revisit it after you do something about the top ones.  I tend to find that you get wildly more bang for the buck by focusing on a couple of things (or one!) rather than dutifully writing down ten “priorities” and feeling overwhelmed so you just go back to ignoring them.

For each of the issues that you’ve picked, you need to figure out what concrete steps you can actually go take to deal with them.  I like to sit down with a piece of paper and do a mind map.  If the issues are worth addressing and you haven’t been doing it, it’s a good bet that you are a bit stuck in figuring out what needs to be done.  Maybe you just need to spend 30 minutes listing next steps or coming up with a plan.  Or, perhaps it will work better to pick somebody you have a good rapport with, and brainstorm about it together.  If it’s a really big issue, you might find it helpful to apply (some of) the framework that I outlined in the series on “Cracking the Nut”.

I’ve done this exercise over a dozen times, and each one has helped me get hard core about tackling something that needed doing and that wasn’t moving forward.  See if it works for you, too!

Filed Under: 1000x, 100x, 10x, 1x Tagged With: ,

Relay Race vs. Diving Competition

May 25, 2012 by 2 Comments

When you are evaluating how people are doing, it seems reasonable to focus on the business results they deliver.  In my previous team, we called this the relay race model.  What matters in a race is finishing, and finishing with the fastest time.  Nobody cares whether your running form was good or terrible – you are measured solely on the results you deliver at the end.  In most work environments, this is appealing, intuitive, measurable … and wrong.

A diving competition is almost the exact opposite of a relay race.  After all, everybody is going to hit the water, and it will take about the same amount of time no matter what you do.  The score is based on your form – how difficult a set of moves you undertake, and how gracefully and perfectly you do them.

Why the Diving Competition Matters

So what does this have to do with measuring job performance?  It’s the difference between assessing the pure business results that somebody delivers vs. the way they delivered those results.

It’s easier to measure (and talk about) the results that were delivered.  Did the product ship?  How much revenue did it generate?  Did you hit your sales quota?  Was the code quality where it needed to be?  Is the product performance hitting the ship goal?  It depends on your job, of course, but in many cases it’s relatively straight-forward to tell whether the person and/or the team achieved the goals that they were striving towards.

But there is another side of things – the way you behaved in pushing towards that goal.  Did you help build up the team?  Do people find you a good person to work with?  Do you help make the whole team better, smarter, and more capable?  I’ve worked with a lot of aggressive young engineers, and they sometimes are very impatient when I bring up these questions.  “Hey, I wrote the code, the product shipped, and it’s good.  I didn’t have time to humor those other idiots – we were on a tight schedule.  And I was right, wasn’t I?”  Often, yes.  But you are still going to get dinged on your review, because you may have been right about the issue, but you didn’t handle it the right way.  By running roughshod over the other person and leaving them feeling dismissed and mistreated, you blew it.  Why?

Well, for one thing, you are only responsible for part of the project.  Checking high quality code into the build is important, but the crucial thing we need to do is solve the customer’s problem.  Which means we need to understand their problem holistically, build a complete solution that meets their needs, test it, explain it and then sell it to them, support it, and integrate with other products.  That calls for a group of people to work effectively together.

Also, a particular deliverable is just one in a long succession of business results that we have to achieve together.  Sure, we shipped the product .. but that was just the beginning.  Even with packaged software, we have to ship patches.  We immediately start building the next version.  If it’s a service, shipping is the beginning of the hard work, not the end – now we have to run it 24/7 and manage the business that is based on it.

All of these ongoing business deliverables rely on the team working smoothly together.  When you are working on a problem that involves groups of people, no single person’s work alone can make the whole group successful.  If you achieve the goal you are focused on but leave a path behind you strewn with dead bodies, you can easily do more harm than good even if you do achieve what you set out to do.  Every team member has as much of an obligation to help ship the team as they do to help ship the product.  Given our ongoing responsibilities, the team is often the more important deliverable.  In the software business, if we create an unpleasant working environment and everyone leaves, we’ll be left with a big pile of code and no ability to run it, fix it, evolve it, support it, and sell it.

So for very hard-headed business reasons, I think it is necessary to evaluate people based on both the relay race model (the explicit results they achieved) and the diving competition model (whether they work effectively with others).  If you only focus on one, you aren’t encouraging and rewarding the behavior that yields the most value for the organization.  And on a more personal note, who wants to be on a team that’s unhappy and mistreats each other?

Filed Under: 100x, 10x, Managing People Tagged With: ,

What Do You Want to Be When You Grow Up?

May 18, 2012 by 2 Comments

When I’m managing or mentoring somebody, this is one of my favorite questions.  It’s a good way to crawl into their heads a bit, understand their motivations, and figure out how I can help them.  It’s also not a bad question to ask yourself!

Framing

There are different ways to approach the answer – you might base it on:

  • A particular person’s abilities: code like Guy Steele, move an audience like Meryl Streep, or paint like Raphael.
  • A particular job: be the VP of your division, the CEO of a Fortune 500 company, or the founder of a growing startup.
  • An achievement:  win an Olympic gold medal in swimming, design a piece of software that millions of people adopt, or take a company public.
  • A state of being: be crazy passionate about your job, feel that you are living in perfect alignment with your principles, or devote your life to serving the community.

How somebody chooses to answer will tell you a lot about their values, ambitions, and self-image.  Be careful about guiding them too much .. we’re talking about their dreams, not yours.

Cheerleader or Critic?

In doing this exercise with many people, it’s usually at least imaginable that they might achieve their ambitions with hard work and some (or a lot) of luck.  However, now and then somebody will come up with a goal where you have to struggle a bit to keep a straight face.  Often it’s about personality and passion – if you are impatient and can’t stay focused on your own, writing novels might not be your best bet.  So what do you do when someone who hates math wants a Nobel Prize in physics?

I try to be a “pragmatic cheerleader”.  I don’t want to be a dream buzzkill – most people are surrounded by plenty of those already.  But I also don’t want to be mindlessly encouraging.  The next question often helps to get things more grounded in reality.

What Does it Take to Achieve That Goal?

Dreaming big is great, but you have to break down the goal and figure out what it takes to achieve it.  One approach I like: if the person wants to be an X, I get them to write down a description of what the absolutely perfect X would be like.  For example, when I was running a relatively large team, I came up with what I thought a perfect team leader should do .. I wrote about that here.

Often, the person doesn’t have a great answer.  They are attracted to the idea of being a CEO, but they don’t actually know much about what the day to day life of a CEO looks like.  After some discussion, we can usually rough out an outline of the skills and qualities needed, but we often agree that a key next step is to validate that list with people who actually have that role or are more familiar with it than we are.  That’s a great practical step forward on the journey.

How Do You Stack Up Today?  How Do You WANT to Stack Up?

The next thing I do is have them assess themselves against those ideal qualities or skills.  At this point, people sometimes realize that their supposed goal doesn’t actually make a lot of sense for them.  I was talking to one person who thought they wanted to be the CFO of a public company.  We talked about the skills you’d need to be a fantastic CFO, and then she thought about when in her work life she had been really happy and why.  And she realized that the things that really did make her passionate and inspired were not the key activities of that job at all.  So I think that was a helpful moment of self-revelation.

This exercise also tests self-awareness.  Sometimes, they are confident that they have some needed quality .. and you think they are wrong.  It’s tricky to handle this.  I tend to tread pretty lightly – the goal here is self-revelation, not a lecture on their weaknesses, which tends to bring the discussion to a halt in a hurry.

But if people have at least a basic level of clue about their true nature and abilities, this exercise can have quite a profound impact.  I’ve watched people realize that they were carrying around outmoded ideals foisted on them by authority figures from their youth, and realize that they have an inspiring path open to them in a quite different direction.  That’s what you hope for – an opportunity to enable somebody to achieve a profound moment of self-discovery.

How Does One Actually Get That Job?

Next, I like to consider how people actually get the job or the opportunity to achieve the goal in question.  If it is a well-defined role, then the answer is usually straight-forward, but many people don’t think about it very practically.  I’ve repeatedly asked, “how do you become a VP” (or whatever).  People go on about a demonstrated track record, a well rounded set of skills, and all sorts of worthy things … but none of those are the real answer.

The actual answer is, “somebody who hires VPs chooses you”!  Now why would they do that?  All these ideas about skills and roundedness are fine, but often the true reason is that the hiring manager has confidence in you to do a great job and trusts you.  They’ll almost always pick somebody with strong skills, whom they deeply trust, over an unknown with a potentially stronger resume but who might be a whacko.  And they trust you because they know you over a long period of time, and have watched how you handle many different situations.  So one of the things that might be necessary to achieve your goal is to deeply commit to a particular team or company for an extended period of time, rather than bouncing around all the time.

Alternatively, the goal might be something that has to be created rather than granted – becoming the CEO of a new startup is self-elected (!).  But they need to have enough money to survive without an income and perhaps to fund the company for a while.  They may need a network of people they can hire.  And so forth.  Pretty much any ambitious goal will require something in order to open up the opportunity.

What Are Your Next Steps?

Now they have identified the goal, what it takes to get that opportunity, what they need to be successful, and where they stand now.  For each skill or ability where they aren’t already strong and experienced, I’ll brainstorm with them about a specific action that will develop it.  For example,  they can often build up skills that aren’t a direct part of their job by volunteering for extra projects – what I like to call “hobbies”.  I’ve learned a ton from hobbies – helping to organize and run a large developer conference, participating in working groups considering radical new directions to take the team, teaching classes, etc.  None of those had much to do with my core job, but I practiced new skills, learned about new areas, met really cool people .. and had a lot of fun.

The goal is to finish the discussion by identifying very concrete steps that will build the skills they need to achieve their dreams.  And if you have an ongoing relationship, hold them to it.  If you can really help somebody achieve their dreams, that is one of the best feelings in the world!

Filed Under: 100x, 10x, 1x, Managing People Tagged With:

We’re Living in Florence, in 1450 A.D.

May 7, 2012 by 2 Comments

If you work in technology, you have the amazing luck to be in the midst of one of the great transformations of human society.  There have been numerous times in history when technology has transformed the way we live, our understanding of the world, and the life we are able to experience.  I’ve had one of them on my mind a lot lately – the Renaissance.  I think there are very interesting parallels to be drawn.  It feels like we’re at the height of the first round .. which would put us in Florence, around 1450.

What Caused The Renaissance?

Many forces came together to enable that astonishing transformation of knowledge, art, architecture, and commerce, whose profound impact shaped the world and still fascinates us.

There were some key technology breakthroughs.  Gutenberg is famous for inventing the printing press in 1440, which greatly expanded our ability to distribute ideas.  But just as important, and less frequently remarked upon, was the spread of paper.  We take it for granted today, but it was a magical boost to our ability to capture and disseminate ideas.  Paper is far less costly to produce and can be manufactured in much greater quantity than alternatives like vellum, made from animal skins.  Invented in China in 100 AD, paper remained a closely held secret for several hundred years, but had become widely used by the time of the Renaissance.

The ability to travel and to trade was revolutionized by ocean-going ships (like the Portuguese caravel), which were capable of navigating the globe.  At the same time, the mariner’s astrolabe made it possible to measure latitude anywhere in the world.  With weapons like the arquebus, small groups were able to wreak havoc among less technologically advanced populations.

New technology called for new technique, as well.  The development of merchant capitalism, along with sophisticated means of tracking commercial activities like double-entry bookkeeping, allowed a modern system of banking to emerge.  It was banking that made Florence so wealthy, and that wealth enabled major investments in science and the arts.

In addition to a major infusion of money, the arts were transformed by new techniques as well.  Paintings became far more realistic using linear perspective.  Architecture took advantage of construction techniques both novel and rediscovered.  Grand new structures were created that finally matched and exceeded marvels (like the dome of the Pantheon in Rome) that had been built 1400 years earlier.

This confluence of breakthroughs in technology and technique allowed Europe to leap forward and become the dominant world power in a remarkably short period of time.  Businesses grew to previously unheard of scale and their activities reached across the globe.  Ideas moved much more quickly, too, because they were carried along by the people who were associated with this explosion of trading activity.  Control of knowledge moved from guilds to a mobile class of experts.  Liquidity, supported by precious metals imported from the New World, began to move the basis of wealth from land to capital.  Our world view also began to shift radically – from deism, centered on God, to humanism.  Our conception of the universe was rocked by the discovery that we were one of many planets orbiting around the sun.

The Modern Renaissance

Fast forward to today, and consider what is happening.  In technology, we are seeing two massive changes: devices and the cloud.  The cloud has transformed the cost and reach of computing.  By most estimates, Google runs well over a million servers in their data centers and handles something like three billion searches per day.  This is technology operating at a global scale that was absolutely inconceivable a few years ago.  At the same time, the programmable cloud has made it extremely cheap to build and deploy software.  For a few hundred dollars a month, a programmer with a laptop can take a program they have written and within minutes make it available to more than two billion people.

Smartphones and similar devices are also astonishing in their capability and their reach.  There are now around 1 billion smartphone users in the world with the Internet in their pocket and accessible at all times.  That number is growing rapidly – estimates are that there will be a billion net new smartphone users in the next few years.  The Apple App Store just crossed 25 billion downloads.  The pace of growth and the scale of what has already happened are just staggering.

As always, new technology encourages innovation in technique.  Open source software has been around for decades, of course, but it has gotten a major boost from its intimate relationship to cloud computing.  A wide body of high quality components are available for free to anyone who wishes to build cloud applications, representing a dramatic reduction in the time and effort required to go from idea to product.  We’re benefiting from that tremendously in our startup.

Another key change is the move from on-premise software to software as a service.  It means that the latest version of every application is available to every customer, without their needing to deploy or manage it.  Services pair nicely with, and encourage, the shift from physical to virtual – instead of manipulating objects, increasingly we’re manipulating data.  We are doing research and developing new products using simulated environments.  We’re transporting knowledge and entertainment as packets over networks, not by sending boxes of plastic and paper around the world.  Increasingly, the basis of value is rooted in virtual goods and services.  I believe that is as profound an economic change as the shift from land-based wealth to capitalism.

Signposts of the Revolution

We have seen some dramatic evidence of the impact that these changes will have, but I think we’re just at the beginning.

  • Facebook has over 900 million active users, and is on track to hit a billion later this year.  It has grown to that size in .. eight years.  To put that in perspective, China’s population today is 1.3 billion; it took around 250 years to grow the last billion (and it took human beings about 12,000 years to hit their first billion).
  • Speaking of Facebook, they recently purchased Instagram.  This company, which serves 30 million users has .. 13 employees.  Two developers run the back-end service for their users.  A few years ago, it would have taken a big company with major resources to support that many users, and now it can be done with a handful of people and no capital expense at all.
  • Consider that icon of the industrial revolution – the car.  A modern premium automobile has something like 100 million lines of code to run the nearly 100 processors distributed throughout it.  It was simulated extensively on supercomputers, is supported by myriad online services, and the supply chain that delivered it to you only works because of massive amounts of software tracking every minute aspect of its progress in real time.

I could go on, but the point is that virtually every industry is in the process of being transformed by the combination of the cloud and the device.  The way we make discoveries and create new inventions.  The way we communicate.  How and what we buy.  How companies interact with each other and with their customers.  And this is all happening incredibly quickly – the cost and effort for new ideas to be tried, refined, and deployed globally has dropped to the floor.  We’ve seen some dramatic changes already, but that was just a warm-up – we’re in for quite a ride.

The Path Ahead is Uncertain

In 1450, Florence was unquestionably at the forefront of the Renaissance, and the city was dominated by the Medici family.  By 1500, the focus of the action had moved elsewhere in Italy and across Europe, and Florence never regained its dominance.  What happened?  Well, for one thing, Charles VIII of France invaded Italy in 1494 and kicked off the Italian Wars, a series of conflicts that involved various city states and several empires.  That first invasion forced the Medici to flee the city, though they returned and ruled it again later.  In the meantime, other parts of Italy and Europe took over and led the Renaissance forward.  I suspect that the fortunes of the early players in our current Renaissance will also dramatically rise and fall.

And worse than losing leadership, there is a darker side to change.  We like to celebrate the Renaissance and the great leap forward in human capability that it represented.  But it wasn’t positive for everyone; it was particularly brutal for indigenous cultures around the world who were now within reach of the Europeans.  Many of them were despoiled and enslaved.  The current changes will not be as violent, hopefully, but we have seen these forces help governments fall and companies be humbled, and there are industries filled with people whose economic future will be dramatically affected.

We can never know ahead of time how things are going to shake out for particular groups during times of great change.  But when transformative forces come along that are this strong, they cannot be denied – they will transform our lives, culturally and economically.

Leonardo Da Vinci.  Michelangelo.  Brunelleschi.  We are still inspired by what they accomplished.  They were amazing people … but they also had amazing luck.  They lived in a magical time and place in the history of mankind.  So do you.  How are you going to be part of this modern Renaissance?

Filed Under: 1x Tagged With: ,

Beware the Shining Paladin From Afar

May 3, 2012 by Leave a Comment

Companies often fall into the trap of believing they can hire some external person to be their salvation – a shining knight who will lead them out of the darkness and into the promised land.  There are times when this works, but in my experience those are overwhelmingly outnumbered by the failures.  Why?  And can we learn something from the failures to try and avoid them?

The idea is very appealing.  If we as an organization are failing – maybe we are getting creamed by some competitor, or our formerly successful product is moldering into obsolescence and we can’t seem to get with the program on a modern reinvention of it.  Shouldn’t we invigorate the tired old blood in the team with a turbo-shot of new thinking?  Well, maybe.  But if we do, we’d better think about the many ways that this tends to fail.

Internal Pathology

We might be failing because of a systemic problem within the company.  This problem might be organizational (there is no team that cleanly owns the charter, so it keeps falling between groups), personal (somebody influential and passionate is preventing us from pursuing a workable strategy), failure of execution (the team that owns it isn’t healthy or effective), failure to focus (the team that owns the problem fundamentally doesn’t care about solving it), or strategic (the team gets pushed by company leaders into building overly general solutions that collapse of their own weight).

Fixing these problems can require a lot of organizational savvy and trust from the rest of the company; a newcomer is often much less capable of solving them than a seasoned employee.  Add to that the burden of expectations, and you have a situation designed to create failure.

Teacher’s Pet

Many times, I’ve watched a senior person get frustrated with a team, so they stick in some new blood and hope for the best.  The team can resent the newcomer as a teacher’s pet or “golden child” who hasn’t paid any dues.  Who are they to come in and start shooting off their mouth, without knowing anything about the team or the product?

For example, a friend of mine was hired by the president of a division to come in and shake things up in a new direction.  My friend gave an exciting pitch about the opportunities the team had and wasn’t taking advantage of.  He was put inside a group that had no buy-in to those new ideas, and which took its cue from somebody who had helped create the division and been responsible for some of its great successes.  Who also wasn’t bought in.  How receptive do you think the team was to my friend’s ideas?  Right, zero.  It was a massively frustrating experience for him and a waste of energy for the team, which of course went on to do exactly as it had before. 

Cultural Rejection

Teams can be a lot like the immune system, which “will try to destroy or neutralize any antigen that is recognized as a foreign and potentially harmful invader.”  People coming into the team, especially people brought in because they think differently and aren’t bound by the dominant assumptions shared by everyone else, are just bristling with antigens.  They use different language and terminology, they say things that seem off-key.  Often the instinctive rejection is so strong that their ideas don’t get a fair hearing.

Missing Key Skills

Having worked at both startups and large companies, I know how deeply different the success factors can be in those two environments.  People who flourish in a smaller community don’t always have (or value!) the skills to negotiate the internal landscape of a large company.  A big company might hire somebody who was winning, and the big company hoped that this person can transform its efforts.  If I’m that new person, why would I abandon my successful approach and adopt ideas from this team that has failed?  What could they have to teach me?

The problem is that the way you run an independent team for success, and the way you run a team inside of a big company, is quite different.  There is some overlap, but there is a lot of non-overlap.  As the new person, I have a lot to learn.  It’s very hard to figure out which traits I bring that are uniquely valuable and must be preserved, and which ones I have to supplement or replace in order to be effective in my new environment.

Wrap Up

I’m certainly not advocating that companies refuse to bring in new blood – it is crucial that they do.  But those people, especially if they are brought in to turn around a failing situation, have to negotiate a minefield.  The company needs to think hard about whether the problem it has is one that an outsider is really equipped to solve.  And it can’t expect that outsider to solve it alone – he or she is likely to need seasoned partners who can help translate that new perspective into a form that others can understand and act on.  Paladins seem like lone crusaders, but they wouldn’t have gotten very far without a host of supporters ranging from squires to armorers (not to mention the most important one – their horse!).  Before you try to hire one, make sure it’s what you really need … and don’t leave them unmounted and unsupported, or you are likely to have a bruised and grumpy knight on your hands pretty soon.

Filed Under: 1000x, 100x, Managing People Tagged With:
«Previous Page
Next Page»
%d bloggers like this: