You are here: Home / 2012 / Archives for June 2012

Manage Your Firedrill Capacity

June 29, 2012 by Leave a Comment

Ah, the firedrill – that urgent project that your team has to scramble to get done by the deadline.  Everywhere I have been, they are an inescapable fact of life.  “We just got a key customer meeting and we’ve got to have that demo ready!”  “There is a VP review of the project next week and we have to nail it!”  “I need this analysis done tonight for the board meeting – they said we have to cover this compete angle!”  “Our customer’s site is down and we have to get them back up ASAP – drop everything!”

Some of this is inevitable – things come up, so you have to rise to the occasion and get it done.  But some teams seem to exist in perpetual firedrill – life is just an endless succession of crises and it feels like a treadmill; you never get the chance to move forward on the really important projects that could be game changers, because you are running as fast as you can to handle the constant stream of do-it-now projects that bombard you.

How much you can control this problem depends on the role.  If your job is to fight fires (literal or metaphorical), then maybe the firedrill is what you do.  But if your work is more project-oriented, my guess is that a lot of them could have been anticipated and avoided.  That’s been my experience.  Test it for yourself – keep track of the firedrills you are pulled into for a couple of weeks, and look over the list.  How many of them really came out of the blue and could not have been avoided?

The Magic Wand: Designing Good Systems

Systems and processes can make life miserable and are one of the things that people complain most about.  They can get rigid, bureaucratic, and generally suck the spirit and energy out of life.  BUT … they are the critical tool that can eliminate firedrills, if they are done right.  What I’ve found is that you must design them thoughtfully, hone them until they cause the minimum of friction, and throw them out when they are no longer adding value. You probably will benefit from a system when something is predictable, repeated, and complex.

  • Predictable: hey, it’s a new year and we have to get our plan landed – wow, who could have seen that coming?  Excuse me, but it’s been on the Gregorian calendar for around 430 years, so this really shouldn’t catch you by surprise.
  • Repeated: if you do something once, then designing a system may or may not pay off.  If you do it over and over again, you don’t want to lose the hard-won lessons of the past.  Figure out how to do it well and bake it into a system so that you don’t have to think about it.  We’d never be able to function as human beings if our bodies didn’t do this constantly – we couldn’t walk, talk, eat, read, or write.  We spend much of our early childhood evolving our neural systems to master the key activities needed for life – think about how helpless an infant is.  Reinventing the wheel every time you do something keeps your team in perpetual infancy.
  • Complex: if it’s trivial, you may (may!) not need to remember how to do it.  But as things get complex, you are wasting enormous amounts of time and operating very inefficiently if you don’t capture that knowledge into a system.

Firedrills are a great indicator that you need to do something – jotting down a line or two about each one will take you less than a minute a day and looking the list over will help you diagnose places where you have a missing system or a failing one.

Don’t Waste Your Firedrill Capacity

Every team has some capacity to absorb firedrills.  That capacity increases as people are more committed to the mission.  It increases with confidence in the team leadership – people figure that if the leaders say something is important, it probably is.  But if you burn through that capacity too often, or you waste it on things that are obviously just screw-ups in planning that could have been prevented, you’ll pay the price.  People will get grumpy, cynical, burned out .. and eventually will leave the team.

So manage that capacity.  Fill the tank by fostering trust and commitment.  Avoid burning it up on dumb things that you could have avoided with some thoughtful planning.  Come clean when you screwed up and the team has to pay for your mistake.  And then when you really do need them to commit heart and soul to pulling off something heroic, the team will be right there with you, ready to dig deep and gut it out.

Filed Under: 1000x, 100x, 10x, Rhythms of Business Tagged With:

Rubber Gloves, Not White Gloves

June 14, 2012 by Leave a Comment

I’ve always loved to be involved in making things happen, on the ground.  The detail of execution, the individual decisions that mount up to determine success or failure – I like to plunge my hands into the engine grease.  Hence, rubber gloves, not white gloves.

That enthusiasm hasn’t always been embraced by the environment I’m in – places where the focus was on Big Ideas™, not the minutia of execution.  In academia, research labs, and staff or management roles, I was often supposed to step back, see the bigger picture, and identify the key trends and leverage points.  I love to do that, but I find that big ideas often become irrelevant if you are disconnected from the details.  In studying many projects that both succeeded and failed, my conviction has only deepened that the combination is where the magic lies.

Combining Vision and Execution

A great example is Brunelleschi, the architect who designed the beautiful Duomo – the dome that sits atop the famous cathedral that dominates the skyline of Florence.

It took 140 years to complete the building; the dome had remained a puzzle for decades because nobody could figure out how to build it.  The solution called for rediscovering ideas that had been forgotten since the days of the Roman Empire, and new inventions that had never been tried before.

Brunelleschi solved the problem in 1419 and got the commission, dedicating most of his adult life to constructing the dome and other parts of the cathedral.  The dome took more than four million bricks (37,000 tons of material!), and with the lantern on top stood 375 feet high.  He got a couple of the first patents ever issued to protect his ideas for a new kind of hoisting machine and a river transport vessel.  Brunelleschi was intimately involved in every step of the construction, laid some of the bricks himself, and immersed himself in every detail (including the work schedule and diet of the people doing the construction).  If you are interested, check out the Wikipedia entry; Ross King did a great job telling the story in more detail in the book, Brunelleschi’s Dome.

Another interesting case study was the race to reach the South Pole first in 1911.   Two expeditions were involved.  One party, led by Robert Scott, was less focused on the details of planning and preparation.  The other, led by Roald Amundsen, was a great example of deep focus on every detail.  Amundsen insisted that every member of the expedition be an expert skier who had been practicing the sport since they were children.  Scott’s team mostly couldn’t ski.  Amundsen prepared navigation sheets so that they wouldn’t have to make complex calculations when they were exhausted.  Four out of five members of his final team were experienced navigators.  Scott had one, and used a technique that required more calculation.  Amundsen relied heavily on sled dogs and figured out how to keep them properly fed and cared for.  Scott’s team didn’t want to rely as much on dogs so they were dependent on horses, which turned out to be poorly adapted for the conditions.

Both teams were staffed with brave and seasoned explorers, but the results starkly illustrate the cost of poor focus on execution basics.  Amundsen got to the pole first and returned successfully without losing a man.  Scott, and the four companions who made it to the pole with him, all died.

Right Vision, Bad Execution

There are many, many examples where a compelling vision but poor execution leads to disaster.  It often happens even if the vision is a great and achievable one.

I thought it was very interesting to learn about the history of the Panama Canal in the excellent book by David McCullough called Path Between The Seas.  Basically, there was a competition to come up with the design for a canal that would connect the Atlantic and Pacific Ocean and avoid the laborious and dangerous trip around South America into the treacherous waters of Drake’s Passage.  A Frenchman, Ferdinand de Lesseps, had been the leading spirit behind the Suez Canal and came up with a hastily slapped together plan that chose a sub-optimal path and paid little heed to the enormous engineering challenges.  An American team very carefully and thoughtfully developed an alternative based on extensive surveys.  Lesseps, who was a charismatic, larger than life figure with little engineering knowledge, carried the day anyway and inspired investors to finance the effort.

An engineering group estimated it would cost $214 million; Lesseps cut that to $120 million with no real justification.  After eight years, 22,000 people had died (mostly of malaria and yellow fever – nobody realized that they were spread by mosquitos) and $235 million dollars had been spent, but the project was only partially done with the hardest issues unresolved.  The company went bankrupt.   The project was eventually taken over by the American government, after a fifteen year delay, and it took another $375 million and nine years to complete (but came in under the budget estimate!).  I think it’s a powerful example of a great vision, uncoupled to strong execution, leading to horrible results.  Then that same vision, carried forward in more pragmatic and expert hands, created a vital global resource.

Vision is Crucial .. but Must Not Become Decoupled From Reality

I’m a great believer in the importance of vision and strategy.  But strategy has to be deeply, deeply informed by all the infinite detail of execution.  And in looking at brilliantly successful strategies in technology, I’ve found that they are often emergent properties.  They were glimpsed, and then refined, by someone steeped in a deep understanding of their markets and the technology forces transforming them.  A kernel of insight evolves into a huge success – it doesn’t leap forth from the mind fully formed, like Athena from the forehead of Zeus.   Having a big and bold goal is fantastic .. but strategy is shaped, and success or failure is determined, by the execution.

Great execution is hard and requires endless focus and obsessiveness to achieve.  I’ve mostly worked on large software projects, and when you are building something that has millions of lines of code in it, the team must work together extraordinary well.  Such projects call for Herculean efforts and tiny bits of friction can lead to huge problems.  The code, the test suites, the shiproom triage meeting, the bug count, the milestone definition .. those are the clay and brick from which the great cathedrals are forged.  Lose your connection to the grit and the glue at your peril.

That’s why I love being in a startup .. we have a bold vision for what we hope to achieve, but our days are full of designing and building the product with endless and loving focus on every detail.  Hopefully we are achieving a combination of vision and execution that will let us have the impact we aspire to make in the lives of our users.

Filed Under: 1000x, 100x, 10x, 1x Tagged With: ,

As the leader of the forces opposing evil in the universe .. you’re fired!

June 7, 2012 by Leave a Comment

Epic tales are a lot of fun to read – the struggle of good and evil, the climactic moment of truth when the future of humanity or the world hangs in the balance.  They also can provide good insight on managing complex projects .. or rather, how not to.  There are some great lessons to be learned from the bungling incompetence of fictional heroes (and yes, this point of view can make you a real buzzkill, so you might want to keep these thoughts to yourself!).

Risk Mitigation and Contingency Planning

The worst sins of omission are basic risk analysis, mitigation, and contingency planning.  Very brittle plans are made, with no effort to figure out a Plan B if something goes wrong.  That makes for great drama, but it’s lousy planning.

Let’s start with Lord of the Rings.  It’s a great story that I have loved since I discovered The Hobbit as a ten year old – it has edge of your seat excitement with a richly detailed universe as the backdrop.  But come on .. what kind of a grab-ass plan was that for saving the world from evil?  We’ve got a group of clueless hobbits wandering to Bree with the Nazgûl charging around and almost catching them. The hobbits are supposed to hook up with Aragorn, but they don’t even know what he looks like – would it have killed Gandalf to give them a description?  How about an escort?  The great civilizations of Middle Earth are facing Armageddon, and they can’t scare up a couple of people to help out?

On Star Trek they are always beaming the ship’s top officers onto a potentially hostile and unknown planet, leaving the Enterprise with mostly junior people to run it.  What’s up with that?  They probably shouldn’t be sending most of the senior officers into harms way in the first place.  And, a Constitution class starship is a massive investment and a jewel in the crown of the Federation .. how come they don’t have enough seasoned officers on board to be fully covered even if four or five of the most senior ones insist on wandering into danger all the time?

In the wonderful fantasy series “The Dark is Rising”, they almost lose the ancient artifacts that determine the victory of good or evil because .. somebody left a note with a family and they happened to forget to deliver it.  Really?  Come on!  If that’s the best you can do, time for a new project manager who has a clue.  To drive a complex and crucial initiative, think about the aspects of your plan that are fragile and could easily go wrong, and build in defense in depth.

The Dark is Rising heroes not only made one dumb and potentially fatal arrangement, they keep doing it.  Don’t be like them.  If you see a particular breakdown, think about the underlying causes and address them.  Don’t settle for the obvious explanation – dig deeper.  Say you are running an online service and it went down .. why did it happen?  Well, there was a bug in the code.

  • Sure, but why did the bug slip through?  Do you need better tests?  More tests?  More realistic load testing?
  • Why did it happen in the first place?  Was there some communication breakdown?  Is the architecture of the system too baroque?  Are there missing levels of abstraction between system components?
  • Why was it hard to find and fix?  Do you need better diagnostics?  Better logging?  Better monitoring?
  • Why did it affect so many people?  Could you have a more loosely connected system?  Could components be more resilient when others fail, and degrade the user experience more gracefully?
  • Why were you down so long?  Does it take too long to deploy a fix?  Too much time to restart components that depend on it?

In running a project, you often have to make bets and take gambles – that’s part of the game.  However, you should think about the key bets you are making, and what will happen if your bet is wrong.  How quickly can you tell that you made a mistake (and be fairly sure about it)?  What will you do to reverse course or mitigate the failure?  Are you keeping anything in reserve so that you have some resources you can apply to help rescue the situation?  If you are making an unrecoverable bet, are you clear about that and about the due diligence you need to do up front?

Defining Roles

One of the common sources of confusion and inefficiency in a team is not knowing what role everyone is supposed to play.  Often, you can muddle along until something really important comes up, and then under stress the team works very poorly to resolve the issue.

Think about Boromir and Aragorn – after Gandalf fell into the cavern fighting the Balrog, they hadn’t resolved who was left in charge of the group.  Boromir deeply disagreed with the strategy Aragorn laid out.  Frodo decided he didn’t want to be with any of them any longer.  Since he was the ring bearer, ultimately it was his decision .. but nobody had given much thought to it.   It’s critical to figure out how the most important decisions are going to get made, and it’s a lot easier to do that before you are in the middle of a stressful situation with emotions running high (though hopefully you won’t be getting attacked by the Uruk-hai).

Thinking Out of the Box

It’s easy to get trapped into conventional thinking.  We’re all prone to unconscious assumptions – how things are supposed to be done, constraints we think we have to live with.  For example, Gandalf is close friends with the eagles, who can .. fly.  While carrying riders, and even outmaneuvering the fell beasts that the Nazgûl are riding later in the story.  So why is the Fellowship slogging their way through the mines of Moria and playing tag with terrifying ancient spiders, when they could get to Orodruin in a couple of hours?  Maybe with some Legolas-class archers along to provide suppressing fire in case anybody tries to interfere?  The whole thing could have been wrapped up and the hobbits tucked cozily back in their beds after a nice end of the day snack, before Sauron had a clue.  This idea is hilariously developed on “How it Should Have Ended”.

In life, it’s impossible to identify and question all your unconscious assumptions .. but you can tease out the most important ones.  Ask yourself what you are assuming, and whether you have the evidence to back it up.  Ask “what do I have to believe?” in order to justify a proposed course of action, and see if you can get some kind of backup that those things are really true.  Or a way to notice that they aren’t, so if you are on a delusional path, you’ll figure it out as quickly as possible.

Learn From Everything

I have found that there are great lessons to be learned about accomplishing your goals from every life experience.  Learn on the job, by all means, but I try to make everything grist for the mill – books, stories, movies, tales from history .. it all provides insight and inspiration to help you pick the goals that matter to you and to find ways to achieve them.  And before you decide that “actually, hope is the plan”, remember that in real life, you aren’t the main character.  There is no author who will turn your heedless folly into an exciting story of success against all odds.

Filed Under: 100x, 10x, 1x Tagged With: ,

Eyes of a Stranger

June 1, 2012 by Leave a Comment

I’m always looking for best practices to adapt and adopt, and I got an idea that I really like from a mentor.  It is a way to combat the complacency that sets in as we settle into any role – that tendency to become accustomed to the way things are, even when they are pretty screwed up.  “Well, of course you stand on one foot and tug on your left ear with your right hand .. that’s just how things are done here.”  With fresh eyes, we might ask “but, umm, isn’t that kind of stupid?”  And, “about the fire burning over there .. maybe throwing some water on it would be good?”

So periodically – maybe once a quarter, or once a year, the idea is to do an exercise I like to call “eyes of a stranger”.  Pretend that you just got your job and are figuring things out – you are in your “first 30 days” and are coming up to speed on the important things you need to focus on.  What is a top priority issue or opportunity that you would see and decide that you absolutely have to pursue?  What inefficiency would you discover that would just bug you until you got it fixed?  What joy-killer is afflicting the team (or you) that needs to get taken care of?

There’s a scene from a book that got stuck in my mind; it’s in Kon Tiki, the really fun story about an anthropologist who gathers a set of kindred spirits to prove that it is possible to sail a raft from the Peruvian coast to the Polynesian islands, using only the technology available in ancient Peru.  At the end of the book, they have crashed on a reef and the raft is smashed, they are pinned down and waves are pounding over them, and one of the people clinging to the remains of the boat says calmly “This won’t do.”  I try to apply that same calm but determined spirit to situations at work that feel desperate.  As you look at your job and the environment around you, what “won’t do” that you’ve gotten used to and have been letting slide?

I’ve found that you probably want to come out of the exercise with a very short list of things you are going to pursue more aggressively than you have been – one is good, three is probably an absolute max.  If you come up with a longer list, revisit it after you do something about the top ones.  I tend to find that you get wildly more bang for the buck by focusing on a couple of things (or one!) rather than dutifully writing down ten “priorities” and feeling overwhelmed so you just go back to ignoring them.

For each of the issues that you’ve picked, you need to figure out what concrete steps you can actually go take to deal with them.  I like to sit down with a piece of paper and do a mind map.  If the issues are worth addressing and you haven’t been doing it, it’s a good bet that you are a bit stuck in figuring out what needs to be done.  Maybe you just need to spend 30 minutes listing next steps or coming up with a plan.  Or, perhaps it will work better to pick somebody you have a good rapport with, and brainstorm about it together.  If it’s a really big issue, you might find it helpful to apply (some of) the framework that I outlined in the series on “Cracking the Nut”.

I’ve done this exercise over a dozen times, and each one has helped me get hard core about tackling something that needed doing and that wasn’t moving forward.  See if it works for you, too!

Filed Under: 1000x, 100x, 10x, 1x Tagged With: ,
%d bloggers like this: