Evolving Systems vs Design Consultants - A Recurring Pattern

I often think of systems architecture as analogous to this word game I played as a child. I don't know if the game has a name, but it is begun by selecting two words, say "cat" and "dog". The goal is to begin with one word, and end with the other. The rules are, you can change only one letter each turn, and at the end of every turn, you must be left with a true word. Hence, one way the game might play out is CAT -> COT -> COG -> DOG. You might also get there through CAT -> COT -> DOT -> DOG. Either path is valid, but there is no direct "upgrade" from CAT to DOG.

This is an apt analogy for the problem of systems architecture when dealing with an operational system. The constraints of the system's operation almost always prevent you from changing more than one component at a time. Every change to any component must result in a system that continues to operate. Real life systems also tend to have far more components than the three-letter word, in fact they comprise sentences, paragraphs, even whole novels.

In my work, I have occasionally had the good fortune to work with some great outside consultants. To date, I have always found these interactions to be productive and educational on multiple levels. It is a remarkable luxury to pick the brain of someone who is truly an expert in their field, and I try to take advantage of such opportunities whenever I can. In those interactions, I have noticed a curious recurring pattern.

Because of my role, I am often dealing with a consultant who is a systems designer. This expert comes in to help us improve the design of our systems. Unfortunately for her (or me), evolving operational systems tend to be more organically grown than designed, and the consultant must infer a design intent from examining the system as built, because the original design intent is lost in the mists of time.

Invariably, a conversation will occur that goes something like this.

"I see that you are using a COT in this part of the system," the consultant will say, attempting to hide a smirk. "A DOG would be much more appropriate. Why don't you try using a DOG?"

Of course, the consultant is being tactful here. No person in his right mind would use a COT as a replacement for a DOG. We, who built the system, are embarrassed even to be showing anyone this particular mangled part of our system. My response, when I have sufficient presence of mind to compose a rational one, always has a similar pattern.

"Well yes, ideally you want a DOG there, but when we were building this aspect of the system, we didn't have enough budget left for a pre-built DOG component. It would have taken us several months to build a custom DOG, which would have caused us to miss our launch deadline. But we had a well-tested CAT component we had built for a different system, and that mostly did the job. We found we could use that if we made some adjustments to the FOOD component to accommodate the CAT, and we could do that faster than building a whole new DOG."

Pause for a breath. Here's where the explanation gets messy. "After we launched, we wanted to come back and fix this to use a DOG, as originally designed, but of course we couldn't switch from a CAT to a DOG without changing the FOOD component again. Since we can only change one component at a time, during the upgrade process either the CAT or the DOG would get the wrong FOOD at some point, breaking the system." Remember that constraint about changing only one component at a time?

"We can't afford to break the system, we have live customers to support now." Here's that other constraint, every change must result in an operational system. Paying customers enforce that pretty strictly. It's hard to say you're lucky if you don't have paying customers, but sometimes it feels that way.

"So instead, we have migrated to using a COT. It's obviously not very efficient, but it fits, and it eliminates the dependency on the FOOD component (a COT does not eat). We're planning to replace the COT with a COG in a future release, which should be a smooth transition, and free up some system resources. Once that's done, we can use those resources to re-engineer the FOOD component to support a DOG, assuming management signs off on the additional cost."

By this time, depending on the consultant's level of experience, she will either be staring at me like I'm a lunatic, or shaking her head with a sympathetic grimace (usually the latter). In either case, the response is usually some variant of "I see." And the final report will advise, "Upgrade from COT to DOG ASAP."

Sigh.

There is no aspect of an organically grown system that could not be better designed in retrospect. The shape of the completed system is not governed solely by the appropriateness of the design/architecture. It is largely shaped by convenience, the accessibility of specific tools or components, the cost-benefit trade-offs and time constraints imposed externally on the design process.

The line between sense and nonsense is squiggly, because it must be drawn through the whole history of the system. And it's not always obvious which side of the line you are on.

Take heed, managers: your "best practices" are killing your company

If you are a manager, you need to understand the ideas of W. Edwards Deming. Deming wrote several books about management, in which he chastised American business schools and American corporate management for perpetuating a failed philosophy and failed management techniques.

Deming proposed a new philosophy of management motivated by quality and grounded in systems theory. The Deming philosophy is too deep, too broad, and too rich to be explained in a mere blog post. Volumes have been written about it, and as I read those volumes I am sharing my thoughts through this venue (with apologies to Mr. Deming if I misrepresent anything, I am still learning.)

Probably the best introduction to Deming and his theories is his Red Bead Experiment. The experiment is detailed in Chapter 7 of his book, The New Economics for Industry, Government, Education. The experiment is extremely educational, and I highly recommend you watch it play out in the video below (you’ll need about an hour).

Systems vs Habits: Why GTD Often Fails

In my previous post, I wrote about David Allen's Getting Things Done book and productivity system. If GTD has a weakness, it is that, although the book describes the system very well, it does a poor job of describing the change of daily habits you'll have to perform if you really want to implement the system. The major reason people fail at implementing a GTD-style productivity system in their lives is that, no matter how simple the system may be, it's a big change from what they are used to.

Leo Babauta is a self-made expert in changing and forming habits. His Zen Habits blog has changed the lives of many of its readers. So when I decided to try getting organized once again, there were two books on my reading list: David Allen's (the System), and Leo Babauta's Zen To Done, Leo's personal take on productivity.

Getting Things Done -- Productivity System

David Allen's Getting Things Done: The Art of Stress-Free Productivity is a phenomenon in the tech community. If you're reading this blog, you've probably already read the book, or at least know something about the productivity system that it defines. I read it years ago, but like many readers never put into practice more than a tiny portion of the system.

As 2012 drew to a close and I looked back on all the things I meant to accomplish, I decided that I should give this productivity bible another look, in the hopes of getting more things done in 2013. I won't bother to summarize the system that David Allen defines. The book is very readable and does a much better job than I could. Instead, I'm just going to note how I decided to apply the principles of his system in my own life, especially given the changes in technology and lifestyle since the book was originally published a dozen years ago.

How to set up a new PC in 12 Steps, or How I spent my evening renewing my disgust with Windows

Step 1: Spend 30 minutes unpacking boxes, peeling plastic, and connecting cables.

Step 2: In breathless anticipation, press the power button.

Step 3: Spend another 30 minutes hunting for the Windows Product Key so you can access the computer you just bought. Find it, finally, on an indelible sticker on the far side of the computer's case.

Step 4: Enlist an assistant to type the Windows Product Key while you hang upside down under the desk using a flashlight to read it out.

Step 5: Insert CD to install hardware drivers, because Windows does not know how to use the network card in your PC.  Try to convince Windows that you know what you are doing and yes, you really want to run that program from the CD.

Step 5b (optional): Wonder at how Windows has not only failed to improve, but has actually gotten worse in the 10 years since you last bought a PC.

Step 6: Using a clunky-looking "wizard" from the CD, attempt to connect to wireless network. Be unable to find your wireless access point in the list because you live in a crowded apartment building, and the list is sorted randomly rather than by signal strength or even alphabetically. Notice that the list has multiple pages, and advance to page two. There it is.

Step 7: Enter password for wireless access point. Curse in frustration when it fails to connect. Blush with embarrassment when you realize CAPSLOCK is on. Turn CAPSLOCK off and try again.

Step 7b (optional): Curse the inventor of the CAPSLOCK key.

Step 8: Start Internet Explorer. Type "google.com/chrome" into the location bar to download a real browser. Try to convince Windows that you know what you are doing and yes, you really want to run that program.

Step 9: Sign into Google Chrome. All extensions and bookmarks are automatically synced. Awesome.

Step 10: Using Google Chrome, visit www.ubuntu.com and download the Windows Installer to install a real operating system. Try to convince Windows that you know what you are doing and yes, you really want to run that program.

Step 11: Let the installer reboot into Linux. Be amazed at how all the hardware is recognized immediately, including the wireless card. Feel like Ubuntu just gave you a warm hug when the wireless network manager pops up on the screen and offers to connect you to your very own wireless access point if you will be so kind as to enter the password. Check to ensure CAPSLOCK is off. Enter password.

Step 12: Click "Install Updates" when the update manager offers to do so. Wait.

Step 12a (optional): Write a blog post about your experience while waiting for updates to download. Feel sorry for people who have not yet discovered Linux.

A Framework for Innovation

How does a large company create an environment that encourages and leverages internal innovation? Here is my checklist of prerequisites for "enterprise" innovation:

Great people. You may think this goes without saying, but it cannot be emphasized enough. You cannot hire drones who put in 8 hours for a paycheck and then head out the door. You need passionate, creative people, people who love their work, people who are impatient with "getting by" and want to be the best at what they do. These are the Innovators. Without them, innovation does not happen.

A clear vision. Innovation happens at the edges. It is not a top-down directed process, it is an organic, bottom-up growth. In order for the innovators at the edge to produce innovations that are relavent to the business, top management must articulate and communicate a clear vision for the direction of the company. If the innovators can see the direction, they will innovate in that direction and get you there faster. If not, they will innovate in random directions, and you won't get the full benefit of innovation. A clear vision is the difference between innovation and distraction.

Spare capacity. Innovation is experimentation. Innovators need time to experiment, and they won't have that if 100% of their time is allocated to executing your current plan. This is the hardest thing for top managers to accept, but it is absolutely essential. You need slack time, or there simply will not be any innovation. Allocate one slice of your capacity for executing the plan. Reserve a second slice for unplanned work and process improvements. Allocate a third slice explicitly to innovation. The relative size of the slices will be entirely dependent on your own business and your desired outcome. My personal preference is 50/30/20.

Freedom to make decisions. Innovators by definition have to make decisions, make changes, form partnerships, and allocate resources from the pool of spare capacity. If permission is required to accomplish these things, then innovation will be quashed before it can succeed.

Accessible Business Intelligence. If you are going to give innovators permission to make decisions, you must give them the information and tools they need to fuel decision making. Innovators need transparent access to customer data, product data, sales data, cost data. Without it, they are shooting in the dark, and the chances of success are low. Innovators also need easy access to tools for gathering their own data, for evaluating experiments and measuring success vs. failure.

Freedom to fail. Innovation is experimentation, and experiments, by their nature, do not always have the expected outcome. When Innovators exercise their power to make decisions, some of the decisions will be wrong ones. Innovators need to feel secure that they will not be punished for taking a chance if it doesn't work out. Remember, these folks are corporate employees, not risk-taking entrepreneurs. They don't stand to make millions if their innovation succeeds, so they shouldn't have to give up their health coverage and pension if it doesn't. Make it clear that failure is a learning opportunity, not a firing offense.

The above are a few requirements for fostering innovation in large companies. Ultimately, innovation only happens where the culture supports it. Managers at all levels build company culture through their hiring and firing practices first, and management styles second. If your managers fear the new and different, your culture will never innovate. Ensuring the above factors at all levels of the organization should help to unchain your hidden innovation potential.

What's missing from this list? How does your company encourage (or discourage) innovation? Drop me a note in the comments.

Systems and Mental Deficiencies

I was surprised when I read some of the things writer Terry Pratchett wrote or said about developing PCA, a form of dementia. I cannot now find the original source that I read, but there are several similar articles. He described some symptoms of the disease slowly robbing him of his own mind. The inability to see certain objects when they are right in front of you. Walking into a room but having no memory of why you went there in the first place. Difficulty comprehending written text despite recognizing every letter and word. Difficulty recognizing people's faces.

I was surprised when I read this, because these "symptoms" have affected me, well, pretty much my whole life. I thought they were normal.

Since I was a child, I have had these problems. Until I have met people many times, I may have difficulty recognizing or remembering them. Sometimes I put something down, and then simply cannot find it again, despite the fact that neither I nor the object have moved at all. I stopped reading books on paper years ago because I just couldn't manage to read one through; I would get distracted in the middle of a paragraph and forget what I was reading. And as for walking into a room and forgetting why you're there? Hardly a day passes without such an event in my life. Sometimes I return to the room two or three times before I manage to complete the task I set out to do. Sometimes I never remember what I was planning to do.

Part of my obsession with systems is the result of this bizarre array of mental quirks that I have slowly realized are not entirely "normal" (whatever that means). Systems are a simple set of rules that I can keep in my head. Lapses in memory become less important when the system is in operation.

Ever lost your car keys? It's bad enough when you can't remember where you left them. It's doubly bad when you can be staring right at them and not see them. (My family calls this quirk "object-blindness" and it drives them crazy. "Why did you put away every dish in the kitchen except that one?!" I didn't see it!)

I no longer have to remember where I left my car keys; the rule says that keys are by the door. If I forget where I left them, I can remember the rule. If I notice them sitting somewhere else (which is rare), I move them into compliance with the rule. When I'm about to put them down, I remember the rule, and I put them by the door. One consistent rule to be applied in all situations. A rule that doesn't even have to be remembered, because it can be derived again and again. (What's the best place to leave the keys? What's the first place I could put them down after entering the house?)

I live my life by rules like this. I have a system for everything I do. A system for packing my laptop bag to be sure I don't forget anything. A system for loading the dishwasher. A system for making breakfast. It sounds ridiculous, but I have rules for all these things so that I don't have to rely on my memory to get them right.

The most important element of my life systems is an attitude I learned only in the most recent quarter of my life, a skill Buddhists describe as mindfulness. Mindfulness means being fully present, in the moment. It releases you from remembering the past or worrying about the future and focuses your attention on the present, the here and the now. Mindfulness is what enables me to obey the rules now, as I perform the actions which later may confound me. It is what allows me to think when I put my keys down and observe the rule of where to place them.

By knowingly placing my keys by the door where they belong, I ensure that later I can find them again. By paying attention to my actions in the present, I ensure success in the future.