Posts Tagged ‘Complexity’

What ITIL and McDonalds has in common!

Comments Off on What ITIL and McDonalds has in common!

Processes and methodologies are great for some kinds of work. If you have a repeatable process that consistently yields a high quality product, you can emulate McDonalds’ approach and hire teens to be chefs. But at McDonalds you will not find the next Gordon Ramsey. People that really want to understand and master their craft will not achieve that in a repeatable industrial process.

Well documented processes are a great foundation for any part of a company when you working with a repeatable process or don’t have a dream team of smart, disciplined and attentive people whose work is more of an exploratory journey. A process helps people to align their collaboration and work at getting better at repeatable tasks. Processes are also good to be run from a checklist so that people don’t forget things, especially when they rarely or never do a specific task.

ITIL is the McDonalds process concept for governance of infrastructure and solutions. ITIL isn’t the process used by creative advertising agency that is looking for best and maximal business value. Software Development is more complex and isn’t a software factory that has a universal solution.

What software development organization need is a reflective improvement framework, with a wide range of tools to choose from. Some tools and practices are designed to help a larger group to collaborate whereas others just focus on quality aspects. There is no universal solutions for complex work, McDonalds approach might work for handling infrastructure and teens in a kitchen but handling complex change in a high tech work where nothing stays long enough to be normal isn’t a solutions.


Craftmanship, Industrialization and Knowledge-Based Work

Comments Off on Craftmanship, Industrialization and Knowledge-Based Work

FW TaylorUp until the nineteenth century, most people were engaged in agriculture; others learned a craft like carpentry. They were organized in small communities and worked together from sowing seeds to harvesting, milling, and baking. To be successful they needed to learn and master a specific craft that was passed from person to person, often within the same family. The real master of the community then decided to leave the comfort zone to explore things that no one dared to explore.  Most of the time, these exploratory ventures ended in failure. But sometimes, the adventurer would stumble upon something that opened new doors for them and eventually for future generations as well.

As late as 1870, 80% of Europe’s population worked in agriculture. In 2010, less than 0.8% of Europeans worked in farming.  This demonstrates an enormous productivity gain for farming. And then in the 18th and 19th centuries, a major change happened: manufacturing shifted from manual labor toward machine-based manufacturing. It started with the mechanization of the textile industries and the introduction of steam power. This also changed the organization of work, where people specialized in a particular phase of the work instead of getting involved in the whole process. By specializing in a specific time/phase-based work, Henry Ford was the first to increase productivity in car manufacturing by using assembly lines for mass production.

To make manufacturing as efficient as possible, managers were needed to oversee the whole process. It was during this era that the first management theories and business schools began teaching what Mary Parker Follett (1868-1993) termed the art of getting things done through people”.

 

Lesson Learned from Craftmanship & Industrialization

Craftsmanship is building on experience and can lead us in the right direction, but experience will only take us so far into uncharted territory. In these instances, we must take what we started with and rely on controlled methodologies and engineering tools. Engineering is the application of tools and methodologies to handle the unexplored.

 

Craftsmanship is critical to knowledge based work in terms of providing quality, whereas in industrialization, quantity was the primary measurement. Allan Cooper, author of “Running with the Inmates” and inventor of Visual Basic, said the following about craftsmanship:

 

Craftsmanship is all about quality – it’s all about getting it right, not to get it fast. It’s measured by quality, not speed. It’s a pure measurement, and a delightful one.”

Craftsmen do it over and over again until they get it right. In their training, they build things over and over so they get the experience they need to get it right.”, more info see.

 

I do not fully agree with Allan Cooper because good craftsmanship must strike a balance between quality and time/cost. This demonstrates the concept of personal competence but this also has a disadvantage.  I’d like to quote one of my favorite bloggers – Joel Spolsky – on his view on craftsmanship.  “Craftsmanship is, of course, incredibly expensive. The only way you can afford it is when you are developing software for a mass audience. Sorry, but internal HR applications developed at insurance companies are never going to reach this level of craftsmanship because there simply aren’t enough users to spread the extra cost out.”

 

A craftsman takes pride in his profession, his experience, and his tools. Because his performance is based on his personal competence, he prioritizes the mastery of his tools, upgrading them and improving his work methods in an evolutionary way. Craftsmen prioritize their continuous improvement and appreciate quality because they know that their product becomes more valuable when done the right way. They know that quick-fixes will rarely be successful in the long run, because that can require more work (and less profit to them) to remedy.

The most important lesson learned from industrialization was that deterministic goals and processes are better achieved by investing in structure capital (developing process / routines and establishing them). This especially applies to a process that is 10% design/analysis and 90% production. A changed requirement after the production phase has begun can be very expensive. Software developments of product and unique system aren’t like this design isn’t just a one thing you do before production is started.

For example, when building a bridge, you can’t consider adding a few new highway lanes when the bridge construction has started.  Complex software development isn’t a deterministic process and is therefore more of 50% design (where you figure out what to do and where to go) and 50% production. Following a plan is therefore not the ultimate solution it’s to be prepared and know you domain and tools so that new knowledge present itself you can take the opportunity.
 
 
Conclusions

Throughout history, success factors for work have changed and gone from individually mastered methods of efficiency and quality to a world where craftsmanship was learned and handed down from generation to generation. During the 18th and 19th centuries, the industrial revolution changed the key success factors into pre-defined processes that were mechanized, automated and as efficient as possible.

 

In a world that changes every day, differentiation and uniqueness come from transforming information into a product through creativity and knowledge. The most important considerations for knowledge-based companies are how good they are in getting the best information and then transforming that information into products or services, through their creativity and knowledge. But in a complex world where goals are uncertain and there is more of a need to explore possibilities, success is increasingly dependent on collaboration, creativity, and knowledge.


Organizing Software Development

Comments Off on Organizing Software Development

Organizing work is a combination of doing the right thing, in the right way in an optimal way. This is hard and what makes it even harder is that the right thing and the right way are constantly changing. Over the years I work with software development; first the traditional project management schools with their view that if you only know what you did, success would come from making good plans and them focusing on executing that plan as effective as possible. Then the agile and lean view became the dominated methodologies that was convinced that if people was just empower, self-organized, became more business oriented and changed to adaptive planning great things would happen.

 

What was the Problem with the Traditional Project Management School?
The traditional view of project management was focusing on administrating passive resources that needed to be managed are not a view that was optimal for knowledge based work. The problem with the traditional view of organizing software development was foremost the underlying view that we were doing something repeatable, if we only was experienced enough we would be as effective as any manufacturing industry. A factory has a deterministic process, where you can expect a certain outcome based on the input, this works when the variation within the process is quite small. But the problem with our craft is that we develop products based on knowledge and collaboration, and then the variations within that process comparing to factories or building buildings are much higher. Software development is therefore a non-deterministic process that actually has more to do with developing new products than building houses or producing something in a factory.

 

The Agile Problem!
The biggest trend in organization of software development the last years are probably transcend to more agile methodologies. My view of the agile manifest is that many people foremost reacted against bad leadership and management style that wasn’t adapted to the knowledge based work people performed. The underlying idea of just empower everyone and give them give them the right to self-organize, isn’t optimal in all situations. I believe that when people realized that the plan-driven view wasn’t working they all started to look on those who shouted most and earliest, (and with all respect to the old gentlemen’s that wrote the agile manifest) they were a little to overrepresented by the anarchists. Don’t miss understand me, I’m a pro agile methodology. I just felt that leadership somewhere was lost in the focus on new process and tools. The problem with scrum trainers is that they value the scrum process and tools over individuals and interactions.

 

The Future
The traditional view of project management and the agile view both have its pros and cons. In the future both worlds will need to connect better in order to get the best from both. I believe that software developments sometimes are a deterministic process where you only install an out of box system and just tweak some things, and sometime it’s more of an exploratory journey into the world complexity.

 

P.S.

I know all Scrum trainers are in London this week, on the Scrum Gathering 2011. Goto Jurgen Appelos sessions he seams to be the only one besides D Snowden that understand the basis of our work, complexity. I’m in London this week as well but unfortunately without attending the Scrum Gathering.

D.S.


Accept that you can’t predict the BEST solution.

Comments Off on Accept that you can’t predict the BEST solution.

“The ones that demand that you as a project manager shall know what behind next rock will probably appreciate a horoscope more than reality.”

If things keep changing, the best solution will not be the best solution in the end of the project, if it even will solve the problem. Instead an empirical process must be taken in complex situations.  You have to evaluate a lot of possible solution, discussing pros and cons with different kinds of designs. A complex problem have multiple solutions some are just better than others, but you can’t predict them. Some solutions are more efficient and more productive than others just like when you solving Rubric’s cube.

 


Accept Complexity and be Free of Guilt

Comments Off on Accept Complexity and be Free of Guilt

“You are in the undiscovered land now, no one expects you to know what behind the next rock!”

 

Realize that we’re all seeking simple truth where there aren’t any. By accepting that you work in a complex situation you let go of the presumption that you can predict things. If project steering groups demand predictability they will get a plan, but they will get a false sense of control because complexity can’t be predicted. This only created guilt for the project manager in charge and then down to the team, when people are expected to commit to a non-deterministic process.


The Benefits of Fast Iterations

2 Comments »

The foundation of Agile methodologies is to work in iterations. This has many benefits:

Benefit # 1: Code that is not used by stakeholders is an inventory cost. If you had a store selling clothes in all models but are not visible to customers, they would never get sold. Their value declines because soon they will be out of fashion.

It’s the same with code. If it isn’t used, you never get real feedback or get the chance to test it. The world around the system constantly changes and soon you will be working on new problems if the code isn’t used. When you move on to new features, this unused code becomes an inventory cost.

A ratio that is used in financial analysis is the stock turnover ratio. Stock turnover ratio = annual sales / average inventory. The average stock is calculated by adding the inventory value at the beginning and end of the year and dividing it by 2. For software development, you could say that the turnover ratio would be “features shipped per year” / “average number of features that are waiting for testing or deployment or not being used by users”.

Benefit # 2: Continuous integration and frequent deliveries generates feedback. Small and fast iterations also mean that the code must constantly be integrated with previous work and frequently delivered to users. This means that the code is tested during integration and also tested by users after it has been developed. If something goes wrong it is better to know about it at once. If you investigate a bug which is the result of work done 6 months ago, the root cause analysis will take some time. But if you get feedback on work done today or recently, you can easily find the root cause.

Benefit # 3: The complexity factor does not become large with small iterations. If you estimate doing two things that are about the same size, you think of the work as a having a linear relationship; that is, if I do twice as much it will take twice as long. If the first thing takes 8 hours and the second one takes 8 hours, you can do it in two days.

But if you have to do both of them in one day with the help of one friend, can it be done in just 8 hours? The answer depends on the dependencies. If both works have dependencies, you need to make sure both parts interact as planned so they don’t interfere or overlap one another.

  • If you have one thing to work with, you’re dealing with only one thing.
  • If you have two things, you deal with them, but you also have to deal with the way they interact and interfere with each other.
  • If you have three things, you have those three things, but at the same time you have three different sets of interaction between pairs of the three, plus you have to deal with the way all three things are combined which leads to seven aspects.
  • If you have four things, you have 15 aspects, and so on.

Therefore, if you have 10 things that are related and dependent on each other, doing them might be 1000 times more complex (or to be exact, 1023 aspects to deal with). The number of dependencies multiplied by the number of things to do result in the following formula: (Aspects to handle = 2 x “the things” – 1). What do these things consist of? It could be number of users’ stories, number of integrated components, number of departments involved, number of people in a discussion.

Benefit # 4: Business value comes from the possibility of adapting to change. When you start a project, the decision to start it comes from the possibility of reaching an expected business value. When the first version of the system is released, not all of the features are perfectly adapted to users’ needs. The system, therefore, doesn’t reach the expected business value.

The longer the system is used, the more the business learns what it takes for the system to reach expectations; however, because the releases are infrequent, the system falls short of its value. After a few years, the system shall have reached its minimum accepted value.

Traditional approach

When an Agile approach is used instead, the first release lacks a lot of expected features but because new versions are frequently released, the system doesn’t fall short of its value over time. Releases are delivered, because new knowledge is used to re-prioritize work. When the system is used for a period of time, expectations tend to rise.

Delivery of Business Value

The most important observation is what this does to the stock turnover ratio, since the delivered value of the system is based on the time the system is used. The delivered business value is the red area, because it creates value when it is used. So using a system early on shows that the value delivered is at least doubled when using the Agile approach.

Conclusion

When the minimum accepted value is reached, business people tend to think they need a whole new system; only this time the original requirement has changed. The priority now is to add features, and to change and adapt the system to new acquired knowledge, re-factoring and updating it with new technologies. In truth, what the business needs is not a completely new system with perfect requirement specifications. What the business needs is to continuously develop their system in new ways.


Noise in Complex Situations

1 Comment »

Noise is something within a specific context that interferes with the sending of signals or messages. When you’re communicating with someone and another person interferes by talking at the same time, it becomes difficult to have a discussion.

When the noise is greater than the sent signal, the sound of what I want to hear is obscured by the sound of what I don’t want to hear – noise. One great example on noise within communications was the following test by the American Psychological Association. Ken Schwaber and Mike Beedle used it in the cover of their book “Agile Software Development with Scrum”. The instruction was easy: “Don’t read the words in the Color Test. Just say out loud what color they’re printed in as fast as you can.”

Noise in Social Complex Situations

If you are like most people, this would be hard because you tend to like reading the text as “Red, Yellow, Green…” rather than reading the colors.

 

As Schwaber and Beedle showed in this example, noise has interfered with your perception. When you look at the words, you see both their color and meaning. When these two pieces of information are in conflict, you need to make a choice. This test showed that most people have learned that the meaning of a word is more important than the color of the word.

 

The interference effect suggests that you’re not always in complete control of what you pay attention to. When projects become more complex, the more conflicts between different learned simplifications systems you will experience. This leads you to make more unconscious decisions. Because you don’t understand the requirement or the technology or even the domain language, your old learned preferences are the ones making the decisions for you.

 


Predicting Complexity Leads to Guilt and Guilt Lead to Apathy

2 Comments »

Predicting Complexity Leads to Guilt

When you work as a project manager you are expected to be in control. To control and review things, you must be able to predict things. To be able to predict things you need a plan.

The most common planning and control tools are GANT and PERT diagrams. But for a PERT diagram to work, it needs to be accurate and must rest on the assumption that it’s predictable.

How is this possible in a complex and complicated world? These types of tools give the project manager a sense of control. With this tool the project manager can control the holy trinity of scope, cost and schedule. Every real project manager has full control of these parameters; otherwise, he would be a fake.

But what happens when the project manager starts to experience the noise? A feeling of being out of control takes over. The second effect is that he feels guilty and asks, “With the information and resources I had, did I do my best? I’m not in control and I should be. How and why did this project fail? Is there any way to save my reputation when this project fails?”


When doubt and guilt get hold of a project manager, he does not focus on making the best software; instead he focuses on regaining control. No process or methodology is a crystal ball where in you can look into the future, only because he made an initial plan.

It may sound like I dislike GANT and PERT diagrams. Not at all! They can be very useful, but if they are your sole foundations and tools for organizing and controlling your project, I’d say you could develop an ulcer!

Guilt may Lead to Apathy

When a project manager’s work is based on a reliable process that produces predictable results and the manager feels that he did everything by the book (which must be right), he starts to wonder if the team followed the process.

Maybe the developers didn’t follow the process or aren’t doing what they are supposes to.  This can lead guilt, doubt and blame to spread within the organization. When people feel they are trying hard and doing their best and still fail to live up to their own expectations, they will eventually stop trying to do their best.

If I am confronted with failure even when I do my best, it’s best to just mind my own business. Why care about things that I haven’t been assigned to work on, and get blamed for things failing?

When this happens, things could be overlooked. They fall through and apathy takes over. To control this situation, the project manager has micro manage everything. The traditionally defined software development process is broken. It’s not a predictable tool that leads to repeatable results.

Let’s stop fooling ourselves, software development isn’t civil engineering. We don’t have 10% construction that leads to a 90% predictable production. When projects are complex and noisy, no process can be defined in advance to obtain a repeatable result. The solutions must be an adaptive and empirical process control.

Predicting Complexity Leads to Guilt

When you work as a project manager you are expected to be in control. To control and review things, you must be able to predict things. To be able to predict things you need a plan.

The most common planning and control tools are GANT and PERT diagrams. But for a PERT diagram to work, it needs to be accurate and must rest on the assumption that it’s predictable.

How is this possible in a complex and complicated world? These types of tools give the project manager a sense of control. With this tool the project manager can control the holy trinity of scope, cost and schedule. Every real project manager has full control of these parameters; otherwise, he would be a fake.

But what happens when the project manager starts to experience the noise? A feeling of being out of control takes over. The second effect is that he feels guilty and asks, “With the information and resources I had, did I do my best? I’m not in control and I should be. How and why did this project fail? Is there any way to save my reputation when this project fails?”

When doubt and guilt get hold of a project manager, he does not focus on making the best software; instead he focuses on regaining control. No process or methodology is a crystal ball where in you can look into the future, only because he made an initial plan.

It may sound like I dislike GANT and PERT diagrams. Not at all! They can be very useful, but if they are your sole foundations and tools for organizing and controlling your project, I’d say you could develop an ulcer!

Guilt may Lead to Apathy

When a project manager’s work is based on a reliable process that produces predictable results and the manager feels that he did everything by the book (which must be right), he starts to wonder if the team followed the process.

Maybe the developers didn’t follow the process or aren’t doing what they are supposes to.  This can lead guilt, doubt and blame to spread within the organization. When people feel they are trying hard and doing their best and still fail to live up to their own expectations, they will eventually stop trying to do their best.

If I am confronted with failure even when I do my best, it’s best to just mind my own business. Why care about things that I haven’t been assigned to work on, and get blamed for things failing?

When this happens, things could be overlooked. They fall through and apathy takes over. To control this situation, the project manager has micro manage everything. The traditionally defined software development process is broken. It’s not a predictable tool that leads to repeatable results.

Let’s stop fooling ourselves, software development isn’t civil engineering. We don’t have 10% construction that leads to a 90% predictable production. When projects are complex and noisy, no process can be defined in advance to obtain a repeatable result. The solutions must be an adaptive and empirical process control.


Summary of the Manuscript

Comments Off on Summary of the Manuscript

The agile leader

Here is a small summary of what I think the book will be about:

This book will be about leadership and management of knowledge-based work.  More particularly, it was  about software development as a knowledge-based discipline. Agile methodologies in software development have implications for organizational agility. The shift to Agile methods signals a larger transformation in the workplace. This transition state is turbulent, marked by continuous change with no clear or easy solutions.

The transformation will result in a state of continuous change. Continuous improvements are indispensable in order to be competitive, whether you like it or not. Changes result in a feeling of being out of control, and when work is about making exploratory journeys you can’t feel comfortable. But you are not alone.

When you leave your comfort zone and undertake an endeavor, it’s natural to feel some hesitation and anxiety.  It’s not a personality flaw, it’s very human to want to maximize our ability to survive, both at sea and in software development projects.

Read the rest of this entry »