Nov 23, 2014

Estimates or #noestimates

Everyone working is Software industry have been involved in estimates in a way or another. Most of the people have known for ages that the one thing that's sure for estimates is that those are wrong. More complex the software or organization creating the software, more difficult it has been to estimate how long creating software takes.

There's been buzzing around for some time the #noestimates movement. The basics of it, as I understand, is that software teams should get rid of making estimates, since those do more harm than good. But I think there goes so much discussion under #noestimates, that I'm not fully sure what all it contains.

I've seen estimates to do lot of harm in many organizations. Estimates have caused poor quality, unhappy people and lot of unnecessary waste on creating those and reacting to those. But then on the other hand, I couldn't see organizations to live without any estimates on software.

Value of estimating versus the time used to it


Value of the estimates versus the time used on creating the estimates is one of the key questions on estimating. Too often estimating takes lot of time. People use hours or easily even days to figure out how much time creating some software takes.

I guess it's not valuable to discuss too much about what's wrong with estimating. People are known to be terrible at estimating. One excellent source to understand more is Nobel prize winner Daniel Kahneman's Thinking, Fast and Slow which introduces at least WYSIATI, anchoring effect and lots of other reasons we can't estimate. So I'm not going to use more time on arguing about the subject, but use is as a given fact that people are bad at estimating.

So what is the reason estimates still exist. Why estimates are still done, even though those are always thought to be wrong. I believe estimating goes to the same category as planning. As Eisenhower said: "Plans are worthless, but planning is everything", there's something similar with estimates. Estimates as such are not accurate, but the journey is valuable and some figures are always needed.

Sometimes the time spent doing the estimates doesn't match the value that estimates can give. Estimates should be used as a tools for decision making. Most often the decision is that should something be tried to do or not. The decision should be about the start, not about the completion. That's what Agile and iterative thinking should help on, there should come multiple decision making points and clarifications about the completion when knowledge increases during the development.

Companies can't live without estimates


Liked it or not, companies can't live without estimates. Most of the companies have limited resources in developing software (and everything else too). Those few who currently seem to have unlimited resources, still need to serve their customers quite often to keep their positions.

In real life, there are customer commitments and internal commitments that need to be handled somehow. Most of these are based on estimates, things that are most probably not accurate. Still those don't change the fact that sometimes these commitments need to be given. Think about yourself, would you be willing to purchase a house building project, if your contractor would say that we don't really know how much it will cost, when it will be ready and what it will contain. Of course you want to get an estimate, or even a fixed price with fixed timeline and features.

Many companies actually are in estimating business. They estimate the costs of development, they estimate their sales, they estimate the timelines and then take known risk with price and profit margin. This is how most of businesses still work. The luxury of making fixed profit is with fixed hour rate businesses. So basically some of consultants, lawyers and couple of others. There the risk is in not getting clients and not in making bad estimates.

Other than actual businesses relying on estimates, many companies have internal dependencies to SW development estimates. Product marketing for example, can't wait for product to be finished before marketing activities can start. They might need easily few months head start to get all the activities ready when product would be ready to ship. There easily are tens of these dependencies inside companies to the software team estimates: sales , marketing, documentation, hardware creation, operations, training, partners and so many others.

Software estimates, as much as software teams hate those, and as much those are wrong are mandatory part of all businesses. Those of you who still hesitate, think about Lean value stream map. For customer, software is just a small part of product. Even a plain software product. I can't even invent a software that wouldn't need other activities to go parallel to software creation to shorten the full cycle.

What's the solution then


Estimates are problematic. Sometimes estimates cause waste, sometimes lack of estimates cause waste. When sales uses SW development estimates as promises, those can create business, but at the same time those might cause quality issues, stress and unhappy people. So what could be done to estimating, when to do estimates and how.

I can reveal already now, that I'm not a believer of black or white solutions, so I don't think there is a one solution for all. But here's couple of thoughts how situation with estimating could be improved.

Estimated targets and predictions


One possible solution is to go towards having targets from early estimates and then predictions separately. At least when estimates are not in the team level, but in the project or feature level, it would be beneficial to have a target and then constantly update the predictions based on progress. Both of these figures should be always visible and that would give to everyone else idea what is the real situation.

In this approach there are many potential problems also. People tend to keep the first estimated targets as the goals and then predictions as a follow-up of the situation. This might give the feeling of failure, when the first estimates will most often be wrong. Still this approach should give everyone the idea, that decisions are made with something that will most probably be wrong.

Software aided and fact based estimating


As discussed, people are terrible at estimating. In a way it's easy to say that computers can't be worse. Saying this, I've never seen a good software that could do this properly. There are quite many variables that needs to be taken in to account to have a proper estimate. Still I believe that it would be possible to create framework that would give rough estimates of projects or features.

This framework should benefit from the historical data that has been stored to many software tools that has been used in the organizations. As an input, the framework would need to get software components that require changes. Then based on throughput times in past and historical progress of projects estimating software could give an idea how long something will take. Even thought these estimates would be quite rough ones, still it would give valuable data for human made estimating. And I believe it still would be closer to the truth than many other estimates. As said, unfortunately I haven't seen system like this yet in place. I would be surprised if there wouldn't be any software projects ongoing in this field. I just haven't bumped in to those yet.

It's ready when it's ready


There are few companies who have been good on preventing the harm what bad estimates have on markets. Apple is the prime example of this. I'm hundred percent sure, that even the guys in apple are bad at estimating. They concentrate on the product quality, so they can't guess when they are ready. They have taken the approach on telling when things are ready when those are ready. In Apple case it works for their benefit, but many other companies need to be able to have some timelines beforehand.

Still the approach it's ready when it's ready has something to learn from. It starts with understanding, that estimating software project length is difficult. When this is realized, then companies should build their activities based on this. Sales shouldn't be selling exact dates too early, handovers and dependencies between different teams should be minimized and people shouldn't be rewarded or punished because the estimates they have given.

Software, even the minimum viable product, won't be ready before it's ready. There isn't any amount of estimating activities before hand which can tell for sure when the ready will be. Activities in companies need to be able to react to each others timelines and not work with fixed timelines.

Summary


Estimating is a key aspect of business in software business (and basically in any other business). There are too many aspects and ideas to improve estimating for a one blog post. Some people believe in time boxed development, others think Minimum Viable Product will help and some believe tools will be the thing that will fix the issues. I don't believe there is one solution to fix it. I see the key thing to be that people start to understand that estimating is really hard and people are bad at it. Most businesses need estimates and can't live without those.

As a feedback to many of Agile and Lean discussion ongoing in different forums, I have to say that too many people forget the business behind software. It's so easy to talk about no estimates thinking, Agile practices or Lean values without understanding the effects of it to the businesses. Software is a key part of many businesses, but it won't make money without the other activities in the companies.

Everyone involved in the estimating processes should acknowledge what is the decision estimates are used for. Minimum Viable Estimate could be a good guidance for everyone. What is the minimum level of estimate we need to know in order to make the decision in hand. And as reminder to everyone, estimates don't get better with estimating, those get better with implementing.


Written by +Henri Hämäläinen

Nov 19, 2014

Book Review: End This Depression Now! by Paul Krugman

Wow, wow, wow. That was the feeling I got from reading the book. Nobel Prize winner from Economics Paul Krugman, gives in the book his view what should be done to End This Depression Now! His view is quite contradictory to the politics that have been ongoing. He rates himself to be New Keynesian and he brings many views from old Keynesian macroeconomics.

First of all, as the best influencers, he knows his stuff well and has the ability to explain those entertainingly and easily. Book about macroeconomics and recession doesn't sound like the most interesting topic, but I wasn't bored even once with this book. He uses examples well and still backs up his stories with hard cold facts.

I don't even think that I would be capable of arguing against any of the ideas he presents in the book. His main idea is that markets do not work perfectly and increasing government spending would help economy (at least US) out from depression.  He brings lot of historical and scientific proof that solution could be so simple.

Other than recession, book is an excellent source to get more understanding about macroeconomics. He is such a good to explain things, that I at least had couple of aha! moments reading the book. It increased my understanding of macroeconomics.

Book is concentrates to US, but is has few chapters to Europe also. I admit being quite supportive for our common currency Euro, but Krugman was able to explain what has been so risky on it. Also he explains why Euro has been one of the main reasons for the recession in Europe and also in Finland where I live. Still he doesn't recommend of getting rid of Euro, but believes that bit higher inflation and few different moves from European Central Bank should do differently to get us out from recession.

I'm so glad I read this book. Few previous books I've read have been good, but this was an eye-opener. These are the feeling you get with best books. You feel that your thinking has changed after reading it. I highly recommend the book to everyone. It's essential book about essential subject.

Written by +Henri Hämäläinen

Nov 16, 2014

Peer training inside organizations can be very effective

In one organization I was working with, there was a desire to improve overall coding skills. The guys had used some external training sessions, but as most often those were in such a general level, that those didn't give much value to the environment they were working in. We started to think about solutions for the problem.

Team had couple of people who were thought to be gurus, who could easily train the others, but as always, the gurus were also the busiest guys in the organization. Creating a proper training takes lot of time and effort. Anyone who have ever created tailored training sessions, know that it takes at least 5-10 times more time to create the training than keeping it. If the training is for 2 hours, you easily need 10 hours of work on creating it.

So the answer didn't lay on the gurus. We also did have a look for external tailored training, but there wasn't much of a budget prepared for this. And also, the external trainers are never as effective as internal ones. So we decided to try peer training. We decided that everyone in the team needs to keep one session to everyone. At that point, we came up with an idea of taking few valued books of the subject and divided chapters from the book to be the subject areas people had to make their training sessions.

In addition there was created a good template for the training. Template made people to explain the basic theory from the book and then it was made mandatory to bring examples from their own production code as a reference learning for the training.

Exercise was successful. Of course people were hesitant in the beginning, but the feedback after the sessions was good. One of the best ways to learn is to teach. When you have to teach others, you have to know the subject much better than you will ever learn from any training sessions.

This example was from actual coding, but in can easily be taken in to use in different professions. I believe the ability to take learning from a valued book and consider and show the examples from organizations daily life is really beneficial. Experienced external trainers are excellent in keeping the atmosphere and proofing points, but they rarely know enough about the daily life to actually be able to make a change to stick.

I have experience in doing tailored training sessions. I've kept close to 50 sessions during change projects inside companies. Even though I normally know quite much about the companies, the challenges people have and the practicalities, still I've seen that my teachings are much more effective when there is internal experts backing me up.

I highly recommend this practice to all teams. I can guarantee it works. It might work differently than you think, but it will definitely make people to learn and find ways to improve.

Written by +Henri Hämäläinen

Nov 4, 2014

Agile State of Mind has been lost in the journey towards Agile

First statement in Agile manifesto is “Individuals and interactions over processes and tools”. When people talk about Agile SW development, much of the talk is about processes and tools. If they are not talking about Scrum or Kanban, then it’s about code reviews or Test Driven Development. Too often the basis for all of this is forgotten.

I think there has been a clear vision of all of this. Back in the years that this movement started, Agile Manifesto was written with careful thinking. Somehow the message got lost in the way. Currently there are so many different methodologies and processes, that have been introduced under the Agile umbrella, that what really is Agile is unclear.

In team sports, there’s always ongoing trends on playing tactics. In football, at the time Barcelona and Spain was dominating, many teams wanted to play like them. What often went wrong was that teams and especially the players didn’t seem to understand the ideology behind the tactics well enough. They were just mimicking the ways others play, but didn’t really understood the reasoning of why to play like that. This lead to the problem, that in a second that there came a problem which didn’t have ready made solution, players didn’t know how to act. This was all because they didn’t understand the reasoning why they had done some of the things in the field like they had.

I believe this same is happening with Agile. People are happy and productive when everything is going like the books have told. At the moment there comes something unexpected, people don’t understand the logic of making choices. Making choices in SW development is never easy. For example in Backlog creation, there needs to be done many guesses and estimates on items when prioritising it. Creating user story items or what ever we call them, is never exact science and always needs judgement from the person in charge.

Focus in many organizations should go towards how to successfully create SW products with Agile State of Mind. That’s at the end what SW development is all about, creating SW for customers. Being Agile in SW development means, that focus should be on creating more value to the customer with improved ways of working. Not just to improve the ways of working.

Written by +Henri Hämäläinen

Oct 26, 2014

Book Review: Implementing Beyond Budgeting by Bjarte Bogsnes

Budgeting has been a subject I've wondered for a long time. I do understand how budgeting works, but I've never really understood why it is there. Implementing Beyond Budgeting by Bjarte Bogsnes didn't explain to me why budgeting should be done, but it told, why it shouldn't be done.

This is essential book for any modern organization. Traditional budgeting will slowly fade away and there will come new ways to lead companies and handle costs. Bjarte says that he might be too hard on some aspects of traditional budgeting and make some things too black and white. I didn't really feel like so. I believe budgets make so much harm in organizations, that Bjarte could have been even more strict on the effects of those.

This book isn't a how to do Beyond Budgeting guide, but a story about the underlying principles and two journeys towards the ideology. Stories and ideas will give a good guidance how to start implementation, but there is definitely lot of thinking needed.

What I love in the book is that tells realistically about the change. Implementing change needs lots of discussions, persuasion and time. Creating a new process or guidelines doesn't really change anything. The mindset behind actions needs to change. Bjarte tells about two organizations that have been on this journey. He admits that other journey was still ongoing as the book was written. The change started many years before, but it was still continuing.

I highly recommend this book to everyone interested on improving the companies they are working at. This topic needs much more attention. There are some valid reasons behind budgets, but most of the reasons are just barriers for creating value to customers or constrains making people unhappy at work or pure waste. Changing traditional budgeting is important topic and this book was a good peek to the topic.

Written by +Henri Hämäläinen