Marcato

Agile is a set of principles to guide how you organize and function as teams and Scrum is a particular system for establishing and building a delivery rhythm within that team.  Part 2 examines 5 more examples of common changes made to Scrum and the reasons why these are counter to what the system is trying to accomplish.

6.  Set “stretch” goals by putting more stories in the sprint than fit within the velocity

To create a sustainable rhythm, you have to set goals for the team that can be met with regularity.  If it requires heroics to accomplish one sprint, then heroics become the norm for future sprints as your team velocity adjusts to account for this.  New teams often fall prey to this phenomena – their energy for the project and desire to make a good impression on their product owner gets the better of them.  They let the product owner put more stories into the sprint than is prudent because their uncertainty about their actual velocity opens the door for the usual optimism of a development team.  They try to shoehorn in just one more story because they are interested in the technology or feature and want to try to make it fit.

This kind of persistent inflation in velocity is unsustainable – like with any economy, real growth can only come from an increase in resources or in the productivity of the existing resources.  You will see this happen through the productivity that comes from climbing the learning curve, but that always levels out eventually.  What’s the downside?  In the short run, disappointments happen, and instead of giving the team the opportunity to establish and celebrate a pattern of successes, they are instead frustrated by a string of missed goals.  In the long run, left unchecked this behavior leads to burnout and turnover as you find yourself trapped in a “death march” of unsustainable expectations.

7.  Give credit for a story when it is not really done (shippable)

Yes, it feels good to show off your work in the sprint review and to claim victory, but with each passing event where “done” is not really done, your team has taken on a future debt that must be repaid.  These schedule bombs don’t go off until much later, long after the fuse is lit, and usually at the most embarrassing and inconvenient times.  Also, once you compromise on this a little it becomes a slippery slope of lowering expectations as individuals vie for credit for their work.  We all know that product owners can be fooled in a demo, so acceptance by the product owner is not really enough.  Instead, the scrummaster and testers should hold firm to the team’s definition of success and only agree to take credit when credit is due.

8.  Increase length of sprint so that more stories can get finished

It’s the day before the end of the sprint and you can see that there are a couple key stories that are not going to make it.  They are big stories, so without these your velocity will be less than half what you expected and the optimistic outlook from the team is that they could get them done if they only just had one or two more days.  Someone on the team suggests that it would be better to extend the sprint than to arbitrarily end it now when you are so close.

Usually this behavior indicates several forces are at work – stories are too big, too many stories are being worked on in parallel, and the team may not be doing enough detailed planning at the start of the sprint to have clarity on what will be built.  Since software is a gas that expands to fill 110% of the space provided, if you extend the sprint to let that last 10% in, then you will learn nothing from your mistakes and just do the same for the next sprint.  You also sacrifice your rhythm, these last minute changes in expectations confuse everyone as you try to figure out who has the authority to change the rhythm and who is responsible for communicating the new plan.  Leave the plan alone, examine in your retrospective what really went wrong, and fix it so it doesn’t happen again.

9.  Cut sprint short because all “bought” stories are complete

This is more of a “minor sin”, but interrupting the rhythm just confuses everyone – you are better off to find other small stories to add to the scope until the time is up than to break the rhythm.  Be careful not to start big stories that you can’t finish – this just leaves the product in an unstable state as you are trying to demonstrate it and threatens the principle that the product be shippable at the end of the sprint.

10.  Estimate during sprint planning meeting

This observation is controversial as there are certainly going to be situations where you are forced to do this by no apparent fault of your own, so if you find yourself in that situation, then by all means just do it.  But first understand the risk you are taking – you are estimating the story in isolation, perhaps by a different collection of people than normally estimate, and you are doing so in the context of already knowing you want it in the sprint and you just need to decide if it will fit.  This probably means you already know how big you need it to be, so you are anchored on this answer.  What happens in this situation?  The estimate comes in to magically fit and round out the sprint and the most common reality is that you underestimated its complexity.

I spent this week with a new client doing training on Agile and Scrum and wrapping up with workshops where we laid out specific plans and timing for rollout of Agile.  It was an exhausting but rewarding experience and it was fascinating to see another organization go through the early stages of the change process.

One aspect of teaching Agile and Scrum that makes it easy is that it is so hands on – the curriculum is designed to get people into teams and to learn how to problem solve, reach consensus, and communicate as a team – and this is part of why it is fun to learn.  The feedback (not surprisingly) is that people enjoy the team exercises and real world examples about what has worked and not worked in other situations.

Since I enjoy thinking about and studying organizational dynamics, one of my favorite parts of teaching Agile is to try to help people understand how the mechanics of the system are designed and why messing with those mechanics in simple ways can have interesting side effects and repercussions.  We teach the values of Agile before the mechanics of Scrum for exactly that reason – most of the explanations for "why?" something is handled the way it is derive from reinforcing the values behind the process.

While there were many enjoyable moments, my favorite was a group discussion we had around a Top 10 List of most common tweeks to Agile/Scrum.  The idea behind the exercise is to throw out things that they are probably already thinking about doing, but then to encourage them to think about what happens if they do them and what the consequences of changing the system might be.  This was probably one of the most interactive parts of the training as you could see the light bulb going on for many in the room.  Here’s what we discussed:

1.  Change the duration of the sprint in calendar days because of known changes to capacity

This is a "minor sin" but still not a good idea – I know that when you are getting cut short on resources during a heavy vacation period, it’s easy to want to stretch things out, but all this does is further disrupt the rhythm and leave people wondering what is going to happen and when.  Better to just drop your velocity a commensurate amount and stay on rhythm.

2.  “Split the difference” in story point estimating when the team gets stuck

Another "minor sin", but still indicative of a problem that can be better solved another way.  Usually this is happening to the larger point values (the jump from 20 to 40 is tough to swallow) and represents a sharp divide in opinion within the team as to how complex the story is because it is so large.  The best remedy is to break the story down into smaller stories, but if that fails you are probably better of using the higher point value.  I know that splitting the difference can help you feel like both parties are giving something up so it seems more fair, but odds are the worst case estimate is actually more accurate, especially if it is coming from the person that happens to know that part of the product well.

3.  Change estimating process because we are consistently under-estimating

This one is more of a "major sin" because it can have serious consequences.  The theory behind estimating is that it does not have to be accurate to work but it does have to be stable.  If the estimating team consciously decides to shift its bias in how it estimates depending on the outcome of the previous sprint, then velocity will become unstable.  If you study system design, you learn that delays in feedback loops always introduce instability into the system.  Far better to keep your estimating process stable and let your velocity change to account for the real variation in productivity.

4.  Give bugs story points

I would call this a "major sin" as well because it strikes right at the heart of Agile values.  Story points are a proxy for "value" – they are the credit or reward we give to a team for doing something that the customer values.  Do customers value bugs in the product?  Absolutely not – in fact, it might even be more accurate to give them negative story points instead of zero.  There is one exception I have made to this rule – when you are trying to get a new team to embrace improving the quality of a legacy product that they did not create, then it is hard to get them motivated to solve the problem unless you give them credit for doing the work.  However, as soon as bugs start to show up in new work they have done or caused by other bug fixes they are making, then you need to go back to giving them zero story points.

5.  Separate feature testing into a later sprint

This one is somewhere in the range of "major sin" to "mortal sin" yet is surprisingly common.  There are kinds of testing (such as the development of new automation testing or full regression testing) that truly are impractical to commit to in every sprint, but I think once a team starts down this slippery slope, they conclude that this should mean that all test activities have until the end of the next sprint to complete.

It’s hard to summarize in a few words why this is so harmful to the system, but my best suggestion is to read the book "The Goal" by Eli Goldratt if you really want to grok the risks to the system.  When you decouple software testing from development, then test activities regress from process control to inspection and you lose your ability to determine if development or testing is your process bottleneck.  Separating when developers and testers work on features also violates one of the most important Agile values – Individuals and Interactions.  If they work on the feature together, they will both be available and motivated to work through it as a team.  If you separate it in time, then the quality of the interaction will drop dramatically.

Stay tuned for 5 more!

We spent yesterday afternoon and evening dodging thunderstorms and tornadoes at the TCF Bank Stadium; while normally this would be entertainment enough, the real reason we were there was to watch the Drum Corps International competition.  If you are unfamiliar with this form of competition, check out this trailer:  http://link.brightcove.com/services/player/bcpid62043804001?bctid=95733275001.  A friend of ours competes throughout the summer in the Minnesota Brass – a remarkable assemblage of talented people that actually PAY for the privilege of spending their summer practicing complex routines in often (especially last night) less than ideal conditions.

This was our second competition we had attended, and this time we sprung for better seats – oddly enough, for sport of this nature, the better seats are actually higher up, and I was amazed at the difference.  Closer to the field you can see the details, but as the old saying goes, “you can’t see the forest through the trees”.  From this perspective, individuals are lost but the design and execution of the whole becomes clear.  The flash of a single cymbal move is lost, but the synchrony of the squad becomes clear.  Excellence in execution is required because errors break the pattern and distract the observer, but execution alone does not make you world class.  Instead, it now becomes a competition of choreography and concept where the constantly shifting patterns communicate a story and accomplish a purpose.

There are lots of elements in the performance to appreciate as there is so much going on – the music itself, the color guard with flags and twirling guns, the visual flash of color and  - but my mind kept observing a simple yet compelling part of how each routine is choreographed.  It seems that the architect of the routine knows that what what will have the most impact is to present a relatively small number of big moments where everything resolves into a sharp image for that “wow” moment – sometimes it is as simple as a single straight line slowly moving forward with all instruments holding a note in unison that sends a shiver down your spine, other times it is a design representative of the theme of the performance and part of telling the story.

But most of the time during the performance itself is actually spent moving between these moments, and during these times, individual execution matters even more.  The patterns dissolve into an almost chaotic scramble as different lines shift and merge; each person performs their work to get from here to there but in a way that cooperates with the others in their immediate vicinity.  This motion has a beauty of its own, but for the most part it is a building phase that we know must be tolerated as we set up the next impressive moment.  I’m sure this is intentional, but often during these transitions the performers are all turned away from you and the music is muted.  It’s messy and in some ways unsatisfying, yet usually worth the wait when the image suddenly comes into sharp focus and all the instruments turn towards you to appeal for your applause.

Now I know that I have a tendency to see everything in life as analogous to software, but there is an important parallel between what I saw on the field last night and what we do when we are using Agile to build software.  It’s very easy to get lost in the details of the execution of each sprint, and when you are down there on the field in the team, it takes everything you have just to get from point A to point B.  In fact, there is a REALLY important difference between what we do when we make software and what we see on the field – we don’t get to practice the same transitions over and over again until they approach perfection.  Instead, we get to do them once and to the best of our abilities, and then we move on to the next one.  Given this important difference, it’s nothing short of miraculous when we do have those sprints where everything comes together well and we have a great sprint review.  This is also why I think it is important to tolerate with grace those times when it doesn’t quite work.  As a coach, what I look for is an honest intent within the team to do better in the future and a honest and healthy spirit of learning from your mistakes.

One final note – in the end, great execution and choreography combine to please the audience, and this is the reason I see so many passionate converts to Agile on the business side.  Once they have experienced this level of involvement in the process and they see the product emerge as a progression of steps that are themselves pleasing to witness, they never want to do it any other way.  In that sense, perhaps with Agile we nerds and geeks have finally found a way to accomplish what we always wanted – to turn what we do into a form of performance art.

We are happy to welcome a new voice to the blog!  Riley Horan is an experienced IT project manager that specializes in Agile and appreciates its focus on the team as the central building block.  Riley and others of similar mindset have been writing about building better teams at http://buildbetterteams.blogspot.com.

Reprinted from:  http://buildbetterteams.blogspot.com/2009/08/where-is-butter.html

——————-

My daughters love butter. They can’t get enough of it. When we have pancakes they love it even more. The problem is though, if they can’t see it they don’t believe it is there.

When you put butter on a hot pancake it melts to the point that you can’t see it. If I bring the pancake to them, without viewable butter, they say “Where is the butter?”. I tell them it is there, but they don’t believe me. So I need to go back and put more butter on it to satisfy their need to “see it to believe it”.

The same thing is true when you are working with someone who doesn’t necessarily behave in a way you would expect or want.

Twice in the last few years I’ve come across circumstances where people I’ve worked with have been steady and even keeled and not subject to excitement or panic. But a perception has been that this person is not driving their team. Not building excitement and momentum.

When this is brought to my attention and I’m told they are not a driver, I ask “why”? What are you seeing or hearing that is making you think that? Because it seems like they are working well and getting their projects done. The response has been that this person is not a driver because they are not pushing their teams.

Ah…..”Where is the butter?”

It took me a while to figure this out when I was working with one person a few years back. We changed one thing. When he was talking in front of people who like action I told him to use stronger words and stronger tone. Result? First time he did this he received a voice mail from someone who thanked him how he was driving on the project and moving things forward.

Did he change any of his other behaviors? No. We did talk about using a slightly stronger choice of words and tone in certain circumstances, but overall his behaviors were the same.

Don’t get confused if at first you don’t see the butter. It may really may be there, you just can’t see it. If you are making a pancake, sometimes you just need to give people what they want to hear or see, and that will keep them satisfied.

Yesterday afternoon I was talking to a friend and client about our efforts over the last six months to build and ship a new product platform and to use that effort to inspire new customers to use our products and services and new investors to help fund us through to profitability.  The result of those efforts recently shipped to our production data center and are now in use by our current customers.  As he has been talking to investors, he is now using this proof point to demonstrate to them that we have an effective team in place that is capable of delivering on the overall vision we have for the product.  More importantly, he has been telling them that we set out six months ago to build this platform and now, after six months, we have shipped it on time.  The problem?  Nobody believes him.  You can see it in their eyes and in the questions they ask – “Nobody ships software on time.  You cheated.”

I:  ”So you say that after six months of development, you have shipped the product that you wanted on schedule.”

A:  ”Yes…well, not exactly.”

I:  ”Oh?”

A:  ”Well, to be more precise – we shipped the product that our customers need on schedule.”

I:  ”And it’s done?”

A:  ”Of course not – we just started.  Show me a software product that is done, and I’ll show you a product that was just turned off by it’s last customer.”

I:  ”OK – fair enough.  As long as you can fund the ongoing effort from your revenue stream and be profitable, I guess I get what I want.  But let me ask the question a different way.  You said it doesn’t have everything you want but what your customers need – does that mean you had to leave out features?”

A:  ”Yes.  There are things we said we wanted to accomplish when we started that were left out.  But we always left the product owner in control of those priorities, and to the degree that they were able to effectively represent our customers, we were always working on what our customers would say was most important.”

I:  ”Aha!  But that’s cheating – you didn’t actually do everything you said you would the way you said you would at the beginning!”

A:  ”No, we didn’t cheat, but yes, we compromised along the way.  To be fair, the business’ understanding of it’s product, position in the marketplace, customer needs, and financial position changed a lot along the way, too.  Staffing and priorities related to other product lines were in flux, and when we started we had not even chosen the two strategic partners we would have to integrate with to provide a significant part of the feature set.”

I:  ”So, it’s somebody else’s fault that you did not get everything done that you said would get done?  I’ve heard that story many times before.”

A:  ”I think you are looking at it wrong.  If your assumption is that everyone starts with a perfect understanding of what is needed and how to do it and all that is needed is to execute on that plan, then I suppose you could say that it is cheating.  But, if you instead assume that everyone involved is simply making the best decisions they can at the time based on the knowledge they have, then the actual path you take to get to the goal isn’t the result of cheating – it simple acknowledges that we are all always acting on imperfect information.”

I:  ”I suppose that’s true – but how can a business make good financial decisions in a situation like that?  If we don’t know what we are going to get, when we will get it, or what it will cost, then we can’t determine if the investment will be profitable.”

A:  ”Well, you can set some constraints on all of those factors so that you know roughly what the business can expect, but are you saying that even if we could somehow tell you all of those things, then you would be able to tell us exactly how much of that product you will be able to sell?”

I:  ”Hmm – not really, at least not exactly.  After all, we’re in a speculative business venture trying to do something that has not been done before, so while we can try to model how the market will behave and how the business will perform, we can’t really be sure until we do it.”

A:  ”So, what you are saying is that while you can’t list the names and dollar amounts of the people that will buy the product, you can make macro level estimates based on related products and industries and overall estimates of the market potential and then correct that model as needed based on what actually happens when you start to sell the product?”

I:  ”Yes – that we can do.”

A:  ”But that’s cheating by your own definition!  Or would you agree that as long as you get the sales you need when you need them, then the model holds together and you get the results you expect without having to try to predict or control the details?”

I:  ”I see what you are doing; don’t try to twist this around!  OK, I’ll concede that point, but you are engineers; you’re supposed to be smart and detail oriented.  Salespeople have to have some latitude for finding creative ways to make their quote – it’s the only way to retain the best.  I don’t want to micro-manage what they do – it’s just the result I care about.  Anyway, let’s switch to the other side of the coin.  You said you got it done on time – so that means everything went to plan?”

A:  ”Nope – there were lots of things that went wrong along the way, and we missed our goals by a couple of months.”

I:  ”There you go – cheating again!  How can you say you shipped on time when you told me you missed your goals?  You can’t have it both ways.”

A:  ”But you can.  There was always two plans – the public plan that we told our current and future customers and investors, and our internal plan.  Our internal goals were much more aggressive because we knew that there would be some unexpected changes along the way.  What matters was that we committed to incremental steps along the way and corrected our plan each time we discovered that it needed to change.”

I:  ”So, let me see if I have this straight.  You intentionally created an internal plan to ship in April so that you could meet your external plan of shipping in June.  The way I see it, you sandbagged by 50% and came nowhere close to meeting your goals.”

A:  ”Or, another way of looking at it is that we intentionally created urgency within the team to make sure we stayed focused only on the most critical features.  Look at it this way – let’s say you have an interview for a job you really want at noon on the other side of town.  You get a map on the Internet and it says you will need 24 minutes to get there.  Do you leave at 11:36?”

I:  ”Of course not.  You might hit traffic, and the map doesn’t account for the construction on Hwy. 36 where they have it down to one lane, plus it always takes and extra 10 minutes or so to find parking near the actual building when you can find that.  I’d probably leave around 11 just to be sure.”

A:  ”Seems reasonable, but you have to admit that you might get there early and have to wait for it to start.”

I:  ”True, but the consequences of being late are more traumatic.  Besides, if I get there early I can always get other work done while I wait.”

A:  ”The same thing applies to our software release.  What if I told you that some of the team finished early and have already started working on features for the next release?  Would you still say we were sandbagging?”

I:  ”It does help, but it still seems like you could just do whatever you want for the first couple of months and then use the last 4 months to actually build what we asked.”

A:  ”It probably seems that way, but what if I also told you that every two weeks we showed the team actual working software?  It seems hard to hide that way.  And what if I told you that every two weeks you get to decide what we will do next based on your current understanding of what is important to our customers and to the business?”

I:  ”Trust, but verify.  I guess I like that I at least get to know how my money is being spent.  I don’t want to have to watch over every little detail along the way, but I need to know I’m getting value for my money.”

A:  ”I wouldn’t want it any other way.  I know there are situations where the technical teams take advantage of the business, but that’s really hard to do with Agile.  In fact, if there is any risk, it is that the business so dominates the agenda that you do not make important but not urgent investments in upgrading technologies or staying true to your overall architectural plan.  You have to make sure you have leadership that understands how to deliver those things along the way while explaining to the business why they are important enough to invest in.”

I:  ”I feel like we’ve been talking in circles, but I at least feel like I understand what happened.  You didn’t ship what you wanted when you wanted, but you did ship what your business and your customers needed when you needed and the business had good visibility into and control over what was happening along the way.  I guess that’s not cheating – it’s just a practical approach given the situation, especially for an early stage company without a proven product or market.”

A:  ”Exactly.”

What happens when you wait too long to ship?  What happens when you go through a dreaded multi-month hardening phase?

We know the benefits of shipping early and often, but it is useful to revisit the pain that occurs when you don’t.

If you don’t ship often, your organization won’t develop the “muscle memory” of going through the ship cycle.  It will be a complete fire drill every time it happens.  When you finally do ship, the rapid response needed to satisfy your long-delayed customers won’t be natural for the org at all.  You’ll need that rapid response, though, because the long ship cycle virtually ensures that there has been a disconnect between customer expectations and the delivered code.

If you don’t ship often, the errors in the assumptions you’ve made have far longer to compound – leading to really big errors in the delivered product.  (All models/assumptions are approximations of the real world your software will live in – but when you ship often, you get more chances to correct.)

If you don’t ship often, you’ll likely talk yourself into maintaining multiple branches of source code that are extremely unwieldy to merge.  Merge hell will be frequent and de-stabilizing.

If you go through a multi-month hardening phase (because you haven’t created the testing infrastructure to avoid it), your organization will assume that is what is “always” required.  The cost of that hardening phase will weigh on project members in a way that causes everyone to grow increasingly conservative about making any future change at all – no matter how badly it’s needed.

Convinced yet? 

The longest any organization will comfortably wait for you to ship something is 4 months – ship at or more often than that rate and you’ll be a hero.  In the long run, ship rate wins out over scope every time – and even if that’s not universally true, the perception will be that you ship more product.

The easiest thing in the world is to "not ship" – we’ve never shipped a perfect product, as we’re always reminded when we decide to ship.  We all have fear, uncertainty and doubt about whether the product will meet the needs of our customers, so we find it easier to keep building to meet the needs of the mythical future market rather than find out with the customers we can have today just how close we are.  In the end, the best reason to ship is simple – because you said you would.