Marcato

I know it has been a long time since any news has leaked out about Marcato – in short, this is because both Scott and I have joined a company we helped start called LiquidSpace.  We have been busy building an initial product, getting funded (woo hoo!) and now starting down the path of maturing the business and product into a real company.  It has been exhausting but a real blast, and part of what we have learned is that there is a wide range of options available to you for deciding where you are going to work, and one LiquidSpace is a practical reality you may even find yourself making that decision on a daily basis.  As part of exploring that workstyle, we have also decided to try offering space for others to use.

If you have never heard of coworking, then let me give you a quick introduction.  There are actually a few existing coworking sites in the Twin Cities area today, but this is still a relatively new phenomena here.  If you were to look at this in San Francisco or Boston or New York (and many other denser urban areas around the world), you would see a growing trend toward providing temporary space to people that sign up for a variety of different membership plans and costs.  The idea is to share the space and use it as needed on an intermittent basis – sort of a cooperative for workspace – and to provide the needed amenities to support your work habits as well as a social outlet that fits with your interests.

LiquidSpace is taking this a step further and providing a mobile and online service to allow you to find and book a cool space nearby that is available now.  As part of this, we at Marcato have decided to use LiquidSpace to allow anyone in the North Metro area the opportunity to see and use our workspace.  We’re still in the process of getting the finishing touches in place, but if you have an interest in trying it out, be sure to let us know and we’ll get you hooked up when the time comes.

See you in Anoka!

You may recall that I have come out in this forum against the traditional XP practice known as Pair Programming (http://marcatopartners.com/2009/09/23/spoiler-alert-im-not-sure-i-believe-in-pair-programming/) – while my fundamental objections to some of the specific parts of the practice stand, I also wanted to tell you about a much more successful experience I have had of late that might seem like Pair Programming.  Perhaps together we can figure out what is working, why it is so productive, and what I have learned that you might be able to copy in your own worklife.  Since I’m not sure what this is or what to call it, for the sake of discussion, let’s just refer to it as “Pear Programming”.

Conservatory at Como Zoo

First, let me explain the situation – I was working with a VERY distributed team; I’m in Minneapolis, MN, we had a product owner and developer in Denver, CO, another developer in Houston, TX, and a team of 3 developers and a tester in Minsk, Belarus.  The very nature of our physical reality forced us to depend on the “virtual lifestyle” – I used Skype IM and voice and GoToMeeting constantly throughout the day to stay in touch with my team mates.  On most days I start at home because that helps increase the overlap I get with the team in Belarus, but I would often transition later in the day to the office or some other location and would work anywhere with bandwidth, a headset, and enough privacy that I would not be bothering others.

I also assumed development responsibility for a portion of the project (yeah, yeah – I know – I’ll save the topic of balancing scrummaster and coding responsibilities for another day) and started working on a new application with a UI and Windows service and with a significant interface to a collection of web services being developed by my team mate in Houston.

As you might expect, our initial efforts were isolated, but after we each had established the basic functionality of our respective pieces, the effort shifted to integration.  Now, there were a lot of technical challenges to overcome, not the least of which was my lack of familiarity with the object persistence framework and data access layer of choice, but the one thing that went remarkably well (and inspired this post) was how we communicated.

Goose Tricks

Both of us were working very hard and it was not unusual for one or the other to be online any time between the hours of 6 am and midnight.  We came to rely on IM for quick questions throughout the day, but we also discovered that it had a different use as a record of sorts for the design discussions we had had, either together or one-sided.  We started using it like a journal to record the thoughts and decisions we had made whether the other person was there or not.  This “thinking out loud but in writing” was informal but very effective – when the other person did rejoin they could get quickly up to speed, provide advice or counterpoint if needed, but easily resume where they had left off with a far better understanding of what had changed since the last merge than one can garner from check in comments.

My take away from the experience is that while I would not universalize pair programming as a necessary practice, there are times where it just sort of happens naturally and when it does, it’s great.  I’ve also seen that when it doesn’t work, it’s mutually frustrating for everybody as they try to force it.  I also learned that an IM “journal” can be a better way of communicating than verbal – my arguments were better expressed and there was a record of thoughts had and decisions made that often escapes you in a verbal exchange.  Bottom line – it was a VERY healthy and productive partnership and I dynamic that I still miss even though I have had other successful projects since.

Check us out in the Fall 2010 issue of the Methods and Tools Magazine with an article on distributed agile:  http://www.methodsandtools.com/ or go directly to the issue at http://www.methodsandtools.com/PDF/mt201003.pdf

Enjoy!

Sign Up!

Please join us Thursday, Sept 2nd for the Twin Cities .NET User Group Meeting! Pizza will be served at 5pm, courtesy of ILM; Beverages will be provided by New Town. The speaker will begin at 5:15pm. The meeting will be held in the Microsoft Offices in Bloomington: 8300 Norman Center Dr., Suite 950.

Title: Everything You Wanted to Know About Velocity But Were Afraid To Cache

Microsoft’s AppFabric Caching (aka Velocity) offers a distributed caching solution, not unlike the popular “memcached” open source library. It allows you to increase scalability and performance by caching data physically (and logically) closer to the consumer. It can help you dramatically increase responsiveness of your apps and services, as well as relieve pressure on back-end resources. Come and hear about the concepts and terminology, as well as deployment considerations, typical usage patterns, pitfalls, and more.

Speaker: Scott Colestock

Scott Colestock lives and works in the Twin Cities. He has been consulting for the past fifteen years, spending time in a variety of areas with performance and process as common threads. He is most recently a partner at Marcato (marcatopartners.com), which focuses on delivering agile coaching services. He is also a BizTalk MVP, performance engineering guy, and “Team Foundation Server + Scrum” resource.

And don’t forget to sign up for these great upcoming conferences:

Minnesota Developer’s Conference, Sept 29th:  http://mdc.ilmservice.com/Register.aspx
Twin Cities Code Camp, Oct 9th – 10th:  http://tccc9.eventbrite.com/

Agile is a set of principles to guide how you organize and function as teams and Scrum is a particular system for establishing and building a delivery rhythm within that team.  Part 2 examines 5 more examples of common changes made to Scrum and the reasons why these are counter to what the system is trying to accomplish.

6.  Set “stretch” goals by putting more stories in the sprint than fit within the velocity

To create a sustainable rhythm, you have to set goals for the team that can be met with regularity.  If it requires heroics to accomplish one sprint, then heroics become the norm for future sprints as your team velocity adjusts to account for this.  New teams often fall prey to this phenomena – their energy for the project and desire to make a good impression on their product owner gets the better of them.  They let the product owner put more stories into the sprint than is prudent because their uncertainty about their actual velocity opens the door for the usual optimism of a development team.  They try to shoehorn in just one more story because they are interested in the technology or feature and want to try to make it fit.

Japanese Lantern Lighting Festival, Como Park

This kind of persistent inflation in velocity is unsustainable – like with any economy, real growth can only come from an increase in resources or in the productivity of the existing resources.  You will see this happen through the productivity that comes from climbing the learning curve, but that always levels out eventually.  What’s the downside?  In the short run, disappointments happen, and instead of giving the team the opportunity to establish and celebrate a pattern of successes, they are instead frustrated by a string of missed goals.  In the long run, left unchecked this behavior leads to burnout and turnover as you find yourself trapped in a “death march” of unsustainable expectations.

7.  Give credit for a story when it is not really done (shippable)

Yes, it feels good to show off your work in the sprint review and to claim victory, but with each passing event where “done” is not really done, your team has taken on a future debt that must be repaid.  These schedule bombs don’t go off until much later, long after the fuse is lit, and usually at the most embarrassing and inconvenient times.  Also, once you compromise on this a little it becomes a slippery slope of lowering expectations as individuals vie for credit for their work.  We all know that product owners can be fooled in a demo, so acceptance by the product owner is not really enough.  Instead, the scrummaster and testers should hold firm to the team’s definition of success and only agree to take credit when credit is due.

8.  Increase length of sprint so that more stories can get finished

It’s the day before the end of the sprint and you can see that there are a couple key stories that are not going to make it.  They are big stories, so without these your velocity will be less than half what you expected and the optimistic outlook from the team is that they could get them done if they only just had one or two more days.  Someone on the team suggests that it would be better to extend the sprint than to arbitrarily end it now when you are so close.

Japanese Lantern Lighting Festival, Como Park

Usually this behavior indicates several forces are at work – stories are too big, too many stories are being worked on in parallel, and the team may not be doing enough detailed planning at the start of the sprint to have clarity on what will be built.  Since software is a gas that expands to fill 110% of the space provided, if you extend the sprint to let that last 10% in, then you will learn nothing from your mistakes and just do the same for the next sprint.  You also sacrifice your rhythm, these last minute changes in expectations confuse everyone as you try to figure out who has the authority to change the rhythm and who is responsible for communicating the new plan.  Leave the plan alone, examine in your retrospective what really went wrong, and fix it so it doesn’t happen again.

9.  Cut sprint short because all “bought” stories are complete

Japanese Lantern Lighting Festival, Como Park

This is more of a “minor sin”, but interrupting the rhythm just confuses everyone – you are better off to find other small stories to add to the scope until the time is up than to break the rhythm.  Be careful not to start big stories that you can’t finish – this just leaves the product in an unstable state as you are trying to demonstrate it and threatens the principle that the product be shippable at the end of the sprint.

10.  Estimate during sprint planning meeting

This observation is controversial as there are certainly going to be situations where you are forced to do this by no apparent fault of your own, so if you find yourself in that situation, then by all means just do it.  But first understand the risk you are taking – you are estimating the story in isolation, perhaps by a different collection of people than normally estimate, and you are doing so in the context of already knowing you want it in the sprint and you just need to decide if it will fit.  This probably means you already know how big you need it to be, so you are anchored on this answer.  What happens in this situation?  The estimate comes in to magically fit and round out the sprint and the most common reality is that you underestimated its complexity.

I spent this week with a new client doing training on Agile and Scrum and wrapping up with workshops where we laid out specific plans and timing for rollout of Agile.  It was an exhausting but rewarding experience and it was fascinating to see another organization go through the early stages of the change process.

One aspect of teaching Agile and Scrum that makes it easy is that it is so hands on – the curriculum is designed to get people into teams and to learn how to problem solve, reach consensus, and communicate as a team – and this is part of why it is fun to learn.  The feedback (not surprisingly) is that people enjoy the team exercises and real world examples about what has worked and not worked in other situations.

Since I enjoy thinking about and studying organizational dynamics, one of my favorite parts of teaching Agile is to try to help people understand how the mechanics of the system are designed and why messing with those mechanics in simple ways can have interesting side effects and repercussions.  We teach the values of Agile before the mechanics of Scrum for exactly that reason – most of the explanations for "why?" something is handled the way it is derive from reinforcing the values behind the process.

While there were many enjoyable moments, my favorite was a group discussion we had around a Top 10 List of most common tweeks to Agile/Scrum.  The idea behind the exercise is to throw out things that they are probably already thinking about doing, but then to encourage them to think about what happens if they do them and what the consequences of changing the system might be.  This was probably one of the most interactive parts of the training as you could see the light bulb going on for many in the room.  Here’s what we discussed:

1.  Change the duration of the sprint in calendar days because of known changes to capacity

This is a "minor sin" but still not a good idea – I know that when you are getting cut short on resources during a heavy vacation period, it’s easy to want to stretch things out, but all this does is further disrupt the rhythm and leave people wondering what is going to happen and when.  Better to just drop your velocity a commensurate amount and stay on rhythm.

2.  “Split the difference” in story point estimating when the team gets stuck

Another "minor sin", but still indicative of a problem that can be better solved another way.  Usually this is happening to the larger point values (the jump from 20 to 40 is tough to swallow) and represents a sharp divide in opinion within the team as to how complex the story is because it is so large.  The best remedy is to break the story down into smaller stories, but if that fails you are probably better of using the higher point value.  I know that splitting the difference can help you feel like both parties are giving something up so it seems more fair, but odds are the worst case estimate is actually more accurate, especially if it is coming from the person that happens to know that part of the product well.

3.  Change estimating process because we are consistently under-estimating

This one is more of a "major sin" because it can have serious consequences.  The theory behind estimating is that it does not have to be accurate to work but it does have to be stable.  If the estimating team consciously decides to shift its bias in how it estimates depending on the outcome of the previous sprint, then velocity will become unstable.  If you study system design, you learn that delays in feedback loops always introduce instability into the system.  Far better to keep your estimating process stable and let your velocity change to account for the real variation in productivity.

4.  Give bugs story points

I would call this a "major sin" as well because it strikes right at the heart of Agile values.  Story points are a proxy for "value" – they are the credit or reward we give to a team for doing something that the customer values.  Do customers value bugs in the product?  Absolutely not – in fact, it might even be more accurate to give them negative story points instead of zero.  There is one exception I have made to this rule – when you are trying to get a new team to embrace improving the quality of a legacy product that they did not create, then it is hard to get them motivated to solve the problem unless you give them credit for doing the work.  However, as soon as bugs start to show up in new work they have done or caused by other bug fixes they are making, then you need to go back to giving them zero story points.

5.  Separate feature testing into a later sprint

This one is somewhere in the range of "major sin" to "mortal sin" yet is surprisingly common.  There are kinds of testing (such as the development of new automation testing or full regression testing) that truly are impractical to commit to in every sprint, but I think once a team starts down this slippery slope, they conclude that this should mean that all test activities have until the end of the next sprint to complete.

It’s hard to summarize in a few words why this is so harmful to the system, but my best suggestion is to read the book "The Goal" by Eli Goldratt if you really want to grok the risks to the system.  When you decouple software testing from development, then test activities regress from process control to inspection and you lose your ability to determine if development or testing is your process bottleneck.  Separating when developers and testers work on features also violates one of the most important Agile values – Individuals and Interactions.  If they work on the feature together, they will both be available and motivated to work through it as a team.  If you separate it in time, then the quality of the interaction will drop dramatically.

Stay tuned for 5 more!

We spent yesterday afternoon and evening dodging thunderstorms and tornadoes at the TCF Bank Stadium; while normally this would be entertainment enough, the real reason we were there was to watch the Drum Corps International competition.  If you are unfamiliar with this form of competition, check out this trailer:  http://link.brightcove.com/services/player/bcpid62043804001?bctid=95733275001.  A friend of ours competes throughout the summer in the Minnesota Brass – a remarkable assemblage of talented people that actually PAY for the privilege of spending their summer practicing complex routines in often (especially last night) less than ideal conditions.

Madison Scouts, Drum Corp Competition, TCF Bank Field

This was our second competition we had attended, and this time we sprung for better seats – oddly enough, for sport of this nature, the better seats are actually higher up, and I was amazed at the difference.  Closer to the field you can see the details, but as the old saying goes, “you can’t see the forest through the trees”.  From this perspective, individuals are lost but the design and execution of the whole becomes clear.  The flash of a single cymbal move is lost, but the synchrony of the squad becomes clear.  Excellence in execution is required because errors break the pattern and distract the observer, but execution alone does not make you world class.  Instead, it now becomes a competition of choreography and concept where the constantly shifting patterns communicate a story and accomplish a purpose.

There are lots of elements in the performance to appreciate as there is so much going on – the music itself, the color guard with flags and twirling guns, the visual flash of color and  - but my mind kept observing a simple yet compelling part of how each routine is choreographed.  It seems that the architect of the routine knows that what what will have the most impact is to present a relatively small number of big moments where everything resolves into a sharp image for that “wow” moment – sometimes it is as simple as a single straight line slowly moving forward with all instruments holding a note in unison that sends a shiver down your spine, other times it is a design representative of the theme of the performance and part of telling the story.

Madison Scouts, Drum Corp Competition, TCF Bank Field

But most of the time during the performance itself is actually spent moving between these moments, and during these times, individual execution matters even more.  The patterns dissolve into an almost chaotic scramble as different lines shift and merge; each person performs their work to get from here to there but in a way that cooperates with the others in their immediate vicinity.  This motion has a beauty of its own, but for the most part it is a building phase that we know must be tolerated as we set up the next impressive moment.  I’m sure this is intentional, but often during these transitions the performers are all turned away from you and the music is muted.  It’s messy and in some ways unsatisfying, yet usually worth the wait when the image suddenly comes into sharp focus and all the instruments turn towards you to appeal for your applause.

Now I know that I have a tendency to see everything in life as analogous to software, but there is an important parallel between what I saw on the field last night and what we do when we are using Agile to build software.  It’s very easy to get lost in the details of the execution of each sprint, and when you are down there on the field in the team, it takes everything you have just to get from point A to point B.  In fact, there is a REALLY important difference between what we do when we make software and what we see on the field – we don’t get to practice the same transitions over and over again until they approach perfection.  Instead, we get to do them once and to the best of our abilities, and then we move on to the next one.  Given this important difference, it’s nothing short of miraculous when we do have those sprints where everything comes together well and we have a great sprint review.  This is also why I think it is important to tolerate with grace those times when it doesn’t quite work.  As a coach, what I look for is an honest intent within the team to do better in the future and a honest and healthy spirit of learning from your mistakes.

Madison Scouts, Drum Corp Competition, TCF Bank Field

One final note – in the end, great execution and choreography combine to please the audience, and this is the reason I see so many passionate converts to Agile on the business side.  Once they have experienced this level of involvement in the process and they see the product emerge as a progression of steps that are themselves pleasing to witness, they never want to do it any other way.  In that sense, perhaps with Agile we nerds and geeks have finally found a way to accomplish what we always wanted – to turn what we do into a form of performance art.