Brian Burson

PRACTICAL AGILE DEVELOPMENT

summary

Almost all development shops now use product development processes based on agile principles. However, my experience has been that these processes must constantly be tuned to meet the changing needs of the company, products, and the makeup of the team. Using new approaches like Kanban to modify tried and true frameworks like Scrum can breath new life and effectiveness into an existing process. Revisiting the 12 Principles of Agile Software Development on a yearly basis is also good practice.

issue

At this point can anyone imagine doing software development using the waterfall model? When the Agile Manifesto was first published back in 2001 it was truly revolutionary and caught most development shops by surprise. People scrambled to figure out how to put it into a “process” and frameworks like SCRUM were born.

Agile’s underlying message is “release relevant software more frequently”—and I think we all benefit from this, both as software creators as well as website consumers. But, are we doing agile “right”? Or, put a different way—could we be getting more out of our agile practices? I think most of us would answer with a resounding “yes!”

SCRUM FRUSTRATIONS

Why? Mainly because every place I’ve been or team I’ve been on feels that existing agile processes/ceremonies get in the way at some point. At my most recent place of employment, even though we followed all the relevant SCRUM best practices, we often struggled to get releases out the door every two weeks. Bottom line, we could never seem to get the right amount of work in a sprint—either we couldn’t get it all out the door on time or we had people sitting idle. Our main issues with SCRUM were:

• Estimation challenges - it was too hard to accurately estimate story points so the team would constantly over-estimate

• Story leakage across sprints - if we kept devs tasked throughout the sprint, then some coding and testing wouldn’t be done in time for the release and this would mess up QA and DevOps.

• Ceremonies were too prescriptive - in the dev team’s eyes meetings didn’t always provide value

• Demoralized team - the team became disheartened when stories weren’t completed and crossed sprints—meaning we didn’t hit our commitment

The Webster definition for agile is “able to move quickly and easily”—so, when agile development becomes hard or slow, perhaps we’ve lost the spirit behind the original manifesto and need to rethink things?

kanban

While still being committed to agile (and SCRUM), we knew we needed to tune our process and ceremonies to get our team back on track to optimally releasing software. Several articles and blogs pointed us towards Kanban - a process that could be applied to an agile framework like SCRUM. The Kanban method is applicable here because by definition it’s an evolutionary process that makes gradual, non-disruptive changes to improve an existing process like SCRUM. The core practices of the Kanban method encourage understanding and managing the flow of work through the entire process, limiting work in progress (WIP), implementing feedback loops, all while improving collaboratively and evolving experimentally.

TUNING OUR AGILE PROCESS

Using the Kanban method, specifically ideas around limiting work-in-progress, managing flow and implementing feedback loops, we rethought several of our agile ceremonies and practices:

Backlog—honestly, we have had to declare backlog bankruptcy about once a year and mercilessly delete unworked stories. Otherwise, an unwieldy backlog slows everyone down in grooming, elaboration and sprint planning.

Elaboration Process—we instituted user story grooming sessions throughout the week—where anyone on the team could participate (this is key!). Architecture would lead these sessions, with Product, Dev, and QA being key contributors. The goal was to make sure that each story had an understandable purpose, minimal notes on how it might be executed, and then clear and detailed acceptance criteria.

Sprint Planning—the main goal of this meeting is to come up with the stories that will be worked in the upcoming Sprint. For us, estimation has always been key here—using story point estimates and previous team velocity has been the classic way of determining what the team is “committing” to release at the end of the Sprint. As mentioned above, this became problematic as the team felt hitting the commitment was more important than releasing software—meaning, high estimations, decreasing velocities, idle team members at the end of the Sprint. The fix for us was to involve more people in the user story grooming process (see item above), then using estimation as more of a way to get everyone on the same page—if you’re going to vote you probably will ask more questions and pay attention! Points became informative (i.e., we probably can’t get 3 “eights” out in one release) as opposed to written in stone and used as the measurement of success. During Sprint planning, discussion became important, the points became informative, and actual stories to be released left a bit looser. With these changes we ended up getting more software out the door as also allowing stories to be started in one Sprint but released in the next freed up everyone to keep coding throughout the entire Sprint time period.

Pre-release Coordination—this meeting is not always called out in the SCRUM framework—however, we have found it to be indispensable for releases where everything can’t be automated (e.g. if dealing with legacy code/systems). Our Dev Manager created a checklist that we always worked from—covering items like regression status, performance testing status, configuration changes documented and approved by DevOps, database scripts approved by DBAs, release date/time on the calendar with adequate operations/monitoring support. This meeting was attended by management, product, dev, QA, DBAs, DevOps and operations personnel and was held to 30 mins max. Investing time here decreased our release durations by nearly 50%.

Retrospective—retrospectives are only worthwhile IF you act on them. Otherwise, they become whine sessions that don’t enrich or satisfy anyone. We have worked to track issues and suggestions more closely—management has owned documenting them, while the team owns suggesting where changes can and should be implemented. The status of previous items are reviewed first at each retrospective.

The Agile Alliance website is a great resource for brushing up on all things agile.

Measuring success - Revisiting the 12 principles of Agile Software Development

As mentioned at the beginning, it’s worthwhile to revisit what the agile founders envisioned when they proposed the agile manifesto and the 12 principles back in 2001. From the list above, I see several that our teams have done well, several that have worked just ok, and some that we have always struggled with. I find teams that have a strong, almost rigid release cadence (e.g. every 2 weeks) struggle with #2 - welcome changing requirements. Encouraging them to consider releasing as soon as there is something valuable to release is one way around this. Automation is a must in this case. You would think that we all naturally would do #6 - convey information face-to-face, but, have you noticed how people sitting 5 feet from each other will email each other? Amazing! If I had to pick the one that I think is the make-or-break one, it would be #4 - business and devs work together daily. In fact, if possible they should sit together. Finally, I personally haven’t seen success with #11- teams self-organize, mainly because someone has to pickup the work people don’t want, and that seems to be hard to do when its peer-to-peer tasking. But, I haven’t given up hope yet!

I think we should all revisit the 12 Principles yearly—revising our practices and ceremonies as necessary to get back on track.