Lessons Learned from Scrum

My Software Development Working Environments

I have been in several different software development environments throughout my career.  It is so interesting to see different company cultures and how they work.  You can learn a lot from it.  Using scrum has given me an appreciation of how it has some ideas to help fix some downfalls I see in other methods of operating.

Some environments are very good for helping people get work done.

Pretty Environment

Others… not so much.

I’m going to spend some time talking about my different work environments in different companies.  Some of the ones I have seen:

  • No process (what they called the “wild-wild west”)
  • Waterfall process (and ‘pretend’ scrum… seriously, both?)
  • Everything is always super-urgent and the top priority, fix it now!!!
  • Agile environment using scrum (or at least a close approximation of it)

Let’s explore these different types of environments, followed by lessons I’ve learned from an agile environment contrasted to these other modes of operation.

No Process

This is similar to everything being super-urgent, except it’s not necessarily urgent.  The big key here is that there aren’t set company-wide priorities.  As a result, there isn’t great communication with different teams.

Communication

This means a lot of things may get started, but not a lot of things get finished.

Do you know how much value the company gets if 90% of things are 90% finished?  0%…

I remember a specific example where I was working on a feature.  95% of the way through it, I discovered another developer on another team was also working on the feature.  We ended up using that other person’s implementation.  Guess how that made me feel?  Not so good.  It was more of a process problem – two people shouldn’t have been working on the same thing in the first place (not unknowingly that is – pair programming would have been great).

That’s not to say there aren’t advantages to this approach.  Without much process, developers potentially have a good deal of freedom to do things the way they want.  Unfortunately, I still don’t think this was the case.  We had priorities, but they may change a good bit.

In light of this, let’s consider a way to fix the priority issue with the waterfall design process.

Waterfall Process

Waterfall

In another position, we did waterfall with a scrum-like aspect to it.  Being a contract position, I think the customer required/wanted use an agile process.

I think using scrum would have gain a lot of benefits for us.  Instead, we had “daily standups” (which were honestly status meetings which lasted way longer than they should) and nothing else.  They called it “scrum waterfall”, but that was more of a joke in my opinion.  It was waterfall, plain and simple.

That’s not to say waterfall is necessarily bad.  I think it could potentially work fairly well.  I just haven’t seen it.

Waterfall dictates that the requirements are well-known before going to the design phase.  I think they were somewhat known, but probably not to the extent that they needed to be.

Design occurred, and then the software was thrown “over-the-wall” to the testers.

The test team generally got very poor-quality builds.  Things would happen such as the software wouldn’t even load.  Or perhaps it would, but the delivery contained code that was a month, two, or three old.

This obviously put some conflict between the test and software team.

In addition, the software team blamed the requirements team for not having clear requirements for their software.

It was the blame game everywhere.

While there’s not as much blame in every waterfall process, the big takeaway for me was that testing comes at the end.  And that’s usually not good.

It’s rare (from what I’ve seen, anyways) to see software pass tests on the first time.  Which means you start going back to the design phase over and over again.

Going back from test to design to test to design isn’t necessarily a bad thing.  The bad part is the cycle time between them.

Whereas in agile you might do this with seconds between the two phases, in waterfall it may be weeks or even months between these cycles times.  That’s really unsustainable if you have lots of bugs in your development.  You have to make that cycle time shorter.

It results in lots and lots of integration issues and missed delivery dates.

So… I’m personally not a big fan of waterfall development.  Just my personal opinion, but I’ve seen it used more as an excuse for allowing poor code quality and ignoring good practices.

Constantly Shifting Priorities

This process is also not a good place to be.

Some jobs want people to be able to multitask.  There’s some good studies out there that show humans aren’t capable of truly multitasking (here’s one from NPR).

Multitask

In addition, especially for programmers, there’s the concept of needing to focus.  It takes time to get all the different functions/objects/algorithms in your head before working on a problem.  An interruption “deletes” that data and requires the setup time “penalty” to occur again.

In the context of a business with constantly shifting priorities, this is chaos for the developer.

You get into the cycle of getting things 20-80% complete and never finishing them.  Or, even worse, you get to 100% completion with poor quality.  Which actually means you are going to be spending more time on the same problem in the future when a test (or worse, the customer) uncovers an issue that shouldn’t have made it in there in the first place.

Shifting priorities is an example of bad business practices.

It is impossible to have two #1 priorities.  But I’ve seen it over and over again because companies don’t want to make hard decisions.  If two things are #1, then neither of them are #1.  It means rather than being focused on finishing one thing, progress is going to be made on two things with neither of them getting done.  It takes longer to finish those things because humans don’t multitask well.  Swapping between the two (or more) tasks repetitively causes context-switching in our brains that costs the business time.

So why don’t we just pick one most important thing?  I’ve seen not wanting to upset the customer.  Or perhaps someone in the company has a louder voice than others.  There are tons of reasons why it happens.

Generally with the shifting priorities comes unreasonable delivery dates based on the circumstances.  The reason something is higher priority is because the company is in reaction mode rather than proactive mode.

Help

If all you are ever doing is reacting, you aren’t going to be well-equipped to handle the future.  And the spiral continues over and over until a breaking point occurs.  Perhaps a major customer drops you, employees burn out, or some other big, bad thing happens.

Another thing that might happen is really good employees don’t want to be a part of a failed project.  So they might work more hours to make it succeed.  This means the business doesn’t change anything because what it is doing is “working”.  It’s not until the breaking point happens that something changes.

Or perhaps a breaking point never occurs, but the business never reaches its full potential.  It continually underperforms due to never focusing on just one thing.  It is always reactive, not planning for the future.  This leaves room for both unsatisfied customers and/or competitors to have an advantage.

Having now explored three different processes, let’s view my experience with scrum.

Scrum

The same company that had no process transitioned to scrum.  This was an extremely interesting experience in and of itself, and probably worth another blog post.  The purpose of this post is to explain my experience with this process.

We quickly realized in order to really be “done” with something, we needed to fit a lot more testing within each sprint (a 2-week cycle period for our team).  This led us to changing our code to allow for compiling smaller chunks of the code and running unit tests against those smaller packages.

In addition, we made documenting what we had done part of the sprint.  And making sure it was ‘deployed’ (which meant integrating the smaller package back into the main codebase).

Whereas with a waterfall process you may do most of your development before testing or documenting, scrum changed that.  Now we did all development, testing, and documenting in a two week cycle.  This meant we had much quicker feedback.  Rather than months before we knew if a feature was working, we could do most of it within two weeks!

Since we fit more testing and documenting than normal into the front part of development, this forced us to be more focused.  We couldn’t ‘do as much’ development right up front.  But the back part of testing and maintenance was much smaller.  (This is proactive vs. the reactive development described in the shifting priorities section.)

If something came up in those two weeks, we pushed it off to the next sprint (most of the time – we still had ‘bug buffers’ to plan for time where stuff just goes wrong and needs to be fixed in that sprint).  This increased focus allowed us to get more done on specific tasks each sprint rather than not finishing several different tasks.

Scrum does a great job of showing problems a lot earlier.  It’s a lot harder to keep secrets.  Which is good!  It fosters an environment of openness.

Secret

If something is at risk of not getting done, more people can focus on the high-priority task to finish it before the sprint is over.

If multiple people cannot work on the same task, it might reveal that training is needed so more people can be cross-functional so they know how to do some task.

Again, scrum is great at revealing pitfalls or other things that need to be fixed.

It doesn’t necessarily fix everything – it does help with some things – but it does a great job of pointing out things that need to be addressed.

Lessons Learned

Owl

By far, the biggest thing I learned from scrum was the huge impact short cycle times can have on development.

Testing things early and often finds problems a lot sooner to when they were created.  This allows those problems to be addressed more rapidly, as we are a lot more familiar with everything that is going on (we forget things as testing finds bugs later in the process).

In addition, I learned that its more important to work together as a team to get things done rather than work individually on several different tasks.  Not only is there more risk when so much work is in progress at once, but team members also don’t cross-train when they are siloed.  Working as a team makes it more likely that the most important business functions get done (vs. multiple other things being partly done).

This isn’t to say that there isn’t room for someone to work on something alone for a short period of time.  That type of concentration is needed sometimes.  But it shouldn’t continue indefinitely.

Finally, open communication is important.  From an engineer’s perspective, they usually don’t have visibility to the most important business functions.  Perhaps project X bring in 100 times more revenue than project Y.  The engineer might not know that.  So if a decision needs to be made on whether to work on a project X feature or project Y feature, management is going to know better what business decision to make.

For this reason, it’s important to be open.  Even if news is difficult to hear (there’s NO WAY we are going to meet that date), it needs to be heard and planned for.

Expecting that ‘magic’ will happen and ignoring a problem IS a decision in and of itself.  It is a decision to fail.  To not plan for what is likely to happen.

This is why scrum is so good in bringing those things to the surface and making them clearly visible.  When business decisions need to be made (put more team members on a project, talking to the customer, or decreasing scope), the right people need to be informed so they can make the right decision.  Hiding these issues is not good for the business.

I know this was a long-winded post, but I hope it gives some perspective on why I believe scrum solves some critical problems found with other processes I have been in.  Although not the ‘sexiest’ topic to talk about, these processes absolutely can make or break a business.  A bad process that receives no attention to improve it is a recipe for disaster.

Please follow and like us:

Leave a Reply

avatar
  Subscribe  
Notify of