Agile in organizations dealing with complex integration

Author: Josip Osrecki

We all know agile works well in a cross-functional team with little or no dependency on people and processes outside the team. The stronger those dependencies are the higher the variation and uncertainty on their impact on team performance become. At a certain point, the overhead of endless firefighting, coordination, and often pointless meetings can become so huge that a day when you manage to spend just 2 hours producing something of value is considered a success. Ultimately, all benefits of agile methods are negated.

Although I have been working in an agile environment my whole career I have found that almost every time organizations go just one step above the classic “let’s do Scrum with 4-9 team members” things do not turn out as expected. So here is my take on what are the most common problems that you should be aware of even before you start your transformation. Most of the stated problems I found to be more pronounced in complex integration projects but are applicable to any project that has a vast number of dependencies that affect the team’s performance. Transformations are a slippery slope and if you do not notice and remedy any of the stated problems the transformation will, most likely, fail.

In the beginning – creating long term agile islands

This is common problem when an organization is trying to do an agile transformation. One or more “Proof of Concept” teams are assembled, often focusing just on development while completely ignoring both vertical (hierarchical) and horizontal (path to production) dependencies. Team’s efficiency might be higher and it could be producing value faster – but how does organization capitalize on that? If downstream code propagation to production uses the same “old” methods and waits several months for next release the whole point of “delivering value faster” is lost. This goes for upstream as well – business requirements sometimes do not translate in technical – meaning some assumptions will fail during implementation.

Sometimes management does not see benefits of new methodologies because, from organizational perspective, time from gathering requirements to production is still the same. The key is ensuring that agile and lean values are lived throughout the lifecycle – (preferably) from higher management decisions to production deployment.

Try to really implement just one flow through the organization, no matter how “thin” that flow thread is. Having fewer people is not necessarily a bad thing. If another team you depend on is unorganized it is difficult to pinpoint a person accountable for the delay (which is when accusations and finger pointing usually start to happen). Including just one dedicated person from another team (let’s say operations) will help tear down barriers and make communication much easier. This is the role that general Stanley McChrystal refers to as “liaison officers” in his book “Team of Teams: New Rules of Engagement for a Complex World”.

There is nothing wrong in starting carefully, with one team (or few), piloting how agile way of working fits in your organization’s mindset. What you should avoid, however, is letting this state run for too long and allow effectively creating isolated agile islands that function differently from the rest of the organization. The friction between these islands and the rest of the organization will become too hard and very soon “organizational gravity” will kick in assimilating agile islands back to old way of working.

Poor definition of a “team”

So, you have decided that your team will consist of 2 business analysts, 4 developers, 2 testers and a designer. Great! Have you given some thought what dependencies your team has to other projects and how it would cope with possible delays? From a technical perspective – are you in charge of just one layer of the application or the entire flow – from the moment user clicks on the web page to the database entry? Also, how often do upstream decisions make your life more difficult than it should be? Make a list and consider how often your team is stuck on each dependency.

When forming cross-functional teams, try to cover as much of the value stream as possible. Parts of the value stream left uncovered by team members will represent external dependencies to your team and will have to be closely managed not to introduce delays.

Ignoring the importance of information flow

Within complex integration projects that have a wide technology stack you will often see people talking different “languages”, not providing any context and not fully understanding each other. It is understandable that people forget dozens of organizational roles and what each role needs in order to do their job. That is why it is crucial to find pieces of information that need to be provided at each step of your value stream. First step would be writing a minimum set of information each team member needs to perform their work (and visualize it on a board!). To make everyone aware why this particular set of information is important it is advisable to explain how every piece of it (for instance variables you pass on or a design requirement) affects teams work. Try to notice if team has become “multi lingual” after a couple of sprints.

It is too often I have seen that working in a team for some people simply means working on the same code (or requirement or test…) with some other person. It is extremely important to treat information the same way (if not even better) then you treat your code. To some this seems pretty obvious. But reality is that often when a new person comes on a project, he or she is not given enough information. Argument being that it takes “some time to become productive” and we treat is as a normal thing – but this is often just a lazy excuse. Next time a new member joins your team, treat it as an opportunity to improve your information flow – try to ask newcomer what does he/she does not understand, what is obvious (the good stuff) and what is missing. Don’t let him or her get caught in a net of “everybody knows this” statements. If you need to explain something to him/her weeks after they have joined the project it means they have lost precious time figuring out things that they should already know.

Make sure everybody understands both the big picture and each step in the development process, as well as information needed in order to progress to next step.

“Implementing” agile as a one-off project with prescribed steps

This has been said too many times but is still not understood by some. A common mistake is thinking that once you “implement” agile the journey is over. After all, agile is the journey, not the destination. Let’s say you want to use Scrum as a framework to “implement” agile. First put emphasis on why the organization needs to transform and state that Scrum is just a starting point of the journey. Agile (or lean) as such will never be “implemented” as this is a process of continuous improvement. Maybe you will conclude that selected framework might not be good for your organization and that you want to change it. You may want to redefine the team size; the way team members and teams in general communicate or find a better way to perform retrospective. And that is perfectly fine if you ask yourself are you doing this just because it is easier shortcut that will cause completely new set of problems or are you really changing for the better.

To assure that you are making progress in the right direction it is important that you pick people who understand the problems both on high and low level and actually have the willpower to fight for change. These individuals will often be misunderstood– especially by the people who focus on local optimums see only local problems and procedures. Procedures that are put in place as a band-aid for other more ingrained problems and constraints because of other problems and barriers… Of course, there will always be some problems – the goal is to find out which ones your organization created because of internal constraints and try to fix the root cause.

Although agile coaches will help in transforming your organization and bring in fresh perspective it is equally important to build internal agile team and identify change agents (to refer to another great book “Lean Thinking”) that has both technical and people skills in order to continue the journey once coaches leave.

If you are interested in learning more or simply deepening your knowledge about lean or agile and want to learn more about Team assessment, Market of Skills, Value Stream analysis or Kanban let us know at learn@croz.net. We have workshops and consultants that will help you on your transformation journey.