Agile Frustrates Project Managers and Software Architects

As if they needed more aggravation

As if they needed more aggravation

photo by Alex Bunardzic himself (and his iPhone helped a bit)

Unlike hardware, software products tend to be nebulous in that we cannot really be in a position to envision their shape nor their boundaries. Because of that, software development efforts have so far been experienced as one long depressing series of failures (with notable exceptions, which seem to be quite rare). When it comes to software development, the prevailing evidence points to the harsh fact that it seems devilishly difficult to predict, with any degree of certainly, the complexity, required effort, and the duration of pretty much any software development project.

This alarming state of affairs necessitated a radical change of direction, whereby leading experts recoiled from advocating the traditional ‘waterfall’ development methodology and instead proposed the so-called agile development methodology. Agile almost flipped traditional waterfall on its head. To begin with, agile’s central axiom is straight-faced admittance that we (i.e. software developers) are not really capable of fully grasping and understanding what is it we are being asked to build. All we can do from that vantage point of self-professed ignorance/impotence is make attempts, in the form of short outbursts of activities, in a vaguely outlined direction provided by the stakeholders and the business analysts. These attempts (iterations, or sometimes called sprints) are ideally brief in nature, followed by immediate feedback in the form of acceptance/rejection from the business side. We then rinse/repeat, and in a nutshell, continue practising this agile discipline until we hear from our sponsors that our delivery is good enough (for the time being).

This agile approach pretty much thwarts our ability to offer any reasonably useful/reliable prognosis. How can we reliably answer the question: “When is the development going to be finished?” when we don’t even know the full scope of what we need to build. As we’re moving forward, we keep discovering, hand-in-hand with the business people, the real requirements of the product or solution we’re building. It is obvious that it is absolutely impossible for anyone to estimate and forecast the effort/timelines based on this loose process of open-ended discovery. And based on the way the cookie crumbles in this agile world, it is also very obvious that the traditional role of project management is placed under a big question mark.

However, not only is this agile methodology frustrating to the project managers, it is proving to be even more frustrating to the software architects. Why is that? Basically, the fundamental stance of agile, that treats business outcomes as the starting point in a software development project, defers any traditional architectural decision into indefinite future. Right there, the traditional role of software architects gets challenged, as we adopt the outside-in development methodology. When starting a software development project in an agile fashion, we focus first and foremost on business outcomes, which means we adopt the use case centric approach. A user story, or a use case scenario, does not care about the traditional trappings of typical software architecture, which is concerned about databases, application servers, network configuration, email servers, etc. The outside-in development methodology focuses instead on asking the question: “What is it that the system/product we are developing is not capable of doing right now, and it is of vital importance that it should be doing?” And then once we remove that hurdle by answering that pressing question and by implementing the solution, we move on to the next pressing question: “Now that we’ve added that vitally important capability to our product, what is the next most important thing this product is not capable of doing, and it absolutely must be able to do?” And on and on, until we run out of such vitally important questions.

In other words, agile development methodology is only concerned with what needs to happen, not how is it going to happen. And that’s precisely the opposite of traditional software architects’ concerns; traditionally, software architects are solely focused on how is something going to happen in the system they are architecting.

Is it then a small wonder that software architects get riled, irritated by the agile development approach? Such approach is clearly squeezing them out of their jobs.

What is a Software Architect to do?

Software architects must adapt with changing times. They must learn to refocus their attention away from the underlying computing infrastructure and straight into the heart of the business concerns. Consider what happens when you work for a business that switched from the old school waterfall approach to the more reasonable agile approach. Such business remains permanently focused on the fundamental unit of delivery, which is always a business outcome. Regardless of the delivery mechanism, i.e. regardless of how is the business outcome going to get delivered, the true value in the software development activity is to keep delivering business outcomes that put the business at a competitive advantage.

Knowing this, software architects must turn their backs on the world of commoditized computing infrastructure and embrace the world of delivering highly specialized, tailored services. Services and capabilities that simply cannot be commoditized, and that require human workforce to apply the hard won business acumen coupled with a healthy dose of common-sense.

How to do exactly that will be the topic of my next post.