Is Agile and Scrum really better than Waterfall?

There are lots of ways to develop software. From the "cowboy coding" that involves no design, no documentation, and no testing to strict and rigorous processes. Almost all software development standards describe how to perform and improve specifications, designs, coding, and testing.

The end goal of all software development processes are: creating software that works as intended, is on time, within budget and can be maintained. But which development approach is the best?

Is this context-dependent or are there any generic aspects so that an overarching software development "framework" can be labeled as "best practice" for the status quo?

All creators and supporters of development methods, claim that their method does deliver better software, with less effort and faster then other approaches. While such an idea is nice and calming, part of your brains want proof and why it is so. Software development process is not a religion and therefore there is no place for faith fanatics. (Although I must admit that most online discussions on the topic suggests otherwise.)

The popularity (or notoriety) of Agile methods are growing and a quick Google Trends query suggests that it's not on its peak.

However, there is one major dilemma. As much of a fan of empirical data with rigorous statistical validity, it is maybe impossible to scientifically prove that one methodology is any better or worse than any other. Software engineering doesn't have (yet) uniform scientific data to compare and back up touted methodologies such as Scrum or Agile. Projects depend on domain, knowledge and skills of the people involved, process methodology, the choices made at various phases of the project from the technology to use to the system architecture and design: It becomes very difficult to look at projects and generalize information in such a way that it is both scientifically valid and useful across the majority of projects.

Besides, the "No true Scotsman" fallacy is overused and is very easy to apply on this subject. It's comparing apples to oranges where everyone can easily say: that is not a real apple and this not an orange neither!

A sufficient definition of the development methods are needed to be able to compare and say something meaningful about them.

This is completely sad when we consider the whole Waterfall method is a straw man issue and that Royce didn't described or intended to follow such a method in his paper.

In general, maybe we can say that it's pretty much impossible to draw any meaningful real-world scientific conclusions because every (non-trivial) software project is unique and software development is, by its very nature, non-deterministic. But this is a fundamental and an even greater statement than we started and is crying for a foundation.

But there are definitely attempts to compare efficiency and effectiveness of software development methods. The book "Making Software: What Really Works, and Why We Believe It" does include some chapters treating the subject. But (regrettably) there is not a sentence stating that Agile methods make a significant difference. There are plenty of anecdotes that suggest that using a specific method in a specific situation has led to great success. But these kind of subjective validation is dangerous and does not in itself prove a causal link.

There are also serious allegations that promoting software development methods is just one big money-making exercise for a group of consultants. Certification has created a small army of consultants and trainers who are constantly busy training and coaching a bigger army of certified Scrum practitioners.

At the other hand, there is nothing wrong with getting some help. Even it is not a thoroughly scientific method, some anecdotal best practices can point in the right direction and can be a push in the back even it is all a bunch of common sense.

Emphasizing the practices of minimizing unnecessary overheads (excessive planning and documentation) while concentrating on incremental delivery of client-specified functions can't be too wrong.

If it is difficult to plan ahead, adopting mechanisms for “empirical process control” that focus on feedback loops sounds alright too. Blind and unthinking practitioners or commercial interests do not alter this conclusion.

In the end, "you get what you measure". Agile is popular, sounds cool, you can't get blamed, it gives some best practices, emphasis on motivation and responsibility. But we must also be honest and acknowledge that it is not based on real evidence that it is better and that agile is maybe a too hollow / empty concept. When there is a great product owner, then Scrum for example can be certainly faster than a waterfall approach with a bad specification document. But a bad product owner can be certainly slower than waterfall with a great specification.

In Rapid Development: Taming Wild Software Schedules, Steve McConnell identifies a number of factors to consider when choosing a lifecycle methodology:
⁃ level of understanding of the requirements
⁃ level of understanding of the architecture
⁃ desired reliability, risk management
⁃ schedule constraints
⁃ amount of process overhead
⁃ mid-project "course corrections"
⁃ ability to provide the customer with visibility
⁃ ability to provide management with visibility
⁃ sophistication of the development team and management

Maybe Agile is just a fad of IT that happens and there will be a new one that gets people excited. Just trying to improve the efficiency of software development is what we all want. Doesn't really matter how it is labeled. I will therefore end this nuanced as a research paper: I hope that this blog post will alert empirical researchers to the possibility that their studies will contribute to the understanding of software development processes.

Is Agile and Scrum really better than Waterfall?

Links