As we all know today (hopefully?) that the methodology Waterfall is actually quite bad. In the original paper outlining what Waterfall was it is described as I believe in this concept, but the implementation described above is risky and invites failure.
. In the same paper many attempts were made to make the process suitable with sufficient proof of success prior to development work beguinning. The methodology recommend by the author was:
Yeah... maybe Waterfall like we think of isn't so bad after all, yuck.
Next up we look at Scrum, the most popular of all "Agile" methodologies. Of importance is to note that even if you "do it", you may not be doing Scrum correctly (seriously!). So Scrum has its own little set of artifacts created, not just "the requirements" or "the design". Of these include:
- The meetings, yup, daily meetings that should not include the entire team! Meetings are costly for developer workflow and in of themselves a product, not part of the daily workings.
- Backlog of work, ever changing lists of tasks that needs to be done. Usuually accompanied with a user story.
- Spring completion work log requirements. Each sprint has a set of goals at hand that must be completed to a set quality level. Before you can move on.
When used Scrum always has somebody in charge to act as a "manager". The Scrum master is trained in Scrum itself and arguably should be left in solving problems as they appear. Their job is to make sure the process is being followed and communication is going on correctly. It is not their job to update project owners, as they should already be part of the process actively. For these reasons Scrum is quite costly to do right and why so many people do it wrong. Finally it still doesn't solve software development process issues, but instead it adds more complexity and cost to the team.
So what other options are there? As it turns out quite a lot. There is Spiral model, RAD, Extreme. Yeah loads with very differing ideas and of course Iterative&Incremental development. So what is Iterative&Incremental model? Well its kinda like Waterfall only with more iteration going on. There are references to iterative development in other fields as early as 1950's.
Each phase of development creates a new artifact based upon the previous ones artifact, except the first. The first iterations job is to create an initial prototype to get feedback on in the next cycle. Each iteration results in artifacts from the difference phases closer to (or stopping all together) the end objective of solving a particular set of problems.
In almost all cases, rules that are set down by methodologies to always be followed only cause problems. If you need a formal design document you add one. Need another process to formalize your testing procedures? Then add it. Tooling cannot help you, unless it actually solves a problem. This is a fairly common issue observed with Scrum in people desiring tools for parts of it, when they don't understand how to use it.
Some key characteristics of Iterative&Incremental is that it pushes to have the user involved where desired. With feedback occuring after every increment/iteration of the cycle. It has less documentation or meetings, work on whatever you can on the work log for that cycle and if you need to meet up. Constant analysis of the artifacts against the goals must occur. If problems occur during development or testing, then redesign/refactor may be required as part of next iteration.
What is important to understanding, is Iterative&Incremental is first and foremost a way of thinking. It hasn't got hard-set rules that you must always follow. This allows you to apply to a many range of activies. For example I have had a very long term goal of having a windowing library. This is quite a hard task to try to accomplish. But that is ok, we can do it! But how though? Do it once and hope for the best? Nope. I started out with porting OOGL to D (called it D_OOGL), followed up by a major refit and rewrite into DOOGLE. All the while the design was horrible and ultimately impossible to use for one reason or another. So I incremented and change my requirements majorly to create SPEW. All the while many years had passed and I had earned my degree, even worked on an honors! The designs are very different, but in the end the goal hasn't changed. Just how I went about it.
For years I have been doing Iterative&Incremental development without doing any form of house keeping. At the end of the day its fairly fast and easy to do without much overhead. It can be applied for tiny projects, really long term projects or extremely large commerical products with high success. Because it deals with the core problem that cause most projects to fail. Human error.