Identifying Reasons Why Agile Isn’t Fast Enough
The Agile process has often been viewed as a “silver bullet” by technical and corporate leaders for delivering software products to market in a timely manner. However, the pace of change in the business world has accelerated over the past few years resulting in rapid obsolescence for even the most carefully planned system and software requirements. Consequently, a view that Agile for any organization isn’t fast enough has begun to gain acceptance in the software/IT world. While it is true that some Agile techniques are beneficial, businesses and their users require significant flexibility in accessing data and the Agile process is often not fast enough to adapt and provide this flexibility in a timely manner. Consequently, overuse and abuse of Agile process elements leave many organizations in a constant struggle with process risk and disgruntled employees.
Problems with the Agile Process and Potential Enhancements
While the Agile process is multi-faceted with the ability to be tailored to meet an organization’s needs, there are three problems that consistently arise when using this process:
- Over-reliance of Scrum as the primary mechanism of Agile
- Difficulty in identifying high and low performers in Agile
- Agile process use produces significant technical debt over the long-term
The Agile Scrum process provokes two wildly divergent sets of options. Management tends to love this process as it allows them to prioritize the software development features and see visible progress after quick two to four-week sprints. On the other hand, developers hate this process since the most visible parts of a software product quite often are of low importance in the overall scheme. For example, sensational video game graphics have a certain ‘wow’ factor but if the artificial intelligence (AI) or other critical gameplay features are substandard, the game will inevitably be a failure. Software engineers and developers often are fairly powerless in the Scrum process and this leads to a very short-term focus where avoiding punishment for not finishing a sprint task takes precedence over developing the best long-term solution.
There is a place for the Scrum process in today’s environment but flexibility is the key to using it successfully. Gathering everyone, or at least all of the necessary stakeholders, in a room is counterproductive over time so allowing many of these stakeholders to communicate via Skype or by phone significantly reduces the overall impact on the software team. In fact, having the engineers co-located eliminates the need for a daily scrum as the problems and challenges that arise are handled “in-house”. This engineer-driven model is not only faster but it fosters ‘out-of-the-box’ thinking that potentially results in innovative solutions for the challenges of today and tomorrow.
The technological evolution requires a very skilled, creative and flexible work force to meet the increasing pace of change. The Agile process, as it is currently constituted, does not lend itself to this environment as creativity is stifled as engineers and developers focus on getting their daily task(s) done to avoid recrimination rather than focusing on the most innovative way to accomplish the task. ‘Good enough’ becomes the mantra rather than the ‘best possible’. Engineers who take the initiative to strive for the best possible solution often will be late on completing the sprint tasks marking them as ‘low-performers’ in the eyes of management. These engineers will be denied promotions and will move on to greener pastures where their talents are appreciated.
Empowering these talented developers and engineers are the key to managing process risk. In a large software project, there are a large number of uncertainties and unknowns that must be identified, analyzed and quantified to accurately gauge their impact. Mediocre engineers will be unable to identify the risk factors in a timely enough manner no matter how good the Agile process is. Therefore, having significant flexibility to address these risk factors outside of the ‘sprint’ concept is key in adopting the process to meet the increasing pace of change today. The motto of the new Agile process is ‘as quickly and correctly as possible, but no faster’.
The last point on why Agile currently is not fast enough for organizations is the long-term technical debt accrued over time. The Scrum/sprint methodology works well for a targeted series of tasks over a short period of time with the duration lasting ten weeks or less. The technical debt accrued to meet this compressed schedule is potentially detrimental to the program and is a rather serious technical risk. Programs that overuse the Sprint/scrum methodology gradually become difficult to maintain and cannot adapt quickly and reliably enough for the current pace of change.
The Agile process is a complex process that is not particularly well suited to the current pace of change. Adopting the process to discard the elements of Agile that are a hindrance, such as Scrum, and hiring a creative and talented workforce gives an organization the best opportunity to handle the process risk associated with the technological evolution.