Once upon a time when enormous mainframe computer systems demanded a modular-based design that resulted in clear requirements handed to programmers who dutifully wrote code and submitted their jobs to the compiler and linker, which then integrated with the mainframe system build in the designated system and user spaces, life was good. Life was predictable. It was deliciously black and white. And green monochrome.
Then chips became smaller and faster. Computers could be found not only in the office, but also in the home. Enterprise-class computer systems became more complex and new computer languages evolved from human minds that combined new ways of thinking about programming. New age thinking and the genius of idea organization that lead to the Rational Unified Process became, with time, seen as a legacy approach of waterfall style software development.
The internet changed expectations – products have to innovate so fast now that designs are often changed before developers can code the initial idea and deliver it to users. Waterfall ideology needed to be set aside and a new process adopted. Agile Scrum seeks to fill that need.
I was attending an outdoor party with some friends last weekend in upstate New York. Sitting on a blanket under a tall oak, the warm sunny breeze rolled over the hills of the horse farm we were at and through the wheat-grass and hay bales. The blue skies twinkled over the rooftops of the horse barn as people chatted, played Frisbee, and snacked on fruits and veggies. A pair of chickens strutted about in the distance, and a friend of mine and I were taking in the scenery and enjoying the tranquility of the afternoon. I sipped a lemonade as she turned to me and mused, “You know, I don’t think Agile really works.”
I raised my eyebrow and cocked my head to the side as I shifted my thought process from cluster-flies to software development practice. “What do you mean,” I laughed. She went on to describe her experience with her Agile Scrum team, who delivered software that her team needed to use. Her experience of the Scrum team was that they went off and developed software that didn’t do what she needed it to do. Her face contorted into a confusing cloud of distaste at the thought of this particular Scrum team. The team seemed to be operating by their own rules and didn’t take kindly to input. I listened and after a while asked, “During the Scrum commitment, what happens?” I purposefully built the assumption into the question that she was invited to the commitment session as a product representative. Silence for a moment, then a look of growing confusion, then the reply, “I don’t think they have those.”
John Paul MacDuffie, codirector of and an associate professor at the Wharton School, and Takahiro Fujimoto, professor at the University of Tokyo and the executive director of its Manufacturing Management Research Center, write in Harvard Business Review about the attractiveness of modular approaches to product engineering in the automobile manufacturing industry. Products that are small and operated in a private space often irk only their owners when they break. Buyer beware is easier to implement with products that either have a single owner or are used by many sets of small number of users. Computer systems, at one point, used to be like this. Legacy software code would often be built in such a way that defects in a piece of code would be exposed only to a certain department using that particular set of screens.
Today, the game has changed, but much of the old guard management philosophy remains. Enterprise web code is often shared amongst many sets of functional code in the name of reducing time to market and engineering effectiveness. These are great ideas, but come with a caveat – defects in code written in a shared module can expose errors to a much larger set of users.
Engineering and manufacturing automobiles, suggests MacDuffie and Fujimoto, requires “intensive coordination to keep track of all its various aspects, and its engineering culture prefers unique (non-modular) solutions.” According to an IEEE report of the Automotive Designline, the number of lines of software code present in Ford motor vehicles rose from 2.4M to 6M between 2005 and 2009. Between 2009 and 2010, the number of lines of code rose from 6M to 10M. The complexity of automobiles, measured in lines of code, is increasing at an impressive rate to meet the demands of their users.
Software development methodologies must adapt to the requirement that working software be delivered faster. The documentation requirements of many legacy processes simply can’t keep up with the changes to design, which now not happen just after product is shipped, but also mid-stream before one idea is fully baked. Agile Scrum has been positioned as a solution to this problem. Does a new process bring with it new problems?
The conversation with my friend was a familiar one. Many people say they’re doing Scrum, but what they really mean is that they’re trying to do Scrum and confusing trying with mastery. Imagine for a moment that instead of giving diplomas for academic achievement, universities simply kept track of attendance and timely payment of tuition. Upon receiving full payment for tuition, and showing up to class, you would receive your diploma. As an employer, which diploma would you value more: the paper received for showing up and paying, or the paper received for showing up, paying, and achievement? As a student, which diploma would you value more: the paper received for showing up and paying, or the paper received for struggling through new course material and pushing through limitations to understand and apply these new concepts?
Certification for some Agile Scrum programs requires that you pay and show up. There is nothing inherently wrong with this model. This approach, in fact, generates some well-qualified and effective Scrum leaders. The weakness in this model isn’t the education, in my opinion. The weakness is confusing the certification received for showing up for Scrum class with the industry experience in the failures experienced and lessons learned leading actual Scrum teams.
Failure is the best induction to effectiveness. I believe that the software development industry is reaching, or will soon reach, the point where we have to stop issuing certifications to people just for good attendance and timely payment. The value of certification is dropping, the ability of managing Scrum is either going sideways or decreasing, and the value of Scrum is decreasing in the minds of team leaders because they confuse actual Scrum with their experience of Scrum failing not due to Scrum itself, but due to poor implementations of Scrum.
If you implement a process poorly, is it the fault of the process or the fault of the implementer? There is a basic sense of ownership and responsibility that comes with software development. Everyone is responsible for the end product delivered to users. One of the base tenets of Agile principles is the empowerment of all team members — ideas, approaches, and input from anyone is designed to be welcomed and integrated into team discussions. Scrum aims to remove blocks and empower developers to craft a product that will serve the business with intimate appreciation of the business vision and strategy.
Adopting Scrum won’t save a project team that has communication issues or team leadership based on fear and punishment. The team follows the leader; if the leadership doesn’t change, the team morale and productivity probably won’t either.
An experiment I would love to see done., especially from a Quality Assurance perspective but also from a process point of view, is to take an effective Scrum team and instead of four 3-week iterations, have them follow a waterfall process for 3 months. During this same time, take a software team struggling to deliver software and have them adopt Agile Scrum for the same 3 months. Measure things like timely delivery, rate of defect discovery, rate of defect closure, amount of work taken on versus amount of work delivered, accuracy of end product to requirements and defects per lines of code.
My predication is that Agile, in this experiment, would lose to waterfall. Wait, what? Isn’t this supposed to be a pro-Agile article? Really think about this thought experiment: The ex-Agile team practicing waterfall in this experiment would have the robust discipline of knowing when to question, know the importance of daily face to face communication, and would have a forward-thinking sense that Agile inspires. The documentation requirements and meeting schedule would be at best an inconvenience but would not block the team from delivering.
The new Agile team, as most teams just starting Agile for the first time without a leader who has successfully implemented Agile, is struggling to adopt to the communication encouraged by Scrum. The team would lumber through the daily stand-ups, project management would still try to micromanage tasks and effectually slow down developers, and product owners would leave too much up to the tech teams to figure out on their own and ultimately be only moderately pleased with the product resulting from the Scrum team. Management, users, and product all walk away thinking that Agile Scrum has problems and doesn’t really work. On the one hand they’re right, they believe they tried Scrum and the failure was obvious. Right? Or did they really all just succeed in fooling themselves?
Consider for a moment why Scrum teams fail to ship product. Is the failure the process, or is the failure the blaming of the process? Where does the responsibility of each team member of a development team begin and end? Agile relies on persistence in applying Agile principles, a team lead trained in the philosophy and people management of software development, and communication. You cannot implement Scrum by printing out a set of rules and expecting people to change their behavior. If a team doesn’t believe in their project manager, get a person in place that can help train that person and/or get the team up to speed. Always be looking for ways to bring in help, be that help education or targeted consulting, to breathe fresh life into a team, especially a team doing real Agile for the first time.