Testing quality into software costs us dearly – is there a better way?

by Prof Barry Dwolatzky

sw_testing

A certain company in Johannesburg has, over the past few years, been outsourcing its software testing to a large Indian company. The value of this contract is R400 million per annum. The Johannesburg company employs several hundred software developers who write applications that support its business operations. It is these applications that are tested in India. 

The major aim of software testing is to expose “defects”. These defects are errors made by analysts, architects, designers and programmers during the software development lifecycle. Various international studies suggest that a piece of software going into system test contains more than 25 defects per thousand-lines-of-code [KLOC]. Data collected by Xerox in the USA concluded that the average time required to remove each defect found in system testing is 1405 minutes (or 23.4 hours). This time includes the time required to find the symptoms of the defect in testing … and then the re-work to be done by the original developer in locating and fixing the error. 

Defects are therefore costing the Johannesburg company mentioned above much more than the cost of the R400 million outsourced testing contract. There is also the large amount of wasted time spent by their in-house developers on re-work and debugging. At 25 defects/KLOC the company is shipping many thousands of defects to India. Some of these (but certainly not all) are then reported back to Johannesburg where tens of programmer-hours are required to fix each one. 

The waste in effort and money is almost mindboggling!! Surely there is a better way?

 I believe that the answer lies in putting higher quality code into system testing. It is obvious that if a way could be found to reduce the number of defects per KLOC from 25 to (say) 10 the saving would be tremendous. Not only that – we would also make software development projects more predictable. 

The reason for this is that the time it takes to find and fix a defect is very unpredictable. Simple defects are found and cleared in minutes. Others may take days or even weeks to resolve. It is therefore obvious that the fewer defects found in testing the more predictable software development projects would become. 

Another key issue about finding defects in system test is that because it is extremely time-consuming and unpredictable the project usually runs out of time and budget before all defects are found. The development team knows that if more tests are run more defects will be found … but who will pay for this extra testing effort? 

There is a proven way of dramatically reducing the number of defects in software before system testing starts. It lies at the heart of the “Team Software Process” (TSP) that is now being used with great success by companies in the USA, Mexico and elsewhere.  We at the JCSE have just run a year long TSP pilot at Nedbank. The results in terms of quality have been extremely encouraging. 

On Tuesday I will be unveiling the JCSE’s “Thousand Job Strategy” which aims to make a significant impact on the South African software development sector. TSP as a way of improving the quality and predictability of software development projects is a central element of the strategy. 

If you are interested in hearing more and debating this strategy with me, please join us at the JCSE’s Annual Process Improvement Symposium on the morning of Tuesday 26th October 2010 (see www.jcse.org.za for more details). If you can’t join us then hopefully the debate will continue on this blog where I will post more details of the strategy after its launch on Tuesday.

My “Thousand Job Strategy” to be launched at JCSE’s Process Improvement Symposium

by Prof Barry Dwolatzky

Barry Fifa

So why do I write this blog? The answer is simple … I’m on a crusade. The sub-title of my blog makes it clear what this crusade is (broadly) about. It says I’m “passionate about the SA software industry”. My focus, however, is much sharper than that. Put quite simply … I’m on a crusade to ensure that the SA software sector grows in size and international reputation over the next 5 years. Furthermore I need to be able to accurately monitor and measure this growth. 

Is this a pointless crusade? Am I a Don Quixote figure tilting at windmills? 

Obviously I believe strongly that my mission is achievable. I also don’t, for one moment, underestimate the difficulties I face. 

So – let me lay it down in front of you!  Here is my action plan: 

1.  My first step is to clearly define what I mean by the “SA software development industry”.

2.  Having agreed what the “industry” is I need to measure its current performance. After considerable thought I’ve decided that the performance of the industry will be determined by collecting a set of 5 numbers from as many software development projects as possible. These numbers are:

  • Size: Number of people in the team.
  • Schedule Performance: What was the difference (in days) between the promised completion date and the actual completion date?
  • Cost performance:  What was the difference (in Rands) between the promised budget and the actual cost?
  • Project size/complexity: How big and how complex was the application developed in the project?
  • Quality: How many defects (or “bugs”) were discovered during system testing?

These – per project – measures will then be averaged to give a measure of the state of software development in South Africa.

3.  I will then implement a strategy (see below) to improve the performance of the industry. My strategy also aims to increase the number of people employed in developing software in South Africa.

4.  On an ongoing basis the measures listed above will be collected and reported on.

5.   If my crusade is to be a success, I would want to see improvements in both performance and the number of jobs.

Before you say that this is “pie-in-the-sky”, or “mission impossible”, let me ask what else we should do to sustain and grow our local software industry?  We need to have ambitious plans, and (I believe) we need to monitor progress. I accept that it’s going to be difficult, but I’m ready to try! 

I’ve developed a strategy (see point 3 above) that aims to achieve my mission. I call it the JCSE’s “Thousand Job Strategy”. It aims to create 1,000 new software development jobs in South Africa over the next 3 years. It also aims to achieve a significant and quantifiable improvement in the performance of local software development teams. 

Are you interested in finding out more about the “Thousand Job Strategy”? It will be unveiled at the annual JCSE Process Improvement Symposium on 26th October 2010 from 8:45 to 12:45 at the Sunnyside Park Hotel, Johannesburg.  I will be inviting comments, both supportive and critical. 

The Symposium will also be addressed by the eminent international software engineer, Prof. Dr.-Ing. Manfred Nagl, Emeritus Professor of Software Engineering, RWTH Aachen University, Germany. 

Visit www.jcse.org.za to find out more about the Symposium. Documents describing the “Thousand Job Strategy” will be posted on this blog after the Symposium.

Where does SA’s software development rank in world terms?

by Prof Barry Dwolatzky

Omni William Penn Hotel, Pittsburgh. Venue of TSP Symposium
Omni William Penn Hotel, Pittsburgh. Venue of TSP Symposium

 

How good is South Africa at software development? Where would we be placed in a league table of software developing nations? I guess the answer would depend on what one means by “good” and what such a “world ranking” would be based on. Continue reading “Where does SA’s software development rank in world terms?”

Public sector strike – dealing with a 21st Century workforce

by Prof Barry Dwolatzky

Public Sector Strike

We live in the “Information Age”. Modern organisations rely – or should rely – on a workforce with specialised knowledge and skills. Information Age employees need to be carefully recruited, grown, nurtured and retained.

These observations are particularly relevant as we enter the third week of major industrial action by unions in South Africa’s public service. Over the past fortnight we’ve seen more than a million government employees – health workers, teachers, and other staff – down tools in support of a demand for higher wages. Continue reading “Public sector strike – dealing with a 21st Century workforce”

Development of World Cup software shows success of balancing Agility and Discipline

by Prof Barry Dwolatzky

 agility and discipline book

Since my visit to the FIFA World Cup “IT Control Centre” (ITCC) last week [see elsewhere on this blog], I’ve thought a lot about the effectiveness of Agile Software Development. Continue reading “Development of World Cup software shows success of balancing Agility and Discipline”

Football and Software Development are both team sports

by Prof Barry Dwolatzky

 

Bafana Bafana

The 2010 World Cup started on June 11th with the mouth-watering prospect of seeing the greatest players on earth doing battle on the football fields of South Africa. Who would be the star of the tournament? Would it be Wayne Rooney, Lionel Messi or Cristiano Ronaldo? As we enter the last week of the tournament all of these stars and many more have returned home, having failed to make much of an impression.

One of the problems with modern football, and how it is marketed, is that the focus is on the star players – the heroes. This is wrong because football is a team sport. Each one of the 11 players has a role to play and the team will only succeed if all players work together on the dual objectives of scoring goals and defending their own goal posts.  At Manchester United Wayne Rooney is an inventive and exciting player. Wearing his England jersey Rooney at the 2010 World Cup was anything but exciting. The same can be said for most of the other football super-stars.

At club level the players work together every day for months and years in training and in matches learning to play as a team. Within the team each player is able to shine in a particular role, but the team provides the environment allowing each player to succeed. At the international level there is very little time for players to work together at becoming teams. It is therefore not surprising that individual skill and excellence doesn’t count for much at the World Cup.

There is a similar lesson to learn in the field of software development. Developing software is, like football, a team sport. Individual developers need to play specific roles in support of the project and its goals. There is – or should be – no room for heroes.

I’ve recently been thinking a lot about teams, both in the context of football and of software development. In terms of software I’ve been involved in coordinating a Team Software Process (TSP) pilot programme in South Africa. TSP is a teaming methodology developed at the Software Engineering Institute (SEI) in the USA. A TSP team focuses on defining and achieving goals together. There is a focus on individual performance, but as inputs to the team’s objectives. TSP teams also work – like a football team – with a skilled and professional coach. While the team works to deliver the project, the TSP coach works to develop the team.

There is a lot of commonality between a game of football and a software development project. I believe that football and software engineering can learn from each other. I wonder if FIFA or some of the national teams have thought about running TSP training for their players and coaches?

If Bafana Bafana are still looking for a new coach I would like to suggest myself! I know very little about football, but I know a lot about teams!