Brian On Software

There were two courses, during my undergraduate studies in Software Engineering, that forever changed my perspective towards software quality: Software Quality Assurance, and Software Quality Control and Testing. The former dealt more with quality assurance and process improvement with various frameworks (CMMI, ITIL, Six Sigma, etc.). The latter dealt more with service level agreements, various testing techniques and quality control. I purposely put “quality assurance” and “quality control” in bold, because I want to emphasize both terms. There is a distinction between these two terms. They do NOT mean the same thing, and they are NOT the same thing.

As you may know, I tend to write more about software development than specifics about software engineering, but I believe that as a professional software engineer, we should use and speak the same lingo, a sort of ubiquitous language within the profession, when defining an intent, a need or a role. In object-oriented programming, we use Design Patterns to reveal an intention behind a specific design or architecture. For example, when someone says “I used the Composite pattern in this situation for X reason” or “We decided to use a Strategy pattern with an Abstract Factory to do X and Y”, it communicates design decisions across the team. Even police officers use their own codes to share their intent amongst their fellow officers. For example, they’ll use a “10-4″ to acknowledge something, a “10-15″ to inform whoever is listening that there’s a prisoner in custody. Many professions have their own codes, standards and guidelines to easily share information between their peers and facilitate communication.

A while back, I wrote a post on “Promoting Professionalism Through A Common Body of Knowledge“. If you read any body of knowledge, you’ll note that each one of them defines a vocabulary that is common to the related profession. For some reason, which is beyond my understanding, the software engineering profession tends to use and express certain vocabularies which are neither standardized nor shared across organizations nor rightfully used in a particular context. For instance, we have titles such as “Software Architects”, “Software Designers”, “Software Specialists” and “Software Analysts” which are used liberally often without knowledge of the role. The worst one is “Software Developer”. Seriously, what does a “software developer” really mean? From what I understand, it’s a catch-all term which means a lot of things. It’s the “default switch case” of software development job titles. Anyhow, that is a discussion for another post…

In this post, I want to concentrate my energy on two terms, or “job titles”, which are often improperly used in our profession. These two terms are important to differentiate and understand, because they both deal with software quality. Quality is a major player in today’s software systems: it greatly influences the economics of the team, the credibility, the reputation and the lives of the people involved in the project, and most importantly it continuously defines the professionalism in our profession. Are we just hacking things together and wishing for the best? I hope not.

In his article, “Cost of Software Quality”, Dan Houston states that:

Software quality is difficult to define because there is no single comprehensive and complete standard definition of its lexicon. Various aspects and terms are found in sources such as ISO 9000-3, IEEE Software Engineering Standards, and various books on the subject. The following are some of the various definitions of software quality.

• Level of satisfaction: it is the degree to which a customer or user perceives that a software product meets his or her composite needs and expectations.
• Product value: it is the degree to which a software product has value relative to its various stakeholders and the competitive situation.
• Key attributes (”ilities”): it is the degree to which a software product possesses a desired combination of properties.
• Free of defects: it is the degree to which a software product works correctly in target user environments, free from operational defects.
• Process quality: in relation to the development process by which the product is produced, it means good people doing the right things in an effective way.

From that definition, we can say that software quality is a composition of defects prevention and defects detection activities. By “defects prevention”, we mean a set of rules and practices that helps us to build the system in way that will minimize problems in the road ahead. In other words, it’s making sure we’re building the system right. This is what we call Software Quality Assurance (SQA). By contrast, when we talk about “defects detection”, we mean a set of rules and practices that helps us to correct, tune and free our system of problems, such as bugs. In other words, it’s making sure we’re building the right system. This is what we call Software Quality Control (SQC). Let us now formally define both of these terms:

Wikipedia defines the term Software Quality Assurance as:

Software Quality Assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. It does this by means of audits of the quality management system under which the software system is created. It is distinct from software quality control which includes reviewing requirements documents, and software testing.

SQA encompasses the entire software development process, which includes processes such as software design, coding, source code control, code reviews, change management, configuration management, and release management. Whereas software quality control is a control of products, software quality assurance is a control of processes.

Wikipedia defines the term Software Quality Control as:

Software Quality Control (also known as Verification and Validation (software)) consists of a means of controlling the quality of software engineering products. It does this by means of tests of the software system. These tests can be unit tests, integration tests, or system tests. It also includes the formal proof of individual pieces of code, and the review of documents and code.

It is distinct from software quality assurance which includes audits of the quality management system against a standard. Whereas software quality control is a control of products, software quality assurance is a control of processes.

The main difference between these two terms were underlined in bold:

Whereas software quality control is a control of products, software quality assurance if a control or processes.

The same way that, in object-oriented programming, an abstract class isn’t the same as an interface, there is a huge difference between “software quality assurance” and “software quality control“. SQA deals more on the methods and processes we’re going to adopt to build the system. SQC deals more on making sure the system is behaving as it was meant and designed to behave.

I used to work for a company that had a QA Director. I once asked him which processes were used to develop their systems. He replied that he didn’t have a clue. I then asked him what was his role as a QA Director if he didn’t know which processes were used in the organization to develop the systems. He replied that he was responsible to manage a team that was testing the system. Had his job title be “Quality Control Director” instead of “Quality Assurance Director”, I could’ve saved myself a couple of minutes that day. There’s something else wrong with this picture. Imagine the following scenario:

Helen, who is member of the QA Director’s team, decides to leave the company to work somewhere else. As she prepares her CV, she writes “QA Engineer” or “QA Tester” as her former job title. As she submits her CV to various interesting companies she would like to work at, she patiently waits for that one phone call. But there’s a problem: the phone doesn’t ring. There might be a few reasons why no potential employer is calling her back for a job interview. Maybe, this is what happened:

Helen submitted her CV to work as a “QA Engineer” at Company X. The only problem is that Company X actually gets the difference between software quality assurance and software quality control. So the recruiter sees Helen’s qualifications on her CV, but her job title doesn’t reflect the work she was doing at her former company. Her former responsibilities don’t fit with the position she applied for. The recruiter shreds Helen’s CV and takes the next one on the pile.

Had Helen wrote “QC Engineer” or “System Tester” as her former job title, the story could’ve ended on a more positive note for her and Company X, because Helen would rightfully be describing her previous role to Company X. Let us flip the coin and imagine for an instance that Helen knew exactly that her former role was “QC Engineer”, but Company X mistakes quality control for quality assurance. In that case, Company X will miss out on opportunities to hire software engineers that really know their stuff. For example, imagine a seasoned professional SQA Engineer being interviewed for a SQA position with Company X. As the interview unfolds, the SQA Engineer realizes that Company X has no idea what SQA is all about. The SQA engineer gets up and walks way from the interview, because as a professional engineer you’re obliged, by law, to “undertake only such work as he or she is competent to perform by virtue of his or her education, training and experience“.

You might think that the previous scenario was just a single event, but in fact a simple search for “quality assurance manager” in Workopolis produced the following result (I highlighted the word “test” in the job description):

The position is clearly one for a Quality Control Manager, NOT a Quality Assurance Manager. Spreading this kind of information, this kind of evangelism, is hard because so much have been written already about SQA being about testing. For instance, software engineer Steve McConnell wrote an article in 1996, “Software Quality at Top Speed“, where he clearly states that:

The most common quality-assurance practice is undoubtedly execution testing, finding errors by executing a program and seeing what happens. The two basic kinds of execution testing are unit tests, in which the developer checks his or her own code to verify that it works correctly, and system tests, in which an independent tester checks to see whether the system operates as expected.

This statement is wrong because it is not compatible with the vocabulary defined by our profession. He’s mixing apples and oranges. Even though Steve McConnell wrote that QA was about testing back in 1996, I know that he gets it today, because you can clearly see the distinction of both terms on Construx’s Testing & QA course plan:

Software testing refers to executing software to detect defects and evaluate features. Software testing includes test planning, test case design, automated test support, and specific kinds of tests including development tests, unit tests, component tests, integration tests, system tests, regression tests, stress tests, and acceptance tests. Software QA refers to activities that assure a specified level of quality in the software, usually prior to testing.

So if you’re a SQA Director or a SQA Engineer and the only work you do is software testing, developing test plans or conduct verification activities on the system, you should revise your title. Maybe you’re more of a SQC Director or a SQC Engineer. As more students graduate from software engineering curriculums around the world, it shouldn’t be too long that someone else will ask you these questions and they’ll expect you to answer them correctly. And if you’re looking for a specific job in software quality engineering, you should be very clear whether you’re looking for a quality assurance position or a quality control position. Furthermore, you should make sure that the company you’re applying at knows the difference too.

In essence, this whole post can be resumed with these words: Software Quality Control is about testing; Software Quality Assurance is NOT.

If you want to read and learn more on the subject of Software Quality Engineering, I invite you to download the Software Engineering Body of Knowledge and check out both Chapter 5: Software Testing and Chapter 11: Software Quality (I still don’t understand why they put this chapter last).