Question

Question 3
(a) Give a description of the Test-Driven Development paradigm and provide
details of two benefits.
[6 marks]
(b) Explain the difference between white-box testing and black-box testing.
Provide an advantage and a disadvantage of white-box testing
compared to black-box testing.
[6 marks]
(c) In black-box testing, it is usually infeasible to test all uses of an interface
of a module. Therefore it is common to test only a subset of all
possible input patterns. Describe a method by which such a subset can
be selected. Provide concrete examples in your answer.
[3 marks]
(d) As a project manager which of the following is preferable:
The test suite covers 100% of the critical code, but currently a
large number of these tests fail.
The test suite covers less than 50% of the critical code, but all tests
pass.
Justify your answer.
[4 marks]
[Total for Question 3: 19 marks]

Answer 1

Question 3

(a)   

The Test Driven Development is a software engineering practice that requires unit tests to be written before the code they are supposed to validate. Coming from the Agile world in which it is a basic practice of the Extreme programming method, TDD is nowadays recognized as a discipline in its own right that is also used outside the agile context.

   TDD = Refactoring + TFD.

TDD completely turns traditional development around. When you first go to implement a new feature, the first question that you ask is whether the existing design is the best design possible that enables you to implement that functionality. If so, you proceed via a TFD approach. If not, you refactor it locally to change the portion of the design affected by the new feature, enabling you to add that feature as easy as possible. As a result you will always be improving the quality of your design, thereby making it easier to work with in the future.

Test-driven development (TDD) is a development technique where you must first write a test that fails before you write new functional code. TDD is being quickly adopted by agile software developers for development of application source code and is even being adopted by Agile DBAs for database development. TDD should be seen as complementary to Agile Model Driven Development approaches and the two can and should be used together. TDD does not replace traditional testing, instead it defines a proven way to ensure effective unit testing. A side effect of TDD is that the resulting tests are working examples for invoking the code, thereby providing a working specification for the code. My experience is that TDD works incredibly well in practice and it is something that all software developers should consider adopting.

Test Driven Development provides its best results when the code is constantly improved. Advantages of test driven development is more than just simple validation of correctness; it can also drive the design of a program. Because of the testing modules that are built into the continuous integration development model, organizations using a TDD approach can easily make changes to their applications – all of that without the fear of ‘breaking’ the application and hamstringing their daily operations.

TDD leads to more modularized, flexible & extensible code. That is due to the fact that the methodology requires that the developers think of the software in terms of small units that can be written and tested independently and integrated together later. This leads to smaller, more focused classes, looser coupling & cleaner interfaces.

Benefits

1: Acceptance Criteria
When writing some new code, you usually have a list of features that are required, or acceptance criteria that needs to be met. You can use either of these as a means to know what you need to test and then, once you've got that list in the form of test code, you can rest safely in the knowledge that you haven't missed any work.

2: Focus
You're more productive while coding, and TDD helps keep that productivity high by narrowing your focus. You'll write one failing test, and focus solely on that to get it passing. It forces you to think about smaller chunks of functionality at a time rather than the application as a whole, and you can then incrementally build on a passing test, rather than trying to tackle the bigger picture from the get-go, which will probably result in more bugs, and therefore a longer development time.

3: Interfaces
Because you're writing a test for a single piece of functionality, writing a test first means you have to think about the public interface that other code in your application needs to integrate with. You don't think about the private methods or inner workings of what you're about to work on. From the perspective of the test, you're only writing method calls to test the public methods. This means that code will read well and make more sense.

4: Tidier Code
Continuing on from the point above, your tests are only interfacing with public methods, so you have a much better idea of what can be made private, meaning you don't accidentally expose methods that don't need to be public. If you weren't TDD'ing, and you made a method public, you'd then possibly have to support that in the future, meaning you've created extra work for yourself over a method that was only intended to be used internally in a class.

5: Dependencies
Will your new code have any dependencies? When writing your tests, you'll be able to mock these out without really worrying about what they are doing behind the scenes, which lets you focus on the logic within the class you're writing. An additional benefit is that the dependencies you mock would potentially be faster when running the tests, and not bring additional dependencies to your test suite, in the form of filesystems, networks, databases etc.

(b)

Criteria	Black Box Testing	White Box Testing
Definition	Black Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is NOT known to the tester	White Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester.
Levels Applicable To	Mainly applicable to higher levels of testing:Acceptance Testing System Testing	Mainly applicable to lower levels of testing:Unit Testing Integration Testing
Responsibility	Generally, independent Software Testers	Generally, Software Developers
Programming Knowledge	Not Required	Required
Implementation Knowledge	Not Required	Required
Basis for Test Cases	Requirement Specifications	Detail Design

Some of the advantages of white-box testing are:

1. Efficient in finding errors and problems
2. Required knowledge of internals of the software under test is beneficial for thorough testing
3. Allows finding hidden errors
4. Programmers introspection
5. Helps optimizing the code
6. Due to required internal knowledge of the software, maximum coverage is obtained

Some of the disadvantages of white-box testing are:

1. Might not find unimplemented or missing features
2. Requires high level knowledge of internals of the software under test
3. Requires code access

(c)

Black Box Testing is also known as behavioral, opaque-box, closed-box, specification-based or eye-to-eye testing.

It is a Software Testing method that analyses the functionality of a software/application without knowing much about the internal structure/design of the item that is being tested and compares the input value with the output value.

The main focus in Black Box Testing is on the functionality of the system as a whole. The term ‘Behavioral Testing' is also used for Black Box Testing. Behavioral test design is slightly different from the black-box test design because the use of internal knowledge isn't strictly forbidden, but it's still discouraged.

Each testing method has its own advantages and disadvantages. There are some bugs that cannot be found using the only black box or only white box technique.

Majority of the applications are tested by Black Box method. We need to cover the majority of test cases so that most of the bugs will get discovered by a Black-Box method.

This testing occurs throughout the software development and Testing Life Cycle i.e in Unit, Integration, System, Acceptance, and Regression Testing stages.

Few major types of Functional Testing are:

• Smoke Testing
• Sanity Testing
• Integration Testing
• System Testing
• Regression Testing
• User Acceptance Testing

Black Box Testing Techniques

Following are the prominent Test Strategy amongst the many used in Black box Testing

• Equivalence Class Testing: It is used to minimize the number of possible test cases to an optimum level while maintains reasonable test coverage.
• Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This technique determines whether a certain range of values are acceptable by the system or not. It is very useful in reducing the number of test cases. It is most suitable for the systems where an input is within certain ranges.
• Decision Table Testing: A decision table puts causes and their effects in a matrix. There is a unique combination in each column.

(d)

Test Coverage and Code Coverage

Test coverage is often confused with Code Coverage. Even though the underlying principles are the same, they are two different things.

Code Coverage really talks about unit testing practices that have to target all areas of the code at least once and is done by developers.

Test Coverage, on the other hand, is testing every requirement at least once and is obviously a QA team activity.

What really qualifies to be a covered requirement depends on the interpretation of each team.

For example, Some teams call a requirement covered if there is at least one test case against it. Sometimes, it is covered if at least one team member is assigned to it. Or, if all the test cases associated with it are executed.

• If there are 10 requirements and 100 tests created – when these 100 tests target all of the 10 requirements and don’t leave out any – we call this adequate testing coverage at the design level.
• When only 80 of the created tests are executed and target only 6 of the requirements. We say that 4 requirements are not covered even though 80% of testing is done. This is coverage statistics at an execution level.
• When only 90 tests relating to 8 requirements have assigned testers and the rest of them are not, we say the test assignment coverage is 80% (8 out of 10 requirements).

It is also important as to when to calculate coverage.

If you do this too early in the process, you will see a lot of gaps because things are still incomplete. So it is generally advised to wait until the Last Build i.e. Final Regression Build. This will give a correct coverage of the Tests performed for the given Requirements.