Types of Software Testing:
As the development style is bending more towards the agile and scrum approaches, the need to test more often and at the minimal level has become a necessity of the industry. In this article, we will dig deep into different types of software testing.
Manual and Automated Software Testing:
Growing up in the era of Waterfall style software development, we have always seen software testing as a separate activity that needs to be performed by a different group of people, at the end of development; to confirm the sanity of our system.
As a result, it is a way of measuring quality but not a way of building quality into the product. We get valuable feedback from this type of software testing, but it comes so late in the development cycle that its value gets greatly diminished.
It also extends the schedule as the problems found are sent back to development for rework, to be followed by another round of software testing.
As the development style is bending more towards the agile and scrum approaches, the need to test more often and at the minimal level has become a necessity of the industry.
It’s more sensible and reliable to test the system at a minimal level to avoid disastrous situations at the later stages.
Self-Testing Code:
As Martin Fowler said, “At the core of my philosophy to testing is that we want to write self-testing code, meaning we have a suite of automated tests that be easily run against the codebase.
We should be confident that this suite will catch almost all bugs in the software, so that when this suite ‘goes green’ (passes), we can release that version of the software into production.”
Self software testing code or Automated software Testing behavior is one of the biggest advances in development methods in the last few decades. This developer-friendly practice has vast benefits in terms of increasing productivity, improving quality, and keeping software from becoming brittle.
The fact that so many developers are now doing it of their own free will speaks for its effectiveness.
What types of software testing should you have?
There are different types of software testing, contingent on what they are trying to verify, the extent of the codebase they cover, and their role in the software process. A popular way to look at your test suite is the “Test Pyramid” (Mike Cohn came up with this concept in his book Succeeding with Agile).
It’s a visual metaphor telling you to think about different layers of testing. It also tells how much testing to do in each layer.
Write lots of small and fast unit tests, some more integration tests, and very few high-level tests that test your application from end to end.
For most regular projects it is sufficient and provides the best possible coverage. People usually get confused by the names of layers in this pyramid.
General Metaphor for Testing:
Remember, this pyramid is a general metaphor for testing; Unit Testing is component-level testing, Service Testing is integration testing (test multiple modules together) and UI Testing is the end-to-end complete application testing.
This pyramid is ignored in applications that are more critical, such as avionics software or industrial applications such as SCADA (ex: concrete batching software).
Because here, we absolutely need to make sure that our software works properly with the hardware. Otherwise, the aircraft might crash and people will die.
Example:
Or for example, the hoppers above a mixer in a plant will not release the correct amount of ingredients for us to make the concrete that we need, resulting in a dead batch.
Unit Testing is the backbone of most of your applications’ test suites. You cannot guarantee the quality of software without unit testing.
As someone said, “You can’t be Agile without Unit Testing.”.
What is Unit Testing? (A type of Software Testing)
Unit testing is a method where components or units of software are continuously tested to determine their compliance with the system specifications.
Unit Testing also includes Data and usage procedure testing. A unit is simply the smallest piece of code that is logically isolated in a system.
The unit test itself is a short piece of code that is designed to verify the behavior of a particular unit to produce a pass or fail result.
Unit Test VS Functional Tests (Types of Software Testing):
When developers write Unit Tests, they often get confused between Functional and Unit Testing (Types of Software Testing).
What is the size of a unit? How can you tell if you’re writing the correct unit tests? Your system can add employees to the database, does testing this functionality can be considered a Unit Test?
- A functional test is when you test one complete functional requirement. e.g. Writing a single test for a User Story can be considered as a Functional Test.
- As for the unit testing, there is no predefined size of it. It is a wide term.
Michael Feathers, the author of Working Effectively with Legacy Code, concurs that there is no “definition of unit test”.
List of Indicators:
He instead offers a list of indicators that one shouldn’t find in a unit test.
A unit test should not:
- Talk to Database (until its sole purpose is to test the database).
- Communicate across the network.
- Touch the file system
- A special environment (e.g. editing in config files) is required to run it.
- It can’t run at the same time as any of your other unit tests
If your code is properly refactored, then 1 Method = 1 Unit Test is often acceptable.
SUT:
In unit testing which is a type of software testing, your SUT (System Under Test) should be fully isolated from other pieces of software.
If it depends on another method, a service, database, or maybe data from an external source then your unit test is not ideal.
Imagine you’re testing an invoice class’s create method. The invoice creates method needs to invoke some functions on the order and customer classes.
Ideally, your test shouldn’t use the real order or customer classes here, because a fault in the order class would cause the invoice class’s tests to fail. Instead, you use Test-doubles for the dependents.
Figure 1: A unit test typically replaces external collaborators/dependents with test doubles
Note: The SUT is whatever class or method(s) we are writing the unit test for.
Test Doubles :
A Test Double is any object/component that we install in place of the actual component to easily run the test. There are a few different kinds of test doubles,
Fake: An actual implementation of the piece of code, but is not suitable for production.
Dummy: Used as a placeholder when an argument needs to be filled in.
Stub: Provides the SUT with fake data.
Spy: Records details about how it is used and will return the information back to the test.
Mock: Defines an expectation of how and for which parameters it will be used. If the requirement is not met, it will cause a test to automatically fail.
The concern of Test Doubles:
As a software engineer, you will mainly be more concerned with mocks and stubs. Later on, maybe spies. A lot of people use the words Mock and Stub interchangeably.
Though at the top they might look the same at the core they have different responsibilities.
The need for Stub/Mocks can be well understood by looking into “Indirect Input” and “Indirect Output” of your SUT.
Indirect Input:
Not all SUT inputs are the output of the test. Some of the indirect inputs come from other SUT-call components in the form of return values, modified parameters, or thrown exceptions.
Here, we don’t really care about what is happening in the dependent component. We care only about what it returns as a result.
Indirect Output:
Not all SUT outputs are visible to the test. In the form of method calls or messages, such indirect outputs are sent to other components. We have to be able to observe the calls that the SUT makes to the dependent component to evaluate the indirect outputs.
In addition, we need to be able to control the values returned if we need the test the progress beyond that stage.
Stubs and Mocks with Indirect Inputs and Outputs:
Stubs:
A Test Stub is an object that acts as a control point (component) to deliver indirect inputs to the SUT when the Test Stub’s methods are called.
It retains predefined data and uses it during testing to answer calls. It is used when we don’t want or can’t involve items that would answer with real data or have unwanted side effects.
The internal structure of this object is not important for our test.
Example:
For example, an entity that needs to fetch certain data from the database to respond to a method call might be an example. We create a stub instead of the individual object and decide what data should be returned.
<?php
public function averageGrades(Student student)
{
return averageStub(student.grades);
}
Public function averageStub(grades){
...
return 8;
}
We can use multiple stubs to test the different execution paths in our SUT. If the method we are testing calls a function that returns information that affects the execution path in our SUT, we need to use stubs.
Mocks:
Mocks are different, here we actually care about the function we’re calling and not what it is returning. A Mock Object can act as an observation point (component) for the indirect outputs of the SUT. In mocks, we care about what that function received from our SUT.
When we don’t use the actual code or when there is no clear way to validate the intended code execution, we use mocks. There no easy way to check return value or system state change.
Imagine we’re writing a test for a method that calls an external API that takes employee information and does a background check on it.
Here we do care that our SUT is calling the API method correctly.
That’s an ideal use case for a mock because we want to make sure our SUT is behaving properly and calling what we need with the right parameters at right time.
Structure of Stubs and Mocks:
Tests written with mocks usually follow an initialize -> set goals/expectations -> exercise -> verify test patterns. Whereas, initialize -> exercise -> verify would be followed by the pre-written stub.
An easy way to remember the difference between mocks and stubs is to remember that state is verified by stubs, and behavior is verified by mocks.
| A great blog post that explains the difference between mocking and stubbing can be found here: https://martinfowler.com/articles/mocksArentStubs.html |
Unit Testing & Laravel:
As for now, we have learned the basics of Unit Testing. You might not have understood everything yet. Unit Testing is a wide concept and it requires practice to be skilled.
PHPUnit is the testing framework for PHP. Like for other languages, JUnit and XUnit. JUnit for Java and XUnit for NET.
Before, we had to do everything with PHPUnit, because that is what was available.
Writing mocks with PHPUnit was painful but with the invention of Laravel; PHP’s most famous framework, testing has become a lot easier.
Laravel Mocks:
Laravel has been built by keeping testing in mind. Support for PHPUnit testing is actually included out of the box, and a phpunit.xml file has already been set up for use.
By default Laravel’s ‘’tests’’ directory contains two sub-directories, “Feature” and “Unit” for feature (functional) and unit tests respectively.
Making tests in Laravel is easy. You only need to run PHP artisan make test –unit command and it’ll create an empty Unit Test in tests>Unit directory.
<?php
namespace Tests\Unit;
use PHPUnit\Framework\TestCase;
class ExampleTest extends TestCase
{
/**
* A basic test example.
*
* @return void
*/
public function testBasicTest()
{
$this->assertTrue(true);
}
}
Laravel offers many built-in methods and assertions to make testing pain-free.
Mockery:
To make mocking easy, Laravel use Mockery. It is a simple mock object framework for PHP which you can be used with PHPUnit.
You can read more about mocking in Laravel by going to this link: https://laravel.com/docs/8.x/mocking
Here, we can see that Laravel gives many built-in Mocks Objects for Notifications, Emails, Queues, and File Uploads, etc. which make our lives easier.
To start writing unit tests in Laravel, the best source of knowledge is https://laravel.com/docs/8.x/testing
Things to Remember in the selection of types of software testing:
This article is about the general idea of types of software Testing. You might not have learned everything but what you’ve learned will help you get started with unit testing.
Unit testing (a type of software testing) is practical knowledge, you can’t excel in it by just reading different books and articles. I recommend you start writing Unit Tests for your application today.
You might not be able to write a perfect test at once but with practice and skills, you’ll gradually succeed in it.
Further Reading about Types of Software Testing:
Testing Guide – Martin Fowler
The Practical test Pyramid – Martin Fowler
Test Double – Martin Fowler
Test Double – xUnit Patterns
Mocks Aren’t Stubs – Martin Fowler
You can read our different articles regarding web development by clicking here.