What Is Manual Testing?
In software development, manual testing, also known as live testing, is the process of testing an application manually. This means that an actual human tester performs the test to validate functionality, user experience, and more.
Manual testing is different than automated testing. With automated testing, automated testing tools and frameworks are leveraged to create and run scripts that can be executed at any point in time to evaluate application functionality without human involvement.
While automated testing has many distinct advantages, manual testing is still a valuable strategy in a variety of scenarios. Below, we will dig into when manual testing makes sense and how it is performed, followed by a deeper comparison of manual versus automated testing.
When Should Live Testing Be Used?
As mentioned at the outset of this post, live testing can be useful in a variety of scenarios. Let’s take a look at a few of these scenarios in greater detail.
Infrequently tested, low-risk functionality
Manual testing makes a lot of sense for testing functionality that will not need to be tested repeatedly. Automated test coverage of 100% is not necessary or realistic. Therefore, for low-risk functionality that will need to be tested infrequently, manual testing can be the more effective and time-efficient approach. This type of functionality is not critical to the application and doesn’t change often enough to justify the development of an automated test script. An example of this would be a holiday-specific banner that shows up at the top of a web page on Thanksgiving. If this failed to appear, all critical application functionality would still work as designed. And this likely isn’t worth the effort to automate.
Functionality that is evolving quickly
For functionality that is evolving very quickly (i.e., UI testing early on in the development lifecycle when the design may frequently be changing), manual testing often makes sense. This is because developing an automated test to validate such functionality might not be a worthwhile endeavor, as it’s almost guaranteed that the test will need to be modified repeatedly before the functionality reaches a greater level of completeness. In this case, it can be difficult to design a test that will remotely meet its intended function, and an effort to automate can be made at a later time.
Evaluation of user experience
Live testing is also an effective solution for evaluating aspects of an application that can be challenging to evaluate in an automated fashion. This includes the usability of an application as it relates to user experience. For example, an application may be working as designed, but the design may have inherent flaws that result in a clunky user experience. While an automated test script would not fail in this scenario, a manual tester may be able to identify this shortfall. In other words, when the outcome of a test is more subjective, having an actual human perform the evaluation is necessary. In this way, a positive user experience can be ensured.
Exploratory testing
Finally, manual testing is a useful strategy for performing exploratory testing of an application. With exploratory testing, testers access the application, gain context of application functionality and flow, and then test functionality in an unscripted manner. The unstructured nature of exploratory testing means that it must be performed manually. This type of testing provides testers with flexibility in how they test application functionality, at times resulting in opportunities for them to use their skill set to identify additional test scenarios that were previously not considered. In this way, it provides a mechanism for discovering problematic edge cases that would potentially have made their way into production undetected.
How to Perform Manual Testing
While there can be some variation, manual testing can generally be performed by taking the following steps.
1. Test case analysis and script design
The first step is to identify the test cases needed to validate the functionality that you desire to test. This requires a complete understanding of the requirements that drive this functionality. Afterwards, step-by-step instructions should be developed to walk a tester through the process of testing each test case.
2. Test execution
Next is test execution. As mentioned above, manual test scripts are executed manually. So this requires that the tester follow the test script step-by-step, performing each step manually and recording the results.
3. Validation of results
Each manual test script will have an expected outcome. After executing the script, the tester will need to verify that the results of the execution match the expected outcome. If they do, the application passes the test. If not, then there is a potential bug that will need to be reported.
4. Reporting of results
After executing all tests, it’s time to report the results for the tests that did not result in the expected outcome. This means providing the steps necessary to reproduce the failed tests as well as providing the actual test results and the behavior that had been expected.
After the bugs have been fixed, re-execution of the failed tests should occur to ensure that a valid solution has been put in place.
Manual Testing in Practice
Now that we have reviewed the steps involved in manual testing, let’s look at a sample test script for manual execution.
In this example, consider a web application with a form for user registration. This form has the following fields:
First name
Last name
Email address
Password
Password confirmation
All fields are required and must contain valid information. For the sake of this example, let’s say the requirements are as follows:
First name field must contain only alphabetic characters.
Last name field must contain only alphabetic characters.
Email address must be in the format xxx@yyy.zzz.
Password field must be a minimum of 8 characters and must contain an uppercase letter, a lowercase letter, a special character, and a numeric character.
The password confirmation field must match the password field.
Let’s consider the scenario of testing user registration functionality. For this example, we will test the case that validates a successful user registration when the above requirements are met.
The design for this test case would look something like this:
Test Steps | 1. Go to www.<example domain name>.com. 2. Click “register” in the top right corner of the web page. 3. Enter valid data in the first name field. 4. Enter valid data in the last name field. 5. Enter valid data in the email address field. 6. Enter valid data in the password field. 7. Enter valid data in the password confirmation field. 8. Click “submit.” |
Test Data | First name: John Last name: Doe Email address: jdoe@<example domain name>.com Password: Abcd*123 Password confirmation: Abcd*123 |
Expected Outcome | User is registered successfully. |
With the above information, a test of a successful registration event can be validated. Keep in mind that this is just an example. And while this functionality can be tested manually, a browser automation framework like Selenium can be used to automate this test case to be executed as frequently as desired with no manual execution necessary.
Furthermore, this test case only validates a successful user registration. To ensure 100% test coverage of user registration functionality, test cases should be designed and executed for every possible failure as well.
Live Testing vs. Automation Testing
Live testing and automated testing have different advantages and disadvantages. Let’s take a look at how these two strategies compare in greater detail.
Method of execution: automated test execution vs. manual test execution
Automated tests are executed in an automated fashion using automated testing tools and frameworks. Manual tests are executed manually. This means that a human tester follows a script of directions dictating how the test should be performed. The human tester executes each step manually and validates the results against an expected outcome.
Risk of error in manual testing vs. automated testing
Manual testing lends itself to a greater risk of errors in the testing process. This is because manual tests are executed by human testers. This human involvement means a greater likelihood of human error (missed steps, inaccurately interpreted results, etc.).
Automated tests, on the other hand, are executed by software automation tools. Once an automated test script has been developed to properly meet requirements, it can be relied upon to run in the exact same manner with each execution, removing the risk of human error and ensuring accuracy during each execution.
Both Strategies Are Better Suited for Particular Types of Application Testing
Manual testing is well-suited for evaluating user experience, exploratory testing, and low-risk functionality that does not require frequent testing.
Automated testing, on the other hand, is all but necessary for efficiently executing unit and integration tests which are typically run with great frequency. These types of tests are critical for software development teams to ensure application quality throughout the development lifecycle as an application evolves. Performing these tests manually in an oft-repeated manner would take far too much time and inefficiently utilize development and QA resources.
Additionally, automated testing is necessary for performance and load testing. In these cases, manual testing is extremely difficult to utilize.
Required staff skill sets for manual testing vs. automated testing
Manual and automated testing requires organizations to hire individuals with specific skill sets. For instance, manual testers must have the ability to interpret application requirements, follow directions, think analytically, and quickly recognize problematic scenarios in an application. They must also be able to convey the results of their testing effectively to the appropriate staff.
For organizations aiming to implement automated testing for their applications, engineers with programming skills are required. They should be familiar (if not proficient) with the organization's automated testing tools and have an understanding of modern development practices (Agile, CI/CD, etc.).
Organizational investment
There is an organizational investment required for both manual and automated testing. However, the investment is quite different in each case.
With manual testing, the major investment is in the time it will take testers to execute tests with regularity. And since actual humans perform the testing, this takes time away from work on other initiatives. This time cost grows as either the number of tests or the frequency with which these tests need to be executed increases.
Automated testing, on the other hand, has a different type of cost, with a good portion of the investment being required upfront (and efficiency improving from there). Organizations need to set up a test infrastructure, train their staff to use their desired testing tools and frameworks, and then develop the automated test scripts they require. However, once this is complete, the process for running the tests is much faster, and the strategy as a whole is easier to scale.
To further illustrate our comparison between manual and automated testing, please see the table below:
Category | Manual Testing | Automated Testing |
Method of execution | Tests performed manually by human testers | Test scripts executed automatically |
Risk of error | Definite risk of error due to human involvement | Low risk of error – as long as the script is developed properly, it will run in exactly the same fashion and report accurate results with each execution |
Types of application testing | Exploratory testing, usability testing, infrequent testing of low-risk functionality, and testing quickly-evolving functionality when the design may be in flux | Valid strategy for most types of application testing; preferred for tests that will need to be executed on a regular basis, and preferred for aspects (such as load testing) where manual testing is not feasible |
Required skills | Experienced testers with the ability to perform the necessary tests and interpret and report the results | Programming expertise and familiarity with popular testing tools/frameworks and development methodologies |
Required investment | Manual testing is time-consuming and expensive in terms of staff hours when performed regularly or for a large number of scenarios | Upfront investment involved in setup, learning necessary technologies, and script development |
Scalability | Not easily scalable – more tests/executions means more time that staff will need to spend testing | Highly scalable – after initial investment, test scripts can typically be developed and folded in to an automated testing strategy with relative ease |
Wrapping Up
Manual, or live testing, is the process of testing an application manually – that is, a tester takes a test script, manually performs each step, validates the end result, and reports the outcome.
Today’s development organizations lean heavily (as they should) on automated testing for their applications. With that said, there are still opportunities for and benefits to performing manual testing on an application. A solution like the Sauce DevOps Test Toolchain offers both capabilities in one platform.
