Emulators have come far since the early days of software testing. Today, emulators work so well that you may wonder why you can’t use them for everything.
Well, the reality is that, although emulators are a great tool for many situations, they’re no magic bullet. There are cases where testing on real devices is just a better approach.
In this post, I explain why you can’t always rely on emulators, and should instead test on real devices. I’m not here to knock emulators—my goal is to help you identify the situations where real-device testing is a better approach.
Overview: We need to test on real devices
Software testing involves a range of steps that must be established and organized to meet all customer requirements. Virtual and real-device testing often yield different results. In many cases, virtual environments cannot mimic real-device problems, like insufficient hardware, firmware version changes, data bandwidth issues, and so on.
Virtual environments also give software testers more control over their configurations. While this is in many ways a good thing, it is a disadvantage in the sense that it tends to lead to tests that miss many of the configuration nuances and oddities of production environments, because software testers are often better at configuring their environments than real-life end users.
The benefits of emulators
Before explaining why real-device testing is important, let me explain the value of emulators—because the point of this article is not to say that you should never use emulators. They often come in handy.)
Why? The main reason is that, when you use a virtual device for testing, you can change the configuration very quickly. You can modify software definitions, connection configuration, firmware and so on in order to mimic different testing environments without having to apply those changes to a real device. In this way, virtual-device testing helps to make tests faster, more flexible and less expensive.
Real-device testing use cases
The above notwithstanding, real-device testing is sometimes a requirement. Consider the following examples of use cases where real devices are a better testing solution:
With applications that make massive use of GPS, testing on real devices will yield more accurate results that better reflect real-world conditions.Some devices require specific tests when using features such as touch screens, touch depths and biometric readers. In these cases, it is fundamental to test the application on the devices because the virtual tests in emulators do not cover everything necessary.
In the case of an application that does facial recognition, the camera quality and image resolution will directly impact the proper functioning of the software. An emulator will use a completely different camera from the camera of the device, and consequently, the result will not be the same as when the test is done on the actual device.
Software that uses GPS for locating and tracking dogs (for example), needs to be tested in the external environment for verification, according to the region where it will be used. The behavior of the application using GPS in an integrated way with the data network can only be simulated and tested safely if the tests are performed with the device itself.
If you want to develop an application for IoT that opens the door of your home with the help of a mobile phone, this specific test can’t be properly done in a virtual environment due to the need for proximity between gadgets. If you only test an application like this in a virtual environment, you risk end users discovering that the application doesn’t always work as expected due to connectivity problems, interference, etc.
Tests on real devices are also helpful for creating documentation with a higher quality of information that anticipates real-life conditions.
It’s also worth keeping in mind that developers need to include project logging mechanisms that can capture and alert on all errors generated during testing and after the app is published in stores. This data also provides insights based on real-device behavior that can assist with testing and quality assurance.
Conclusion
The key takeaway is that only tests performed on real devices put you in the place of the flesh-and-blood end user. Yes, virtual test environments are useful and necessary in order to meet the challenges of continuous delivery. But you need the proper balance between virtual testing and real-device testing.
And remember, there’s no reason you can’t do both. You can test on virtual devices earlier in the release cycle, then do real-device testing for the applications that require it before release.
Brena Monteiro is a software engineer with experience in the analysis and development of systems. She is a free software enthusiast and an apprentice of new technologies.
