Why we prefer Espresso to Appium
I am a QA engineer who kickstarted my testing career in 2011. Like many other QA engineers, the monotonous life of manual testing drove me to drift to Automation in 2014, when Appium and Espresso were competing to be the winner in the field of test automation. Back then, I was working in a QA work pattern where the whole test automation suite across platforms was handled by the same QA engineer, and hence I opted for Selenium/Appium. Over the years, I have developed test frameworks with Java and Python for multiple cross-platform applications using Selenium/Appium.
A year ago I joined Busuu and I got the opportunity to explore the world of Espresso,which is a test framework created by Google for Android, to write UI tests. Here, we follow an Agile sprint methodology, and one independent QA is assigned per platform. Therefore, in contrast to having a common framework across platforms, we manage different test frameworks for different platforms. The reason behind this is, we have adopted the Developer-QA collaborated methodology of automated test development. The UI test PRs are reviewed and improvements are suggested by the Dev team, along with the QA team. Sometimes the Dev team also pitch in to help QA maintain existing UI tests as well. Hence each framework must be comfortable for the corresponding developers’ team to use. Espresso is an Instrument based test automation framework for Android, built on Java and Junit. And these two, being the core skillset for any Android developer, makes it perfect for the work process we follow at Busuu. And now if you ask me whether I will go back to Appium, I would say NEVER! Here I will brief why.
To begin with, Espresso is part of the Android Jetpack and integrated with the Google Android IDE — Android Studio. Also, the UI tests are written within the application source code. This provides a seamless start for any Android developer, without any extra setup or ramping up required, in contrast to Appium, which has a long complicated initial setup procedure to enable a test environment. And also Developers will have to switch to another framework to access the UI tests. As a result, with Espresso, developer and tester can use the same environment simultaneously for app development and automation suite development.
Since Espresso is part of the Android Jetpack and directly open-sourced by Google, it is always ahead of the curve. It provides frequent updates and provides enhanced support for newer Android versions promptly. Whereas Appium, which is also an open-source tool, is more hacky and breaks very easily when a new Android version is released. We need to depend on various discussion forums to follow up on deprecations and bugs introduced after the Android releases. And there were times when I had to wait long till a new version of Appium was released with an assured fix, for some of my tests to work as expected.
The next major attraction of Espresso is the fact that it is super fast. Appium indeed has the advantage of using a prebuilt APK to test, which saves a lot of time, unlike Espresso which uses a reasonable amount of time to build the APK. But this time saved by Appium is wasted in every request the Appium client makes: like find a view, request a click, perform a swipe. For each of these actions, the client has to request the Appium server running on the localhost and then the server invokes the action on the device. If this is the case, you can imagine how much time will it require for Appium to complete one test. Whereas Espresso tests live in the app and build the test suite as a standalone app alongside the AUT. Hence it is compiled as an application along with your AUT, which makes the test suite run faster than a test suite in Appium.Also the AUT is also a small native APK, separate from the app, which makes it even more faster.
A test which takes 30 sec to run in Appium, runs in 5 seconds in Espresso.Having a faster test framework is highly recommended for our accelerated deployment process as well, which means at least 1 release every 2 weeks. If the test suite is quick to run, the developers can regularly use it against their branche as and when they finish an implementation, before they give the branch for testing, and confirm immediately whether the branch has caused any regression issues.
Test flakiness is a prime factor when comparing the stability provided by different automation frameworks. While using Appium I had to rely on wait/sleep statements with timeouts to handle flakiness in the UI tests. Such tests does not scale well when the test device or the network is slow during the test run. Also, your tests will take longer time to complete depending on the timeout durations. Espresso introduces Automatic Synchronisation to help you here. It is true that Espresso gracefully waits for the main UI thread to be idle before any test actions are performed. However, results of asynchronous operations (say loading data, establish connection, or any complex business logic) always affect the subsequent test and may result in test flakiness. In such circumstances, Espresso does not need workarounds like wait/sleep. It uses an interesting feature called the Idling Resource mechanism. By registering an idling resource for every asynchronous activity in your app, forces the system to wait for the activity to complete, and the subsequent test step is not affected. With this phenomenon, I did avoid sleep statements in the UI tests, and it resulted in a perfect framework with no flaky tests.
The total time taken for a test suite to run is vital when comparing different test frameworks available in the industry. Avoiding sleep statements has importance here, as this is also one of the factors for Espresso test suite to take much less time compared to the Appium, which does not have a mechanism to handle this.
Native alerts in an Android app are a key factor that may interfere with your UI tests as well. To handle them, the test framework needs access to the notification. So getting permission to access them is significant. Espresso has the permission to handle them directly like any other element on the AUT. Appium can also perform actions on native alerts or buttons, but it uses a long workaround to do so.
Pros of Appium and how it doesn’t affect us
Appium supports every programming language that is supported by WebDriver. But Espresso is restricted to Java. This is not a con for us since the Busuu Android app is developed partly in Java and partly in Kotlin. And since the automation framework also is in Java, the UI tests are compatible with the app source code, and also Dev can pitch in anytime to help the QA maintain or troubleshoot issues with the test framework.
Appium is suitable for the QA team that handles cross-platform automation testing using a single framework. But here at Busuu, we have one QA per platform.
With Appium we can have a shared code base across platforms. But this does not affect us again since we prefer to maintain separate frameworks for separate platforms. Let it be Android/iOS/Backend. We want the dev team to be well acquainted with the automation framework, so either the dev/QA both can understand and use the test framework effortlessly.
Advanced Espresso
Espresso tests are written within the application source code, so it has access to all the resources a developer use: views, assets, colors, strings, etc, which are shared between the application code and the UI tests. This facilitates hassle-free and in-depth UI verification possible using Espresso.
Espresso has the flexibility to mock different layers of the app for the sake of testing. This is possible with the help of different tools like, Espresso intents (an extension to Espresso) which can be used to isolate a specific app screen or screen state. Another way is to use Mockito, a Java mocking framework, that can be used to mock a specific state of the app or user. Additionally, you can use compile-time dependency injection, with the help of the Dagger framework. This helps tests not to rely on the live application data. For instance, Busuu has a lot of learning content in the form of a variety of exercises like typing, matchup, fill gaps, dialog, etc. And the Education team regularly updates the content for all the 12 language courses we provide via Busuu, depending on popular demand. And we want to avoid indeterministic behaviour from our tests, whenever the content is altered. So we swap some dependencies in order to get a predefined response for some network requests. Hence they run independently and successfully verify all the expected behaviours of the corresponding exercise, regardless of the changing content in the app. Also, Busuu provides free content as well as premium content for all courses. We can mock the response of a free and premium user as well, to verify if the app behaviour is as expected for the two users respectively.
These capabilities facilitate black box testing, white box testing as well as hermetic testing possible using Espresso. Hermetic testing means isolating certain app areas in specific states and testing them. This helps prevent latent/masked defects in the app. Applying hermetic testing also enables us to write a lot of small UI tests, across self-contained app screens/states. Hence even if the app behaviour or user flow changes in the future, test maintenance is easy, since we choose isolated application states in every UI test. This support provided by Espresso towards hermetic testing keeps the tests completely independent and self-sufficient.
Espresso emphasizes providing as descriptive error messages as possible for a test failure, along with a dump of the view hierarchy that caused the error, so it help us to troubleshoot right away.
In short…
At Busuu, we follow 2 weeks-long sprints and each sprint has a minimum of 1 release, hence we won’t have time for longer QA cycles. An effective and fast test framework like Espresso, with extreme flexibility on test isolation, is the best for us to use in such an accelerated development process.
With Espresso, QA can play around with the actual app source code with free access to its assets, also isolate, inject and mock exclusive application states. Hence QA becomes more acquainted with the Android environment. At the same time, a QA using Appium is ignorant about the code beneath the application. On an interesting note, a expert in Espresso automation can eventually evolve into a Junior Android developer, if he/she wishes.
