At Google, we understand how important the quality of your application is in order to grow your user base, increase customer satisfaction and boost your revenues. When we took a closer look at data from 1-star reviews on Google Play, we observed that more than 50% of these reviews are related to bugs and crashes in the app.
Sampling of reviews in the Google Play Store, May 2016
For many developers, finding and resolving these problems prior to release can be very challenging. As the Android ecosystem continues to grow rapidly, it can be a daunting task to maintain a high quality, performant app that scales well for your users. As your user base grows, you may only have access to a handful of devices to use for testing. Accessing devices that are unavailable in your country is also difficult. Moreover, setting up a proper testing infrastructure that facilitates pre-release testing is a very expensive and time-consuming process that requires continual maintenance.
To help streamline the testing of mobile applications, we’re introducing Firebase Test Lab for Android, a platform where you can automate testing with the same tools that Google uses to test its own products. With a few easy steps, you can launch your tests on our physical devices to help ensure the best possible quality for your applications.
Your tests, our devices
Testing Android applications normally involves writing instrumentation tests to script the interactions with the app. If you’ve already written tests using Espresso, UI Automator 2.0, or Robotium, you can begin running those tests today on devices hosted by Firebase Test Lab.
Don’t write tests - record them
Authoring instrumentation tests is easier now with Android Studio 2.2 (or newer) using the new Espresso Test Recorder tool. All you have to do is launch your app in recording mode, and the test recorder will observe and remember all your interactions with the app, then generate test code in Espresso that duplicates those interactions. You can then turn around and run these tests in Firebase Test Lab.
No written tests? Use ours
Even if you’re not writing your own tests, you can still use Firebase Test Lab. We have a fully automated, intelligent test called a Robo test that will crawl your app by performing interactions with your app’s user interface. You don’t have to write a single line of code to gain the benefits of this automated coverage.
Run tests at scale - you choose the devices
With each test you run, you will select from a variety of device manufacturers and models, Android versions, and configurations (Virtual devices are also now available as a beta offering). Firebase Test Lab will then run your tests on multiple devices simultaneously to satisfy your selections as quickly as possible. When the tests are complete, the results from the test will be stored in your Firebase project.
Test early and often
Testing works best when it happens throughout the development process, not just before you publish your app on Google Play. To simplify this process, Firebase Test Lab gives you the ability to invoke tests directly from within the tools you’re already using, such as Android Studio and the Firebase Console. Our command-line interface allows you to run your tests through your continuous integration setup. Additionally, after you opt-in to receive the pre-launch report in the Google Play Developer Console, every new version of an app you publish to an Alpha or Beta channel will receive a free, five minute Robo test. These test results will become visible in the Play Store Developer Console.
Pricing
The Play pre-launch report generated from running the Robo tests is currently available at no cost! For more customized testing using Firebase Test Lab, customers on the Blaze Billing Plan will be charged $5 per physical device hour, and after October 1st, 2016, $1 per virtual device hour. Prior to that, virtual devices are available at no charge. See our pricing page for details.
Start improving your app quality today!
Running your first test on Firebase Test Lab is easy. You can follow our step-by-step codelab which walks you through various scenarios for running your first test. You can refer to our full documentation here. If you have questions or encounter any issues, please ask in the Firebase Google group.
Happy testing!