Ensuring your apps are high quality before release is crucial. Manual testing is a standard practice but it is expensive and time-consuming. Automated testing frameworks can have flaky tests and are difficult to maintain. To smooth out this part of the development process, we’re excited to share a preview of the App Testing agent, which uses the power of AI to help test your app directly within App Distribution. It provides test coverage over your app’s core user journeys and since the agent will do the testing for you, your team has more time to focus on building exciting features that your users will love.

Meet the App Testing agent

The App Testing agent is a Gemini-powered test case generation, management, and execution agent designed to work directly within App Distribution. You define test goals in natural language, and the agent uses AI to understand and navigate your app, simulate user interactions, and provide detailed test results.

The App Testing agent is designed to:

• help you automate and expand your functional and UI testing efforts
• be codeless, making the agent accessible to a wider range of team members
• be low maintenance by adapting to changes in your app so you spend less time fixing broken tests
• run tests simultaneously against various physical and virtual devices

Let’s say you are building a travel app that lets users describe their ideal trip in natural language and receive a matching itinerary. With the App Testing agent, you can create a test case focused on their core app flow.

First, you create a test case with two steps. Each test step has a goal that tells the agent what to do. In this case, the two goals are:

1. “Start a search using a description of a dream trip to Greece."
2. "Open the first result.”

Next, to help the App Testing agent successfully navigate a few tricky parts of the UI, you can add hints. For example, you add a hint that the agent must hide the keyboard after entering text in order to expose the submit button.

Finally, you can optionally tell the agent what to expect during a passing test run by adding success criteria, such as “The search has been started”.

Once this test case is created, you can tell the App Testing agent, powered by Gemini, to run this test on a set of physical and/or virtual devices with different locales and orientations.

After the test runs, you can see if the test:

• passed: the success criteria were met, and the itinerary was displayed
• failed: there is a potential regression or issue in the latest build

To dig into a particular test run, you can view a step-by-step breakdown of the actions the agent took. You can review screenshots and explanations to understand how the agent navigated the app and why the test passed or failed.

The agent’s AI capabilities power intuitive navigation of many app flows on its own. But as you can imagine, it’s not perfect, and sometimes it will take different actions for the same input. In the case where the agent has trouble navigating the app, you can refine the test case by adding hints or splitting the test case into smaller, more manageable steps.

Looking ahead

The App Testing agent offers a valuable way to enhance your app testing workflow, helping you release more stable apps while providing gains in productivity. Try it out with your Android apps today in App Distribution. We would love your feedback to help further shape the future of this feature, and help us make the agent even smarter and more reliable. We can’t wait to see what you build with it!