Introducing Unity support in Firebase AI Logic

May 20, 2025

We’re thrilled to announce the Preview of the Firebase AI Logic for Unity! This new client-side SDK makes it easy for you to integrate powerful generative AI capabilities, powered by Gemini models, directly into your games and Android XR applications built with Unity.

For developers building immersive experiences with Unity and targeting platforms like Android XR, integrating AI often requires complex backend setups to interact with models like Gemini. This adds extra development steps and can make leveraging advanced features like live audio streaming challenging. The Firebase AI Logic for Unity simplifies this process, providing a direct and secure way to access Gemini’s features within the Unity runtime. It’s designed to be a core component for building next-generation immersive AI experiences.

Get started at no cost, then choose the API that’s right for you

The SDK supports access to Gemini models via two pathways:

The Gemini Developer API is a great fit if you’re just starting to add generative AI in your app. It has a no-cost tier to help you start prototyping and exploring possibilities quickly and without friction. It also has a pay-as-you-go plan suitable for production apps. Support for this API is released in Preview, pursuing GA soon.
The Vertex AI Gemini API remains the go-to for apps needing enterprise-grade features. When availability and governance are critical for your production app, Vertex AI is built for those demands. Support for the Vertex AI Gemini API is already available in GA.

You can start developing at no cost with the Gemini Developer API then optionally transition to the Vertex AI Gemini API if that meets your needs better.

Uplevel your gaming and XR experiences with Gemini

Integrating generative AI through the Firebase AI Logic opens up a variety of possibilities for creating truly immersive and dynamic experiences in games and XR applications. Gemini’s capabilities, including multimodal input and output and bi-directional streaming for immersive conversational interfaces, are fully supported.

Imagine enhancing your applications with features like:

Interactive conversational storytelling: Provide users with an audio conversational interface, which can be used to ask questions about text-based content or metadata, adding a new layer of interaction to linear media, games, and XR experiences. This takes advantage of Gemini’s multimodal bidirectional streaming capabilities, which enables users to upload text, talk to Gemini, and get real-time voice responses.
Dynamic image generation: Allow users to describe in natural language a desired element and use AI to generate 2D graphics to create more immersive and dynamic environments that’s customized to the user’s experience utilizing Gemini’s image generation capabilities.
Intelligent NPCs: Make NPCs aware of the user’s context by providing them with text and images to describe the player and its environment so they can have more dynamic conversations and storylines throughout the experience. This takes advantage of Gemini’s multimodal input and text output feature set.

These are just a few examples. By bringing Gemini’s power directly into the Unity runtime, you can explore new forms of player interaction, create responsive and evolving game worlds, and build personalized experiences. The basic snippet below shows how easy it is to instantiate an NPC with recent context and memory.

var model = FirebaseAI.DefaultInstance.GetGenerativeModel(
  modelName: "gemini-2.0-flash",
  systemInstruction: ModelContent.Text(
    "You are a medieval innkeeper. The user is an adventurer looking for a room."/* + any other additional context */)
);

var chat = model.StartChat(/* Optional chat history */);

var response = await chat.SendMessageAsync("How much for a room?");
Debug.Log(response.Text);

Beyond just model access, the Firebase AI Logic for Unity is built with production readiness in mind. Out of the box it includes integrations with core Firebase services such as Authentication, App Check, and Cloud Storage. These integrations help enhance security and data management within your AI-powered features.

Support for Android XR Developer Preview 2

The Firebase AI Logic for Unity is compatible with Android XR projects in Unity 6! Add the SDK to your Android XR project to provide customized AI-powered experiences.

Generate text using multimodal input

In this example, we’re extracting colors and themes based on the user’s environment and generating artwork that matches it. This is done by passing an image of what the user sees and a prompt that tells Gemini to extract primary colors from the room so they can be used in the artwork.

var model = Firebase.DefaultInstance.GetGenerativeModel(
  modelName: “gemini-2.0-flash”
);

var multimodalPrompt = new [] {
  ModelContent.Text(textPrompt),
  ModelContent.InlineData("image/png",
    UnityEngine.ImageConversion.EncodeToPNG(unityTexture2D))
};

var response = await model.GenerateContentAsync(multimodalPrompt);
Debug.Log(response.Text);

Generate abstract artwork to place in the environment

Use the color palette that’s returned in the previous example to generate abstract images for the user.

var generationConfig = new GenerationConfig(
  responseModalities: new ResponseModality[] {
    ResponseModality.Text, ResponseModality.Image }
);

var model = FirebaseAI.DefaultInstance.GetGenerativeModel(
  modelName: “gemini-2.0-flash-preview-image-generation”,
  generationConfig: generationConfig
);

var response = await model.GenerateContentAsync(
  "Generate abstract artwork for a room that complements a color palette: " + roomColors
);

// Save the image into a local file.
var imagePart = response.Candidates.First().Content.Parts
  .OfType<ModelContent.InlineDataPart>()
  .First(part => part.MimeType.Contains("image"));
var imageFile = File.Create("output/path/to/image");
imageFile.Write(imagePart.Data.ToArray());
imageFile.Close();

Let the user see what the artwork would look like in the real world

Blend the AI-generated content with the real world to create an immersive XR experience.

The SDK supports Unity 2021 LTS and onward on both Android and iOS!

Build the future of AI-powered gaming and XR experiences

The launch of the Firebase AI Logic SDK for Unity is a significant step towards making cutting-edge generative AI accessible for games and immersive experiences. We’re excited to see what you’ll build! Check out our getting-started guide to learn more about using the Firebase AI Logic SDK for Unity. We welcome your feedback as we continue to evolve the SDK towards GA in the future. Start experimenting today, and we can’t see wait to see you build with AI in Unity and Android XR!

The Firebase Blog