Google went big at this year’s Game Developer Conference (GDC), focusing on the ways generative AI (“gen AI”) is profoundly impacting the gaming industry. A three-part series of presentations featured gaming and gen AI leaders across Google who dove into the company’s AI and gen AI development and publishing solutions, the transformative impact of AI on real-time, in-game experience, and actionable insights for developers ready to get started on their own AI journey.
Gen AI enabling “living games”
“Games are entering a new era,” said Jack Buser, director for games at Google Cloud, with “living games” expected to hit the market within the next three to five years. These incorporate many of the fundamentals of live service games or live ops games, but are also underpinned by generative AI.
“Ultimately games will be able to respond directly to players’ implicit or explicit instructions,” he explained. “The game will be able to create content on the fly to immediately meet the needs of an individual, or even a small cohort of users. You’ll see something coming very soon from my colleagues at Google on exactly that.”
Gen AI in the game development lifecycle
In the meantime, many studios are integrating generative AI into the production pipeline, often using tools like Google Cloud’s Vertex AI, a managed platform for AI and gen AI, to develop games, localize games for other markets and transform in-game experiences. Developers are even creating their own LLMs, Buser added — for instance, Google Cloud recently worked with NCSOFT to develop its VARCO LLM.
Generative AI is also impacting the publishing and distribution space, says Lei Zhang, director of Play Partnerships, global generative AI, and greater China BD at Google.
“We’re moving from just distribution to covering the entire lifecycle for both game developers and gamers,” Zhang said. “We use generative AI to enhance game discovery. We use it to enhance developers’ experience, helping them create marketing assets on the Play Store. In the foreseeable future, your [ games’ ] store descriptions and graphical assets can be generated with generative AI.”
“Fundamentally, I think generative AI can help across the board when you look at the business,” added Simon Tokumine, director of product management at Google AI. “But just focusing on the creative side of things for a second, we put a lot of our cutting-edge models up on our Labs portal — these are all tools to help with the creative flow.”
He pointed to recent work from Lupe Fiasco, who partnered with Google to explore linguistic flow and new creative ways of expression, and from electronic musician Dan Deacon, who uses AI assistance to fine-tune live performances.
Gemini 1.5 offering new game development opportunities
Gemini 1.5 Pro, Google’s mid-size multimodal model, was recently released in more than 180 countries and territories via the Gemini API in public preview. It offers new features like native speech understanding, it’s optimized for text, image, video, audio and coding tasks, and can process up to 1 million tokens in production (and up to 10 million in the lab, Tokumine added).
“The models are finally able to maintain context and use that to generate a cogent response,” he said. “If you’ve ever tried to do RAG-based solutions for information retrieval, you’ll know that’s a major issue and a real challenge. I’m excited about the models from the perspective of creating new things.”
Google Cloud offers developers the benefits of a managed platform to keep data secure and fine-tune models, as well as providing access to a huge library of third-party models, Tokumine added, pointing to Google’s recent deal with Hugging Face, which has more than 100,000 models available.
The potential of LLMs for game developers
Glenn Cameron, product marketing manager at Google, dug into the possibilities of LLMs for game development. These massive models can understand complex requests and return accurate, nuanced responses — which makes them particularly effective as creative assistants via chat interfaces, Cameron said.
“They can be your inspiration engine, get you past creative blocks, especially when you’re fleshing out quests or character stories. They can be your collaborator. You can ask them anything and they can help you come up with ideas,” he explained. “These things are transforming the way that developers approach game creation, especially in the early stages.”
Text-to-image models, as well as technology like DreamBooth, Google’s deep learning generation model which can be used to personalize existing text-to-image models by fine-tuning, can be used to visualize characters, render images from text, backgrounds, videos and more.
It can be used as a natural code assistant, to produce the code you want for a specific function in the coding language you prefer, or as a code completion assistant. With the groundbreaking 1 million context window, models like Gemini 1.5 Pro can help track all the details of your game’s lore and ensure a seamless, believable narrative experience — including dynamic, unscripted dialogue and open-ended chats from NPCs that remember previous encounters.
Google’s lightweight, open model for development
Google offers two model families, Gemini and the newly released Gemma, both of which can be used in games. Gemma is based on the research and tech used to create Gemini models, but is more lightweight and open. It was launched in all of the major libraries and frameworks, including Keras, JAX, TensorFlow, PyTorch and Hugging Face, while Gemma can be run as a binary on a CPU.
The model comes in two sizes: a 2-billion parameter size which can run locally on a mobile phone or a laptop, and the 7 billion parameter model, which requires the power of a desktop gaming GPU. Gemma also comes with a responsible generative AI toolkit, which helps developers put guardrails on content to help ensure a safe, enjoyable experience for players.
But with great power comes great responsibility, Cameron added.
“LLMs are trained on massive amounts of data, and this data has bias in it. It has toxic content in it. These things can make biased content, which is dangerous,” he said. “You need to be careful when you deploy these things. You need to make sure that you monitor them, that you’re taking care of that. There are a lot of reputational and financial risks that you could incur if you don’t do that.”
Getting started with gen AI for living games
In the final presentation, Dan Zaratsian, AI and machine learning solutions architect at Google Cloud and Giovane Moura Jr., hybrid cloud specialist at Google, demonstrated how generative AI and Google Cloud can support innovative player interactions, with the example of a multiplayer game powered by Google Cloud. They built it on managed Kubernetes with Google Kubernetes Engine (GKE) behind the scenes, making it easy to scale and distribute AI workloads across the globe. It offers low latency and flexible interoperability, which means it can easily switch between open models — for example, from Vertex to a local open model, and can run anywhere without lock in.
Spanner stores embeddings, which is important for very quick lookups of RAG-based approaches to not only retrieve relevant embedding information to pass into a prompt, but can also integrate with other structured data — and still have global scale and fully asset-compliant consistency from a database that can serve a global community.
Maintaining long-term memory with a database like Spanner — especially in ephemeral types of services that are globally scalable — is critical for creating lifelike NPCs. The chat history and interactions for each NPC can be stored in Spanner. Then, when a new question is asked of the NPC, we’re able to semantically search historical player conversations in order to recall relevant conversations. Those conversations are then used by the LLM to intelligently answer the player’s question, taking into consideration the question itself as well as the learned knowledge that it gathered through past conversations. Powering hundreds of thoughts of NPCs globally, at scale, and with a low latency makes Spanner a top choice for this application.
The future of generative AI in game dev
Going forward, new tools and services aimed at changing the way developers work and think will continue to appear. For instance, the Google Cloud team is working on federated queries to make NPC behavior far more complex. Rather than passing queries back and forth to a single limited LLM, chained or sequential groups of LLMs can federate queries to a variety of endpoints — another LLM, a SQL database, an API and so on. In other words, if a player asks an NPC a question that the LLM isn’t set up to answer, or is asked to take an action they’re not designed to take, a background query could be sent against a SQL database with that information or an API request might be triggered.
“If you think of chaining together these NPCs, these LLMs, to do something very sequential and very powerful, you can start to get a lot more out of LLMs than just one pass back and forth to the LLM,” Zaratsian said.
