Leaked video appears to show Sony director conversing with an AI Aloy prototype

PlayStation is reportedly using characters from its biggest franchises in an AI prototype.

A leaked video appeared to show PlayStation’s director of software engineering, Sharwin Raghoebardajal, conversing with an early AI Aloy prototype.

The video – sent to The Verge but since taken offline following a copyright claim from Sony – showed Raghoebardajal give a brief tech demo of Aloy communicating with him voice prompts both within the AI prototype and in the game Horizon Forbidden West itself.

“This is just a glimpse of what is possible,” Raghoebardajal said, before demonstrating how Aloy can give players lore and background information upon request.

Whilst the voice does not match that of the in-game Aloy, the character model responds intelligently to questions with some synchronised facial expressions and movement. According to Raghoebardajal, this is only an internal demo developed at Sony.

According to The Verge’s reporting, the AI system uses OpenAI’s whisper for speech-to-text and GPT-4 and Llama 3 for its communication and decision-making, whilst Sony’s own Mockingbird tech powers the audio-to-facial animation and Emotional Voice Synthesis (EVS) powers speech.

Although the prototype was running on a PC, Sony believes it can run the same tech on PS5 consoles with “little overhead.”

Last week, the voice talent team that brings Apex Legends’ characters to life in French reportedly refused to sign contracts that would authorize their voices to be used to train AI.