Navigation

© Zeal News Africa

Indian Sibling Duo Unleashes 'Koyal' AI: Turning Audio into Blockbuster Movies!

Published 2 days ago4 minute read
David Isong
David Isong
Indian Sibling Duo Unleashes 'Koyal' AI: Turning Audio into Blockbuster Movies!

Koyal.AI, an innovative audio-to-video filmmaking platform, was officially announced in its beta version on September 9 by co-founder Mehul Agarwal. The inspiration for Koyal.AI stemmed from Agarwal's stand-up comedy days at Carnegie Mellon University (CMU) in 2019, where he envisioned transforming his jokes into films without the hefty production costs. Developed by Mehul, 23, and his sister Gauri Agarwal, 26, Koyal is presented as the fastest and most effective method to convert audio into cinematic video, maintaining consistent characters, settings, and storylines.

The platform is designed for a broad audience with extensive audio content, including musicians, podcasters, and storytellers. It offers the unique ability to transform music into music videos, podcasts into TV shows, or bedtime stories into animated films. Utilizing Artificial Intelligence (AI), Koyal can generate custom characters and animations in minutes, providing users with significant creative control. Creators can personalize avatars, alter scenery, makeup, and costumes, and even adjust the tone and mood of dialogue. For instance, a scene from the 'Barbie' movie could be set in a castle in Northern Ireland, or a bespoke music video with a choreographed flash mob could be created, bringing users' imaginations to life rapidly.

Koyal, identified as a next-generation GenAI audio-to-video storytelling platform, made its public debut at the World Audio Visual and Entertainment Summit (WAVES) 2025 in Mumbai earlier this year. During the inaugural day, a series of music videos created using Koyal were launched, demonstrating its capability to generate personalized, narrative-driven videos from audio in diverse and engaging styles. Mehul emphasized that the platform uses state-of-the-art AI models to extract emotions, context, and storytelling elements directly from music, transforming audio tracks into rich, emotive video narratives.

Before its public beta release, Koyal collaborated with renowned artists such as playback singer and composer Shankar Mahadevan and music maestro AR Rahman. These collaborations were instrumental in creating and supporting video content for the WAVES Album, which also featured artists Ricky Kej and Meet Brothers. The founders highlighted Koyal's multimodal AI suite and state-of-the-art character consistency as features setting new benchmarks for how artists and creators visualize their work.

The technological foundation of Koyal is deeply rooted in academic research, specifically a paper presented at the NeurIPS 2024 AI conference, a leading event in machine learning and computational neuroscience. Furthermore, the platform incorporates a patented secure personalization protocol named CHARCHA, ensuring that video personalization occurs only with explicit user consent. Gauri Agarwal, who also worked at Meta (then Facebook) on Instagram reel generation, explained that CHARCHA, which stands for Computer Human Assessment for Recreating Characters with Human Actions, acts as a safeguard. It allows users to create personalized avatars if they wish their likeness to appear in the video, or they can opt to use only AI-generated models, prioritizing consent throughout the process.

The Agarwal siblings, who now reside in the San Francisco Bay Area, California, bring strong academic backgrounds to Koyal. Mehul completed his master's degree at the Robotics Institute (RI) at Carnegie Mellon University in 2024, while Gauri earned her bachelor's degree in computer science in 2020 before pursuing further studies at the Massachusetts Institute of Technology (MIT). Their educational pursuits at CMU facilitated connections with Jean Oh, an associate research professor in the RI, and Michael Hilton, a teaching professor in the Software and Societal Systems Department, who were instrumental in Koyal's collaborative development.

Having previously engaged in seven paid pilots with major music labels, podcast, and storytelling companies in India, including Universal Music, T-Series, Maddock Entertainment, and Collective Artists Network, Koyal has now transitioned to a public-facing prosumer product. Gauri expressed the hope that Koyal will broaden opportunities for a wide range of creators, democratizing filmmaking beyond those who can afford large-scale productions. She added that the platform offers a safe avenue to explore the rapidly evolving potential of AI, allowing even Mehul to fulfill his dream of converting his stand-up jokes into a movie.

Recommended Articles

Loading...

You may also like...