What is AI VTuber?
Powered by AI, unlike the usual VTubers (Virtual YouTubers) controlled and piloted by humans, AI VTubers are Artificial Intelligence Virtual YouTubers that can autonomously interact with viewers based on a large language model (LLM). They can respond and engage with the audience in real-time, play video games, and share their personal anecdotes. Best of all, they can evolve over time as their developers feed them with updated data or a language model.
Pros and Cons of AI VTubers
Pros: Operated by AI, AI VTubers don’t require a crew during live streaming, thereby saving on labor costs. The unpredictability of their responses keeps the audience on their toes, wondering about the fantastical twists AI VTubers might introduce next.
Cons: There is currently no AI VTuber generator capable of meeting all the requirements needed to create an AI VTuber. It is a rather complex process that demands a significant amount of effort to train the AI before unveiling it to the public. Additionally, the unfiltered use of language, although considered frank and honest, might be perceived as insensitive when discussing controversial issues.
Who is the First AI VTuber?
Neuro-sama is the first AI VTuber created by computer programmer and AI developer Jack Vedel. Initially created to play the rhythm game Osu! and later Minecraft, Neuro-sama can engage with her viewers by responding to questions or spontaneously uttering a range of statements. She took the Twitch world by storm with her unfiltered expression. Neuro-sama boasts a direct yet polite demeanor, creating an intriguing contrast with the unpredictable nature of her statements.
AI VTubers Worth Checking Out
Other than Neuro-sama, the first-ever AI VTuber mentioned earlier, here are a few more AI VTubers worth checking out.
1. Hilda
(Source: Hilda Ch. AI VTuber)
Born under the influence of Neuro-sama’s skyrocketing fame, Hilda is an English-speaking AI VTuber. Her development aimed to break new ground by making her code open source for everyone. Hilda possesses the abilities to engage in chats, respond with her Text-to-Speech voice (TTS), sing, retain long-term memory, and convey visual feedback based on her simulated “emotions.”
2. Ubi-chan
(Source: Ubitus)
Debuting just a month ago, Ubi-chan leverages Uibitus’ Ai technology, including Retrieval-augmented Generation (RAG). Excelling in anime concepts and conversations, her YouTube channel highlights original music videos and dance routines, all using AI for music, lyrics, and choreography.
3. Camila
(Source: Typecast)
You don’t often come across non-Japanese anime-style VTubers, but Camila sets herself apart with her dusky skin tone. Powered by ChatGPT, she streams 24/7 on the Typecast Global Channel. Using the Camila Voicebank, she seamlessly transitions between various AI voices, creating an entertaining and educational experience for her audience.
How to Make an AI VTuber?
First, when crafting your AI VTuber, decide on its personality, appearance, and backstory. Take into account the specific audience you aim to captivate, and tailor the persona to resonate with their preferences.
Second, create an avatar that aligns with your VTuber's personality. Tools like VRoid Studio, VIVERSE Avatar Creator or Live 2D Cubism can come in handy as they enable you to create adorable cartoonish characters.
Next, pick an AI platform or framework for your AI VTuber, such as OpenAI's GPT (Generative Pre-trained Transformer), Unity ML-Agents, or other versatile machine learning frameworks.
After that, ensure a comprehensive and diverse dataset is employed during the AI training process to capture a broad spectrum of speech patterns and behaviors. This approach enhances the adaptability of your AI VTuber, allowing it to resonate with a wider audience by embracing linguistic and behavioral diversity.
Then, connect your AI model with the streaming software to synchronize the virtual avatar's movements and expressions with your speech and actions.
Lastly, choose a voice synthesis system that aligns with your VTuber's persona. You can employ tools like Google Text-to-Speech or specialized voice synthesis programs like voice.ai. Don’t forget to adjust the voice parameters to match your VTuber's character, including pitch, speed, and tone.
Is Neuro-sama a real AI?
Certainly. Neuro-sama primarily employs computer software to carry out on-stream activities such as speaking, singing, moving, and playing games. This stands in contrast to being directly controlled by a human. Twitch banned her for participating in hateful conduct, attributed to controversial statements made by the AI, such as casting doubt on the Holocaust's authenticity during a live stream.
Is Kizuna AI a real AI?
Although Kizuna AI claims to be an independent artificial intelligence, she has been directly managed by the company that created her, Activ8, since her debut. Later, she was under the management of its in-house agency, upd8, and Kizuna AI Inc. before going on hiatus. Additionally, it was revealed that Japanese voice actress Kasuga Nozomi provides the voice for Kizuna AI. Consequently, while Kizuna AI may be considered the world’s first VTuber, she does not quite fit the definition of an AI VTuber, so to speak.