Text lip sync

This thread is a few years old. Some details may refer to an earlier version of Visionaire Studio. Current tutorials

1. Sunday, 03. January 2021, 23:17 · 6 years ago#1

   Hello. Perhaps another stupid question, but still. My question is about lip sync. I have reviewed topics related to Rhubarb Lip Sync, but I have no audio files. Is there a way to dynamically change the mouth animation based on the text of the printed line?
2. × Working on a game?

   The free version covers up to 25 scenes per project. For more scenes and to sell your game, you need a license. Forum readers get the Indie Bundle 15 % off with the code FORUM15: €84.15 instead of €99, valid until 31 Dec 2026.

   Get 15 % off Try it for free
3. Monday, 04. January 2021, 03:34 · 6 years ago#2

   probably with lua.

   There are text started hooks you can use to get the event of a character speaking. Then regarding to the text you can retrieve it from that event and go on from there and set the talk animation frames depending on your algorithm...

   
   
4. Monday, 04. January 2021, 12:18 · 6 years ago#3

   Okay, thank you, I will try.
5. Monday, 04. January 2021, 15:45 · 6 years ago#4

   It's possible to create them manually with Lua script, but it would be a lot of work. Another thing you could do is to still use Rhubarb Lip Sync, without actually using Rhubarb to generate the tsv files. What you would need to do is create audio files that just contain silence in them, if you don't want to actually add any voices to the game & then create a tsv file per audio file with the same name & manually enter the times & mouth shapes/animation frame numbers you want to trigger - it's still a lot of work mind, & it would also end up bulking up the required storage space required for your game.
   
   Having said that, the solution Sebastian suggested would probably be the best approach. Daedalic Entertainment did something similar to that years ago before Rhubarb support got implemented into Visionaire. What they did was create a load of external Lua files & inside of those they created a load of tables, with each one linked to the ID of an in-game display text & then they iterated over the relevant table & forced which animation frame should currently be displayed at x time.
   
   The thing is though that their game had voice overs, & for a text only game I don't see the point in trying to add complex lip sync to it because the players aren't going to be watching the mouths of the characters move, as they will be too busy reading the subtitles instead.

   
   

   What I would highly suggest doing is creating a bunch of frames & ticking the random play order option. If you want to make it even more dynamic then you could for each frame have it specify a random amount of time between a min & max value that the current frame could play for. It wouldn't be super accurate like lip sync, but it would be better then just playing an animation in forward mode in a constant loop.
6. Monday, 04. January 2021, 19:50 · 6 years ago#5

   Oh, thank you!