Text lip sync

  • #1, by ivan-parketovSunday, 03. January 2021, 23:17 3 years ago
    Hello. Perhaps another stupid question, but still. My question is about lip sync. I have reviewed topics related to Rhubarb Lip Sync, but I have no audio files. Is there a way to dynamically change the mouth animation based on the text of the printed line?

    Newbie

    9 Posts


  • #2, by sebastianMonday, 04. January 2021, 03:34 3 years ago
    probably with lua.
    There are text started hooks you can use to get the event of a character speaking. Then regarding to the text you can retrieve it from that event and go on from there and set the talk animation frames depending on your algorithm...

    Thread Captain

    2346 Posts

  • #3, by ivan-parketovMonday, 04. January 2021, 12:18 3 years ago
    Okay, thank you, I will try.

    Newbie

    9 Posts

  • #4, by afrlmeMonday, 04. January 2021, 15:45 3 years ago
    It's possible to create them manually with Lua script, but it would be a lot of work. Another thing you could do is to still use Rhubarb Lip Sync, without actually using Rhubarb to generate the tsv files. What you would need to do is create audio files that just contain silence in them, if you don't want to actually add any voices to the game & then create a tsv file per audio file with the same name & manually enter the times & mouth shapes/animation frame numbers you want to trigger - it's still a lot of work mind, & it would also end up bulking up the required storage space required for your game.

    Having said that, the solution Sebastian suggested would probably be the best approach. Daedalic Entertainment did something similar to that years ago before Rhubarb support got implemented into Visionaire. What they did was create a load of external Lua files & inside of those they created a load of tables, with each one linked to the ID of an in-game display text & then they iterated over the relevant table & forced which animation frame should currently be displayed at x time.

    The thing is though that their game had voice overs, & for a text only game I don't see the point in trying to add complex lip sync to it because the players aren't going to be watching the mouths of the characters move, as they will be too busy reading the subtitles instead.

    What I would highly suggest doing is creating a bunch of frames & ticking the random play order option. If you want to make it even more dynamic then you could for each frame have it specify a random amount of time between a min & max value that the current frame could play for. It wouldn't be super accurate like lip sync, but it would be better then just playing an animation in forward mode in a constant loop.

    Imperator

    7278 Posts

  • #5, by ivan-parketovMonday, 04. January 2021, 19:50 3 years ago
    Oh, thank you! 

    Newbie

    9 Posts