Visionaire Studio Bugfix Update 5.0.5

  • #20, by sebastianMonday, 04. June 2018, 18:33 6 years ago
    So you basically have each written word inside a big table which Includes its phonemes? wow

    Thread Captain

    2346 Posts


  • #21, by afrlmeMonday, 04. June 2018, 18:52 6 years ago
    I did a bit of googling on text to phoneme convertors & it seems they base pronounciations on some dictionary. I think it would be quite complicated to program something that can accurately decode words into phonemes without some kind of reference for it work from. We know that x word has y syllables, but your script won't know that.

    This text to pronounciation generator for example uses this dictionary.

    Here is some small paper they or someone wrote about generating letters to pronounciation sounds.

    Simplest thing would probably be to use the tsv files & just include paFallbacktime tags in your display texts.

    Imperator

    7278 Posts

  • #22, by sebastianMonday, 04. June 2018, 19:01 6 years ago
    the thing is that a fallback animation has no sound and also no tsv file you can rely on. Just the textstring

    Thread Captain

    2346 Posts

  • #23, by afrlmeMonday, 04. June 2018, 19:55 6 years ago
    You could build a custom dictionary maybe? Not like you need to include every single word. Only the ones that you know will be in the game & a bunch of those words will likely be used multiple times.

    I'm  | going | to | the | cinema
    C > A | F > C | B | C | B > C > A

    That took for bloody ever for me to think about which phonemes should represent each word. Damn I'd hate to be the person that wrote the dictionary I linked earlier.

    Imperator

    7278 Posts

  • #24, by darren-beckettMonday, 04. June 2018, 23:58 6 years ago
    What animation settings do we need to use for the talking animation frames for the lipsync?

    What milliseconds? Do we need to set the pause for each frame?

    Great Poster

    384 Posts

  • #25, by afrlmeTuesday, 05. June 2018, 00:48 6 years ago
    What animation settings do we need to use for the talking animation frames for the lipsync?

    What milliseconds? Do we need to set the pause for each frame?

    Doesn't matter. You just have to select/create animation frames to match mouth shapes A to G. X mouth shape should be your characters mouth shape when idle.

    Scroll down to mouth shapes on this page here: https://github.com/DanielSWolf/rhubarb-lip-sync to learn which mouth shapes correspond to which letter. Animation frame 1 should be the A mouth shape. 2 should be B mouth shape & so on. X (idle mouth shape) should be frame 8.

    As for how to use Rhubarb, check out the screenshots & info I posted somewhere in this thread. I might create a wiki page later in the week with some screenshots to show how to install Rhubarb (on windows) & use it.

    Imperator

    7278 Posts

  • #26, by darren-beckettTuesday, 05. June 2018, 10:51 6 years ago
    I had a working LipSync solution, so i just reordered my mouth frames and set it all up with the tsv.
    But it doesn't look good, so i though i was missing something.

    Changing the milliseconds did affect how it looked, so i thought it may be related.

    I will investigate further this evening.

    Great Poster

    384 Posts

  • #27, by sebastianTuesday, 05. June 2018, 11:22 6 years ago
    when you use your own lipsync solution you would need to set the frame pause time very low (e. g. 10ms) to allow the animation to quickly adapt to the changed frame settings. 

    Thread Captain

    2346 Posts

  • #28, by afrlmeTuesday, 05. June 2018, 11:46 6 years ago
    I had a working LipSync solution, so i just reordered my mouth frames and set it all up with the tsv.
    But it doesn't look good, so i though i was missing something.

    Changing the milliseconds did affect how it looked, so i thought it may be related.

    I will investigate further this evening.
    Rhubarb Lip Sync is kind of basic as it uses the Hanna-Barbera animation approach to talk animations. 7-8 animations to cover some basic mouth shapes & then 1 animation frame for idle, however...

    Simon reworked it so that we don't have to use the ABAC values that Rhubarb generates. You can replace the letters with frame numbers instead, which means that you can include as many mouth shapes as you want. You don't technically have to use Rhubarb to generate the tsv files, they just need to written out in the same format.

    If you wanted to spice things up then you could create the same 8 frames with different moods on the characters face(s) & use animation frame values instead of ABAC values. It would be one way to make it look a lot more dynamic without having to manually determine which mouth shapes you need to use for each word, though I suppose you could use another lip sync tool to generate time stamps & the phonetic alphabet to help you figure out the mouth shapes for each word.

    Imperator

    7278 Posts

  • #29, by darren-beckettTuesday, 05. June 2018, 12:45 6 years ago
    when you use your own lipsync solution you would need to set the frame pause time very low (e. g. 10ms) to allow the animation to quickly adapt to the changed frame settings. 
    I think i have each of my 10 Viseme frames set at about 100ms, which happens to be about right (most of the time).

    I cut them down to the Rhubarb 8 and re-ordered them to match the rhubarb setup, but they don't look good - Not appearing to lipsync at all.
    As you guys have said that there's nothing more to the setup, I will just investigate further and work out what's wrong.

    Great Poster

    384 Posts

  • #30, by afrlmeTuesday, 05. June 2018, 15:01 6 years ago
    when you use your own lipsync solution you would need to set the frame pause time very low (e. g. 10ms) to allow the animation to quickly adapt to the changed frame settings. 
    I think i have each of my 10 Viseme frames set at about 100ms, which happens to be about right (most of the time).

    I cut them down to the Rhubarb 8 and re-ordered them to match the rhubarb setup, but they don't look good - Not appearing to lipsync at all.
    As you guys have said that there's nothing more to the setup, I will just investigate further and work out what's wrong.
    Can you post some screenshots please? Have you made sure to include the tsv file in the same folder as the speech audio file? The tsv file should have the same name as the audio file. Let's say you have an audio file called: "hello_world.ogg", now the tsv file should be named like this: "hello_world.ogg.tsv", with ogg.tsv being the format name & not the file name.

    Imperator

    7278 Posts