Mouth animation using Rhubarb Lip Sync

  • #10, by AFRLmeThursday, 04. October 2018, 11:28 4 months ago
    Sorry, there was a tutorial I wrote & shared on the wiki, but something happened & it somehow got completely erased, which means I have to write it all again & dig out the image files I created for it - the latter will be fun.

    Imperator

    6432 Posts


  • #11, by AFRLmeWednesday, 17. October 2018, 21:33 4 months ago
    The lip sync tutorial is now back on the wiki - had to write it again from scratch. I have no idea whether or not it all makes sense, but at least it's somewhere to start...

    Imperator

    6432 Posts

  • #12, by stothewThursday, 18. October 2018, 09:06 4 months ago

    Forum Fan

    122 Posts

  • #13, by daniel-wolfThursday, 18. October 2018, 09:54 4 months ago
    Great to see the tutorial up again! Thanks for re-writing it (I know that must be annoying). And it's good to know that this time there's a backup.

    Newbie

    9 Posts

  • #14, by AFRLmeThursday, 18. October 2018, 11:41 4 months ago
    haha, a backup. grin

    funny story... David actually wrote something or installed something that was supposed to back the wiki up every day - I don't know why it didn't restore all of the data when he updated the wiki. I also have no idea if anything else got deleted or any recent edits I'd done got undone, but whatever you know.

    Imperator

    6432 Posts

  • #15, by chronologicWednesday, 07. November 2018, 15:55 3 months ago
    As I was using the lip sync, I noticed that the character makes a momentary mouth shape at the start of the text which is not in the .tsv file. 

    After some trial and error, I found that setting the characters talk animation to "Play Backwards" ensures that this additional frame uses the rest animation, and so is then hidden.

    Newbie

    16 Posts

  • #16, by AFRLmeWednesday, 07. November 2018, 17:45 3 months ago
    mouth shape x, which is the idle/default mouth shape should be frame 9 I believe - if you use all mouth shapes. If you didn't then you need to edit the tsv file to change all instances of X to the correct frame number.

    Imperator

    6432 Posts

  • #17, by chronologicWednesday, 07. November 2018, 18:14 3 months ago
    Hi, I am using 8 shapes, and change the X to be H in the file...

    But for me, regardless of the first shape in the .tsv file, there is always one additional frame shown at the start of the animation.

    It seems to be showing the first frame directly from the talk animation before starting to read the .tsv

    Initially for me, it was a random shape each time because I had it set to play in random order.
    So changing to play backwards ensured that this extra frame is the H idle shape, so goes unnoticed...

    Newbie

    16 Posts

  • #18, by AFRLmeWednesday, 07. November 2018, 20:03 3 months ago
    The tsv should start with a 0.00 time frame that's linked to the x mouth shape.

    0.00    X
    0.03    E
    0.13    B
    0.27    C
    0.41    B
    0.48    E
    0.55    F
    0.62    C
    0.76    B
    1.19    A
    1.24    C
    1.43    B
    1.50    D
    1.64    B
    1.85    X
    1.89    X
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16

    taken from one of the files I have generated.  I didn't include H in that example because Simon hadn't added it back then. Anyway, you don't necessarily have to change x in the tsv file if you haven't generated h mouth shapes, you could just insert one of the other mouth shape animations in as padding or a blank animation frame as frame 8 - probably easier, no?

    Anyway, I think it's supposed to start & end with x - but not 100% sure.

    Imperator

    6432 Posts

  • #19, by chronologicWednesday, 07. November 2018, 21:46 3 months ago
    Yes it would have been easier ;-) but it doesn't make any difference if I have 9 animations and use X as the idle, or 8 animations and use H as the idle. The same issue happens... 

    I begin and end the .tsv file with the idle frame, and starting at 0.00

    0.00 H
    1.00 B
    1.72 C
    1.86 G
    1.93 E
    2.00 B
    2.14 C
    2.28 B
    2.35 F
    3.12 H

    My point is that visionaire studio seems to be including one extra frame before it starts to follow the tsv instructions. 

    So in this example, before the character is idle for the first second, there is a single animation frame being shown which is not in the tsv file, but is being taken from the animation.

    But you can workaround it by setting the animation to play backwards so that it is forced to uses the idle frame.

    Newbie

    16 Posts

  • #20, by AFRLmeWednesday, 07. November 2018, 23:51 3 months ago
    Are you sure you are actually even using the tsv file & not just playing through the animation normally? The name of the tsv file has to be in the same folder as the audio file & needs to contain the exact same filename + extension with .tsv on the end of it, like this...

    example.ogg would example.ogg.tsv
    example.wav would be example.wav.tsv
    foo bar.ogg would be foo bar.ogg.tsv

    & so on.

    Also the example you have shared has a 1 second delay between showing H @ 0.00 to B .

    Imperator

    6432 Posts

Write post