Visionaire Studio Bugfix Update 5.0.5

haha. aye.

Here's what you could do if you still only want to have a basic open & close mouth... generate the tsv files with Rhubarb. Edit the file with sublime text, atom or notepad++ or something similar & just replace all the instances that aren't X with A B A then replace all X with C, so you end up with something like this instead...

0.00    C
0.03    A
0.13    B
0.27    A
0.41    C

C being closed mouth shape. A being slightly open mouth. B being fully open mouth.

 - The 2nd video shows my own lipsync

In the example shown in the videos, I think I prefer my original lipsync, due to it using more visemes.

Note: The only time mine will fail is if the speech isn't in a continuous flow, that's when the timing in the TSV would be the perfect solution

Also, The TSV solution is a lot more elegant and easier to setup for the users.

- An action to be called after each frame of the talk animation

- Tables containing translations of all Words/Phonemes/Visemes

Thanks for all your help

Regarding OGG files: I'm myself not a Visionaire user. So I cannot estimate how important OGG support is to you. What's the workflow in Visionaire Studio? Is it always OGG-based?

Regarding non-English dialog: This is absolutely something I'm planning to support. But it's a lot of work and I currently have very little free time. So I can't make any promises as to when multi-language support will be available. In the mean time, you can get decent results (though not great) by just omitting the --dialogFile option. Rhubarb will take a sound file containing German/Italian/... dialog and understand some gibberish English words that sound similar.

Regarding lip sync based on phoneme-viseme tables: This is where I myself started off two years ago. It turned out that convincing lip sync is much more complex. Depending on the word, we often anticipate certain sounds, forming them with the mouth while still speaking a previous sound. Or we hold a certain mouth shape even though we're not speaking the corresponding sound any more. Or in the case of plosives ("P", for example), we form the corresponding mouth shape (closed mouth) before the sound is heard, and open the mouth when the sound is heard. Rhubarb has a large set of rules to cover these cases, and I'm still a long way from finished.

OGG has good compression and works great on Android.

Regarding OGG: I understand the benefits of OGG for finished games. I'd certainly never dream of publishing a game with uncompressed audio resources. During production, however, I prefer a lossless, uncompressed format like WAVE. Storage space usually isn't an issue during production, and a lossless format allows you to make unlimited edits without loss of quality.

On Thimbleweed Park, they managed all recordings as uncompressed WAVE files. Converting the audio format was only the last step, and was done automatically as part of the build process.

I'm completely new to Visionaire Studio. Reading between your lines, it seems that Visionaire Studio doesn't support this kind of conversion. So if you want OGG files in the finished game, you need to work with OGG files during production. And this means that it's a major inconvenience to you that Rhubarb only supports WAVE files. Did I get that right?

Hi Daniel  

If you have a bit knowledge it is possible to open the game project file  in a texteditor (as its basically an xml file) and replace all .wav strings to .ogg, but its not that convenient for most users. 

So using the compressed files in the first place would be much more "easy" for most of us. 

Currently using Rhubarb would require to have two files (. wav for creating lipsync/. ogg for actual usage inside the engine) if you use other formats than. wav in your project. . 

A'llo.

To be honest I think most of us work with wav or aiff to begin with as speech recordings are probably exported out in those formats, it's just that we directly work in Visionaire Studio with ogg/opus for audio, png/webp for images, mkv for videos, etc because it allows us to work on optimization & see how our games will run while we are working on them.

ogg vorbis isn't a requirement of the engine, we can technically import wav, aiff, ogg or mp3 files. ogg is just the recommended format because it's an open container format unlike mp3 & it's compressed unlike wav, aiff or flac.

In regards to what you mentioned about Thimbleweed Park converting at the end during the exporting/compiling process. We have some options like that, but unfortunately not for audio. Only for converting images to WebP, but the results for that aren't as reliable as converting yourself with XnConvert.

@darren-beckett: I think I actually prefer the tsv version more as it's less flappy. You can edit the tsv & insert additonal frames (mouth shapes) or time stamps if you want.

@machtnix: I keep forgetting that you are still using 4.2.5. I guess maybe you can use Sebastian's script, if he decides to share it. Or you could write a small script that returns a frame per character & idle mouth shape for anything other than a letter. I dunno.

In a 3D-programm like Maya or Cinema you could make jigsaw-animation: you can crossfade all movements without breaks, calls oneself "nonlinear animation". I think today all CG animations use this way.