Dynamic Sound Balancing

  • #1, by BenmirathSaturday, 14. June 2014, 08:06 10 years ago
    Hi all,
    I've seen the topic of dynamically balancing game sound to the playable character's position thrown around a bit in the past, and am currently thinking through implementing it in my game. I haven't turned up anything concrete on it yet though, and the post on it in the wiki list of scripts is empty. I have a few idea of how I might go about it bumping around in my head, but wanted to see if anyone has tackled this already before I go flailing around in my code trying things out.
    Thanks!

    Newbie

    31 Posts


  • #2, by BigStansSaturday, 14. June 2014, 13:49 10 years ago
    HuHi,

    the wiki is empty?
    http://wiki.visionaire-tracker.net/wiki/Player_Commands
    Have a look at the player commands and sound functions.

    The walking sound is panning already, but with VS4 you need to create an mono Sound for panning.

    - David

    Great Poster

    361 Posts

  • #3, by afrlmeSaturday, 14. June 2014, 13:56 10 years ago
    Yeah I wrote the script for 3.8 but since then the data structure entry that I got added into 3.8 (unreleased build) was changed from integer (round number) to float (decimal) which means I need to rewrite the script a little bit & make sure my edits work before I post it on the wiki.

    I will sort it out when I can.

    Have you read my post in the manual section in regards to approaching sound design? it explains about various aspects of sound design & implementation, such as: is the sound balance & volume controlled by the characters position or the camera position. These 2 perspective points greatly influence the volume & balance of certain sounds.

    For instance if you were to use the characters perspective then all sounds made by the character would be centered & played at approx. 100% volume level, whereas sounds not made by the character would be panned to the left or right respective of the characters position on the screen, as would the volume be higher or lower based on distance of character from the sound source.

    http://wiki.visionaire-tracker.net/wiki/Approaching_Sound_De...

    Imperator

    7278 Posts

  • #4, by BenmirathSaturday, 14. June 2014, 14:30 10 years ago
    Thanks for the prompt responses,
    -David: The part I was referring to on the wiki was the script AFRLme mentions. I did see the player commands though, which I'll make use of. Just trying to figure out the most efficient means of accessing any playing sounds to do so.

    -AFRLme: Ah, makes sense, gotcha. That same issue caught me while trying to make use of the CharacterSize property, which is mislabeled as an integer on the wiki. Your post on sound design was actually what got me thinking through how to pursue this! I would most likely set it to be dependent on the camera perspective, as the screen can pan with the mouse, freeing you to look away from the player character. Still thinking through how to actually go about it though.
    The part that I'm unsure of is how to properly access each sound to dynamically change its balancing. I activate all sounds via action parts instead of scripting, and am not sure how to make a script aware of every sound playing in the scene. I suppose I could manually add the file path to a table in lua whenever I add a sound to the scene, but that seems clunky. I'd much rather there was a way of checking for active sounds via a looping script and adjusting them there based on the x position of the camera, or something similarly dynamic. Any thoughts on how best to approach this issue?

    Thanks!
    -Ben

    Newbie

    31 Posts

  • #5, by afrlmeSaturday, 14. June 2014, 15:06 10 years ago
    the sounds have to be started via the startSound() function & the file paths are best stored in tables or variables.

    I can share the script I wrote for 3.8, so you can get an idea of what I did, but like I said, it is currently invalid at the minute.

    --[[
Character footstep sound & property functions (v3.0) [functions]
Written by AFRLme
-- * --
label@alternatingfrequencies.com, aim/skype: AFRLme
--]]

-- * begin footstep sound code * --

-- let's create a function in which we can set the footstep ground type! (execute a script > footstepType("wood"))
function footstepType(typ)
 groundType = typ
end

-- let's create the function which will play the correct footstep sound! (execute a script > footstepSounds("character_name"))
function footstepSounds(char)
 local tbl = {} -- create an empty table
 tbl["_temporary_"] = "" -- set table as temporary
 tbl["character"] = getObject("Characters[" .. char .. "]") -- store current character
 tbl["character_size"] = tbl["character"]:getInt(VCharacterSize) -- get linked characters current scale
 tbl["character_pos"] = tbl["character"]:getPoint(VCharacterPosition).x -- get linked characters current x pos
 tbl["screen_center"] = (game:getPoint(VGameWindowResolution).x / 2) + game:getPoint(VGameScrollPosition).x -- find current scene center x pos
 tbl["v_random_val"] = math.random(-5, 5) -- for adding/subtracting from sound volume (dynamic reasons)
 tbl["s_random_val"] = math.random(1, table.maxn(tblSounds[groundType])) -- for selecting a random file number from sound table
 tbl["volume"] = tbl["character_size"] + tbl["v_random_val"] -- adds or subtracts random value from volume level
 if tbl["volume"]  100 then tbl["volume"] = 100 end -- check volume isn't greater/lower than 0 or 100%
 -- * check if position of character to screen center to determine if audio balance % should be a positive or negative value
 if tbl["character_pos"] < tbl["screen_center"] then tbl["balance"] = - ((tbl["screen_center"] - tbl["character_pos"]) / (game:getPoint(VGameWindowResolution).x / 2) * 100) end -- inverse balance %
 if tbl["character_pos"] >= tbl["screen_center"] then tbl["balance"] = ((tbl["character_pos"] - tbl["screen_center"]) / (game:getPoint(VGameWindowResolution).x / 2) * 100) end -- positive balance %
 -- * break * --
 startSound(tblSounds[groundType][(tbl["s_random_val"])], {flags=1, volume=tbl["volume"], balance=tbl["balance"]}) -- play the linked sound & set volume / balance according to characters scale & x position
 -- * break * --
 setWalkSpeed(char) -- call the walk speed function; which determines the characters walk speed based on character scale
 -- * debug * --
 --[[
 if getObject("Conditions[debug_mode]"):getBool(VConditionValue) then
  print("footstepSound function") 
  print("screen left: " .. game:getPoint(VGameScrollPosition).x .. ", screen center: " .. tbl["screen_center"] .. ", screen right: " .. tbl["screen_center"] + (game:getPoint(VGameWindowResolution).x / 2))
  print("character name: " .. char .. ", size: " .. tbl["character_size"] .. ", x pos: " .. tbl["character_pos"])
  print("filename: " .. groundType .. ", file number: " .. tbl["s_random_val"] .. ", balance: " .. tbl["balance"] .. ", volume: " .. tbl["volume"] .. ", original volume: " .. tbl["character_size"] .. ", random value: " .. tbl["v_random_val"] .. "\n")
 end
 --]]
end

-- * end footstep sound code * --
--
-- * begin walk speed code * --

-- let's create the function in which we set the walk speed of the current character
function setWalkSpeed(char)
 local tbl = {} -- empty table
 tbl["_temporary_"] = "" -- set as temporary table
 tbl["min_speed"] = 120 -- minimum walk speed value
 tbl["max_speed"] = 350 -- max walk speed value
 tbl["character"] = getObject("Characters[" .. char .. "]") -- store linked character
 tbl["character_size"] = tbl["character"]:getInt(VCharacterSize) / 100 -- store linked character scale & divide by 100
 tbl["character_outfit"] = tbl["character"]:getLink(VCharacterCurrentOutfit) -- store linked character outfit
 tbl["old_walkspeed"] = tbl["character_outfit"]:getInt(VOutfitCharacterSpeed) -- store linked outfit speed
 tbl["new_walkspeed"] = (tbl["character_size"] * tbl["max_speed"]) + tbl["min_speed"] -- calculate the new walk speed
 -- * break * --
 if tbl["new_walkspeed"] > tbl["max_speed"] then tbl["new_walkspeed"] = tbl["max_speed"] end -- check if new walk speed is more than max walk speed
 if tbl["old_walkspeed"] ~= tbl["new_walkspeed"] then tbl["character_outfit"]:setValue(VOutfitCharacterSpeed, tbl["new_walkspeed"]) end -- set the new walk speed value (only if not same as old walk speed value)
 -- * debug * --
 --[[
 if getObject("Conditions[debug_mode]"):getBool(VConditionValue) then
  print("setWalkSpeed function")
  print("character: " .. char .. ", outfit: " .. tbl["character_outfit"]:getName())
  print("old walkspeed value: " .. tbl["old_walkspeed"] .. ", new walkspeed value: " .. tbl["new_walkspeed"] .. "\n")
 end
 --]]
end

-- * end walk speed code * --


    --[[
Character sound & property functions (v1.0)
Written by AFRLme
-- * --
label@alternatingfrequencies.com, aim/skype: AFRLme
--]]

-- let's create the function that will allow us to play any character sound file dynamically based on character position & scale
function playCharSound(char, sound, val, random)
 local tbl = {} -- create an empty table
 tbl["_temporary_"] = "" -- set table as temporary
 tbl["character"] = getObject("Characters[" .. char .. "]") -- store current character
 tbl["character_size"] = tbl["character"]:getInt(VCharacterSize) -- get linked characters current scale
 tbl["character_pos"] = tbl["character"]:getPoint(VCharacterPosition).x -- get linked characters current x pos
 tbl["v_random_val"] = math.random(-5, 5) -- for adding/subtracting from sound volume (dynamic reasons)
 if random then tbl["s_random_val"] = math.random(1, val) else tbl["s_random_val"] = math.random(val, val) end -- for selecting a random file if "random" = true else select specified file number from sound table
 tbl["volume"] = tbl["character_size"] + tbl["v_random_val"] -- adds or subtracts random value from volume level
 if tbl["volume"]  100 then tbl["volume"] = 100 end -- check volume isn't greater/lower than 0 or 100%
 tbl["screen_center"] = (game:getPoint(VGameWindowResolution).x / 2) + game:getPoint(VGameScrollPosition).x -- find current scene center x pos
 -- * check if position of character to screen center to determine if audio balance % should be a positive or negative value
 if tbl["character_pos"] < tbl["screen_center"] then tbl["balance"] = - ((tbl["screen_center"] - tbl["character_pos"]) / (game:getPoint(VGameWindowResolution).x / 2) * 100) end -- inverse balance %
 if tbl["character_pos"] >= tbl["screen_center"] then tbl["balance"] = ((tbl["character_pos"] - tbl["screen_center"]) / (game:getPoint(VGameWindowResolution).x / 2) * 100) end -- positive balance %
 -- * break * --
 startSound(tblSounds[sound][(tbl["s_random_val"])], {flags=1, volume=tbl["volume"], balance=tbl["balance"]}) -- play the linked sound & set volume / balance according to characters scale & x position
 -- * debug * --
 if getObject("Conditions[debug_mode]"):getBool(VConditionValue) then
  print("character sound function")
  print("screen left: " .. game:getPoint(VGameScrollPosition).x .. ", screen center: " .. tbl["screen_center"] .. ", screen right: " .. tbl["screen_center"] + (game:getPoint(VGameWindowResolution).x / 2))
  print("character name: " .. char .. ", size: " .. tbl["character_size"] .. ", x pos: " .. tbl["character_pos"])
  print("filename: " .. sound .. ", file number: " .. tbl["s_random_val"] .. ", balance: " .. tbl["balance"] .. ", volume: " .. tbl["volume"] .. ", original volume: " .. tbl["character_size"] .. ", random value: " .. tbl["v_random_val"] .. "\n")
 end 
end


    -- let's create a list of sounds in a table that we can play with the startSound() function
tblSounds = {}

-- * begin footstep sounds * --
--
-- * detrius [forest/woods undergrowth] (normal) * --
tblSounds["detrius"] = {}
tblSounds["detrius"][1] = "vispath:sounds/footsteps/detrius/detrius_1.ogg"
tblSounds["detrius"][2] = "vispath:sounds/footsteps/detrius/detrius_2.ogg"
tblSounds["detrius"][3] = "vispath:sounds/footsteps/detrius/detrius_3.ogg"
tblSounds["detrius"][4] = "vispath:sounds/footsteps/detrius/detrius_4.ogg"
tblSounds["detrius"][5] = "vispath:sounds/footsteps/detrius/detrius_5.ogg"

-- * grass (normal) * --
tblSounds["grass"] = {}
tblSounds["grass"][1] = "vispath:sounds/footsteps/grass/grass_1.ogg"
tblSounds["grass"][2] = "vispath:sounds/footsteps/grass/grass_2.ogg"
tblSounds["grass"][3] = "vispath:sounds/footsteps/grass/grass_3.ogg"
tblSounds["grass"][4] = "vispath:sounds/footsteps/grass/grass_4.ogg"
tblSounds["grass"][5] = "vispath:sounds/footsteps/grass/grass_5.ogg"

-- * grit (normal) * --
tblSounds["grit"] = {}
tblSounds["grit"][1] = "vispath:sounds/footsteps/grit/grit_1.ogg"
tblSounds["grit"][2] = "vispath:sounds/footsteps/grit/grit_2.ogg"
tblSounds["grit"][3] = "vispath:sounds/footsteps/grit/grit_3.ogg"
tblSounds["grit"][4] = "vispath:sounds/footsteps/grit/grit_4.ogg"
tblSounds["grit"][5] = "vispath:sounds/footsteps/grit/grit_5.ogg"
-- * grit (reverb) * --
tblSounds["v_grit"] = {}
tblSounds["v_grit"][1] = "vispath:sounds/footsteps/grit/v_grit_1.ogg"
tblSounds["v_grit"][2] = "vispath:sounds/footsteps/grit/v_grit_2.ogg"
tblSounds["v_grit"][3] = "vispath:sounds/footsteps/grit/v_grit_3.ogg"
tblSounds["v_grit"][4] = "vispath:sounds/footsteps/grit/v_grit_4.ogg"
tblSounds["v_grit"][5] = "vispath:sounds/footsteps/grit/v_grit_5.ogg"

-- * mud (normal) * --
tblSounds["mud"] = {}
tblSounds["mud"][1] = "vispath:sounds/footsteps/mud/mud_1.ogg"
tblSounds["mud"][2] = "vispath:sounds/footsteps/mud/mud_2.ogg"
tblSounds["mud"][3] = "vispath:sounds/footsteps/mud/mud_3.ogg"
tblSounds["mud"][4] = "vispath:sounds/footsteps/mud/mud_4.ogg"
tblSounds["mud"][5] = "vispath:sounds/footsteps/mud/mud_5.ogg"

-- * sand (normal) * --
tblSounds["sand"] = {}
tblSounds["sand"][1] = "vispath:sounds/footsteps/sand/sand_1.ogg"
tblSounds["sand"][2] = "vispath:sounds/footsteps/sand/sand_2.ogg"
tblSounds["sand"][3] = "vispath:sounds/footsteps/sand/sand_3.ogg"
tblSounds["sand"][4] = "vispath:sounds/footsteps/sand/sand_4.ogg"
tblSounds["sand"][5] = "vispath:sounds/footsteps/sand/sand_5.ogg"

-- * snow (normal) * --
tblSounds["snow"] = {}
tblSounds["snow"][1] = "vispath:sounds/footsteps/snow/snow_1.ogg"
tblSounds["snow"][2] = "vispath:sounds/footsteps/snow/snow_2.ogg"
tblSounds["snow"][3] = "vispath:sounds/footsteps/snow/snow_3.ogg"
tblSounds["snow"][4] = "vispath:sounds/footsteps/snow/snow_4.ogg"
tblSounds["snow"][5] = "vispath:sounds/footsteps/snow/snow_5.ogg"

-- * stone (normal) * --
tblSounds["stone"] = {}
tblSounds["stone"][1] = "vispath:sounds/footsteps/stone/stone_1.ogg"
tblSounds["stone"][2] = "vispath:sounds/footsteps/stone/stone_2.ogg"
tblSounds["stone"][3] = "vispath:sounds/footsteps/stone/stone_3.ogg"
tblSounds["stone"][4] = "vispath:sounds/footsteps/stone/stone_4.ogg"
tblSounds["stone"][5] = "vispath:sounds/footsteps/stone/stone_5.ogg"
-- * stone (reverb) * --
tblSounds["v_stone"] = {}
tblSounds["v_stone"][1] = "vispath:sounds/footsteps/stone/v_stone_1.ogg"
tblSounds["v_stone"][2] = "vispath:sounds/footsteps/stone/v_stone_2.ogg"
tblSounds["v_stone"][3] = "vispath:sounds/footsteps/stone/v_stone_3.ogg"
tblSounds["v_stone"][4] = "vispath:sounds/footsteps/stone/v_stone_4.ogg"
tblSounds["v_stone"][5] = "vispath:sounds/footsteps/stone/v_stone_5.ogg"

-- * wood (normal) * --
tblSounds["wood"] = {}
tblSounds["wood"][1] = "vispath:sounds/footsteps/wood/wood_1.ogg"
tblSounds["wood"][2] = "vispath:sounds/footsteps/wood/wood_2.ogg"
tblSounds["wood"][3] = "vispath:sounds/footsteps/wood/wood_3.ogg"
tblSounds["wood"][4] = "vispath:sounds/footsteps/wood/wood_4.ogg"
tblSounds["wood"][5] = "vispath:sounds/footsteps/wood/wood_5.ogg"
-- * wood (reverb) * --
tblSounds["v_wood"] = {}
tblSounds["v_wood"][1] = "vispath:sounds/footsteps/wood/v_wood_1.ogg"
tblSounds["v_wood"][2] = "vispath:sounds/footsteps/wood/v_wood_2.ogg"
tblSounds["v_wood"][3] = "vispath:sounds/footsteps/wood/v_wood_3.ogg"
tblSounds["v_wood"][4] = "vispath:sounds/footsteps/wood/v_wood_4.ogg"
tblSounds["v_wood"][5] = "vispath:sounds/footsteps/wood/v_wood_5.ogg"
-- * end footstep sounds * --
--
-- * begin character sound fx * --

-- * scanner * --
tblSounds["scanner"] = {}
tblSounds["scanner"][1] = "vispath:sounds/characters/scanner.ogg"

    The last edit date on this is from september 2013, so it's quite an old script from me & maybe not how I would write scripts these days.

    Ok the first script played character footstep sounds which are triggered via a script inside of the contact foot frames of each walk animation. I didn't use the built in footstep sound system because I wanted to be able to randomly select from a list of footstep sounds based on floor type & then set volume & balance based on character position on scene (based on current position of camera, not the whole scene, meaning the closer you get the edge of the visible part of the scene the more the sound will pan left or right). The volume also uses a random value to add/subtract from the actual volume amount so that you never get the same sound at the exact same volume twice in a row. This creates a more natural feel to the sound.

    The second script works in a similar way to the footsteps but was for character sounds such as noises made by the character (coughing, talking, farting, whatever...).

    The final script was just an example of the some tables I used for the sound paths & the table prefix names were how I randomly selected from the specified table based on ground type + random number between 1 & 5.

    As you can see: it is no simple task for adding dynamic sound to your game & this script took me quite a while to type up & get working, without any bugs (at the time).

    * edit: I think the second script is probably an older version of the first script (based on the header comment title in the script).

    Imperator

    7278 Posts

  • #6, by BenmirathSaturday, 14. June 2014, 15:18 10 years ago
    Yikes, that's about as complicated as I'd feared. Thanks for passing it along though, it will be helpful for thinking through if and how I'll implement it. I was afraid that I'd have to start sounds via lua. I guess I'll go through and ctrl-f any instance of "play sound" in the xml and use that to track down any I need to replace. I'll report back if I manage anything decent with it.
    -Ben

    Newbie

    31 Posts

  • #7, by afrlmeSaturday, 14. June 2014, 15:25 10 years ago
    That's not the entire script either... don't forget I was manually triggering the function via the animation frames themselves & also I think I included scripts at begin of scene for setting ground type & also inside of action areas too for updating ground type.

    It was quite a complex task.

    The bottom half of the first script can be ignored... it was for adjusting character speed based on character size but is no longer relevant due to the new walk system & character size now being float instead if integer.

    Imperator

    7278 Posts