Asset Deduplication?

  • #1, by KisaiSaturday, 15. June 2013, 14:51 11 years ago
    Hello,

    I was wondering (particularly if Visionaire will be able to target a iOS device) if the development environment is able to deduplicate the assets used.

    Like, if the same images, or audio files are re-used in multiple places, have the game development environment check the hash of the file if it's already used. When packing/encrypting the game, instead of storing an audio or image file used in two scenes once in each scene, the first and second reference points to a common use file instead.

    I don't quite know how the Visionaire internal formats work, but the topic came up when talking about how steam, iOS and game consoles patch things.

    Newbie

    3 Posts


  • #2, by afrlmeSaturday, 15. June 2013, 15:59 11 years ago
    iOS target port is currently in development - I can't really specify any more details on how it will function/work - as I don't know - but David (BigStans) or SimonS will be able to provide an answer for everything else you've asked.

    Imperator

    7278 Posts

  • #3, by SimonSSaturday, 15. June 2013, 16:26 11 years ago
    I don't really see the point of generating hashes and comparing them, it's very time intensive, that would slow the export process down. If it's the same file (same path and name), it won't be packed two times, but if you have two equal files there is nothing done about that. That's something the game developer must handle by not duplicating the files.

    Thread Captain

    1580 Posts

  • #4, by KisaiSaturday, 15. June 2013, 17:13 11 years ago
    I don't really see the point of generating hashes and comparing them, it's very time intensive, that would slow the export process down. If it's the same file (same path and name), it won't be packed two times, but if you have two equal files there is nothing done about that. That's something the game developer must handle by not duplicating the files.


    Hmm interesting. What I was thinking of though was just hash the files at the finalization/packing step so that the files are only stored once. I don't know how other developers workflows are, but if other tools are used to break apart animation frames to use in the game engine, there may inevitably be the same images or audio reused a dozen times with different filenames.

    Anyway it was just a suggestion I thought I would mention. I'm still evaluating if Visionaire is the right tool for a project. iOS target being one of the consideration points.

    Newbie

    3 Posts

  • #5, by SimonSSaturday, 15. June 2013, 17:48 11 years ago
    I think I understood what you meant, but as I said hashing is a bit time intensive. You could try a program like dupehunter to see how much potential there is (and to see the time needed). Even if the duplicates wouldn't stored twice there would be still some things to do, that the image won't be two times in the memory. I think a bit about that and look at some benchmarks, if I use it it'll be done only by choice.

    Thread Captain

    1580 Posts