Add event for when a game save is updated

[cryptonaus]

Adds an event, saveUpdate that triggers when 1) a new save occurs and 2) the hash of the new save is new or differs from the prior hash. The dependency on subtle crypto for hashing necessitates a secure context to use this functionality.

Combined with #1095, can provide a pretty reliable save update event. I tested with 80ms and had no issues.

[michael-j-green]

Any performance impact to this? Do you have any fallback for if EJS is not being run on https?

[cryptonaus]

I didn't notice any performance impact (and it should be handled asynchronously, so I don't think it should matter). I don't have a fallback for not HTTPS, unfortunately—I briefly examined it, but I didn't want to do something overly complicated.

Edit: And just to clarify a bit more, it seems like an own fallback would require an external library or writing our own algorithm. Subtle crypto is a native browser implementation, and so is likely also the most performant way to achieve this.

[michael-j-green]

In that case; can you do a check for https and bypass (return to existing behaviour) this code if it's running in plain http?

[cryptonaus]

I'm not sure what you mean. It's only adding a new event in the case where it can be compared by hash because of HTTPS/localhost—or are you saying to invoke their callback in the saveSave event (with a null hash)?

[michael-j-green]

Yup... null hash.

Currently in Gaseous, I'm simply polling EJS every x seconds for the file and if the last updated datetime is different uploading it to my C# server. My C# server is then doing the hashing (no https required) to determine if the file is different. I do it this way because a) there's no way to tell if the file has changed or not (currently), and b) not everyone is running in https.

[ethanaobrien]

I do think there should be some method of fallback here.. not sure how

[cryptonaus]

Comments on the review so far are all fine. Will think about a fallback. It's late here, so I'll submit a revision tomorrow.

[cryptonaus]

Switched to Cyrb53, per this StackOverflow question: https://stackoverflow.com/questions/7616461/generate-a-hash-from-string-in-javascript. It also eliminates the dependency on subtle crypto. I did run a few tests with millions of random elements and did not encounter even a single collision: Good enough for not cryptographic purposes, anyway.

Some performance notes on my M2 MacBook Air:

• In my testing, Cyrb53 was faster than the accepted, Java-based answer, and supposedly more random
• Slower than SHA-1, especially on larger buffer sizes, but still fast enough for practical usage
  • At 1MB/save (about the largest PPSSPP save from a quick Google search, which I assumed was the largest of EmulatorJS), Cyrb53 takes about 1 millisecond (SHA-1 takes ~0.5)
  • At smaller values, they are much faster so there's no concern—32KB took about 50 microseconds on Cyrb53, and 30 for SHA-1.

In other words, this hash should be fast enough for all cores flushing saves at any sensible frequency. If we were to poll even every 100ms on PPSSPP, the update callback would be delayed by only about 1% of that cycle time. (The bigger concern is how frequently the cores themselves can dump.)

Rudimentary performance testing code

var testExponent = 3;
var randomStrings = []
var testCount = () => 10 ** testExponent

var encoder = new TextEncoder();
var generateRandomString = () => (Math.random()*10).toString(36).substring(4).repeat(80000);
for (let i = 0; i < testCount(); i++) {
    randomStrings[i] = encoder.encode(generateRandomString()); // Since saves are buffers
}
console.log(`Preparing test of ${randomStrings.length} buffers of approximate size ${randomStrings[0].length}`)



// --- CYRB TEST---
var generateHashCyrb = async(str, seed = 0) => {
    let h1 = 0xdeadbeef ^ seed, h2 = 0x41c6ce57 ^ seed;
    for(let i = 0, ch; i < str.length; i++) {
        ch = str[i];
        h1 = Math.imul(h1 ^ ch, 2654435761);
        h2 = Math.imul(h2 ^ ch, 1597334677);
    }
    h1  = Math.imul(h1 ^ (h1 >>> 16), 2246822507);
    h1 ^= Math.imul(h2 ^ (h2 >>> 13), 3266489909);
    h2  = Math.imul(h2 ^ (h2 >>> 16), 2246822507);
    h2 ^= Math.imul(h1 ^ (h1 >>> 13), 3266489909);
  
    return 4294967296 * (2097151 & h2) + (h1 >>> 0);
};

var startTime = performance.now();
await (async () => {for (i = 0; i < testCount(); i++) {
    (await generateHashCyrb(randomStrings[i])).toString(16);
}})()
var endTime = performance.now();
console.log(`cyrb53 test done, took ${endTime - startTime} milliseconds`);



// --- SHA TEST ---
function digestAsHexString(buffer) {
    const hashArray = Array.from(new Uint8Array(buffer));
    return hashArray.map((b) => b.toString(16).padStart(2, "0")).join("");
}

var startTime = performance.now();
await (async () => {for (i = 0; i < testCount(); i++) {
    await window.crypto.subtle.digest("SHA-1", randomStrings[i]).then(digestAsHexString);
}})()
var endTime = performance.now();
console.log(`SHA-1 test done, took ${endTime - startTime} milliseconds`);

[ethanaobrien]

I do think this looks good, thank you

[michael-j-green]

Looks good... happy to approve once documentation changes have been proposed :)

[cryptonaus]

Proposed a bit of documentation

[michael-j-green]

Awesome... thanks @cryptonaus!