How To Generate Checksum & Convert To 64 Bit In Javascript For Very Large Files Without Overflowing RAM?

Question: How to generate a checksum correctly, which is unique, consistent independent of browsers? Also, I would like to convert a SHA256/MD5 checksum string to 64-bit. How t

Solution 1:

There is a logical issue in the code at below line:

start = chunkCount * SIZE_CHECKSUM;  // <--- bug

The variable start is initialized to 0 and then again reset to 0 in the 1st iteration, which is not right.
Following is the way to get a 32 bytes SHA5 checksum with the same library mentioned in the question: "emn178/js-sha256".

That library doesn't provide a Typescript interface, but we can define trivially as following:

// Sha256.d.ts  (also name the corresponding JS file as "Sha256.js")
declare class Sha256 {
  update (data: ArrayBuffer): Sha256;
  hex (): string;
}

declare var sha256: any;
declare interface sha256 {
  create (): Sha256;
}

Then use it as following:

import "./external/Sha256"

async function GetChecksum (file: File):
Promise<string>
{
  let algorithm = sha256.create(); 
  for(let chunkCount = 0, totalChunks = Math.ceil(file.size / SIZE_CHECKSUM); 
      chunkCount < totalChunks;
      ++chunkCount)
  {
    let start = chunkCount * SIZE_CHECKSUM, end = Math.min(start + SIZE_CHECKSUM, file.size); 
    algorithm.update(await (new Response(file.slice(start, end)).arrayBuffer()));
  }
  return algorithm.hex();
}

Above code generates same checksums in all my browsers for any chunk size.