Geek asks and answers

Fast hashing with fuzzy critical error in JavaScript for files

I am looking for a fast, low collision hash implemented in JavaScript. It should not be a crypto hash. I mainly use it as a way to see if a given file has already been downloaded (or partially downloaded) to a user account in order to save some loading time in large (video) files.

I am using the new HTML5 file API to read in file slices. Then I pass this to SparkMD5 to give me a hash of the file. I like the fact that SparkMD5 allows me to do an incremental hash, so I don't need to read everything in memory.

In general, SparkMD5 works for my needs, but it may take some time for large files to get my hash (about 30 seconds for a 300 MB file). Ideally, I would like to reduce this. I am not so versed in hash functions, so I am not looking for something in the port, and I am fine looking for an already implemented library.

+5
source share
1 answer

I compares various hashing algorithms, and here are the quickest options I found:

  • If you only need 32-bit digests, use iMurmurHash . Note that this will give you collisions after about 2 ** 14 (16,000) hashes.

  • SparkMD5, 32 . 64 128- Murmur, SparkMD5 (75 /).

    • , , SparkMD5, SparkMD5, , .

JavaScript. , SparkMD5 ArrayBuffers.

Node, :

  • : crypto MD5.

    • : , 500 /, npm, murmurhash-native . 128 , 128 , .

      ( , murmurhash-native , , Node MD5 .)

  • , .

  • :

    • 32 , iMurmurHash. , 2 ** 14 (16 000) .

    • 32 : - MD5.

      • , , -, . , , , .
+1

More articles:


All Articles