Geek asks and answers

File server caching

I have a java file server that serves the file via http. Each file is uniquely addressed by an identifier, for example:

http://fileserver/id/123455555

I want to add a cache layer to it so that files with the most frequently used access are stored in memory. I would also like to control the overall cache size. I am thinking of using ehcache or oscache for this, but I only used them to cache the serialized object before. Are they a good choice and are there any additional considerations for creating a file cache?

Edit

Thanks for all the answers. Some file server details simplify (or complicate) the problem:

  • After saving the file, it never changes.
  • MD5 hash to avoid duplicate files when saving. (I am aware of potential collisions and security issues)
  • A file server running in Linux boxes.

Edit 2 Although the server itself does not impose any restrictions on the type of file that it supports, files are mainly images (jpg, gif, pgn), Word, excel, PDF no more than 10 MB.

+3
source share
4 answers

Use the HTTP protocol

The most effective caching mechanism will be to switch caching from your own server and as close to the client as possible (data locality;)). Use the HTTP protocol effectively so that clients and caching proxies perform caching when they can do this:

  • ETag, (, MD5Sum) - , !
  • / /

edit: , , , Expires .

... , ...

Ehcache

EhCache - , .

, " ", , , , ( ), LFU ( ) , LRU ( ) - - 30 . LFU LRU.

, , , . LRU , LFU , , .

OSCache

OSCache, .

  • . , Java , , , Java: . ( , , BigMemory, )
  • , , , (gzip'd) ( , !). , . , , (, , ), , .
+1

IMHO, , .

  • , , .
  • Os can use all available free memory, which can vary depending on what else the system does.
  • You do not double the disk cache (since this is the disk cache).

In any case, the OS will store all the least used files in memory.

+1
source

More articles:


All Articles