Geek asks and answers

How do I upload files larger than 5 GB in Amazon S3?

I am currently using Rails 3.2 with the Carrierwave stone to upload files to Amazon S3. Now I need to be able to process user-submitted files larger than 5 GB, while using the Carrierwave gem. Are there any other gems or branches of Carrierwave or Fog that can handle uploading 5 GB + files to S3?

Edit: I would prefer not to rewrite the complete solution for downloading Rails, so links like this won't help: https://gist.github.com/908875 .

+5
source share
3 answers

I figured out how to do it, and now it works. In the correct config/environmentfile, add the following to send files of 100 MB fragments to Amazon S3:

CarrierWave.configure do |config|
  config.fog_attributes = { :multipart_chunk_size => 104857600 }
end

Since foggy pearls have built-in multi-line downloads (thanks to Veraticus pointing this out), the corresponding configuration attributes just need to be passed into the fog through Carrierwave. When sending to S3, I received frequent errors Connection reset by peer (Errno::ECONNRESET), so parts of the download may be repeated.

+6
source

You want to use S3 multi-page download . It is useful that Fog can actually handle multi-page S3 downloads, as you can see in this stretch request .

, Carrierwave , . Carrierwave, Fog, .

+6

Before downloading, you need to break the file into small pieces.

Take a look at the following:

http://www.ruby-forum.com/topic/1282369

http://joemiller.me/2011/02/18/client-support-for-amazon-s3-multipart-uploads-files-5gb/

In any case, you need to split the file.

-2
source

More articles:


All Articles