I need to get files in my web application larger than 64 KB. Right now, in the v1beta1 JSON API, Google is allowing you to upload / download 64KB files through its JSON API. I figured out how to upload files larger than 64 KB using "renewable" downloads (and not via the v1beta1 interface in the JSON API, but manually).
What I can’t understand is a good way to download. Right now, I'm making the ACL public for the object that I want to upload, upload the file, and then delete the public ACL on the object. This is not only ineffective, but also not very clean. Is there a better method I could use, or am I stuck until Google becomes the best tool in a future version of my API?
Background information
I am writing a GAE application and I know the google.appengine.api.files interface. Unfortunately, this does not work on live buckets when using the local development environment, and for testing we need my team to check the local development (too cumbersome to deploy for GAE among other restriction / security factors). We can interact with all other APIs, with the exception of cloud storage, so I am writing a class that will use the JSON API or AppEngine when reading / writing / deleting from cloud storage. I got a working implementation, but I am not happy with the way I extract the files.
Explained from the comment below : We upload large amounts of information, massage it and store it in Cloud Storage for consumption in BigQuery. We need to use live buckets from the dev environment, because if we do not, BigQuery will not be able to use the data we want to check. No need to maintain these files, just process them
The solution from the comment on the accepted answer is below:
I was able to reuse my authenticated object httplib2from my code that interacts with the JSON API to execute an authenticated GET request with the URL endpoint https://{bucket_name}.storage.googleapis.com/{object_name}, adding only the headers Content-Length: 0and x-goog-api-version: 2.
source
share