I have a Django view that processes downloaded files, and when it is called multiple times, we always run into memory errors. (We are on Heroku, so we get 512 MB of memory for every web dino).
Errors over memory indicate that each call increases memory usage by the size of the file, and memory usage is never returned. I think the server reads the image into memory to save it in the Django model, but then never releases the memory when it is done.
I think there should be a link to the image that is saved after the function returns, preventing garbage collection. But I can’t think of what it can be.
Here is a simplified version of the code that I tested to make sure it has the same problem:
def simplified_add_image(request, biz_id):
if request.is_ajax():
newBI = NewBusinessImage(business_id=biz_id, name=request.GET.get("name"), primary_image=True)
newBI.id = uuid.uuid4()
newBI.save()
uniquename = biz_id + ".." + get_a_uuid() + ".jpg"
newBI.original_image.save(uniquename, ContentFile(request.read()))
tasks.image_meta_task.delay(uniquename, newBI.id)
return StockJSONResponse(request, {
"success" : True,
})
I would really appreciate any help. Many thanks!
Clay
Below is the additional information requested in the comments:
We save our files on AmazonS3:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATIC_URL = 'https://s3.amazonaws.com/%s/' % AWS_STORAGE_BUCKET_NAME
from S3 import CallingFormat
AWS_CALLING_FORMAT = CallingFormat.SUBDOMAIN
And here is the NewBusinessImage model
class NewBusinessImage(models.Model):
id = UUIDField(auto=True,primary_key=True,version=4, default=uuid.uuid4())
business = models.ForeignKey('Business')
name = models.CharField(max_length=100)
original_image = models.ImageField(upload_to='photos/originals/')
thumbnail = models.ImageField(upload_to='photos/thumbnails/')
display_image = models.ImageField(upload_to='photos/displays/')
enlarged_image = models.ImageField(upload_to='photos/enlarged/')
num_views = models.PositiveIntegerField(editable=False, default=0)
primary_image = models.BooleanField(default=False)