Serving dynamically generated ZIP archives in Django

How to serve users a dynamically generated ZIP archive in Django?

I'm making a site, where users can choose any combination of available books and download them as ZIP archive. I'm worried that generating such archives for each request would slow my server down to a crawl. I have also heard that Django doesn't currently have a good solution for serving dynamically generated files.

Asked by: Sienna147 | Posted: 28-01-2022

Answer 1

The solution is as follows.

Use Python module zipfile to create zip archive, but as the file specify StringIO object (ZipFile constructor requires file-like object). Add files you want to compress. Then in your Django application return the content of StringIO object in HttpResponse with mimetype set to application/x-zip-compressed (or at least application/octet-stream). If you want, you can set content-disposition header, but this should not be really required.

But beware, creating zip archives on each request is bad idea and this may kill your server (not counting timeouts if the archives are large). Performance-wise approach is to cache generated output somewhere in filesystem and regenerate it only if source files have changed. Even better idea is to prepare archives in advance (eg. by cron job) and have your web server serving them as usual statics.

Answered by: Max444 | Posted: 01-03-2022

Answer 2

Here's a Django view to do this:

import os
import zipfile
import StringIO

from django.http import HttpResponse


def getfiles(request):
    # Files (local path) to put in the .zip
    # FIXME: Change this (get paths from DB etc)
    filenames = ["/tmp/file1.txt", "/tmp/file2.txt"]

    # Folder name in ZIP archive which contains the above files
    # E.g [thearchive.zip]/somefiles/file2.txt
    # FIXME: Set this to something better
    zip_subdir = "somefiles"
    zip_filename = "%s.zip" % zip_subdir

    # Open StringIO to grab in-memory ZIP contents
    s = StringIO.StringIO()

    # The zip compressor
    zf = zipfile.ZipFile(s, "w")

    for fpath in filenames:
        # Calculate path for file in zip
        fdir, fname = os.path.split(fpath)
        zip_path = os.path.join(zip_subdir, fname)

        # Add file, at correct path
        zf.write(fpath, zip_path)

    # Must close zip for all contents to be written
    zf.close()

    # Grab ZIP file from in-memory, make response with correct MIME-type
    resp = HttpResponse(s.getvalue(), mimetype = "application/x-zip-compressed")
    # ..and correct content-disposition
    resp['Content-Disposition'] = 'attachment; filename=%s' % zip_filename

    return resp

Answered by: Maddie435 | Posted: 01-03-2022

Answer 3

Many answers here suggest to use a StringIO or BytesIO buffer. However this is not needed as HttpResponse is already a file-like object:

response = HttpResponse(content_type='application/zip')
zip_file = zipfile.ZipFile(response, 'w')
for filename in filenames:
    zip_file.write(filename)
response['Content-Disposition'] = 'attachment; filename={}'.format(zipfile_name)
return response

Answered by: Dainton853 | Posted: 01-03-2022

Answer 4

I used Django 2.0 and Python 3.6.

import zipfile
import os
from io import BytesIO

def download_zip_file(request):
    filelist = ["path/to/file-11.txt", "path/to/file-22.txt"]

    byte_data = BytesIO()
    zip_file = zipfile.ZipFile(byte_data, "w")

    for file in filelist:
        filename = os.path.basename(os.path.normpath(file))
        zip_file.write(file, filename)
    zip_file.close()

    response = HttpResponse(byte_data.getvalue(), content_type='application/zip')
    response['Content-Disposition'] = 'attachment; filename=files.zip'

    # Print list files in zip_file
    zip_file.printdir()

    return response

Answered by: Emily844 | Posted: 01-03-2022

Answer 5

For python3 i use the io.ByteIO since StringIO is deprecated to achieve this. Hope it helps.

import io

def my_downloadable_zip(request):
    zip_io = io.BytesIO()
    with zipfile.ZipFile(zip_io, mode='w', compression=zipfile.ZIP_DEFLATED) as backup_zip:
        backup_zip.write('file_name_loc_to_zip') # u can also make use of list of filename location
                                                 # and do some iteration over it
     response = HttpResponse(zip_io.getvalue(), content_type='application/x-zip-compressed')
     response['Content-Disposition'] = 'attachment; filename=%s' % 'your_zipfilename' + ".zip"
     response['Content-Length'] = zip_io.tell()
     return response

Answered by: Aldus330 | Posted: 01-03-2022

Answer 6

Django doesn't directly handle the generation of dynamic content (specifically Zip files). That work would be done by Python's standard library. You can take a look at how to dynamically create a Zip file in Python here.

If you're worried about it slowing down your server you can cache the requests if you expect to have many of the same requests. You can use Django's cache framework to help you with that.

Overall, zipping files can be CPU intensive but Django shouldn't be any slower than another Python web framework.

Answered by: Freddie975 | Posted: 01-03-2022

Answer 7

Shameless plug: you can use django-zipview for the same purpose.

After a pip install django-zipview:

from zipview.views import BaseZipView

from reviews import Review


class CommentsArchiveView(BaseZipView):
    """Download at once all comments for a review."""

    def get_files(self):
        document_key = self.kwargs.get('document_key')
        reviews = Review.objects \
            .filter(document__document_key=document_key) \
            .exclude(comments__isnull=True)

        return [review.comments.file for review in reviews if review.comments.name]

Answered by: Kevin469 | Posted: 01-03-2022

Answer 8

I suggest to use separate model for storing those temp zip files. You can create zip on-fly, save to model with filefield and finally send url to user.

Advantages:

Serving static zip files with django media mechanism (like usual uploads).
Ability to cleanup stale zip files by regular cron script execution (which can use date field from zip file model).

Answered by: Kimberly716 | Posted: 01-03-2022

Answer 9

A lot of contributions were made to the topic already, but since I came across this thread when I first researched this problem, I thought I'd add my own two cents.

Integrating your own zip creation is probably not as robust and optimized as web-server-level solutions. At the same time, we're using Nginx and it doesn't come with a module out of the box.

You can, however, compile Nginx with the mod_zip module (see here for a docker image with the latest stable Nginx version, and an alpine base making it smaller than the default Nginx image). This adds the zip stream capabilities.

Then Django just needs to serve a list of files to zip, all done! It is a little more reusable to use a library for this file list response, and django-zip-stream offers just that.

Sadly it never really worked for me, so I started a fork with fixes and improvements.

You can use it in a few lines:

def download_view(request, name=""):
    from django_zip_stream.responses import FolderZipResponse
    path = settings.STATIC_ROOT
    path = os.path.join(path, name)

    return FolderZipResponse(path)

You need a way to have Nginx serve all files that you want to archive, but that's it.

Answered by: Audrey185 | Posted: 01-03-2022

Answer 10

Can't you just write a link to a "zip server" or whatnot? Why does the zip archive itself need to be served from Django? A 90's era CGI script to generate a zip and spit it to stdout is really all that's required here, at least as far as I can see.

Answered by: Kimberly219 | Posted: 01-03-2022

Similar questions

python - How do i output a dynamically generated web page to a .html page instead of .py cgi page?

So ive just started learning python on WAMP, ive got the results of a html form using cgi, and successfully performed a database search with mysqldb. I can return the results to a page that ends with .py by using print statements in the python cgi code, but i want to create a webpage that's .html and have that returned to the user, and/or keep them on the same webaddress when the database search results return. th...

python - Serve a dynamically generated image with Django

How do I serve a dynamically generated image in Django? I have an html tag <html> ... <img src="images/dynamic_chart.png" /> ... </html> linked up to this request handler, which creates an in-memory image def chart(request): img = Image.new("RGB", (300,300), "#FFFFFF") data = [(i,randint(100,200)) for i in range(0,300,10)] draw = I...

python - Django and dynamically generated images

I have a view in my Django application that automatically creates an image using the PIL, stores it in the Nginx media server, and returns a html template with a img tag pointing to it's url. This works fine, but I notice an issue. For every 5 times I access this view, in 1 of them the image doesn't render. I did some investigation and I found something interesting, this is the HTTP response header when the...

io - Anagram Code in Python - Compare dynamically generated strings to a txt file

python - iterate a list when list name is dynamically generated

How do I iterate through a list whose name will be dynamically generated? boneList_head =['def_neck', 'def_armbase']#hard coded list itemType='head'# result of a user button press ... def selectBones(): global itemType bones =('boneList_'+itemType)# evaluates as a string , not name of a list for bone in bones: cmds.select(bone, tgl=True) the problem is bones is...

python - Flask: How to get url for dynamically generated image file?

from flask import Flask, redirect, url_for app = Flask(__name__) @app.route('/') def index(): generate_img("test.jpg"); return '<img src=' + url_for(filename='test.jpg') + '>' generate_img() will output a random image to the current directory (same folder as this script). I am getting 404, I navigate to mydomain.com/test.jpg but it's not there.

web scraping - Reading dynamically generated web pages using python

I am trying to scrape a web site using python and beautiful soup. I encountered that in some sites, the image links although seen on the browser is cannot be seen in the source code. However on using Chrome Inspect or Fiddler, we can see the the corresponding codes. What I see in the source code is: <div id="cntnt"></div> But on Chrome Inspect, I can see a whole bunch of HTML\...

python - Serve dynamically generated file instead of htm

I'm completely new to twisted and need a little push forward. I'm looking for a way to serve a dynamically generated file other than htm page - for example csv file. EDIT: It happened that while change of 'content-type' would suffice with csv file, what I needed was a change of 'content-disposition' of request: request.responseHeaders.setRawHeaders('content-disposition', ['attachment; filename="file...

python - serving dynamically generated static img file in template django 1.6 dev server

I am trying to serve a static img file in a template that I have just had Django generate. The template code looks like this. <img src="{% static {{ output }} %}" height="50px" width="50px" alt="hi"> output is a context variable that I passed to the template through my views. When I try and load the page I get this error. Exception Type: TemplateSyntaxError Exception Va...

python - Not able to recognise url variable in html page in django. Html page is getting generated dynamically

here is the views.py def home (request): now = datetime.datetime.now() t = get_template('index.html') html = t.render(Context({'current_date': now})) return HttpResponse(html) def run(request, id): table="" table + = "<table> <tr> <td> <a href= \" /run/{{ id }} \" > "+ linkname + "</a></td></tr></table>" ret...

reflection - How can I dynamically get the set of classes from the current python module?

I have a python module that defines a number of classes: class A(object): def __call__(self): print "ran a" class B(object): def __call__(self): print "ran b" class C(object): def __call__(self): print "ran c" From within the module, how might I add an attribute that gives me all of the classes? dir() gives me the names of everything from withi...

python - Dynamically update ModelForm's Meta class

I am hoping to dynamically update a ModelForm's inline Meta class from my view. Although this code seems to update the exclude list in the Meta class, the output from as_p(), as_ul(), etc does not reflect the updated Meta exclude. I assume then that the html is generated when the ModelForm is created not when the as_*() is called. Is there a way to force the update of the HTML...

reflection - How to dynamically load a Python class

Given a string of a Python class, e.g. my_package.my_module.MyClass, what is the best possible way to load it? In other words I am looking for a equivalent Class.forName() in Java, function in Python. It needs to work on Google App Engine. Preferably this would be a function that accepts the FQN of the class as a string, and returns a reference to the class: my_clas...

python - How do I retrieve a Django model class dynamically?

Without having the full module path of a Django model, is it possible to do something like: model = 'User' [in Django namespace] model.objects.all() ...as opposed to: User.objects.all(). EDIT: I am trying to make this call based on command-line input. Is it possible to avoid the import statement, e.g., model = django.authx.models.User

python - Jython, Query multiple columns dynamically

I am working with a oracle database and Jython. I can get data pulled from the database no problem. results = statement.executeQuery("select %s from %s where column_id = '%s'", % (column, table, id)) This works fine if I want to pull one column of data. Say I wanted to loop threw a list like this: columns = ['column1', 'column2', 'column3', 'column4', 'colum...

python - Django: How to create a model dynamically just for testing

I have a Django app that requires a settings attribute in the form of: RELATED_MODELS = ('appname1.modelname1.attribute1', 'appname1.modelname2.attribute2', 'appname2.modelname3.attribute3', ...) Then hooks their post_save signal to update some other fixed model depending on the attributeN defined. I would like to test ...

python - Doctest for dynamically created objects

What is the best way to test code like this (the one below obviously fails while object is created in different block every time): def get_session(db_name, verbose, test): """Returns current DB session from SQLAlchemy pool. >>> get_session('Mmusc20090126', False, True) <sqlalchemy.orm.session.Session object at 0xfb5ff0> """ if test: engine = create_engine('sqlite:///:memory:', echo=verb...

python - Dynamically specifying tags while using replaceWith in Beautiful Soup

Previously I asked this question and got back this BeautifulSoup example code, which after some consultation locally, I decided to go with. >>> from BeautifulSoup import BeautifulStoneSoup >>> html = """ ... <config> ... <links> ... <link name="Link1" id...

python - How to make these dynamically typed functions type-safe?

Closed. This question needs to be more focused. It ...

Still can't find your answer? Check out these communities...

top