How to write a download progress indicator in Python?

I am writing a little application to download files over http (as, for example, described here).

I also want to include a little download progress indicator showing the percentage of the download progress.

Here is what I came up with:

    sys.stdout.write(rem_file + "...")    
    urllib.urlretrieve(rem_file, loc_file, reporthook=dlProgress)

    def dlProgress(count, blockSize, totalSize):
      percent = int(count*blockSize*100/totalSize)
      sys.stdout.write("%2d%%" % percent)
      sys.stdout.write("\b\b\b")
      sys.stdout.flush()

Output: MyFileName... 9%

Any other ideas or recommendations to do this?

One thing that's somewhat annoying is the blinking cursor in the terminal on the first digit of the percentage. Is there a way to prevent this? Is there a way to hide the cursor?

EDIT:

Here a better alternative using a global variable for the filename in dlProgress and the '\r' code:

    global rem_file # global variable to be used in dlProgress

    urllib.urlretrieve(rem_file, loc_file, reporthook=dlProgress)

    def dlProgress(count, blockSize, totalSize):
      percent = int(count*blockSize*100/totalSize)
      sys.stdout.write("\r" + rem_file + "...%d%%" % percent)
      sys.stdout.flush()

Output: MyFileName...9%

And the cursor shows up at the END of the line. Much better.


Asked by: John734 | Posted: 01-10-2021






Answer 1

There's a text progress bar library for python at http://pypi.python.org/pypi/progressbar/2.2 that you might find useful:

This library provides a text mode progressbar. This is tipically used to display the progress of a long running operation, providing a visual clue that processing is underway.

The ProgressBar class manages the progress, and the format of the line is given by a number of widgets. A widget is an object that may display diferently depending on the state of the progress. There are three types of widget: - a string, which always shows itself; - a ProgressBarWidget, which may return a diferent value every time it's update method is called; and - a ProgressBarWidgetHFill, which is like ProgressBarWidget, except it expands to fill the remaining width of the line.

The progressbar module is very easy to use, yet very powerful. And automatically supports features like auto-resizing when available.

Answered by: Ryan989 | Posted: 02-11-2021



Answer 2

You might also try:

sys.stdout.write("\r%2d%%" % percent)
sys.stdout.flush()

Using a single carriage return at the beginning of your string rather than several backspaces. Your cursor will still blink, but it'll blink after the percent sign rather than under the first digit, and with one control character instead of three you may get less flicker.

Answered by: Kellan819 | Posted: 02-11-2021



Answer 3

For what it's worth, here's the code I used to get it working:

from urllib import urlretrieve
from progressbar import ProgressBar, Percentage, Bar

url = "http://......."
fileName = "file"
pbar = ProgressBar(widgets=[Percentage(), Bar()])
urlretrieve(url, fileName, reporthook=dlProgress)

def dlProgress(count, blockSize, totalSize):
    pbar.update( int(count * blockSize * 100 / totalSize) )

Answered by: Edward377 | Posted: 02-11-2021



Answer 4

If you use the curses package, you have much greater control of the console. It also comes at a higher cost in code complexity and is probably unnecessary unless you are developing a large console-based app.

For a simple solution, you can always put the spinning wheel at the end of the status messge (the sequence of characters |, \, -, / which actually looks nice under blinking cursor.

Answered by: Lenny733 | Posted: 02-11-2021



Answer 5

I used this code:

url = (<file location>)
file_name = url.split('/')[-1]
u = urllib2.urlopen(url)
f = open(file_name, 'wb')
meta = u.info()
file_size = int(meta.getheaders("Content-Length")[0])
print "Downloading: %s Bytes: %s" % (file_name, file_size)

file_size_dl = 0
block_sz = 8192
while True:
    buffer = u.read(block_sz)
    if not buffer:
        break

    file_size_dl += len(buffer)
    f.write(buffer)
    status = r"%10d [%3.2f%%]" % (file_size_dl, file_size_dl * 100. / file_size)
    status = status + chr(8)*(len(status)+1)
    print status,

f.close()

Answered by: Elise398 | Posted: 02-11-2021



Answer 6

def download_progress_hook(count, blockSize, totalSize):
  """A hook to report the progress of a download. This is mostly intended for users with slow internet connections. Reports every 5% change in download progress.
  """
  global last_percent_reported
  percent = int(count * blockSize * 100 / totalSize)

  if last_percent_reported != percent:
    if percent % 5 == 0:
      sys.stdout.write("%s%%" % percent)
      sys.stdout.flush()
    else:
      sys.stdout.write(".")
      sys.stdout.flush()

    last_percent_reported = percent

urlretrieve(url, filename, reporthook=download_progress_hook)

Answered by: Patrick380 | Posted: 02-11-2021



Answer 7

For small files you may need to had this lines in order to avoid crazy percentages:

sys.stdout.write("\r%2d%%" % percent)

sys.stdout.flush()

Cheers

Answered by: Charlie599 | Posted: 02-11-2021



Answer 8

Thats how I did this could help you: https://github.com/mouuff/MouDownloader/blob/master/api/download.py

Answered by: Julian173 | Posted: 02-11-2021



Answer 9

Late to the party, as usual. Here's an implementation that supports reporting progress, like the core urlretrieve:

import urllib2

def urlretrieve(urllib2_request, filepath, reporthook=None, chunk_size=4096):
    req = urllib2.urlopen(urllib2_request)

    if reporthook:
        # ensure progress method is callable
        if hasattr(reporthook, '__call__'):
            reporthook = None

        try:
            # get response length
            total_size = req.info().getheaders('Content-Length')[0]
        except KeyError:
            reporthook = None

    data = ''
    num_blocks = 0

    with open(filepath, 'w') as f:
        while True:
            data = req.read(chunk_size)
            num_blocks += 1
            if reporthook:
                # report progress
                reporthook(num_blocks, chunk_size, total_size)
            if not data:
                break
            f.write(data)

    # return downloaded length
    return len(data)

Answered by: Carlos524 | Posted: 02-11-2021



Answer 10

To avoid progress values like 106% or similar, follow the below logic

percent = min(int(count * blockSize * 100 / totalSize), 100)

Answered by: Arthur631 | Posted: 02-11-2021



Similar questions

python - GUI Progress Indicator using command line tools

I would like to know a good way to make a GUI progress bar. I'm using python, Glade/GTK. An example of a program I'd like to get more than just a pulse bar. Something that could give the user a more accurate way to know the progress. The program I'm trying to capture this for is cdparanoia, but would like to know how to do this in general for other programs like wget. Important to note on these two programs is the the info...


python - Volume level indicator in wxPython

I'm writing a simple sound player, and I'd like have a volume level indicator that would show the current signal level of played sound. I'm able to feed the widget with the number from some range or percentage or anything. Actually as I have the information from the decoding process, I just need the widget to visualize the level. Is there such a widget in


python - progress of loop indicator

This question already has answers here:


python - Apply Indicator Function to List

Is there an easy way to apply an indicator function to a list? i.e. I have a list with some numbers in it and I would like to return another list of the same size with, for example, ones where the positive numbers were in the original list and zeros where the negative numbers were in the original list.


pygtk - Python unity indicator applet and glade child window

I have created an unity indicator applet with python and glade. Here is the screenshot that appears when indicator applet is clicked. You can see the preferences menu. When this preferences menu is clicked, it opens a new window. Indicator Applet Menu Preferenc...


python - Combined multiple Indicator Columns

I have a dataframe with multiple columns representing categorical variables and the data in the columns is either 1 or 0 as an indicator like below: ID A B C D 100 1 0 0 0 101 0 0 1 0 102 0 1 0 0 103 0 0 0 1 I would like to represent this information as one column of data such as: ID VALUE 100 A 101 C 102 B ...


Mayavi, python axes INDICATOR

does anyone know how to switch on the axes INDICATOR in python code? I can switch it on when the graph is plotted, but I wanna know how to do it in code. I have found only how to switch on the axes: mlab.axes() but it is not the indicator (I mean that three arrows X,Y and Z) Thank you


list - Creating an indicator for strings of numbers, ignoring a particular one in python pandas

I have no idea how to title this question, edits welcome. I have a pandas series of numbers (could be made into a list, the type isn't important really). The numbers go from 1 to 13. For example: 13,13,1,1,1,1,13,2,1,1 I want to find strings of the same numbers but NOT counting 13 if it's in between or at the beginning. I want to return...


python - Create indicator list

I have 5 lists which have following records: list1 = [1,2,3,4,7,13,19,21] list2 = [1,2,7,13,14,19,21] list3 = [3,5,6,15] list4 = [17,18] list5 = [20] I want to create a single list with the union of all the elements in the lists and indicate in which of the original lists they are present. output_list = [[1,True,True,False,False,False],[2,True,True,False,False,False],[3...


python - Logical OR of 2 indicator columns in pandas

I have a dataframe with two indicator variable columns. I would like to create a 3rd column that is the logical OR of the two existing columns. I am able to, like this: df = DataFrame ({"a" : [1,0,0] , "b" : [0,0,1] }) df["c"] = (df["a"] | df["b"]).apply(lambda x : 1 if x else 0) The series produced by (df["a"] | df["b"]) is of type bool. This ...






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top