How to redirect stderr in Python?

I would like to log all the output of a Python script. I tried:

import sys

log = []

class writer(object):
    def write(self, data):
        log.append(data)

sys.stdout = writer()
sys.stderr = writer()

Now, if I "print 'something' " it gets logged. But if I make for instance some syntax error, say "print 'something# ", it wont get logged - it will go into the console instead.

How do I capture also the errors from Python interpreter?

I saw a possible solution here:

http://www.velocityreviews.com/forums/showpost.php?p=1868822&postcount=3

but the second example logs into /dev/null - this is not what I want. I would like to log it into a list like my example above or StringIO or such...

Also, preferably I don't want to create a subprocess (and read its stdout and stderr in separate thread).


Asked by: Audrey276 | Posted: 30-11-2021






Answer 1

I have a piece of software I wrote for work that captures stderr to a file like so:

import sys
sys.stderr = open('C:\\err.txt', 'w')

so it's definitely possible.

I believe your problem is that you are creating two instances of writer.

Maybe something more like:

import sys

class writer(object):
    log = []

    def write(self, data):
        self.log.append(data)

logger = writer()
sys.stdout = logger
sys.stderr = logger

Answered by: Aida883 | Posted: 01-01-2022



Answer 2

You can't do anything in Python code that can capture errors during the compilation of that same code. How could it? If the compiler can't finish compiling the code, it won't run the code, so your redirection hasn't even taken effect yet.

That's where your (undesired) subprocess comes in. You can write Python code that redirects the stdout, then invokes the Python interpreter to compile some other piece of code.

Answered by: Roland216 | Posted: 01-01-2022



Answer 3

I can't think of an easy way. The python process's standard error is living on a lower level than a python file object (C vs. python).

You could wrap the python script in a second python script and use subprocess.Popen. It's also possible you could pull some magic like this in a single script:

import os
import subprocess
import sys

cat = subprocess.Popen("/bin/cat", stdin=subprocess.PIPE, stdout=subprocess.PIPE)
os.close(sys.stderr.fileno())
os.dup2(cat.stdin.fileno(), sys.stderr.fileno())

And then use select.poll() to check cat.stdout regularly to find output.

Yes, that seems to work.

The problem I foresee is that most of the time, something printed to stderr by python indicates it's about to exit. The more usual way to handle this would be via exceptions.

---------Edit

Somehow I missed the os.pipe() function.

import os, sys
r, w = os.pipe()
os.close(sys.stderr.fileno())
os.dup2(w, sys.stderr.fileno())

Then read from r

Answered by: Audrey213 | Posted: 01-01-2022



Answer 4

To route the output and errors from Windows, you can use the following code outside of your Python file:

python a.py 1> a.out 2>&1

Source: https://support.microsoft.com/en-us/help/110930/redirecting-error-messages-from-command-prompt-stderr-stdout

Answered by: Agata301 | Posted: 01-01-2022



Answer 5

For such a request, usually it would be much easier to do it in the OS instead of in Python.

For example, if you're going to run "a.py" and record all the messages it will generate into file "a.out", it would just be

python a.py 2>&1 > a.out

The first part 2>&1 redirects stderr to stdout (0: stdin, 1:stdout, 2:stderr), and the second redirects that to a file called a.out.

And as far as I know, this command works in Windows, Linux or MacOS! For other file redirection techniques, just search the os plus "file redirection"

Answered by: Stuart743 | Posted: 01-01-2022



Answer 6

Since python 3.5 you can use contextlib.redirect_stderr

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        help(pow)

Answered by: Walter797 | Posted: 01-01-2022



Answer 7

I found this approach to redirecting stderr particularly helpful. Essentially, it is necessary to understand if your output is stdout or stderr. The difference? Stdout is any output posted by a shell command (think an 'ls' list) while sterr is any error output.

It may be that you want to take a shell commands output and redirect it to a log file only if it is normal output. Using ls as an example here, with an all files flag:

# Imports
import sys
import subprocess

# Open file
log = open("output.txt", "w+")

# Declare command
cmd = 'ls -a'
# Run shell command piping to stdout
result = subprocess.run(cmd, stdout=subprocess.PIPE, shell=True)
# Assuming utf-8 encoding
txt = result.stdout.decode('utf-8')
# Write and close file
log.write(txt)
log.close()

If you wanted to make this an error log, you could do the same with stderr. It's exactly the same code as stdout with stderr in its place. This pipes an error messages that get sent to the console to the log. Doing so actually keeps it from flooding your terminal window as well!

Saw this was a post from a while ago, but figured this could save someone some time :)

Answered by: Julia610 | Posted: 01-01-2022



Answer 8

import sys
import tkinter

# ********************************************

def mklistenconsswitch(*printf: callable) -> callable:
    def wrapper(*fcs: callable) -> callable:
        def newf(data):
            [prf(data) for prf in fcs]
        return newf
    stdoutw, stderrw = sys.stdout.write, sys.stderr.write
    funcs = [(wrapper(sys.stdout.write, *printf), wrapper(sys.stderr.write, *printf)), (stdoutw, stderrw)]
    def switch():
        sys.stdout.write, sys.stderr.write = dummy = funcs[0]
        funcs[0] = funcs[1]
        funcs[1] = dummy
    return switch

# ********************************************

def datasupplier():
    i = 5.5
    while i > 0:
        yield i
        i -= .5

def testloop():
    print(supplier.__next__())
    svvitch()
    root.after(500, testloop)

root = tkinter.Tk()
cons = tkinter.Text(root)
cons.pack(fill='both', expand=True)
supplier = datasupplier()
svvitch = mklistenconsswitch(lambda text: cons.insert('end', text))
testloop()
root.mainloop()

Answered by: David410 | Posted: 01-01-2022



Answer 9

Python will not execute your code if there is an error. But you can import your script in another script an catch exceptions. Example:

Script.py

print 'something#

FinalScript.py

from importlib.machinery import SourceFileLoader

try:
    SourceFileLoader("main", "<SCRIPT PATH>").load_module()
except Exception as e:
    # Handle the exception here

Answered by: Nicole990 | Posted: 01-01-2022



Answer 10

To add to Ned's answer, it is difficult to capture the errors on the fly during the compilation.

You can write several print statements in your script and you can stdout to a file, it will stop writing to the file when the error occurs. To debug the code you could check the last logged output and check your script after that point.


Something like this:

# Add to the beginning of the script execution(eg: if __name__ == "__main__":).
from datetime import datetime
dt = datetime.now()
script_dir = os.path.dirname(os.path.abspath(__file__))      # gets the path of the script
stdout_file = script_dir+r'\logs\log'+('').join(str(dt.date()).split("-"))+r'.log'
sys.stdout = open(stdout_file, 'w')

This will create a log file and stream the print statements to the file.


Note: Watch out for escape characters in your filepath while concatenating with script_dir in the second line from the last in the code. You might want something similar to raw string. You can check this thread for this.

Answered by: Julian573 | Posted: 01-01-2022



Similar questions

How to redirect stderr in Python? Via Python C API?

This is a combination of my two recent questions: [1] Python instance method in C [2] How to redirect stderr in Python? I would like to log the output of both stdout and stderr from a python script. The thing I want to ask is...


redirect API Stderr python

I am using a python package https://pypi.python.org/pypi/mygene to do some gene id mapping. I have code below: #! /usr/bin/env python # ID mapping using mygene # https://pypi.python.org/pypi/mygene # http://nbviewer.ipython.org/gist/newgene/6771106 # http://mygene-py.readthedocs.org/en/latest/ # 08/30/14 __author__ = 'tommy' import mygene i...


How to redirect stderr to a file in Python?

The following code I got from http://forums.devshed.com/python-programming-11/redirect-stdout-stderr-file-500952.html which tells how to redirect a stderr to a file. I tried it but the error message is not getting saved to the file. While I am not using sys.stderr = file_err


How to redirect stderr in Python? Via Python C API?

This is a combination of my two recent questions: [1] Python instance method in C [2] How to redirect stderr in Python? I would like to log the output of both stdout and stderr from a python script. The thing I want to ask is...


redirect API Stderr python

I am using a python package https://pypi.python.org/pypi/mygene to do some gene id mapping. I have code below: #! /usr/bin/env python # ID mapping using mygene # https://pypi.python.org/pypi/mygene # http://nbviewer.ipython.org/gist/newgene/6771106 # http://mygene-py.readthedocs.org/en/latest/ # 08/30/14 __author__ = 'tommy' import mygene i...


How to redirect stderr to a file in Python?

The following code I got from http://forums.devshed.com/python-programming-11/redirect-stdout-stderr-file-500952.html which tells how to redirect a stderr to a file. I tried it but the error message is not getting saved to the file. While I am not using sys.stderr = file_err


Redirect stderr output of c module to custom python writer

I would like to capture the stderr output of my program and apply some custom pre-processing before displaying it on the terminal. The stderr is written by some C/C++ module that I do not control. with redirect_stderr(): # foo() execute `std::cerr &lt;&lt; "some log" &lt;&lt; std::endl` some_module.foo() # I would like to redirect the `std::cerr` string to MyCustomLogger.write


bash - Redirect command to input of another in Python

I would like to replicate this in python: gvimdiff &lt;(hg cat file.txt) file.txt (hg cat file.txt outputs the most recently committed version of file.txt) I know how to pipe the file to gvimdiff, but it won't accept another file: $ hg cat file.txt | gvimdiff file.txt - Too many edit arguments: "-" Getting to the python part... # hg...


Python + Django page redirect

How do I accomplish a simple redirect (e.g. cflocation in ColdFusion, or header(location:http://) for PHP) in Django?


python - How to pass information using an HTTP redirect (in Django)

I have a view that accepts a form submission and updates a model. After updating the model, I want to redirect to another page, and I want a message such as "Field X successfully updated" to appear on this page. How can I "pass" this message to the other page? HttpResponseRedirect only accepts a URL. I've seen this done bef...


python - Django: Redirect to previous page after login

I'm trying to build a simple website with login functionality very similar to the one here on SO. The user should be able to browse the site as an anonymous user and there will be a login link on every page. When clicking on the login link the user will be taken to the login form. After a successful login the user should be taken back to the page from where he clicked the login link in the first place. I'm guessing that I ...


How to redirect the output of .exe to a file in python?

In a script , I want to run a .exe with some command line parameters as "-a",and then redirect the standard output of the program to a file? How can I implement that?


python - How to Redirect To Same Page on Failed Login

The Django framework easily handles redirecting when a user fails to log in properly. However, this redirection goes to a separate login page. I can set the template to be the same as the page I logged in on, but none of my other objects exist in the new page. For example, I have a front page that shows a bunch of news articles. On the sidebar is a login form. When the user logs in, but fails to authenticate, I wou...


cgi - How to redirect to another page in Python in CGIAr

If I want to redirect the user from a cgi script to a HTML page using core libraries from the following: import cgi Please could someone point me in the right direction. Nothing more. Simply redirect from cgi script to html page. How ever you would do this Python. If you have to physically write out the HTTP Response Headers to achieve this, then I would appreciate any i...


How to redirect stderr in Python? Via Python C API?

This is a combination of my two recent questions: [1] Python instance method in C [2] How to redirect stderr in Python? I would like to log the output of both stdout and stderr from a python script. The thing I want to ask is...


linux - Redirect stderr to stdout on exec-ed process from python?

In a bash script, I can write: exec 2&gt;&amp;1 exec someprog And the stderr output of someprog would be redirected to stdout. Is there any way to do a similar thing using python's os.exec* functions? This doesn't have to be portable, just work on Linux.


python - How to set cookies with redirect in Pylons

In light of the cookie-handling bugs affecting Safari and Chrome (see this thread), and Pylons implementation of redirect_to as an exception, is it possible to reliably set a tracking cookie and redirect at the same time? Is the META refresh method looked down upon?






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top