How to redirect stderr in Python? Via Python C API?

This is a combination of my two recent questions:
[1] Python instance method in C
[2] How to redirect stderr in Python?

I would like to log the output of both stdout and stderr from a python script.

The thing I want to ask is, to create a new type according to [1] seems fairly complicated. Does it simplifies the things if there was no need to expose the new type to Python, i.e. it would only exist in C?

I mean, when Python prints something it goes to "Objects/fileobject.c" and there in "PyFile_WriteObject" it check whether it is possible to write to its argument:

writer = PyObject_GetAttrString(f, "write");
if (writer == NULL)
...

Also, it is possible to get stdout and stderr like this:

PyObject* out = PySys_GetObject("stdout");
PyObject* err = PySys_GetObject("stderr");

My question is then, is it somehow possible to construct necessary PyObject which satisfies the above 'PyObject_GetAttrString(f, "write")' and is callable so I can write:

PySys_SetObject("stdout", <my writer object / class / type / ?>);

http://docs.python.org/c-api/sys.html?highlight=pysys_setobject#PySys_SetObject

This way, there would be no need to expose the new "writer type" to the rest of Python script so I thought it might be a bit simpler to write the code...?


Asked by: Michelle574 | Posted: 30-11-2021






Answer 1

Just make a module object (you're doing that anyway, if you're using the C API!-) and make it have a suitable write function -- that module object will be suitable as the second argument to PySys_SetObject.

In my answer to your other question I pointed you to xxmodule.c, an example file in Python's C sources, which is a module with a lot of examples including types and functions of various kinds -- you can work from there even if (mysteriously to me) you consider the "make a new type" part too difficult;-).

Edit: here's a trivial working example (aview.py):

#include "Python.h"
#include <stdio.h>

static PyObject *
aview_write(PyObject *self, PyObject *args)
{
    const char *what;
    if (!PyArg_ParseTuple(args, "s", &what))
        return NULL;
    printf("==%s==", what);
    return Py_BuildValue("");
}

static PyMethodDef a_methods[] = {
    {"write", aview_write, METH_VARARGS, "Write something."},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initaview(void)
{
    PyObject *m = Py_InitModule("aview", a_methods);
    if (m == NULL) return;
    PySys_SetObject("stdout", m);
}

Once this aview module is properly installed:

$ python
Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import aview
>>> print 'ciao'
==ciao====
==>>> 

...any string emitted to standard output is written with == signs around it (and this print calls .write twice: with 'ciao', and then again with a newline).

Answered by: Melissa782 | Posted: 01-01-2022



Answer 2

Based on Alex's answer, here is fully working C code, without the Python "import aview", in Python 3 (so no Py_InitModule), and with stderr redirection :

#include <functional>
#include <iostream>
#include <string>
#include <Python.h>


PyObject* aview_write(PyObject* self, PyObject* args)
{
    const char *what;
    if (!PyArg_ParseTuple(args, "s", &what))
        return NULL;
    printf("==%s==", what);
    return Py_BuildValue("");
}


PyObject* aview_flush(PyObject* self, PyObject* args)
{
    return Py_BuildValue("");
}


PyMethodDef aview_methods[] =
{
    {"write", aview_write, METH_VARARGS, "doc for write"},
    {"flush", aview_flush, METH_VARARGS, "doc for flush"},
    {0, 0, 0, 0} // sentinel
};


PyModuleDef aview_module =
{
    PyModuleDef_HEAD_INIT, // PyModuleDef_Base m_base;
    "aview",               // const char* m_name;
    "doc for aview",       // const char* m_doc;
    -1,                    // Py_ssize_t m_size;
    aview_methods,        // PyMethodDef *m_methods
    //  inquiry m_reload;  traverseproc m_traverse;  inquiry m_clear;  freefunc m_free;
};

PyMODINIT_FUNC PyInit_aview(void) 
{
    PyObject* m = PyModule_Create(&aview_module);
    PySys_SetObject("stdout", m);
    PySys_SetObject("stderr", m);
    return m;
}


int main()
{
    PyImport_AppendInittab("aview", PyInit_aview);
    Py_Initialize();
    PyImport_ImportModule("aview");

    PyRun_SimpleString("print(\'hello to buffer\')");
    PyRun_SimpleString("make a SyntaxException in stderr");

    Py_Finalize();

    return 0;

}

Note, though, that if you plan to have several distinct interpreters, this won't be enough, because aview_write won't be able to know which buffer to append into. You'll need something like that.

Here is an awesome reference on how to add new modules and types, btw.

Answered by: Kimberly851 | Posted: 01-01-2022



Similar questions

How to redirect stderr in Python?

I would like to log all the output of a Python script. I tried: import sys log = [] class writer(object): def write(self, data): log.append(data) sys.stdout = writer() sys.stderr = writer() Now, if I "print 'something' " it gets logged. But if I make for instance some syntax error, say "print 'something# ", it wont get logged - it will go into the console instead. Ho...


redirect API Stderr python

I am using a python package https://pypi.python.org/pypi/mygene to do some gene id mapping. I have code below: #! /usr/bin/env python # ID mapping using mygene # https://pypi.python.org/pypi/mygene # http://nbviewer.ipython.org/gist/newgene/6771106 # http://mygene-py.readthedocs.org/en/latest/ # 08/30/14 __author__ = 'tommy' import mygene i...


How to redirect stderr to a file in Python?

The following code I got from http://forums.devshed.com/python-programming-11/redirect-stdout-stderr-file-500952.html which tells how to redirect a stderr to a file. I tried it but the error message is not getting saved to the file. While I am not using sys.stderr = file_err


How to redirect stderr in Python?

I would like to log all the output of a Python script. I tried: import sys log = [] class writer(object): def write(self, data): log.append(data) sys.stdout = writer() sys.stderr = writer() Now, if I "print 'something' " it gets logged. But if I make for instance some syntax error, say "print 'something# ", it wont get logged - it will go into the console instead. Ho...


redirect API Stderr python

I am using a python package https://pypi.python.org/pypi/mygene to do some gene id mapping. I have code below: #! /usr/bin/env python # ID mapping using mygene # https://pypi.python.org/pypi/mygene # http://nbviewer.ipython.org/gist/newgene/6771106 # http://mygene-py.readthedocs.org/en/latest/ # 08/30/14 __author__ = 'tommy' import mygene i...


How to redirect stderr to a file in Python?

The following code I got from http://forums.devshed.com/python-programming-11/redirect-stdout-stderr-file-500952.html which tells how to redirect a stderr to a file. I tried it but the error message is not getting saved to the file. While I am not using sys.stderr = file_err


Redirect stderr output of c module to custom python writer

I would like to capture the stderr output of my program and apply some custom pre-processing before displaying it on the terminal. The stderr is written by some C/C++ module that I do not control. with redirect_stderr(): # foo() execute `std::cerr &lt;&lt; "some log" &lt;&lt; std::endl` some_module.foo() # I would like to redirect the `std::cerr` string to MyCustomLogger.write


bash - Redirect command to input of another in Python

I would like to replicate this in python: gvimdiff &lt;(hg cat file.txt) file.txt (hg cat file.txt outputs the most recently committed version of file.txt) I know how to pipe the file to gvimdiff, but it won't accept another file: $ hg cat file.txt | gvimdiff file.txt - Too many edit arguments: "-" Getting to the python part... # hg...


Python + Django page redirect

How do I accomplish a simple redirect (e.g. cflocation in ColdFusion, or header(location:http://) for PHP) in Django?


python - How to pass information using an HTTP redirect (in Django)

I have a view that accepts a form submission and updates a model. After updating the model, I want to redirect to another page, and I want a message such as "Field X successfully updated" to appear on this page. How can I "pass" this message to the other page? HttpResponseRedirect only accepts a URL. I've seen this done bef...


python - Django: Redirect to previous page after login

I'm trying to build a simple website with login functionality very similar to the one here on SO. The user should be able to browse the site as an anonymous user and there will be a login link on every page. When clicking on the login link the user will be taken to the login form. After a successful login the user should be taken back to the page from where he clicked the login link in the first place. I'm guessing that I ...


How to redirect the output of .exe to a file in python?

In a script , I want to run a .exe with some command line parameters as "-a",and then redirect the standard output of the program to a file? How can I implement that?


python - How to Redirect To Same Page on Failed Login

The Django framework easily handles redirecting when a user fails to log in properly. However, this redirection goes to a separate login page. I can set the template to be the same as the page I logged in on, but none of my other objects exist in the new page. For example, I have a front page that shows a bunch of news articles. On the sidebar is a login form. When the user logs in, but fails to authenticate, I wou...


cgi - How to redirect to another page in Python in CGIAr

If I want to redirect the user from a cgi script to a HTML page using core libraries from the following: import cgi Please could someone point me in the right direction. Nothing more. Simply redirect from cgi script to html page. How ever you would do this Python. If you have to physically write out the HTTP Response Headers to achieve this, then I would appreciate any i...


How to redirect stderr in Python?

I would like to log all the output of a Python script. I tried: import sys log = [] class writer(object): def write(self, data): log.append(data) sys.stdout = writer() sys.stderr = writer() Now, if I "print 'something' " it gets logged. But if I make for instance some syntax error, say "print 'something# ", it wont get logged - it will go into the console instead. Ho...


linux - Redirect stderr to stdout on exec-ed process from python?

In a bash script, I can write: exec 2&gt;&amp;1 exec someprog And the stderr output of someprog would be redirected to stdout. Is there any way to do a similar thing using python's os.exec* functions? This doesn't have to be portable, just work on Linux.


python - How to set cookies with redirect in Pylons

In light of the cookie-handling bugs affecting Safari and Chrome (see this thread), and Pylons implementation of redirect_to as an exception, is it possible to reliably set a tracking cookie and redirect at the same time? Is the META refresh method looked down upon?






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top