Different behavior of python logging module when using mod_python

We have a nasty problem where we see that the python logging module is behaving differently when running with mod_python on our servers. When executing the same code in the shell, or in django with the runserver command or with mod_wsgi, the behavior is correct:

import logging
logger = logging.getLogger('site-errors')
logging.debug('logger=%s' % (logger.__dict__))
logging.debug('logger.parent=%s' % (logger.parent.__dict__))
logger.error('some message that is not logged.')

We then the following logging:

2009-05-28 10:36:43,740,DEBUG,error_middleware.py:31,[logger={'name': 'site-errors', 'parent': <logging.RootLogger instance at 0x85f8aac>, 'handlers': [], 'level': 0, 'disabled': 0, 'manager': <logging.Manager instance at 0x85f8aec>, 'propagate': 1, 'filters': []}]

2009-05-28 10:36:43,740,DEBUG,error_middleware.py:32,[logger.parent={'name': 'root', 'parent': None, 'handlers': [<logging.StreamHandler instance at 0x8ec612c>, <logging.handlers.RotatingFileHandler instance at 0x8ec616c>], 'level': 10, 'disabled': 0, 'propagate': 1, 'filters': []}]

As one can see, no handlers or level is set for the child logger 'site-errors'.

The logging configuration is done in the settings.py:

MONITOR_LOGGING_CONFIG = ROOT + 'error_monitor_logging.conf'

import logging
import logging.config


   DB_LOGLEVEL = logging.INFO

The second problem is that we also add a custom handler in the __init__.py that resides that in the folder as error_middleware.py:

import logging
from django.conf import settings
from db_log_handler import DBLogHandler

handler = DBLogHandler()

The custom handler cannot be seen in the logging!

If someone has idea where the problem lies, please let us know! Don't hesistate to ask for additonal information. That will certainly help to solve the problem.

Asked by: Eric925 | Posted: 06-12-2021

Answer 1

It may be better if you do not configure logging in settings.py.

We configure your logging in our root urls.py. This seems to work out better. I haven't read enough Django source to know why, precisely, it's better, but it's working out well for us. I would add custom handlers here, also.

Also, look closely at mod_wsgi. It seems to behave much better than mod_python.

Answered by: Ted614 | Posted: 07-01-2022

Answer 2

The problem is not solved by using mod_wsgi.

I could solve the problem by placing the complete configuration into one file. Mixing file and code configuration seems to create problems with apache (whether using mod_wsgi or mod_python).

To use a custom logging handler with file configuration, I had to do the following:

import logging
import logging.config
logging.custhandlers = sitemonitoring.db_log_handler

From the settings.py I cannot import the sitemonitoring.db_log_handler, so I have to place this code in the root urls.py.

In the config file, I refer to the DBLogHandler with the following statement


PS: Note that the custhandler 'attribute' is created dynamically and can have another name. This is an advantage of using a dynamic language.

Answered by: Anna682 | Posted: 07-01-2022

Answer 3

You don't appear to have posted all the relevant information - for example, where is your logging configuration file?

You say that:

When executing the same code in the shell, or in django with the runserver command or with mod_wsgi, the behavior is correct

You don't make clear whether the logging output you showed is from one of these environments, or whether it's from a mod_python run. It doesn't look wrong - in your code you added handlers to the root, not to logger 'site-errors'. You also set a level on the handler, not the logger - so you wouldn't expect to see a level set for the 'site-errors' logger in the logging output, neh? Levels can be set on both loggers and handlers and they are not the same, though they filter out events in the same way.

The issue about custom handlers is easily explained if you look at the logging documentation for configuration, see

http://docs.python.org/library/logging.html (search for "the class entry indicates")

This explains that any handler class described in the configuration file is eval()'d in the logging packages namespace. So, by binding logging.custhandlers to your custom handlers module and then stating "custhandlers.MyCustomClass" in the config file, the eval() produces the expected result. You could just as well have done

logging.sitemonitoring = sitemonitoring

and specified the handler class as


which would work just as well (as long as the db_log_handler subpackage has been imported already).

BTW the reason why people sometimes have problems configuring logging in settings.py is due to Django's import magic causing circular import problems. I generally configure logging in settings.py and it works fine unless you want to import certain bits of Django (e.g. in django.db - because the app import logic is in django.db, you can run into circular import issues if you try to import django.db.x in settings.py).

Answered by: Sawyer827 | Posted: 07-01-2022

Similar questions

python - Django, mod_python, apache and wacky sessions

I am running a Django through mod_python on Apache on a linux box. I have a custom authentication backend, and middleware that requires authentication for all pages, except static content. My problem is that after I log in, I will still randomly get the log in screen now and again. It seems to me that each apache process has it's own python process, which in turn has it's own internals. So as long as I get served b...

python - Failed to get separate instances of a class under mod_python

I'm trying to run some python code under Apache 2.2 / mod_python 3.2.8. Eventually the code does os.fork() and spawns 2 separate long-run processes. Each of those processes has to create a separate instance of a class in order to avoid any possible collision in the parallel flow. class Foo(object): pass kidprocs = [] for kid in ('kid1', 'kid2'): pid = os.fork() if pid: # parent kidprocs.a...

python - Running Django with FastCGI or with mod_python

which would you recommend? which is faster, reliable? apache mod_python or nginx/lighttpd FastCGI?

python - How to access to the root path in a mod_python directory?

In my Apache webserver I put this: &lt;Directory /var/www/MYDOMAIN.com/htdocs&gt; SetHandler mod_python PythonHandler mod_python.publisher PythonDebug On &lt;/Directory&gt; Then I have a handler.py file with an index function. When I go to MYDOMAIN.com/handler.py, I see a web page produced by the index function (just a plain vanilla HTML page). Every other page is of th...

python - How do I modify sys.path from .htaccess to allow mod_python to see Django?

The host I'm considering for hosting a Django site has mod_python installed, but does not have Django. Django's INSTALL file indicates that I can simply copy the django directory to Python's site-packages directory to install Django, so I suspect that it might be possible to configure Python / mod_python to look for it elsewhere (namely my user space) by modifying sys.path, but I don't know how to change it from .htaccess ...

php - mod_php vs mod_python

Why mod_python is oop but mod_php is not ? Example :We go to www.example.com/dir1/dir2 if you use mod_python apache opens www/dir1.py and calls dir2 method but if you use php module apache opens www/dir1/dir2/index.php

python - How do I upload a file with mod_python?

I want to create a simple file upload form and I must be completely incapable. I've read docs and tutorials,but for some reason, I'm not getting the submitted form data. I wrote the smallest amount of code I could to test and it still isn't working. Any ideas what's wrong? def index(): html = ''' &lt;html&gt; &lt;body&gt; &lt;form id="fileUpload" action="./result" method="post"&gt; ...

python - mod_python publisher and pretty URLs

I am new to Python (I am getting out of PHP because of how increasingly broken it is), and I am racing through porting my old code. One thing: I have a file /foo.py with functions index() and bar(), so, with the publisher I can access http://domain/foo/bar and http://domain/foo as the documentatio...

python - Configure Apache to recover from mod_python errors

This question already has an answer here:

python - mod_python caching of variables

I'm using mod_python to run Trac in Apache. I'm developing a plugin and am not sure how global variables are stored/cached. I am new to python and have googled the subject and found that mod_python caches python modules (I think). However, I would expect that cache to be reset when the web service is restarted, but it doesn't appear to be. I'm saying this becasue I have a global variable that is a ...

Still can't find your answer? Check out these communities...

PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python