Using locale.setlocale in embedded Python without breaking file parsing in C thread

We're using a third-party middleware product that allows us to write code in an embedded Python interpreter, and which exposes an API that we can call into. Some of these API calls allow us to load various kinds of file, and the loading code is implemented in C. File loading happens in a separate thread, and calls back into Python when the data is available. So far, all well and dandy.

We've been i14ing (heh) our product, and one thing we'd like to do is format user-facing numerical output according to the user's locale settings. So, from Python, we do this:

import locale
locale.setLocale( locale.LC_ALL, '' )

Now, this works (in that the user-facing numbers are formatted correctly for their locale). However, if the user's locale differs from the default C locale, any files that are subsequently loaded will return incorrect data, presumably because all string-to-float conversion has been affected, right down to the metal.

We can't work around this by implementing locale aware file loading, so our current workaround is to only set the locale when formatting output for the user, and then set it back again afterwards. That is, something like:

import locale
currentLocale = locale.getLocale( locale.LC_ALL )
locale.setLocale( locale.LC_ALL, '' )
locale.setlocale( locale.LC_ALL, currentLocale )

This seems a bit clunky, and I was wondering whether this is a common approach to formatting locale-aware output for the user? My other concern is that this is obviously not thread safe, so we'll presumably still get problems if any file parsing occurs in a separate thread when the locale is changed.

Any information on best practice is appreciated - I don't have much experience with this sort of thing.

Asked by: Daniel400 | Posted: 05-10-2021

Answer 1

Setting the locale after multiple threads have started operating may have unexpected results. Unless I could figure out a more subtle approach, I'd probably just split file loading and the user interface into separate processes, communicating through a pipe or a file socket.

Answered by: Nicole531 | Posted: 06-11-2021

Similar questions

internationalization - Python's os.path choking on Hebrew filenames

I'm writing a script that has to move some file around, but unfortunately it doesn't seem os.path plays with internationalization very well. When I have files named in Hebrew, there are problems. Here's a screenshot of the contents of a directory: (source: ...

internationalization - Can I call and set the Python gettext module in a library and a module using it at the same time?

Im a coding a library including textual feedback that I need to translate. I put the following lines in a module that I import everywhere in my app : import gettext, os, sys pathname = os.path.dirname(sys.argv[0]) localdir = os.path.abspath(pathname) + "/locale" gettext.install("messages", localdir) I have the *.mo files in ./locale/lang_L...

internationalization - Python urllib, minidom and parsing international characters

When I try to retrieve information from Google weather API with the following URL,,Germany&hl=de and then try to parse it with minidom, I get error that the document is not well formed. I use following code sock = urllib.urlopen(url) #...

Python internationalization, local setting independent

I need the return of a strftime() call being in a language different at the one set on my local machine/OS. Is that possible to choose the language of the return?

internationalization - Python: Get system calendar format

Is it possible to return the current system calendar format using Python? For example non-Gregorian calendar formats such as the Thai Buddhist calendar.

internationalization - Japanese in python function

I wrote a function in Python which is used to tell me whether the two words are similar or not. Now I want to pass Japanese text in my same function. It is giving error "not a ascii character." I tried using utf-8 encoding, but then it giving the same error Non-ASCII character '\xe3' in file Is there any way to do that? I cant generate the msg file for that since the 2 keyword will...

internationalization - Python encoding ISO to UTF8

I am trying to read my emails using a Python script (Python 2.5 and PyPy) Some of my results are not in ASCII and i get strings like this: =?ISO-8859-7?B?0OXm7/Dv8d/hIPP07+0gyuno4enx/u3h?=' Is there any way to decode it and convert to utf-8 so that i can process it? I tried .decode('ISO-8859-7') but i got the same string

internationalization - Internationalizing a Python 2.6 application via Babel

We're evaluating Babel 0.9.5 [1] under Windows for use with Python 2.6 and have the following questions that we we've been unable to answer through reading the documentation or googling. 1) I would like to use an _ like abbreviation for ungettext. Is there a concencus on whether one should use n_ or N_ for this? n_ does not appear to work. Babel does not extract text. N_ appears to partially work. B...

python - Django internationalization language codes

Closed. This question does not meet Stack Overflow guid...

internationalization - How can I programmatically find the list of codecs known to Python?

This question already has answers here:

Still can't find your answer? Check out these communities...

PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python