A database for python 3?

I'm coding a small piece of server software for the personal use of several users. Not hundreds, not thousands, but perhaps 3-10 at a time.

Since it's a threaded server, SQLite doesn't work. It complains about threads like this:

ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140735085562848 and this is thread id 4301299712

Besides, they say SQLite isn't great for concurrency anyhow.

Now since I started working with Python 3 (and would rather continue using it) I can't seem to get the MySQL module to work properly and others seem equally frustrated.

In that case, is there any other DB option for Python 3 that I could consider?


Asked by: Arthur306 | Posted: 30-11-2021






Answer 1

First note that sqlite is thread safe

$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite
>>> sqlite.threadsafety
1

Then just make sure you open a new handle to the database in each thread.

I do this using thread local storage to cache the database handle so there is only one per thread. Something like this... (from a py2.5 prog - hopefully it will work with 3.0!)

import threading

class YourClass:
    def __init__(self):
        #...
        self.local = threading.local()  # Thread local storage for db handles
        self.db_file = "/path/to/db"
        #...
    def db_open(self):
        if not getattr(self.local, "db", None):
            self.local.db = sqlite3.connect(self.db_file)
        return self.local.db

Answered by: David262 | Posted: 01-01-2022



Answer 2

I did a port of psycopg2 to Python 3.

Answered by: Aida665 | Posted: 01-01-2022



Answer 3

pymongo works with Python 3 now.

Answered by: Hailey238 | Posted: 01-01-2022



Answer 4

I know there is some Python driver for Firebird but I don't know if some exist for Python 3. May be you can ask in the Firebird-Python support list

Answered by: Rebecca998 | Posted: 01-01-2022



Answer 5

You could create a new sqlite object in each thread, each using the same database file. For such a small number of users you might not come across the problems with concurrency, unless they are all writing to it very heavily.

Answered by: David760 | Posted: 01-01-2022



Answer 6

Surely a pragmatic option is to just use one SQLite connection per thread.

Answered by: Robert937 | Posted: 01-01-2022



Similar questions

python - Sometimes can't delete an Oracle database row using Django

I have a unit test which contains the following line of code Site.objects.get(name="UnitTest").delete() and this has worked just fine until now. However, that statement is currently hanging. It'll sit there forever trying to execute the delete. If I just say print Site.objects.get(name="UnitTest") then it works, so I know that it can retrieve the site. ...


python - How do I test a django database schema?

I want to write tests that can show whether or not the database is in sync with my models.py file. Actually I have already written them, only to find out that django creates a new database each time the tests are run based on the models.py file. Is there any way I can make the models.py test use the existing database schema? The one that's in mysql/postgresql, and not the one that's in /myapp/models.py ?


python - Using user input to find information in a Mysql database

I need to design a program using python that will ask the user for a barcode. Then, using this barcode, it will search a mysql to find its corresponding product. I am a bit stuck on how to get started. Does anyone have any tips for me?


python - Import XML into SQL database

I'm working with a 20 gig XML file that I would like to import into a SQL database (preferably MySQL, since that is what I am familiar with). This seems like it would be a common task, but after Googling around a bit I haven't been able to figure out how to do it. What is the best way to do this? I know this ability is built into MySQL 6.0, but that is not an option right now because it is an alpha development rel...


python - Using 'old' database with django

I'm using a hand built (Postgres) database with Django. With "inspectdb" I was able to automatically create a model for it. The problem is that some tables have multiple primary keys (for many-to-many relations) and they are not accessible via Django. What's the best way to access these tables?


database - python orm

This is a newbie theory question - I'm just starting to use Python and looking into Django and orm. Question: If I develop my objects and through additional development modify the base object structures, inheritance, etc. - would Django's ORM solution modify the database automatically OR do I need to perform a conversion (if the app is live)? So, I start with a basic Phone app Object person: name, address, city, s...


python - Large Sqlite database search

How is it possible to implement an efficient large Sqlite db search (more than 90000 entries)? I'm using Python and SQLObject ORM: import re ... def search1(): cr = re.compile(ur'foo') for item in Item.select(): if cr.search(item.name) or cr.search(item.skim): print item.name This function runs in more than 30 seconds. How sh...


abap - Query SAP database from Python?

Closed. This question needs details or clarity. It ...


python - URLs stored in database for Django site

I've produced a few Django sites but up until now I have been mapping individual views and URLs in urls.py. Now I've tried to create a small custom CMS but I'm having trouble with the URLs. I have a database table (SQLite3) which contains code for the pages like a column for header, one for right menu, one for content.... so on, so on. I also have a column for the URL. How do I get Django to call the information in...


python - Using Django database layer outside of Django?

I've got a nice database I've created in Django, and I'd like to interface with through some python scripts outside of my website stuff, so I'm curious if it's possible to use the Django database API outside of a Django site, and if so does anyone have any info on how it can be done? Google hasn't yielded many hits for this.


database - Problem opening berkeley db in python

I have problems opening a berkeley db in python using bdtables. As bdtables is used by the library I am using to access the database, I need it to work. The problem seems to be that the db environment I am trying to open (I got a copy of the database to open), is version 4.4 while libdb is version 4.6. I get the following error using bsddb.dbtables.bsdTableDB([dbname],[folder]): (-30972, "DB_VERSION...


database - Store simple user settings in Python

I am programming a website in which users will have a number of settings, such as their choice of colour scheme, etc. I'm happy to store these as plain text files, and security is not an issue. The way I currently see it is: there is a dictionary, where all the keys are users and the values are dictionaries with the users' settings in them. For example, userdb["bob"]["colour_scheme"] would have the value "b...


python - Sometimes can't delete an Oracle database row using Django

I have a unit test which contains the following line of code Site.objects.get(name="UnitTest").delete() and this has worked just fine until now. However, that statement is currently hanging. It'll sit there forever trying to execute the delete. If I just say print Site.objects.get(name="UnitTest") then it works, so I know that it can retrieve the site. ...


python - How do I test a django database schema?

I want to write tests that can show whether or not the database is in sync with my models.py file. Actually I have already written them, only to find out that django creates a new database each time the tests are run based on the models.py file. Is there any way I can make the models.py test use the existing database schema? The one that's in mysql/postgresql, and not the one that's in /myapp/models.py ?


python - Using user input to find information in a Mysql database

I need to design a program using python that will ask the user for a barcode. Then, using this barcode, it will search a mysql to find its corresponding product. I am a bit stuck on how to get started. Does anyone have any tips for me?


python - Given an rpm package name, query the yum database for updates

I was imagining a 3-line Python script to do this but the yum Python API is impenetrable. Is this even possible? Is writing a wrapper for 'yum list package-name' the only way to do this?


python - Import XML into SQL database

I'm working with a 20 gig XML file that I would like to import into a SQL database (preferably MySQL, since that is what I am familiar with). This seems like it would be a common task, but after Googling around a bit I haven't been able to figure out how to do it. What is the best way to do this? I know this ability is built into MySQL 6.0, but that is not an option right now because it is an alpha development rel...


python - Using 'old' database with django

I'm using a hand built (Postgres) database with Django. With "inspectdb" I was able to automatically create a model for it. The problem is that some tables have multiple primary keys (for many-to-many relations) and they are not accessible via Django. What's the best way to access these tables?


python - How do I notify a process of an SQLite database change done in a different process?

Let's say I have two or more processes dealing with an SQLite database - a "player" process and many "editor" processes. The "player" process reads the database and updates a view - in my case it would be a waveform being mixed to the soundcard depending on events stored in the database. An "editor" process is any editor for that database: it changes the database constantly. Now I want the player t...


database - python orm

This is a newbie theory question - I'm just starting to use Python and looking into Django and orm. Question: If I develop my objects and through additional development modify the base object structures, inheritance, etc. - would Django's ORM solution modify the database automatically OR do I need to perform a conversion (if the app is live)? So, I start with a basic Phone app Object person: name, address, city, s...






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top