DB Permissions with Django unit testing


I'm very new to Django. I must say that so far I really like it. :)

(now for the "but"...)

But, there seems to be something I'm missing related to unit testing. I'm working on a new project with an Oracle backend. When you run the unit tests, it immediately gives a permissions error when trying to create the schema. So, I get what it's trying to do (create a clean sandbox), but what I really want is to test against an existing schema. And I want to run the test with the same username/password that my server is going to use in production. And of course, that user is NOT going to have any kind of DDL type rights.

So, the basic problem/issue that I see boils down to this: my system (and most) want to have their "app_user" account to have ONLY the permissions needed to run. Usually, this is basic "CRUD" permissions. However, Django unit tests seem to need more than this to do a test run.

How do other people handle this? Is there some settings/work around/feature of Django that I'm not aware (please refer to the initial disclaimer).

Thanks in advance for your help.


Asked by: Briony240 | Posted: 06-12-2021

Answer 1

Don't force Django to do something unnatural.

  1. Allow it to create the test schema. It's a good thing.

  2. From your existing schema, do an unload to create .JSON dump files of the data. These files are your "fixtures". These fixtures are used by Django to populate the test database. This is The Greatest Testing Tool Ever. Once you get your fixtures squared away, this really does work well.

  3. Put your fixture files into fixtures directories within each app package.

  4. Update your unit tests to name the various fixtures files that are required for that test case.

This -- in effect -- tests with an existing schema. It rebuilds, reloads and tests in a virgin database so you can be absolutely sure that it works without destroying (or even touching) live data.

Answered by: Anna159 | Posted: 07-01-2022

Answer 2

As you've discovered, Django's default test runner makes quite a few assumptions, including that it'll be able to create a new test database to run the tests against.

If you need to override this or any of these default assumptions, you probably want to write a custom test runner. By doing so you'll have full control over exactly how tests are discovered, bootstrapped, and run.

(If you're running Django's development trunk, or are looking forward to Django 1.2, note that defining custom test runners has recently gotten quite a bit easier.)

If you poke around, you'll find a few examples of custom test runners you could use to get started.

Now, keep in mind that once you've taken control of test running you'll need to ensure that you someone meet the same assumptions about environment that Django's built-in runner does. In particular, you'll need to someone guarantee that whatever test database you'll use is a clean, fresh one for the tests -- you'll be quite unhappy if you try to run tests against a database with unpredictable contents.

Answered by: Brad567 | Posted: 07-01-2022

Answer 3

After I read David's (OP) question, I was curious about this too, but I don't see the answer I was hoping to see. So let me try to rephrase what I think at least part of what David is asking. In a production environment, I'm sure his Django models probably will not have access to create or drop tables. His DBA will probably not allow him to have permission to do this. (Let's assume this is True). He will only be logged into the database with regular user privileges. But in his development environment, the Django unittest framework forces him to have higher level privileges for the unittests instead of a regular user because Django requires it to create/drop tables for the model unittests. Since the unittests are now running at a higher privilege than will occur in production, you could argue that running the unittests in development are not 100% valid and errors could happen in production that might have been caught in development if Django could run the unittests with user privileges.

I'm curious if Django unittests will ever have the ability to create/drop tables with one user's (higher) privileges, and run the unittests with a different user's (lower) privileges. This would help more accurately simulate the production environment in development.

Maybe in practice this is really not an issue. And the risk is so minor compared to the reward that it not worth worrying about it.

Answered by: Arthur303 | Posted: 07-01-2022

Answer 4

Generally speaking, when unit tests depend on test data to be present, they also depend on it to be in a specific format/state. As such, your framework's policy is to not only execute DML (delete/insert test data records), but it also executes DDL (drop/create tables) to ensure that everything is in working order prior to running your tests.

What I would suggest is that you grant the necessary privileges for DDL to your app_user ONLY on your test_ database.

If you don't like that solution, then have a look at this blog entry where a developer also ran into your scenario and solved it with a workaround:


Personally, my choice would be to modify the privileges for the test database. This way, I could rule out all other variables when comparing performance/results between testing/production environments.



Answered by: Eric366 | Posted: 07-01-2022

Answer 5

What you can do, is creating separate test settings. As I've learned at http://mindlesstechnology.wordpress.com/2008/08/16/faster-django-unit-tests/ you can use the sqlite3 backend, which is created in memory by the Django unit test framework.


Create a new test-settings.py file next to your app’s settings.py containing:

from projectname.settings import * DATABASE_ENGINE = 'sqlite3'

Then when you want to run tests real fast, instead of manage.py test, you run

manage.py test --settings=test-settings

This runs my test suite in less than 5 seconds.

Obviously you still want to run tests on your real db backend, but this is awesome for sanity checks, and while you’re doing test development.

To load initial data, provide fixtures in your testcase.

class MyAppTestCase(TestCase):
    fixtures = ['myapp/fixtures/filename']

Answered by: Julian859 | Posted: 07-01-2022

Similar questions

zip - Set permissions on a compressed file in python

I have a file test.txt that is inside a zip archive test.zip. The permissions on test.txt are out of my control when it's compressed, but now I want them to be group-writeable. I am extracting the file with Python, and don't want to escape out to the shell. EDIT: Here's what I've got so far: import zipfile z = zipfile.ZipFile('test.zip', 'w'...

python - Will Django be a good choice for a permissions based web-app?

I've been exploring the details of Django for about a week now and like what I see. However I've come upon some.. negativity in relation to fine-grained control of permissions to the CRUD interface. What I'm writing is an Intranet client management web-app. The organisation is about 6 tiers, and I need to restrict access to client groups based on tiers. Continually expanding. I have a fairly good idea how I'm going...

python - Amazon S3 permissions

Trying to understand S3...How do you limit access to a file you upload to S3? For example, from a web application, each user has files they can upload, but how do you limit access so only that user has access to that file? It seems like the query string authentication requires an expiration date and that won't work for me, is there another way to do this?

chmod - check permissions of directories in python

i want a python program that given a directory, it will return all directories within that directory that have 775 (rwxrwxr-x) permissions thanks!

python - Permissions for a site only

I have a multilingual Django project. Every language is a different subdomain. So we've decided to use the "sites" application and to create one different site for every language. On that project, I also have a "pages" application, which is quite similar to a CMS. The user can create pages with content and they'll be displayed in the appropriate language site. Now I'm looking to be able to manage advanced p...

python - How do I change permissions to a socket?

I am trying to run a simple Python based web server given here. And I get the following error message: Traceback (most recent call last): File "webserver.py", line 63, in <module> main() File "webserver.py", line 55, in main server = HTTPServer(('', 80), MyHandler) File "/usr/lib/python2.5/SocketSe...

linux - Dropping Root Permissions In Python

I'd like to have a Python program start listening on port 80, but after that execute without root permissions. Is there a way to drop root or to get port 80 without it?

setting permissions of python module (python setup install)

I am configuring a distutils-based setup.py for a python module that is to be installed on a heterogeneous set of resources. Due to the heterogeneity, the location where the module is installed is not the same on each host however disutils picks the host-specific location. I find that the module is installed without o+rx permissions using disutils (in spite of setting umask ahead of running setup.py). One solutio...

python - Django 1.2 object level permissions - third party solutions?

python - Django Inlines user permissions + view only - permissions issues

I'm not sure if this is a bug or I'm just missing something (although I have already parsed the documentation about inlines), but: Let's say I have a model A. Model A is an inline of model B. User U has full access to model B, but only change permissions to model A (so, no add, nor delete). However, when editing model B, user U can still see the "Add another A" link at the bottom, although U hasn't add perm...

Still can't find your answer? Check out these communities...

PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python