Migrating from CPython to Jython

I'm considering moving my code (around 30K LOC) from CPython to Jython, so that I could have better integration with my java code.

Is there a checklist or a guide I should look at, to help my with the migration? Does anyone have experience with doing something similar?

From reading the Jython site, most of the problems seem too obscure to bother me.

I did notice that:

  • thread safety is an issue
  • Unicode support seems to be quite different, which may be a problem for me
  • mysqldb doesn't work and needs to be replaced with zxJDBC

Anything else?

Related question: What are some strategies to write python code that works in CPython, Jython and IronPython


Asked by: Marcus158 | Posted: 24-09-2021






Answer 1

First off, I have to say the Jython implementation is very good. Most things "just work".

Here are a few things that I have encountered:

  • C modules are not available, of course.

  • open('file').read() doesn't automatically close the file. This has to do with the difference in the garbage collector. This can cause issues with too many open files. It's better to use the "with open('file') as fp" idiom.

  • Setting the current working directory (using os.setcwd()) works for Python code, but not for Java code. It emulates the current working directory for everything file-related but can only do so for Jython.

  • XML parsing will try to validate an external DTD if it's available. This can cause massive slowdowns in XML handling code because the parser will download the DTD over the network. I reported this issue, but so far it remains unfixed.

  • The __ del __ method is invoked very late in Jython code, not immediately after the last reference to the object is deleted.

There is an old list of differences, but a recent list is not available.

Answered by: Marcus819 | Posted: 25-10-2021



Answer 2

So far, I have noticed two further issues:

  • String interning 'a' is 'a' is not guaranteed (and it is just an implementation fluke on CPython). This could be a serious problem, and really was in one of the libraries I was porting (Jinja2). Unit tests are (as always) your best friends!
Jython 2.5b0 (trunk:5540, Oct 31 2008, 13:55:41)
>>> 'a' is 'a'
True
>>> s = 'a'
>>> 'a' is s
False
>>> 'a' == s   
True
>>> intern('a') is intern(s)
True

Here is the same session on CPython:

Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49)
>>> 'a' is 'a'
True
>>> s = 'a'
>>> 'a' is s
True
>>> 'a' == s
True
>>> intern('a') is intern(s)
True

  • os.spawn* functions are not implemented. Instead use subprocess.call. I was surprised really, as the implementation using subprocess.call would be easy, and I am sure they will accept patches.

(I have been doing a similar thing as you, porting an app recently)

Answered by: Madaline518 | Posted: 25-10-2021



Answer 3

I'm starting this as a wiki collected from the other answers and my experience. Feel free to edit and add stuff, but please try to stick to practical advice rather than a list of broken things. Here's an old list of differences from the Jython site.

Resource management

Jython does not use reference counting, and so resources are released as they are garbage collected, which is much later then you'd see in the equivalent CPython program

  • open('file').read() doesn't automatically close the file. Better use the with open('file') as fp idiom.
  • The __ del __ method is invoked very late in Jython code, not immediately after the last reference to the object is deleted.

MySQL Integration

mysqldb is a c module, and therefore will not work in jython. Instead, you should use com.ziclix.python.sql.zxJDBC, which comes bundled with Jython.

Replace the following MySQLdb code:

connection = MySQLdb.connect(host, user, passwd, db, use_unicode=True, chatset='utf8')

With:

url = "jdbc:mysql://%s/%s?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull" % (host, db)
connections = zxJDBC.connect(url, user, passwd, "com.mysql.jdbc.Driver")

You'll also need to replace all _mysql_exception with zxJDBC.

Finally, you'll need to replace the query placeholders from %s to ?.

Unicode

  • You can't express illegal unicode characters in Jython. Trying something like unichr(0xd800) would cause an exception, and having a literal u'\ud800' in your code will just wreak havoc.

Missing things

  • C modules are not available, of course.
  • os.spawn* functions are not implemented. Instead use subprocess.call.

Performance

  • For most workloads, Jython will be much slower than CPython. Reports are anything between 3 to 50 times slower.

Community

The Jython project is still alive, but is not fast-moving. The dev mailing list has about 20 messages a month, and there seem to be only about 2 developers commiting code lately.

Answered by: Max221 | Posted: 25-10-2021



Answer 4

When I switched a project from CPython to Jython some time ago I realized a speed-down of up to 50x for time-critical sections. Because of that I stayed with CPython.

However, that might have changed now with the current versions.

Answered by: Roland410 | Posted: 25-10-2021



Answer 5

You might also want to research JPype. I'm not sure how mature it is compared to Jython, but it should allow CPython to access Java code.

Answered by: Alfred491 | Posted: 25-10-2021



Answer 6

Recently, I worked on a project for a professor at my school with a group. Early on, it was decided that we would write the project in Python. We definitely should have used CPython. We wrote the program in Python and all of our unit tests eventually worked. Because most people already have Java installed on their computers, and not Python, we decided to just deploy it as a Jython jar. We therefore wrote the GUI with Swing, because that's included in Java's standard library.

The first time I ran the program with Jython, it crashed immediately. For one, csv.reader's ".fieldnames" always seemed to be None. Therefore I had to change several parts of our code to work around this.

A different section of my code crashed as well, which worked fine with CPython. Jython accused me of referencing a variable before it was assigned anything (which drove me nuts and really wasn't the case). This is one example: ActiveState's Code Recipe's external sort

Worse yet, the performance was awful. Basically this code combined several CSV files, one of which was about 2 GB. In CPython, it ran in 8.5 minutes. In Jython, it ran in 25 minutes.

These problems happened with 2.5.2rc2 (the latest at the time of writing this post).

Answered by: Emma498 | Posted: 25-10-2021



Similar questions

django - migrating from one framework to another in python

I'm having trouble deciding which python framework to use for my website. So I've decided to bite the bullet and use Django. My question is how easy (or difficult) will it be to migrate to a different framework in future if I have issues with Django ?


python - migrating django 1.1.1 -> 1.2.1: {% url %} doesn't work

I am migrating a django project from 1.1.1 to 1.2.1 Now neither the {% url %} tag works nor the @models.permalink-decorated get_absulute_url works i.e. I get TemplateSyntaxError at / Caught TypeError while rendering: __init__() got an unexpected keyword argument 'error_message' for <li><a href="{% url archive_...


python - Migrating to Linux from Windows

I have a python gui that access files on windows as C:\data and C:\MyDIR all outside my doc's. On, a linux sys i created /data and /MyDIR. My gui cant access. I foresee always using C:\data and C:\MyDIR on both systems. How do I fix code or Linux permissions to have access to both dir and sub directories.


python - Django tests failing after migrating to 1.2.5 - primary key issue for child model

I have a model ThreadedComment that inherits from model Object. ThreadedComment does not have a unique primary key of its own, relying on Object's promary key ("ID"). This is how the model is constructed: class Object(models.Model): permalink = models.CharField(max_length=128) status = models.IntegerField() version = models.IntegerField() class ThreadedComment(Object): parent = models.Forei...


python - Django South: Migrating FK from another table

I was doing some system and my table design was like this: Table A: - ID - ForeignKey(C) Table B: - ID - ForeignKey(A) Table C: - ID Due to some requirement changes, I need to move ForeignKey(C) from A to B, so it goes like this: Table A: - ID Table B: - ID - ForeignKey(A) - ForeignKey(C) -Table C: - ID The schema mig...


java - Password hashing on 32bit system then migrating to 64bit systems

I came across a potential issue when migrating from a 32bit system to a 64bit system. the system runs two 'one way' encryption algos on the string. However when I tried the same code on a 64 bit system the hashes were different, now thats fine but does anyone have any method on how to ensure the the hash can still be retreived and matched in the future when upgrading to 64bit and beyond? Without ri...


python - Migrating django auto model loading to iPython .11+ - how to execute code after script loads?

I'm having trouble migrating my old iPython ipy_user_conf.py script to the new .11+ iPython. Before, loading models was as easy as this snippet in ipy_user_conf.py My manage.py sets the django environment so iPython could import django successfully regardless of project / not having djang...


python - Migrating data from one sqlite database to multiple SQLite databases using SQLObject

Till now our application has been using one SQLite database with SQLObject as the ORM. Obviously at some point we knew we had to face the SQLite concurrency problem and so we did. We ended up splitting the current database into multiple databases. Meaning each table schema remained the same but we distributed different tables into multiple databases keeping tightly coupled tables together. Now this works v...


Python - Migrating program from Linux to Windows, in Windows it doesn't work

This has been bugging me for a few days now. I'm trying to migrate my utility, LScreamer, to Windows 7, yet it doesn't seem to be cooperating. This utility transmits a Atmega firmware file to the bootloader running in the atmega. In Linux when I run this utility, it is able to send the entire file with only a handful of retransmits necessary. In Windows, almost every other transmit is a retransmit, and most of t...


Migrating C# to Python - Random class

I need to migrate some C# code to Python. The original code makes use of the Random class. The migrated code must be cycle-accurate (namely consecutive calls to Next() must produce the same results in both codes). Some questions: Is there a call-equivalent to C#'s Random in Python? Alliteratively, assuming I can modify both sources, is there a pseudo-random libr...






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top