An evening with threads

Wednesday, October 21, 2009 9:52 - Code

I noted on Monday how I have been working on jack mixer recently; to be honest I didn't do much work up to now, replacing some widgets, adding a preferences dialog, easy stuff.

But as I am starting to use it seriously I realized it didn't scale well, I would add five channels and it would burn CPU and memory in a terrific manner. Without much investigation I decided it was caused by the polling for MIDI events, and set out to fix this.

Well, jack_mixer is written in Python and C, Python for GUI stuff and C for jack and computer intensive stuff, and it turned out it's not possible to call back from a C extension to Python code, when using SWIG, so I got to write a manual binding, easy enough, and I got it feature complete quite fast, gaining good looking code along the way, from here:

mixer = jack_mixer_c.create("test")
print "Channels count: %u" % jack_mixer_c.get_channels_count(mixer)
channel = jack_mixer_c.add_channel(mixer, "Channel 1", True)

to there:

mixer = jack_mixer_c.Mixer("test")
print "Channels count: %u" % mixer.channels_count
channel = mixer.add_channel("Channel 1", True)

All was left was to replace the polling by a callback system, and it worked, then it crashed, randomly. And I realized I stepped in the dreaded thread country.

There is the main thread, it's Python, and PyGTK, but then there are threads created by jack, and the callback is called from one of them, from jack thread to Python, where the Global Interpreter Lock reigns, to PyGTK, where you should do everything in a single thread.

After much reading and pestering I believe I reached a working state doing the following things:

  • calling gtk.gdk.threads_init();
  • enclosing the extension PyObject_CallObject call (which calls a Python function from C) between PyGILState_Ensure() and PyGILState_Release();
  • emitting a gobject signal from the Python callback;
  • calling the GTK stuff from a function connected to that signal.

With all of that in place I can now turn knobs and push faders all I want, without crashing.

Unfortunately it didn't address the performance issue, which was in fact much simpler (some widgets were invalidated every 60ms, as their "value changed" computation was wrong).

Lessons learned (again): 1) don't jump to conclusion on the cause of performance issues, 2) it's possible to tame threads.

Last Modification: Wednesday, October 21, 2009 9:53

Did you look into Cython for generating Python bindings to C code? It's really a powerful solution.

Comment by Daniele on October 21, 2009 11:09

I wanted to write about Cython but someone was faster than me :-) It really sounds like it could help you.

Comment by Martin Soto on October 21, 2009 13:01

Don't fear the threads. They're really not all that hard with GTK+, once you write a couple of simple apps, and understand what GTK+ needs exactly.

Comment by Rodney Dawes on October 21, 2009 17:20

So are you taking the GIL from the jack thread? Can that block? The jack thread should never block. The Right Way is to pass the data from the jack thread to the other thread via jack_ringbuffer (or some other lock-free FIFO).

Comment by Paul on October 21, 2009 22:24

Comments on this entry have been closed.