Coin web de Frédéric Péters

An evening with threads

I noted on Monday how I have been working on jack mixer recently; to be honest I didn't do much work up to now, replacing some widgets, adding a preferences dialog, easy stuff.

But as I am starting to use it seriously I realized it didn't scale well, I would add five channels and it would burn CPU and memory in a terrific manner. Without much investigation I decided it was caused by the polling for MIDI events, and set out to fix this.

Well, jack_mixer is written in Python and C, Python for GUI stuff and C for jack and computer intensive stuff, and it turned out it's not possible to call back from a C extension to Python code, when using SWIG, so I got to write a manual binding, easy enough, and I got it feature complete quite fast, gaining good looking code along the way, from here:

mixer = jack_mixer_c.create("test")
print "Channels count: %u" % jack_mixer_c.get_channels_count(mixer)
channel = jack_mixer_c.add_channel(mixer, "Channel 1", True)

to there:

mixer = jack_mixer_c.Mixer("test")
print "Channels count: %u" % mixer.channels_count
channel = mixer.add_channel("Channel 1", True)

All was left was to replace the polling by a callback system, and it worked, then it crashed, randomly. And I realized I stepped in the dreaded thread country.

There is the main thread, it's Python, and PyGTK, but then there are threads created by jack, and the callback is called from one of them, from jack thread to Python, where the Global Interpreter Lock reigns, to PyGTK, where you should do everything in a single thread.

After much reading and pestering I believe I reached a working state doing the following things:

  • calling gtk.gdk.threads_init();
  • enclosing the extension PyObject_CallObject call (which calls a Python function from C) between PyGILState_Ensure() and PyGILState_Release();
  • emitting a gobject signal from the Python callback;
  • calling the GTK stuff from a function connected to that signal.

With all of that in place I can now turn knobs and push faders all I want, without crashing.

Unfortunately it didn't address the performance issue, which was in fact much simpler (some widgets were invalidated every 60ms, as their "value changed" computation was wrong).

Lessons learned (again): 1) don't jump to conclusion on the cause of performance issues, 2) it's possible to tame threads.

21 octobre 2009, 09:52