Writing Extension Modules
to be Interruptible

Zack Weinberg

〈zack@millionconcepts.com〉

〈zack@owlfolio.org〉

https://www.owlfolio.org/

A Common Bug

(in compiled extension modules)

>>> import numpy as np
>>> rng = np.random.default_rng()
>>> def g(n=1000000000):
...     return rng.random(n)
...
>>> g()
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in g
KeyboardInterrupt
>>>

3.5 second delay

What went wrong

NumPy didn’t ever call PyErr_CheckSignals
Common bug in compiled-code extensions
This talk is about:
1. Why there’s no way to avoid needing to do this
2. Why, today, many extensions don’t do this
3. How we can make it easier to do this

Short version of what went wrong: NumPy’s “fill array with random numbers” loop never called PyErr_CheckSignals.

I’m not here to pick on NumPy specifically. Lots of compiled code extensions have the same bug. But it is a bug.

The long version, now? That’s the rest of this talk.
Why is it necessary for extensions to do this?
Why don’t extensions do this?
How can we make it easier for extensions to do this?

There is no easy fix for this bug

There’s no workaround possible for users of buggy extensions
Core CPython changes will help but cannot eliminate the bug
Many extensions will need to change

This talk is about development of Python

You’ll get increasingly more out of it if…

you’ve written code in compiled languages
you’ve written compiled-code extensions for CPython
you’ve looked at or worked on the code of CPython itself
(or PyPy, Perl, Ruby, Lua, etc.)
you’ve written code that uses Unix signal handlers

So, the rest of this talk is going to be about those changes that need to be made, and why—and this necessarily gets into details of the innards of the CPython interpreter, and what happens at a low level when you type control-C.

I’m going to do my best to introduce these things gently, but I only have half an hour, so I’ll need to assume a bunch of background.

You’ll get more out of this talk if you have written compiled code extensions for CPython—or just if you have written code in a compiled language at all. You’ll get more out of it if you know the essence of how a bytecode interpreter works. And if you already understand the dangers of Unix signal handling, there’s a big chunk that won’t be news to you.

By the way, all the measurements I made, all the code I’m going to show you, it was all prepared using Python 3.12. I haven’t had a chance to try anything newer.

So if you’re excited about the free threading project, I’m excited about it too, but you’re gonna hear me say “global interpreter lock” a lot and, thing is, I have the impression free threading doesn’t actually change the rules much about when you can and can’t call C-API functions. Please mentally substitute “attached thread state” each time I say it. Come find me afterward if you have notes.

Why does this bug exist?

Not a problem for shell scripts
… but shell scripts don’t have structured exceptions
… and they get more help from the kernel

Let’s start with the big obvious question: Why do extensions need to call PyErr_CheckSignals in the first place? Some other programming languages don’t have this problem at all—take shell scripts for example: the sort utility doesn’t, as far as I know, do anything special with control-C, but if you control-C a shell pipeline with a big sort in there somewhere, you get the shell prompt back immediately!

One answer to that is, Python is trying to be fancier than shell. Python lets you “unwind” part way out of what you were doing, catch KeyboardInterrupt, and keep going. Shell scripts don’t have anything like that.

Another answer is, shell scripts get more help from the OS kernel in dealing with this, which I’ll come back to at the very end of this talk.

What `Control-C` Does¹

Starts out like any keystroke
Converted to a signal, SIGINT
SIGINT delivered to Python interpreter
Python interpreter reacts by raising KeyboardInterrupt

C signal handler only sets a flag
Nothing more happens until control returns to the main interpreter loop

Main loop checks the flag in some (not all!) bytecode ops
If the flag is set, raises KeyboardInterrupt

¹ Except on Windows

To understand why CPython needs compiled code extensions to call PyErr_CheckSignals in order to be responsive to control-C, we need to understand what control-C actually does.

Control-C isn’t something Python invented, it’s at least a decade older, and part of how it works is built into the Unix kernel.

Footnote, what I’m about to tell you is how this works on every OS we care about except Windows. I don’t know what Windows does instead, I just know it’s different. But, it’s not different in a way that helps with the bug we’re talking about, so it’s moot.

Control-C is the same as any other keystroke to your keyboard and to a whole bunch of processing stages we’re going to skip over. ssh for instance just passes it along. But right before it gets to the CPython process, a chunk of code inside the kernel, called the “terminal line discipline,” intercepts it, takes it out of the input stream and turns it into a thing called a “SIGINT signal” and delivers that to the process. And the interpreter reacts to the signal by raising KeyboardInterrupt.

These words here. The signal is “delivered” and the interpreter “reacts.” The delay we saw earlier is in between these words, and here’s why.

I’ll go into more detail about what “delivered” means in a moment, but the gist is, at the moment of delivery, a special function inside CPython runs and … all it does is set a flag. Nothing more happens until CPython gets back to its main interpreter loop.

The main loop checks that flag as part of some bytecode instructions—if you look at bytecodes.c, it’s the ones that include the “check_periodic” “op”—and if they detect the flag is set, then the reaction happens, raising KeyboardInterrupt.

And this is also what PyErr_CheckSignals does: it checks the flag and it carries out the reaction of raising KeyboardInterrupt.

Signal Delivery

>>> import numpy as np
>>> rng = np.random.default_rng()
>>> def g(n=1000000000):
...     return rng.random(n)
...
>>> g()
^C
Program received signal SIGINT
random_standard_uniform_fill ()
    at 0x00007ffff00b23a7
(gdb) signal SIGINT
Continuing with signal SIGINT.
Breakpoint 2, signal_handler (sig_num=2)
    at ./Modules/signalmodule.c:347
(gdb)

So why can’t the special signal handler function just raise KeyboardInterrupt itself, at the moment of delivery? Let’s look at what actually happens when a signal is delivered. Here’s that random number generation function again. I’m going to run it and interrupt it again…

but this time, I’ve hooked up CPython to the C-level debugger, GDB. It intercepts the actual signal, and it gives you a chance to do something about it before the program ever sees it.

GDB is a very talkative program. I’ve cut a lot of unimportant detail out of the transcript on these slides. If you try this yourself, it’ll look differently.

Notice what GDB says is on top of the C call stack just now. random_standard_uniform_fill, that sounds like part of NumPy’s code for filling arrays with random numbers, yeah?

Now let’s let the signal go through to Python. In advance, I set a breakpoint on the special signal handler function, so we immediately get told that we’re about to execute that function. The signal has been delivered at this point.

Signal Delivery

Breakpoint 2, signal_handler (sig_num=2)
    at ./Modules/signalmodule.c:347
(gdb) backtrace
#0  signal_handler (sig_num=2)
#1  <signal handler called>
#2  random_standard_uniform_fill (…)
#3  __pyx_f_5numpy_6random_7_common_f…
#4  __pyx_pw_5numpy_6random_10_genera…
#5  method_vectorcall_FASTCALL_KEYWOR…
#6  _PyObject_VectorcallTstate (…)
#7  PyObject_Vectorcall (…)
#8  _PyEval_EvalFrameDefault (…)
#9  PyEval_EvalCode (…)

I can’t fit the whole debugger transcript on one slide, so let’s scroll away the first bit. We’re still right where we were at the end of the previous slide, stopped at a breakpoint, about to execute CPython’s special signal handling function. Now let’s ask GDB for the C-level traceback.

GDB prints tracebacks the other way around from Python—the innermost frame is on top.

There’s a lot in there, let’s focus on the top four stack frames. The innermost is the special signal handler function. The next level up doesn’t seem to be a function at all, it just says “signal handler called”, and above that is, huh, random_standard_uniform_fill, where we were just before the signal was delivered, and above that is more of NumPy’s random numbers module.

Signal Delivery

Signal delivery interrupts normal code execution
- Exact analogy to hardware interrupts
- Kernel “preempts” execution of CPython
- Creates fake stack frames, to make CPU return to preemption point later
- Resumes execution at beginning of signal_handler
Preempted code could have been doing anything
Therefore, unsafe to do much within a signal handler

Signal delivery was invented by operating system programmers from the 1970s. They thought in assembly language. They designed signals to be like hardware interrupts.

At a completely arbitrary point between two machine instructions—it happened to be in random_standard_uniform_fill, but it could have been anywhere—the OS kernel has “preempted” execution of CPython, messed with the saved register state, written fake frames to the hardware call stack, and then restarted execution at the beginning of the signal handler function.

When signal_handler is done, the fake stack frames will make the CPU run a little bit of private code from the C library to clean up the mess—that’s what that “signal handler called” stack frame is about—and then return to exactly where it was in random_standard_uniform_fill, which will go on with whatever it was doing when it was preempted.

This is why CPython needs to have signal_handler just set a flag, with all further reaction postponed till we get back to a safe point, like the main interpreter loop. The code that was interrupted could have been doing anything at all, we don’t know what, it’s not safe to do anything else.

What `Control-C` Does¹

Starts out like any keystroke
Converted to a signal, SIGINT
SIGINT delivered to Python interpreter
Python interpreter reacts by raising KeyboardInterrupt

C signal handler only sets a flag
Nothing more happens until control returns to the main interpreter loop

Main loop checks the flag in some (not all!) bytecode ops
If the flag is set, raises KeyboardInterrupt

¹ Except on Windows

Why don’t extensions check the flag?

Need to call PyErr_CheckSignals is poorly documented
Adding calls may require tricky refactoring
- to make safe places to call it
- to propagate a −1 return all the way up
Calling it is expensive
- especially if you released the global interpreter lock

int PyErr_CheckSignals()

Part of the Stable ABI.

This function interacts with Python’s signal handling.

If the function is called from the main thread and under the main Python interpreter, it checks whether a signal has been sent to the processes and if so, invokes the corresponding signal handler. If the signal module is supported, this can invoke a signal handler written in Python.

The function attempts to handle all pending signals, and then returns 0. However, if a Python signal handler raises an exception, the error indicator is set and the function returns −1 immediately (such that other pending signals may not have been handled yet: they will be on the next PyErr_CheckSignals() invocation).

If the function is called from a non-main thread, or under a non-main Python interpreter, it does nothing and returns 0.

This function can be called by long-running C code that wants to be interruptible by user requests (such as by pressing Ctrl-C).

Note: The default Python signal handler for SIGINT raises the KeyboardInterrupt exception.

Here’s the documentation for PyErr_CheckSignals. It’s a big wall of text. I’m gonna highlight the crucial sentence:

“This function can be called by long-running code that wants to be interruptible.” That doesn’t sound nearly as important as it is. I might have written something like “Long-running code needs to call this function periodically in order for the interpreter to be responsive to user interrupts.”

And I’d have made it the first sentence!

Not to mention, before you can read this wall of text and realize its implications, you need to find it.

It’s buried in the “signal handling” section of the exception-handling chapter of the C-API reference manual. The “extending and embedding” guide doesn’t mention it at all.

There’s another sentence in here that’s very important and very easy to miss. This one.

“If the signal module is supported, this can invoke a handler written in Python.”

PyErr_CheckSignals might run arbitrary Python code before returning to you.

Why don’t extensions check the flag?

Need to call PyErr_CheckSignals is poorly documented
Adding calls may require tricky refactoring
- to make safe places to call it
- to propagate a −1 return all the way up
Calling it is expensive
- especially if you released the global interpreter lock

How to check: a simple example

void
random_standard_uniform_fill(
    bitgen_t *rng,
    npy_intp cnt, double *out
) {


  npy_intp i;
  for (i = 0; i < cnt; i++) {
    out[i] = next_double(rng);




  }

}

int
random_standard_uniform_fill(
    bitgen_t *rng,
    npy_intp cnt, double *out
) {


  npy_intp i;
  for (i = 0; i < cnt; i++) {
    out[i] = next_double(rng);
    if (PyErr_CheckSignals())
      return -1;


  }
  return 0;
}

Here’s probably the simplest case: the source of the random_standard_uniform_fill function from NumPy. This is what got interrupted to deliver the signal, earlier. If we want this to be interruptible at all, we have to put a call to PyErr_CheckSignals inside the loop…

like this, maybe.

Notice that I changed the return type from void to int. Just like any other C-API function that can fail, when it returns −1, it’s telling you that the error indicator has been set—to KeyboardInterrupt—and you need to clean up and return immediately, and tell your caller to clean up and return immediately, and so on all the way up the call stack. You might need to deallocate resources. You might need to roll back transactions. The cascade of changes is probably not too bad for this function, though.

Also, by making this function interruptible we’ve introduced a new bug.

How to check: a simple example

int
random_standard_uniform_fill(
    bitgen_t *rng,
    npy_intp cnt, double *out
) {


  npy_intp i;
  for (i = 0; i < cnt; i++) {
    out[i] = next_double(rng);
    if (PyErr_CheckSignals())
      return -1;


  }
  return 0;
}

int
random_standard_uniform_fill(
    bitgen_t *rng,
    npy_intp cnt, double *out
) {
  PyGILState_STATE st;
  int err;
  npy_intp i;
  for (i = 0; i < cnt; i++) {
    out[i] = next_double(rng);
    st = PyGILState_Ensure();
    err = PyErr_CheckSignals();
    PyGILState_Release(st);
    if (err) return err;
  }
  return 0;
}

Remember PyErr_CheckSignals can run arbitrary Python code before coming back to you? That means it needs the global interpreter lock, and this function was called with the global interpreter lock released. NumPy is diligent about releasing the lock when it’s running loops like this.

So we can’t just call PyErr_CheckSignals. We need to use PyGILState_Ensure to reclaim the lock for just long enough to call PyErr_CheckSignals, and then release it again.

PyGILState_Ensure and _Release are buried pretty deep in the manual, too, and using them can be troublesome, see PEP 788 for gory details—but they’re the only functions I can find that do what we need in this case, namely reclaim the lock if it was released, but don’t deadlock if this thread already holds the lock, and don’t require a PyThreadState argument. If you have a better idea, please come tell me about it afterward.

Why don’t extensions check the flag?

Need to call PyErr_CheckSignals is poorly documented
Adding calls may require tricky refactoring
- to make safe places to call it
- to propagate a −1 return all the way up
Calling it is expensive
- especially if you released the global interpreter lock

A more complicated example

I didn’t measure the changes to the NumPy “generate a zillion random numbers” loop that I’ve been showing you, though, I measured the changes to this. It’s more complicated in terms of what needs to be done to it, but still simple enough to cram onto one slide. It’s the core of the simplest fast Fourier transform implementation I could find. Originally written by a fellow name of Mark Borgerding. Are you in the audience today, Mark?

[if he is] Thank you for …
[if he isn’t] Well, I’d like to thank him for …
writing an FFT implementation I could easily make sense of.

What I want you to see is that this is a recursive function—the recursive call is inside this loop in the middle, highlighted in brown—the base case is up top, highlighted in green, and after either of those finishes it always does one of these calls down at the bottom, highlighted in purple.

Each of these blocks does a linear pass over the input array.

Now we know checking could be expensive, and if we studied the algorithm in detail we’d see that the compiler might be able to vectorize the innermost loops. That’s great for speed, we don’t want to lose that. Adding a function call to any loop pretty much kills its chances of getting vectorized. And the innermost loops run fast enough that even at the biggest problem sizes, they complete in microseconds. It’s fine to have a delay as much as a few tens of milliseconds when responding to control-C.

So I think a reasonable first cut at making this responsive to control-C without hurting performance much would be to add checks in these two places: right after each pass over the input.

Cost of checking for signals

I wrote a test module wrapping Mark Borgerding’s FFT that lets me benchmark it with or without signal checks, and with or without releasing the GIL around the whole computation.

All benchmarks were run single-threaded, meaning the GIL was never contended, so the numbers you’re going to see are the best possible case for GIL-related overhead.

Without any signal checks, and without releasing the GIL, I measured this nice linear relationship between FFT problem size and execution time. Note both axes are on a log scale.

Releasing the GIL once at the beginning and reclaiming it once at the end doesn’t add any measurable overhead.

If we don’t release the GIL, enabling signal checks is pretty cheap. No signals are actually sent in this test, we’re measuring how much it costs to have the checks even when they have nothing to do, like they would most of the time.

But if we do release the GIL, suddenly the signal checks are a lot more expensive.

Cost of checking for signals

What if we didn’t check every time?

Claw back more speed with coarse clock

It’s hard to compare the slopes of lines by eye, so this plot shows what we get if we divide out the problem size. The recursive structure of the FFT implementation means there’s still an effect of the problem size, but we can see the effect of turning on checks, more easily. Up top, here, the difference between the top two strips shows the overhead of signal checks with the GIL held. I’m still calling PyGILState_Ensure and _Release around each call to PyErr_CheckSignals, but they never have any real work to do. The cost is small but it’s still there.

On the other hand, down below, this shows the overhead of signal checks with the GIL released, so we have to reclaim it for each call to PyErr_CheckSignals, and it’s a lot. The FFT is something like three to four times slower in this mode.

The obvious thing to do about this is not check as often. We already aren’t checking after every iteration; but what if we literally look at the clock? Ask the OS if at least a millisecond has gone by since the last time we called PyErr_CheckSignals; only reclaim the GIL and call it, if it has. Human responsiveness will be plenty good enough.

The new third strip on each half of this plot shows what we get if we do that. Looking at the clock slows us down a little more when we didn’t release the GIL, but speeds us up a whole lot when we did; the overhead with GIL released is now basically the same as the overhead with it held. That’s great.

Most of the remaining overhead is the cost of looking at the clock all the time. Some operating systems offer a “coarse” realtime clock that’s plenty good enough to tell us if a millisecond has gone by. That version is efficient enough that the performance cost is barely measurable except for the biggest problem sizes.

Longer intervals don’t help

Recap

CPython interpreter cannot raise KeyboardInterrupt directly from its C-level signal handler
- Must return to main interpreter loop before raising exceptions
- Prompt return needs cooperation from compiled-code extensions
Many extensions do not cooperate
- Need to do this has not been well advertised
- Retrofit requires finicky code changes
- Substantial runtime costs

Short term: look at the clock

int CheckSignalsOftenEnough(void) {
  static struct timespec last_check = { 0, 0 };
  struct timespec now;
  clock_gettime(CLOCK_MONOTONIC_COARSE, &now);

  if (timespec_difference_at_least(&now, &last_check, ONE_MS_IN_NS)) {
    last_check = now;
    PyGILState_STATE st = PyGILState_Ensure();
    int err = PyErr_CheckSignals();
    PyGILState_Release(st);
    return err;
  }
  return 0;
}

Full version: https://github.com/MillionConcepts/cpython-ext-ctrl-c/blob/main/CheckSignalsOftenEnough.c

Here’s my short-term performance fix. Look at the clock, reclaim the GIL once a millisecond to check for signals.

You can grab the full code for this from the link at the bottom of this slide. Copy and paste it into your extension. Call CheckSignalsOftenEnough at appropriate moments. You still have to figure out where to do that, and how to propagate the failure back out of your extension, but it takes care of looking at the clock and being GIL-safe for you. It’s only been tested on Linux and NetBSD but I sincerely hope it doesn’t need any more portability gunk than what it has now.

3-clause BSD license. If you want a different license contact my boss about it.

How can we make this better?

Improve core documentation
- Highlight need to call PyErr_CheckSignals regularly
- Explain how to do so safely
- Explain how to do so efficiently
- gh-134075

zaniness: 0

How can we make this better?

Don’t require GIL to call PyErr_CheckSignals
- Cost of frequent checks is mostly cost of locking
- Signal flag is an atomic variable! Don’t need GIL to test it
- Have PyErr_CheckSignals reclaim GIL itself, before running Python signal handlers, only if flag is set
- (Still needs to be safe to take GIL at any callsite)
- Proposed for Python 3.14: gh-133465

zaniness: 0

Maybe we could push the PyGILState_Ensure calls down into PyErr_CheckSignals, making it safe to call with or without the GIL, as long as it’s safe to take the GIL at the callsite? The signal flag itself is an atomic variable, so PyErr_CheckSignals could test the flag without taking the GIL, and only take the GIL if the flag is set.

This would eliminate any need to look at the clock and space out the calls. CheckSignalsOftenEnough could just be PyErr_CheckSignals. You’d still want to avoid calling it on every single iteration, though, if only because that would block the compiler from vectorizing your loops.

Also not a zany idea at all in my opinion, I have written a patch for Python 3.14 that does this.

How can we make this better?

Make Cython, Numba, and similar tools insert checks for you

cdef random_standard_uniform_fill(
    bitgen_t rng,
    double [:] out,
) nogil:
    for i in range(len(out)):
        out[i] = next_double(rng)

int random_standard_uniform_fill(
    bitgen_t rng,
    __Pyx_memviewslice out
) {
  Py_ssize_t i;
  Py_ssize_t len;
  len = __Pyx_MemoryView_Len(out);
  for (i = 0; i < len; i += 1) {
    *((double *)
      ((out.data +
        i * out.strides[0]))) =
      next_double(rng);
    if (CheckSignalsOftenEnough())
      return -1;
  }
  return 0;
}

zaniness: 1

How can we make this better?

In tools like PyO3, model control-C as async cancellation

#[pyfunction]
async fn random_standard_uniform_fill(
    #[pyo3(cancel_handle)] mut cancel: CancelHandle,
    rng: BitGen,
    out: &mut f64[],
) {
    futures::select! {
        cancel.cancelled.fuse() => {},
        _ => {
            for i in 0..out.len() {
                out[i] = rng.next_double();
            }
        }
    }
}

zaniness: 5

How can we make this better?

Turn Python into a shell-structured language
Evaluate each step in a subprocess

>>> import numpy as np
>>> rng = np.random.default_rng()
>>> def g(n=1000000000):
...     return rng.random(n)
...
>>> g()

Pass results back via pipes or shared memory or something
Let the subprocess die on SIGINT
This is why the Unix shell doesn’t have the same problem

zaniness:

Remember I said shell scripts get more help from the kernel? Remember I said the sort utility probably doesn’t have any special handling for control-C, and yet, when you control-C a pipeline with sort in it, it stops immediately? How does that work, anyway?

The answer is, subprocesses.

Because (we presume) the sort process doesn’t have a handler for SIGINT, instead of “delivering” the signal the kernel just kills off the process, along with the rest of the pipeline, and the shell gets control back immediately. We could, in theory, restructure the CPython virtual machine to work the same way.

Here be dragons, right? This would be a huge project. There’s implications for language semantics. There’s probably at least two “how do we even do this” research studies in there somewhere. But: it’s the only idea I’ve got that would let us not have to add PyErr_CheckSignals calls to every compiled code extension on PyPI!

How can we make this better?

Brainstorming time!
- Call out your ideas
- Or ask a question
- One sentence per person

Acknowledgments

Slides, code samples, raw data, and analysis scripts may be found
at https://github.com/MillionConcepts/cpython-ext-ctrl-c
or https://git.sr.ht/~zackw/cpython-ext-ctrl-c
and may be reused under Million Concepts’ usual 3-clause BSD license
KISS FFT originally written by Mark Borgerding
Data visualization thanks to ggplot2 and cowplot
Graphics postprocessed to various degrees in Inkscape
Slide deck rendered with reveal.js
Fonts are B612 Mono, Libertinus Sans, and Quando
Dragon head by Lazur from openclipart.org
Financial support from NASA ADAP grant 80NSSC21K1421

Writing Extension Modulesto be Interruptible

A Common Bug

What went wrong

There is no easy fix for this bug

This talk is about development of Python

Why does this bug exist?

What Control-C Does1

Signal Delivery

Signal Delivery

Signal Delivery

What Control-C Does1

Why don’t extensions check the flag?

int PyErr_CheckSignals()

Why don’t extensions check the flag?

How to check: a simple example

How to check: a simple example

Why don’t extensions check the flag?

A more complicated example

Cost of checking for signals

Cost of checking for signals

What if we didn’t check every time?

Claw back more speed with coarse clock

Longer intervals don’t help

Recap

Short term: look at the clock

How can we make this better?

How can we make this better?

How can we make this better?

How can we make this better?

How can we make this better?

How can we make this better?

Acknowledgments

Writing Extension Modules
to be Interruptible

What `Control-C` Does¹

What `Control-C` Does¹

`int PyErr_CheckSignals()`