Rusty Russell's Coding Blog | Stealing From Smart People

May/10

24

Typesafe callbacks in C (and gcc)

A classic pattern in C is to hand a generic callback function around which takes a “void *priv” pointer so the function can take arbitrary state (side note: a classic anti-pattern is not to do this, resulting in qsort being reimplemented in Samba so one can be provided!).

The problem with this pattern is that it breaks type safety completely, such as in the following example:

int register_callback(void (*callback)(void *priv), void *priv);
static void my_callback(void *_obj)
{
struct obj *obj = _obj;
...
}
...
register_callback(my_callback, &my_obj);

If I change the type of my_obj, there’s no compiler warning that I’m now handing my callback something it doesn’t expect.

Some time ago, after such a change bit me, I proposed a patch to lkml to rectify this, using a typesafe callback mechanism. It was a significant change, and the kernel tends to be lukewarm on safety issues so it went nowhere. But these thoughts did evolve into CCAN’s typesafe_cb module.

The tricksiness…

More recently, I tried to use it in libctdb (the new async ctdb library I’ve been toying with), and discovered a fatal flaw. To understand the problem, you have to dive into how I implemented typesafe_cb. At its base is a conditional cast macro: cast_if_type(desttype, expr, oktype). If expr is of type “oktype”, cast it to “desttype”. On compilers which don’t support the fancy gcc builtins needed to do this, this just becomes an unconditional cast “(desttype)(expr)”. This allows us to do the following:

#define register_callback(func, priv) \
_register_callback(cast_if_type(void (*)(void *), (func), void (*)(typeof(priv)))

This says that we cast the func to the generic function type only if it exactly matches the private argument. The real typesafe_cb macro is more complex than this because it needs to ensure that priv is a pointer, but you get the idea.

Now, one great trick is that the callback function can take a “const” (or volatile) pointer of the priv type, and we let that work as well: we have a “cast_if_any” which extends “cast_if_type” to any of three types:

#define typesafe_cb(rtype, fn, arg) \
cast_if_any(rtype (*)(void *), (fn), \
rtype (*)(typeof(*arg)*), \
rtype (*)(const typeof(*arg)*), \
rtype (*)(volatile typeof(*arg)*))

The flaw…

If your private arg is an undefined type, typeof (*arg) won’t work, and you need this to declare a const pointer to the same type. I have just filed a bug report, but meanwhile, I need a solution.

The workarounds…

Rather than use cast_if_any, you can insert an explicit call to the callback to evoke a warning if the private arg doesn’t match, then just cast the callback function. This is in fact what I now do, with an additional test that the return type of the function exactly matches the expected return type. cast_if_type() now takes an extra argument, which is the type to test:


#define typesafe_cb(rtype, fn, arg) \
cast_if_type(rtype (*)(void *), (fn), (fn)(arg), rtype)

cast_if_type does a typeof() on (fn)(arg), which will cause a warning if the arg doesn’t match the function, and the cast_if_type will only cast (fn) if the return type matches rtype. You can’t test the return type using a normal test (eg. “rtype _test; sizeof(test = fn(arg));”) because implicit integer promotion makes this compile without a warning even if fn() returns a different integer type.

Unfortunately, the more general typesafe_cb_preargs() and typesafe_cb_postargs() macros lose out. These are like typesafe_cb but for callbacks which take extra arguments (the more common case).


/* This doesn't work: arg might be ptr to undefined struct. */
#define typesafe_cb_preargs(rtype, fn, arg, ...) \
cast_if_any(rtype (*)(__VA_ARGS__, void *), (fn), \
rtype (*)(__VA_ARGS__, typeof(arg)), \
rtype (*)(__VA_ARGS__, const typeof(*arg) *), \
rtype (*)(__VA_ARGS__, volatile typeof(*arg) *))

We can’t rely on testing an indirect call: we’d need example parameters to pass, and because they’d be promoted. The direct call might work fine but an indirect call via a different function signature fail spectacularly. We’re supposed to increase type safety, not reduce it!

We could force the caller to specify the type of the priv arg, eg. “register_callback(func, struct foo *, priv)”. But this strikes me as placing the burden in the wrong place, for an issue I hope will be resolved soonish. So for the moment, you can’t use const or volatile on callback functions:


/* This doesn't work: arg might be ptr to undefined struct. */
#define typesafe_cb_preargs(rtype, fn, arg, ...) \
cast_if_type(rtype (*)(__VA_ARGS__, void *), (fn), (fn), \
rtype (*)(__VA_ARGS__, typeof(arg)))

RSS Feed

7 Comments for Typesafe callbacks in C (and gcc)

Noone | May 24, 2010 at 1:55 pm

Or use a language that supports typesafe callbacks instead of needing a “module”.

Anonymous | May 24, 2010 at 7:10 pm

This seems like a task better solved by fixing the compiler. In theory, you could use some compiler attribute to tell the compiler that you want a limited form of templating: void call_my_callback(void (*)(T *), T *)

Alex Besogonov | May 24, 2010 at 8:19 pm

Ah…

It’s fun to see what people are going to do just to avoid C++…

Brendan Miller | May 25, 2010 at 8:27 am

the void* priv pattern is really just a simulation of a closure… but you are already using GNU C, which supports closures on nested functions.

closure.c:
#include
#include

int main() {
int count = 0;
int compar(const void *left_ptr, const void *right_ptr) {
int left = *(int*)left_ptr;
int right = *(int*)right_ptr;
++count;
return left – right;
}
int array[] = {8, 2, 4, 2, 7};
qsort(array, sizeof array / sizeof(int), sizeof(int), compar);
int ii;
for(ii = 0; ii < sizeof array / sizeof(int); ++ii) {
printf("%d ", array[ii]);
}
printf("\n");
printf("count: %d\n", count);
}

compile's without any special flags on gcc 4.4:
gcc closure.c

Produces output:
./a.out
2 2 4 7 8
count: 8

The limitation of this is that count is on the stack, so if you are registering a callback, say for a GUI app, that may outlive main's stack frame, this won't work. However, it's fine for traditional higher order functions like qsort, and is typesafe.

Author comment by rusty | May 25, 2010 at 9:54 am

This is slightly different: I actually use nested functions in CCAN’s asort module.

And yes, though any simple example here can be implemented by closures, as you point out that can’t be done in general for libraries which need callback functions. That was my target with this.

But there are two issues here. Firstly, it’s not typesafe. Change array[] to char and watch it compile perfectly and fail miserably. That’s precisely the point about avoid void *.

Secondly, it can’t be simulated by non-gcc. CCAN uses gcc where available, but I want the modules to work without it. Thus, typesafe_cb() et al can be simulated (without type safety) on lesser compilers.

Hope that helps!
Rusty.

Author comment by rusty | May 25, 2010 at 10:03 am

Several people suggested other languages or extensions: I completely agree that this is a C limitation. If you’re in C++, I’m fairly sure (without having implemented it) that templating would provide a neat typesafe callback solution. Other languages might punt this to runtime (which is not an equivalent solution in my book!).

But my aim with this post (and CCAN in general) is not to convert people to another language, but improve state of the coding in C. If you’re going to write callbacks, try to avoid sprinkling around void *, ok?

Arnd | May 25, 2010 at 8:03 pm

I’ve hit the same problem in gcc with my sparse annotations for an __rcu namespace, trying to make that typesafe in the kernel. Fortunately, the kernel source is completely available, so I could make sure that the structure are in fact defined in all places that use rcu_dereference().

Leave a comment!

«

»

Find it!

Theme Design by devolux.org

Tag Cloud