Re: compiler optimization causing issues with glib

[Colin Walters <walters verbum org>]

On Tue, 2012-06-05 at 13:20 -0500, Mike wrote:

> I'm fairly convinced at this point that it is something in libc, but
> I'm not sure that it isn't acting to spec with regards to allowed
> behavior.  

Just to be clear, *which* libc are we talking about?  I'm guessing
eglibc?

> I've walked through the glib code for spawning a process,
> and everything looks good with regards to the requirements of
> fork/exec as well as the signal handler for SIGCHLD. 

Yeah, I did a pass through the spawning code last year and fixed
a few minor cases where we were missing EINTR handling (which only
matter on non-Linux UNIX), but other than that it's always been pretty
robust.

>  The fact that it
> gets caught in a futex for malloc makes me think that the issue
> relates to the usage of multiple threads and then the fork inside of
> g_spawn_async.  Most documentation I've read says fork and pthreads
> don't always work well together. 

Yes.  See https://bugzilla.gnome.org/show_bug.cgi?id=659326 for some
useful discussion.  In particular
https://bugzilla.gnome.org/show_bug.cgi?id=659326#c24 was a major
discovery for me at least.

Basically, for each process, you need to choose between one of:

* no shared address space between schedulable entities
* forking (without a following exec)
* threads

GLib uses threads, so it excludes you from using fork-without-exec.

> In any case, I've worked around it by forking a process early before
> glib init/main thread that is connected via a pipe.  This child
> process handles all of the forking requests of the parent and does not
> use pthreads or gdbus.  I have run it non-stop for a couple days, no
> issue.  Previously I would have issues after a few minutes.

If that works, then OK, but I'd be pretty curious what commit a
bisection of glib landed you on.  It's fairly automatable.