Re: Merging gio into glib

[Behdad Esfahbod <behdad behdad org>]

On Thu, 2007-11-08 at 18:45 +0100, Alexander Larsson wrote:
> On Wed, 2007-11-07 at 16:31 -0500, Behdad Esfahbod wrote:
> > On Wed, 2007-11-07 at 21:46 +0100, Alexander Larsson wrote:
> > > 
> > > This leads me to belive it should be possible to create a configure
> > > option for glib such that libglib, libgmodule, libgobject are in the
> > > same libglib.so file, and with symlinks for gmodule and gobject. This
> > > setup should allow all old binaries to work as they used to, plus apps
> > > linked against this setup will continue to work in systems using the old
> > > setup.
> 
> After talking to jakub I came up with a better approach for this. 
> Consider the case of merging libglib and libgobject. We make libglib.so
> and libgobject.so small libraries that only contain the symbols from the
> respective real library. These libraries are what are picked by the
> linker (ld) when you specify -lglib and -lgobject, and the symbols in
> them are verified against what the app uses (so you get errors if you
> reference symbols that are not in the libs). 

A bit fragile as you are linking to something, running against something
else.  It sure works.


> However, in the resulting binary the DT_NEEDED is encoded based on the
> DT_SONAME of these libraries. In our case the sonames of these libraries
> is libglib.so.0 and libgobject.so.0. These are the things looked up by
> the runtime linker (ld.so) when finding the library to load for the app.
> However, we make both libglib.so.0 and libgobject.so.0 symlinks to the
> same file, libglib_combined.so.0, which contains all the object files
> from glib and gobjects in one shared object.

Yep, nice.  That's the part I was not sure about, that how to make sure
DT_NEEDED is correct.  Your approach takes care of it.

> So, the result is:
> a) binaries have the same dependencies as before (i.e on both libs)
> b) however, we only load one library, meaning less linker overhead
> c) and that library is actually smaller than the two separate ones as
> in-library calls can now be optimized
> 
> There is a slight problem with ldconfig. It goes throught the
> $prefix/lib directory and "fixes" up all symlinks based on the actual
> soname in the files. This means it will change the libglib.so.0 symlink
> to point to libglib.so, totally breaking things. I've worked around this
> by having the dummy library be in another directory
> ($libdir/glib-2.0/dummy/) which causes ldconfig not to change anything.

Makes sense to have them in a separate lib as they are not actual
libraries.  $libdir/glib-2.0/stubs or something... and pkg-config makes
it completely transparent.  Though the pkg-config --libs should probably
give full path to the stub object instead of -L followed by -l.

> I've attached a patch that gives a --enable-combined feature which
> combines glib, gobject and gmodule. (Not gthread, because linking in
> libpthreads causes slowdown since e.g. all mallocs etc suddenly become
> threadsafe with locking.)
> 
> I tested this patch by just starting gedit and then looking
> at /proc/pid/smaps. The total combined rss/vmsize before combination was
> 712kb/984kb, and afterwards it was 688kb/964kb. In addition each of the
> three libraries had 4k private dirty memory, which was reduced to only
> one in the combined case. So, 24kb less RSS, 20k less vmsize and 8k less
> private dirty memory (this is per process using glib).

Yes, saving the private page-per-.so-per-process .data is nice.  I
should do this in pango too, combine pango/pangoft2/pangocairo.  gtk/gdk
too...  You get an immediate 20kb/process saving right there.

You can gain even more.  I guess with your approach, gobject is not
linking to "internal" glib objects, but to the public aliases.  That is,
it's going through PLT entries.  That can be optimized too.  We need to
find a extensible scheme to include the aliases files from all modules
being combined and do all the work...

I really wish someday this could be all done by libtool...

> For some reason the combined version also uses 8k less RSS in libgtk.
> I'm not sure why that is. Maybe just random fluctuations.
> 
> Anyway, I can't think of any negative aspects about this, appart from
> the additional dependency on libdl.so for apps linking to glib. Sounds
> like a good idea to me.

So, with this approach, the cost to having a separate pkg-config for any
glib feature is exactly zero.  On the good side, embedded people are
free to only include modules/components that they are interested in.

-- 
behdad
http://behdad.org/

"Those who would give up Essential Liberty to purchase a little
 Temporary Safety, deserve neither Liberty nor Safety."
        -- Benjamin Franklin, 1759