Re: KDE 2.0 impressions

["Padraig O'Briain" <Padraig Obriain ireland sun com>]

Your versioning script sounds very similar to scope mapfiles on Solaris. I 
assume that it is not a coincidence that the syntax of your versioning script is 
the same as that of a scope mapfile.

I cannot speak for whether The Gnome project is interested in this although I 
would assume from a statement I saw on another message that GTK 2.0 will use 
symbol scope reduction that the intention is to use it.

As to the meaing of "load":

On Solaris the default method for loading an object (executable or shared 
library) is to map it into memory and then examine it for any dependencies and 
if it has any they are immediately loaded. These dependencies are then, in turn, 
examined and additional objects continue to be loaded until all inter-object 
dependencies are resolved. This may have the effect that objects are loaded into 
memory whether or not the code in the objects will actually be referenced by the 
program being run.

Solaris does have an option which allows for lazy loading of objects. Objects 
which have been labelled for lazy loading will not be loaded unless explicitly 
To use Solaris-speak it seems that on Linux the efect is the same as if all 
shared libraries are labelled for lazy loading.

Solaris has another linking option, called Direct Bindings. What direct bindings 
doies is decide which symbol a specific symbol reference will bind at link-edit 
time. This is done by having the link editor record for a symbol from which 
object from which object it is made available. and record that information in 
the newly built object. This can give much faster symbol resolution and permit 
multiple symbols with the same name to exist on the same link map without name 
space collision. Does Linux have a similar feature?

Padraig


> Delivered-To: gnome-private gnome org
> To: "Padraig O'Briain" <Padraig Obriain ireland sun com>
> Cc: gnome-private gnome org
> Subject: Re: KDE 2.0 impressions
> MIME-Version: 1.0
> X-BeenThere: gnome-private gnome org
> X-Loop: gnome-private gnome org
> X-Mailman-Version: 2.0beta5
> List-Id: <gnome-private.gnome.org>
> 
> On Tue, 31 Oct 2000, Padraig O'Briain wrote:
> 
> > One of the things that we found on Solaris which improved startup 
performance 
> > for applications was the use of scope mapfiles. I have not seen these used 
in 
> > GNOME so perhaps scope mapfiles are not available on Linux. Could someone 
> > educate me on this point? One can think of a scope mapfile for a shared 
library 
> > as being similar to a .def file for a Windows DLL in that it defines the 
> > interface provided by the library, i.e. the functions in the library which 
can 
> > be acccessed outside the library. The problem with a shared library is that, 
by 
> > default, i.e. without a scope mapfile, every function in the library is 
exported 
> > but in Windows the default is that no functions are exported.
> > 
> > If using C++, the function names, those mangled ones, tend to be even longer 
> > than the function names used in C and the improvement in startup time from 
using 
> > scope mapfiles for the shared libraries for a large C++ application, i.e. 
> > StarOffice was significant. The size of the symbol table reduced 
significantly 
> > which reduced the cost of loading it and the cost of finding external 
references 
> > was reduced as the symbol tables were smaller. It is worth noting that 
external 
> > references are to symbol names, without the shared library which contained 
them 
> > at link time being specified. On Windows external references are to a 
reference 
> > number in a specific Windows DLL. This makes fixing up references much 
faster
> > on Windows although if someone changed the order of the functions in the DLL 
> > since your application was built you will have some problems.
> 
> This was a *huge* problem in mozilla. Due to XPCOM each component dll
> really did only export four functions, but by default each bloody non-
> static function/method got exported. And due to the c++ name mangling the
> dynamic symbol table got *really* large.
> 
> The first step towards fixing this was to use a hack with the gnu linker
> using ELF symbol versioning. I came up with this hack and ramiro
> implemented it in Mozilla.
> 
> Basically you create a versioning script like this:
> (mozilla/build/unix/gnu-ld-scripts/components-version-script in mozilla
> cvs):
> 
> EXPORTED {
>    global:
>                 NSGetFactory;
>                 NSGetModule;
>                 NSRegisterSelf;
>                 NSUnregisterSelf;
>    local: *;
> };
> 
> This script makes all symbols except the specified 4 local. As a bonus
> these 4 get an ELF version called "EXPORTED", but ignore that.
> 
> The .so's are then linked using an extra gcc flag:
>  -Wl,--version-script,$(BUILD_TOOLS)/gnu-ld-scripts/components-version-script
> 
> When you look at these files with "nm --dynamic" everything looks just
> peachy, the unwanted symbols are gone. This is in fact not 100% true. Due
> to a bug in the gnu linker (it's marked with "TODO:" in the binutils
> code) the extraneous symbol strings in the .dynstr segment are not
> removed. This is a bit bad, because it is mostly the strings that are
> large, not the symbol table entries.
> 
> To fix this i had to write an amazing little utility called elf-gc-dynstr
> that removes all unreferenced strings from an ELF shared library, and then
> compacts the file. This utility can be found in
> mozilla/tools/elf-gc-dynstr in the mozilla cvs tree. Don't look at the
> code, your brain will melt.
> 
> I've got local modifications to elf-gc-dynstr which makes it work on ppc
> to (the current cvs version does x86 only), which i will check in soon.
> 
> I don't know if the Gnome project is interested in using this. It was done
> mostly to minimize Mozilla for use in embedded systems (desktop systems
> gain a lot too of course). 
> 
> > Solaris also have a feature which allows the code in shared libraries to be 
> > reordered at link time so that frequently accessed code is clustered 
together 
> > and code which is very infrequently used is at the end of the file and is 
never 
> > paged in. Windows also has a similar feature for DLLs. Use of such a feature 
can 
> > reduce the working set size and paging activity for memory constrained 
systems. 
> > This feature was invaluable when trying to get CDE performance to an 
acceptable 
> > level on 32 Mb systems.
> This is basically a description of nat's program "grope". Unfortunately he
> is to busy hacking gnome to have any time to finish it.
>  
> > Another feature which is the default on Windows and is available as an 
option on 
> > Solaris and I do not know its state on Solaris is to load shared libraries 
only 
> > when required rather than automatically on startup. This can make a 
significant 
> > difference to the user-perceived startup time if not all shared libraries 
have 
> > to be loaded before the user perceives the application to have started.
> What do you mean by "load"? Linux typically mmaps the libraries into
> memory, so no page is loaded from disk until it is read from or written
> to. Fixup of symbols is lazy by default on linux (each symbol is fixed up
> the first time it is referenced), but you can force non-lazy binding by
> defining some environment variable (LD_BIND_NOW?).
>  
> > It is not clear to me whether there is a performance problem on GNOME but if 
> > there is, hopefully, we have the tools to fix it.
> 
> We have some tools.
> 
> / Alex
> 
> 
> 
> 
> _______________________________________________
> gnome-private mailing list
> gnome-private gnome org
> http://mail.gnome.org/mailman/listinfo/gnome-private