Re: memory allocations.

[Ulrich Drepper <drepper redhat com>]

Alan Cox <alan redhat com> writes:

> Why does locking come into this. Is glibc not itself keeping per thread
> pools to cut down locking overheads ?

You cannot have one pool per thread.  That would cause too much
fragmentation and is even impractical with thousands of threads.
There is a pool of memory pools and access obviously has to be
controlled.

We had lots of complains from people writing big applications (e.g.,
Mozilla) about the fragmentation.  This has been addressed.  Locking
now (for some platforms) avoids using the pthread calls and instead
uses spinlocks.

The whole thing would be mostly void if we'd have the fast locking
with kernel help.  If there is no contention the overhead is minimal.
Plus the one or other optimization in this area which we already
talked about.

> Is that perhaps something that it should do rather than g_malloc ?

I don't understand what you want to do.  Keeping per-thread pools?
This is unlikely to be better.  You'd have to create and destroy them
with each new thread.  You have to keep track of thread creation etc
etc.  The current implementation is already quite good and 2.3 will
help a bit better.  Plus, you have severe fragmentation problems.

What an application with thousands of memory requests should do (and
not at the g_malloc level) is to keep memory pool for object of one
type.  These could eventually be kept in per-thread.  If applications
are allocation (and not freeing) thousands of little blocks this
should be handled by analyzing the object's lifetime and putting them
in one struct.

This is all common programming knowledge.  Nobody seems to have done
any work here and instead now tries to blame other parts of the
system.  First clean up your yard before complaining about your
neighbor.

If you cannot do this use an automatic garbage collection mechanisms.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------