Re: Memory leaks in gnome_apps

[Owen Taylor <otaylor gtk org>]

Jason Nordwick <nordwick@scam.xcf.berkeley.edu> writes:

> Manish Vachharajani wrote:
> > 
> > You seem to be correct, that it doesn't seem to matter, I have plenty of
> > memory.  I was just concerned that there were no full pages being freed.
> > It could potentially be a problem after an application that does a lot of
> > widget allocation and freeing has been running for a long time.  If the
> > pages cannot be returned to the system, then the OS must keep them in
> > swap, since it doesn't know anything about what will happen when the
> > memory is referenced.  This could fill up swap, unnecessarily.
> 
> Not really, just becaus I sbrk() 100M, it doesn't fill up swap.  And with
> memory overcommits, you are allowed to allocate more memory than you have,
> just don't use it, or you'll get something killed off.

Different issue. As long as an application has touched a page, that
will page will use swap. However, filling up swap assumes that the
memory use will keep on increasing continually. I don't think that
is true in this case.
 
> > Now, on the other hand, it is also disturbing that the application creates
> > 3000 or so widgets to display the directory, then ALL of those widgets and
> > associated data are freed, but only about 1/2 of it gets reused, the rest
> > seems to get reallocated and never reused, why?  Does GTK+ never return to
> > the same state after a widget is created and then destroyed?

[ Perhaps you should consider revising your application not to use
  3000 widgets? I know that sounds like a poor solution, but you'll
  get better performance if you can avoid that many widgets (and
  especially all the related X stuff), and you are most likely
  toying with X's limit that a window can only be 32768 pixels
  in any dimension.

  GtkCList, GtkCTree and GnomeIconList are all suitable for displaying
  large amounts of data, since they don't create a widget for each
  rowl or cell ]

I think you'll find that if you create and destroy repeatedly,
the memory use will stabilize - at least in my test, when
repeatedly creating and destroying 10,000 widgets, memory
use stabilized at about 1.5 times the amount necessary to
create the widgets the first time. While this isn't all that
great, and probably could be improved with some profiling of
the allocation growth, it is a long way from run-away memory growth.
 
> It is reused, just that you have fragmented memory, so you have
> holes in you free list that aren't big enough for the next
> allocation that you want.  It is a common problem.  Maybe somebody
> should look at the small allocation code (memory chuncks?), because
> the small allocated should do a better job.  Better yet, they should
> mmap the memory they require, insteadm of mallocing it, so it
> can be retured to the system.    

I doubt that mmapping would help with memory reuse. Since memory
allocated through GMemChunks is allocated in large constant-sized
pieces, it should have no problem being reused. (It could, in some
cases, improve the efficiency of GMemChunk, because you can get
aligned memory.  But that is a different issue.)

> Gtk since I can remember has alot of memory problems for such a
> large project, maybe its time to go make the allocations better?

Umm, what? Evidence? And since when did smaller projects have
less memory problems...

                                        Owen