Re: [Evolution-hackers] Memory consumption and virtual machines

[Philip Van Hoof <spam pvanhoof be>]

On Tue, 2006-07-18 at 16:05 -0500, Federico Mena Quintero wrote:
> On Tue, 2006-07-18 at 14:46 -0400, Jeffrey Stedfast wrote:
> > I have to wonder if it's even worth ever merging the mmap hack into
> > Evolution at all. If the plan is to finish Zucchi's disk-summary branch,
> > which also solves the memory problems (afaik) as well as:
> > 
> > 1. introducing an API for using cursors to get at message infos
> > 2. better designed on-disk format that uses B-Trees
> 
> Let's put that discussion on Pause until someone actually starts
> resurrecting the disk-summary branch.
> 
> If Philip's patch turns out to work well for daily use, it may be a good
> stopgap measure.  The disk-summary branch will need way more QA time
> than the mmap() stuff, and a lot more performance tuning work as well :)
> 
> > the problem with philip's mmap file format is that the strings that will
> > be hit for sorting/viewing/etc are all spread out over a huge number of
> > pages. I just see this being re-examined later to try and design the
> > format to better optimise it by compacting all the strings into a strtab
> > type thing.
> 
> I've been talking to Philip on IRC, and gave him these requirements for
> his patch:
> 
> 1. Don't change the external ABI of Camel, so that Evo needs no changes,
> *OR* also submit a patch to update Evo for the changed API.

The external API nor ABI has changed by the patches.

> 2. Make sure the summary format on disk works with older Evos without
> making *them* rewrite the summaries.  This is for deployments which have
> machines with old and new versions of GNOME, but NFS homedirs accessible
> from any machine.

What about making an internal little tool that converts any summary file
it detects to a summary.mmap file, and after that never touches the
original summary file.

This will allow the format of the mmaped summary file to change, and
only an evolution with the mmap patches will only the first time have to
perform things.

We can also simply change the file-name and let the older evolutions
continue using the "summary" file, and the mmaped evolutions the new
file.

The older evolutions will not accept any changes to the summary files
(for example version changes). They will reload the summary file if it
has defects it can't handle, however.

> 3. Keep the coding style, variable naming convention, indentation, etc.

Of course.

> It may be possible to change the summary format by *just* adding a
> nul-terminator to strings; that may work with older Evos if we are lucky
> enough that they'll just ignore the nul byte at the end.  This needs
> testing.
> 
> I'd say that (1) and (2) are hard requirements.  (3) is the usual stuff.
> 
> > I just don't get the feeling this is really all that well thought out
> > and it scares me.
> > 
> > I'd just hate to see a rush job come out of this
> 
> Yeah, it needs good testing.  Philip says he'll cook a patch so that I
> can use it with my system's e-d-s RPM for daily use.  Then I can test it
> with my normal mailbox.

It might take me a few days as in fact I was planning to give it a rest
for a few days. I've been caring to much about it.

I don't know, it might also be finished in a few hours. Oh, no .. it's
to late now. 



-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be