Re: [RFC] : New features (search, filters, virtual folders...)

[James McPherson - TSG Engineer <James McPherson Sun COM>]

On 11 Nov 2001, 11:41:30 AM Carlos Morgado wrote:
[snip]
> > > 
> > > and then there is procmail style filtering, but that sounds easy :)
> > > and then there is knowing what messages you already filtered and which
> > > are new ones for copy/move rules. this scares me.
> > 
> > Yeah, this one scares me too, and that's why I implemented automatic 
> > filtering upon reception only for pop3, because with pop3. For local 
> > mailboxes in general you could have filtered yet a message, already copied 
> > in another mailbox, but next time you'll filter it again (I think the only 
> > solution is to use a finer flag systems for messages : a flag indicating 
> > that filters have been applied to it; perhaps this is yet possible, I saw 
> > that libmutt flags are finer that balsa's, balsa has only NEW flag, but 
> > libmutt has new and unread flags, if I understand it well).
> > 
> 
> i'm not sure if this will hold for imap. don't some imap servers look at
> flags or somesuch ? libmutt knows about 6 or so flags



> > > > 	* virtual folders (I don't know Evolution but everyone speaks
> > > > about it, shame on me :) : as I have thought about it, it is a "fake"
> > > > mailbox, and, when you launch it, it will be filled in by all messages
> > > > (from one or several mailboxes) matching the rules you had set up. Also
> > > iirc evolution uses the neat-o mbox indexing system to do vfolders. i'm
> > What is this indexing system?
 
> binary indexes of regular mbox files. basic ripoff of jwz's indexing system
> for netscape3 + some sort of content indexing for searches.
> in a nutshell, you make records with message headers (whatever you need for
> display/threads) and offset pointers to the mbox file.
> there is been talk on and off to implement this on balsa, but neve happened.


Hmmm, now we're approaching the interesting set of features that Solaris' dtmail 
has - for values of "interesting" which go from "fscking awful" to "really quite
neat." 

dtmail uses $HOME/$MAILDIR/.<viewname>.index for keeping track of the matching
messages that are currently in your inbox - dtmail "views" == balsa filters on a
virtual folder. I just ran strings over one of these index files -- it's totally
binary and I have no desire to go through the dtmail code to find out how it is
written out (apart from the company confidential thing).

dtmail updates the views / virtual folders each time it grabs an updated index from
your inbox, which can get very slow ;| even though I'm pretty sure that's another
thread that's doing it. hmm, think there could be a locking issue with this. 

How about this for a way forward:

virtual folders to be updated on demand by the user -- either by explicitly requesting
an update, or after moving (a) message(s) to disk-based folder / trashcan etc.

updates done with a separate thread, which _either_
   (1) also grabs the inbox so that the regular inbox checking gets postponed 
or (2) forces an inbox check, then locks the inbox and updates the virtual folders

Of course, messages can be visible in multiple virtual folders - eg if I filter 
separately on "to me" and "cc me" - something to be careful of. 

If keeping an index on disk, what sort of file format would be needed? Berkeley db
or plain text? I think Berkeley db - somewhat easier to read and parse quickly than
plain text imho.


just a few thoughts - hope they help.

James

-- 
TSG Engineer (Kernel/Storage)           828 Pacific Highway
APAC Customer Care Centre               Gordon NSW 
Sun Microsystems Australia              2072

Failfast panic: those controlling voices in my head have 
stopped telling me what to do.....

Read about the VOS Initiative at http://www.vosinitiative.com