Bins Architecture Re: TRS component Re: IM log browser mockup, call for hackers?

[Srikant Jakilinki <sriks dcs gla ac uk>]

Cool Raphael. Glad to have you on board mate :-)

I was thinking of a simple "bins" architecture. Let me explain -

1) We have "fixed" global bins like TODAY, YESTERDAY, THIS WEEK, LAST 
WEEK, THIS MONTH, LAST MONTH, THIS YEAR, LAST YEAR, LONG-LONG-AGO

2) After seeing TODAY's value, all bins are initialized to time value 
ranges. When any item's timestamp is extracted, it will go into the 
corresponding bin. So, we can do a real-time sort in an interpretative 
fashion on a first-come-first-serve basis. Of course we can use a cache 
mechanism for displaying results (if it is too old beyond a threshold we 
will leave the processing for later so that the latest items are always 
displayed in a snap) and others are kept accruing. And if we indeed want 
to go for XML, we can only generate the full document when we process 
all values. Ideally, it should be a mixture. TRS could hold the bins in 
an array or a table temporarily for applications like Best to use 
directly and from this the XML is generated. We should be able to handle 
the concept of a "saved search/query" as well

3) Extraction could happen by parsers/filters as you say. The minimum 
thing parser has to output is a tuple - <TimeStamp, Snippet, Link>. TRS 
will only use the TimeStamp when sorting into bins. The rest of the 
tuples could be used to display the results. So, instead of putting the 
entire email in the results, we will only display the snippet and point 
to the original mail when the link is clicked. Snippets could be minimal 
email headers or the start-point of a conversation or anything the 
parser defines for itself. Best should be hidden from this. It just sees 
a <snippet> tag and displays it - text or thumbnail or preview

4) TRS should be recursive as well. For example, when going into the 
THIS WEEK bin and say it is Thursday today, we should have different 
color/markings for Sunday, Monday, Tuesday, Wednesday. Similarly, if we 
go in for TODAY, the markings would be for Morning, Afternoon, Evening, 
Night.  So, the bins are more like Russian dolls. Similarly for LAST 
YEAR, we would have January, February... All this is in reverse 
chronological order. Key is to give such "sense" subtly and in ways that 
we ourselves remember naturally

5) I wonder if the processing of TRS could be stored i.e. if an item 
gets processed by TRS at some point, its timestamp and the bins it 
belongs to should be retained in an index so that lookups can be easier 
in the future. Wait, this could be very difficult to manage and refresh 
but worth an implementation as a seperate timestamp index. Hmmm...

6) I hope that this is all comprehensible. Maybe Tuomas can do a mockup 
sometime to make things clearer. Implementation should be in C# for 
Beagle I suppose but it would be cool if we worked on the design and 
pseudo-code first so that anyone could implement it in any language in a 
days time. I left programming a long time ago I am afraid and I hope 
someone who enjoys coding would be up for the challenge :-)

Sorry guys for making bloating a simple IM log browser into something so 
big and complicated. But I think this is a generic need for many. As a 
first instance, we can implement an IM log browser (input from GAIM, a 
parser for GAIM logs, simple TRS, output to arrays) using this 
architecture and see how things go. Comments?

----
Regards-Cheers-Sincerely,
Srikant
http://sriks6711.blogspot.com
http://www.dcs.gla.ac.uk/~sriks
"Mythology... it's all Greek and Hindu to me"
-=-Sriksisms powered by Life, TagZilla and QOTD-=-

Raphaël Slinckx wrote:
> Hi i'm too interested in this thing, will it be written in Csharp ?
>
> Concerning your figure you mean something like:
>
> GaimLogParser-|
> FileAttr------||
> ...-----------|||
> MailParser----||||
>               ||||
>               ||||
>               ----
>     All provide a well defined (timestamp,content(object or string ?)) 
> added into a list. Should the list be sorted "real-time" as the items 
> are added, or we send every item, then we ask to mangle the result ?
>                 |
>                TRS
>                 |
>               ------
> * "Live query" like getTodayEvents(),getLastWeekEvents()
> * Output to an XML file with a well defined DTD, that can be parsed 
> (problem: only strings are allowed as events..)
>
> Raf