FYI: GtkEtext final design

[Derek Simkowiak <dereks kd-dev com>]

	I am one of the people working on a new text widget.  This email
summarizes the design I am going to use.  I have made up example
struct names as I was writing the message, so the names will likely
change.

	Over the past 2-3 months, I have looked at many text widgets and
editors.  I have looked at parallel style buffers, linked lists of lines
(both char buffers and style tags), gapped text buffers, TkText's BTree
design, and even designed a SkipList-based version of the TkText.

	About 2 weeks ago, I was planning on a SkipList version of the
TkText design.  It would have been less of a port of the TkText (which
Havoc has done) and more of a re-implementation, with the data structs
encapsulated in the Glib style, more of a Gtk code look, etc.

	The TkText design is very powerful, because it abstracts the data
stored by the widget.  Instead of a "text" widget, we may as well call it
a "line-based display widget", since the underlying structures handle
images, windows, text, and other "segments" all in the same manner.  It
also has a powerful overlapping tag-based mechanism for styling the
text--indeed, the TkText design is well-suited for a word processor or
even a simple HTML renderer.

	Unfortunately, this flexibility comes at a severe memory cost.  If
you have the following line of C code:

while(done==FALSE)

	...and the while, ()s, and == all have their own
syntax-highlighted color, you will be using about 300 bytes of "segment"
overhead (7*40 bytes) just to store 18 bytes of text.  That doesn't count
the RAM used for the actual style tags, the characters themselves, or the
line information. Havoc has posted some numbers lately which are very
eye-opening--see the README of his TkText port for details.

	Furthermore, the character data is not in a contiguous buffer.
Doing any regex matching, file i/o, searches, etc. requires splicing all
the character "segments" together into a buffer.  To me, this feels less
like a text widget and more like a word processor.

	My design goals are more oriented towards a programmer's text
editor: lighter, with features such as line numbering, line marking, block
highlighting, and easy hooks for a lexical-scanner based syntax
highlighter (NOT regex-based).  I have no need for embedded widgets,
overlapping tags, or even superscript/subscript or right-aligned or
centered text.  (Embedded images, especially icons, may come later
though).  Therefor, I have settled on the following design:

	I will store the character data in gapped text buffer (the source
of which I posted yesterday).  The property information will be stored in
a manner similar to the GtkText's "TextProperty" struct, but instead of a
linked list, I will be using the Glib-styled SkipList I wrote.

	Also, the TextPropertyTags will hold only an offset ("length") and
a pointer to a TextProperty-like structure, which I am calling a "Style":

struct _TextPropertyTag {

	TextStyle *style;
	guint length;
}

	The SkipList will hold a "Line" at each Level[0] node, where a
Line holds:

1) A linked list of the TextPropertyTags in that particular line
2) A linked list of DLines (Displayed Lines), so if the line wraps there
will be more than one DLine for that Line.  This will be effectively the
same as the LineBufferCache in the current GtkText.

	This will allow me to scroll quickly and find the correct
TextPropertyTags quickly (by line number, character number, or even pixel 
height), with a relatively minor memory increase over the current GtkText
widget.

	The widget will provide hooks for the style tags, so you can find
out the character position of nearest TextPropertyTag.  This is very
important for fast (optimal?) scanner-based highlighting.

	I will also be writing a new lexical scanner (called the gescanner
:) which will be more configurable than the current GScanner.  I am basing
its design off the scanner that the win32 program "TextPad" uses.  It
should support multiple grammers and languages, based on config files.
This is what I plan to use for syntax highlighting--it's very fast and
extensible.  But it will be a seperate module from the text widget.

	This is more or less the GtkText with some ideas from the TkText
thrown in.  One design goal is that the code is clean and *maintainable*.
Modularizing the buffer gap was only the first step in that direction.  

	I expect to finish the Style API and code, and hopefully the line
drawing API/code by the end of next week.  It may go a few days beyond
that, I haven't put together a project schedule yet (and this is being
done in my spare time).

	Comments/suggestions are welcome.


Thanks,
Derek Simkowiak
dereks@kd-dev.com