Re: memory allocations / libxml

[Michael Meeks <michael ximian com>]

On Tue, 2002-03-05 at 01:09, Daniel Veillard wrote:
>   Libxml works on a set of platform far larger than glib. 
>     http://mail.gnome.org/archives/gdome/2002-February/msg00008.html
>     Enjoy !

	Fascinating, AFAIK Dan Winship has got evolution running on MacOS X -
which presumably entails porting glib, is it possible that there have
been some portability regressions in glib-2.0 ? The bug report is
incredibly vague and unusable to the glib team to help fix anything
though.

> in a lot of case the length of the buffer ain't available so add an
> extra xmlStrlen() penalty to your alloca path

	In general I was proposing optimizing the path that showed on the
profile if it could be found; inasmuch that xmlStrndup showed on the
profile, it might seem that it was an area ripe for some alloca action.

	Incidentally it seems that xmlStrdup and most of the xml family malloc
type string handlers chain to xmlStrndup - thus it might seem that the
length is ~always calculated.

	Of course - quite apart from any clever malloc/alloca tricks, it would
quite possibly be sufficient to have a stack managed arena, referenced
from the xmlParserCtxtPtr that provided the equivalent of a more
portable / ideal alloca type impl. where one could decide the
granularity of allocation far more effectively, and implement it using
malloc.

	An API like this might suffice:

gulong         mstack_alloc_begin_scope   (xmlParserCtxtPtr ctxt);
const xmlChar *mstack_alloc_strndup       (xmlParserCtxtPtr ctxt,
					   const xmlChar   *buf,
					   guint            length);
void           mstack_alloc_restore_scope (xmlParserCtxtPtr ctxt,
                                           gulong           offset);

	And require an addition to the parser context of perhaps:

	xmlChar *base_buffer;
	gulong   base_buffer_size;
	xmlChar *cur_ptr;

	Oh, and of course assuming we did fairly few reallocs we would have ~0
malloc overhead, and of course save a lot of 'free' invocations, and get
more inlining type optimization (perhaps).

> also from where do you extract that magical 64 value ?

	Well, 42 is allegedly the meaning of life etc., add my current age and
subtract 2 and there you are - obviously optimal no ? ;-)

>   Maybe there is something to save from this but it would need quite some
> work to become maintainable and portable.

	How does the above sound ?

>   What I love about this rant on libxml2 performances looking like it
> simply sucks rocks

	I don't believe I ever said that or insinuated it. Certainly I think
the cost of allocation in libxml is quite high, but I'm prepared to
believe that was the best way to do it. The thing is that the bonobo
code does rather hammer the libxml SAX parser, with lots of rather small
fragments of xml.

>   Are you by chance looking for a problem to solve ???? I'm tempted to say
> you could find other places.

	Well to the extent that gedit spends only ~3% of it's time doing
mallocs, and of those xmlStrndup is no. 4 on the list it seems to me
that there are much better places to look for efficiency problems. :-)

	There was merely the suggestion that it was libXML's fault, and I
wondered if we could easily speed libXML up even more, it appears we
can't - so whatever. I'm not loosing sleep over it, it is the case
however that we use libXML a lot around the place, so if we can could it
up easily we all win lots.

>   Okay got it, I will process it, should not be too hard.

	Thanks for that,

	Regards,

		Michael.

-- 
 mmeeks gnu org  <><, Pseudo Engineer, itinerant idiot