Re: memory allocations / libxml

[Daniel Veillard <veillard redhat com>]

On Tue, Mar 05, 2002 at 12:24:29PM +0000, Michael Meeks wrote:
> On Tue, 2002-03-05 at 01:09, Daniel Veillard wrote:
> >   Libxml works on a set of platform far larger than glib. 
> >     http://mail.gnome.org/archives/gdome/2002-February/msg00008.html
> >     Enjoy !
> 
> 	Fascinating, AFAIK Dan Winship has got evolution running on MacOS X -
> which presumably entails porting glib, is it possible that there have
> been some portability regressions in glib-2.0 ? The bug report is
> incredibly vague and unusable to the glib team to help fix anything
> though.

  there is another report on that thread about the pain of trying to build it
on RiscOS. I think that report was relatively detailed.
  In both case you have the people email and can ask for more specific
feedback.

> > in a lot of case the length of the buffer ain't available so add an
> > extra xmlStrlen() penalty to your alloca path
> 
> 	In general I was proposing optimizing the path that showed on the
> profile if it could be found; inasmuch that xmlStrndup showed on the
> profile, it might seem that it was an area ripe for some alloca action.
> 
> 	Incidentally it seems that xmlStrdup and most of the xml family malloc
> type string handlers chain to xmlStrndup - thus it might seem that the
> length is ~always calculated.

  Depends, not always.

> 	Of course - quite apart from any clever malloc/alloca tricks, it would
> quite possibly be sufficient to have a stack managed arena, referenced
> from the xmlParserCtxtPtr that provided the equivalent of a more
> portable / ideal alloca type impl. where one could decide the
> granularity of allocation far more effectively, and implement it using
> malloc.

And when that string ends up in a DOM representation which then may be passed
from a document to another, libxml2 just segfault ?

> 	An API like this might suffice:
> 
> gulong         mstack_alloc_begin_scope   (xmlParserCtxtPtr ctxt);
> const xmlChar *mstack_alloc_strndup       (xmlParserCtxtPtr ctxt,
> 					   const xmlChar   *buf,
> 					   guint            length);
> void           mstack_alloc_restore_scope (xmlParserCtxtPtr ctxt,
>                                            gulong           offset);
> 
> 	And require an addition to the parser context of perhaps:
> 
> 	xmlChar *base_buffer;
> 	gulong   base_buffer_size;
> 	xmlChar *cur_ptr;
> 
> 	Oh, and of course assuming we did fairly few reallocs we would have ~0
> malloc overhead, and of course save a lot of 'free' invocations, and get
> more inlining type optimization (perhaps).

  perhaps. Okay if libxml2 is too slow for you. As I said and will repeat
show me the code, have it run in non-trivial apps, prove it's portable and
then I will certainly incorporate it. People from Sun went through that
process for threading support, feel free to go ahead.

> 	How does the above sound ?

  Lot of hard work for still unclear result. Still I take patches
if they look maintainable.

Daniel

-- 
Daniel Veillard      | Red Hat Network https://rhn.redhat.com/
veillard redhat com  | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/