Re: simplifying closures

[Karl Nelson <kenelson ece ucdavis edu>]

[...]

> But I'm saying there may only be one parent.

I was stating what the design plan was, just for clarification.  It
wasn't really clear to others what the use of the lists where. (IMO)


[...]
> > (chaining is a major pain.  You can see chaining used in a number
> > of places in gtk+ currently where the user asked for a connection
> > and we instead create another connection to proxy over to the 
> > first function.) 
> >
> 
> It isn't a major pain in any cases I see. It's just a bit of trivial
> typing, and it's simple to understand. Concrete examples in existing
> GTK+ code?
> 
> Do you agree however, that chaining _can be used_ to always solve the
> problem, unless we have a case where user and library code are each 
> adding separate notifiers. 

That is a very big unless.  I don't see how you can solve the problem
of the object which created the closure used up the invalid and free,
and the connected object now wishs to add something.  It now
must add an intermediate proxy which is twice the cost and then it
needs to know how to chain.  It does not seem like a very nice solution.

Note, that my original closure proposal was for a (invalid, dtor, marshal)
with a data node chain to store the chains to get over the limitation.
But data node chains take more memory than what he implemented because
each data node was one pointer, the malloc overhead and data itself.


> However, if you make the changes I've suggested (no closure reuse, one
> piece of user data) I can't think of such a case. Can you give a
> concrete example of a case where the library would supply a notifier,
> and also the user, given my changes (single parent, single user data),
> which could not be handled by chaining.

Where do you store the chains?


[...]
> I said "we define closure to be connectable to only one parent and
> user data" and you say "this is bad because you can have more than one
> parent or user data."  Not convincing!

You would need to ask the potential users and get their option.  I
personally can live without it, however, I will end up providing it
in libsigc++ anyway.  I merely suggest that multiple parents is
likely going to make the closure system more flexible. 

Is treating closures like function pointers a worthy goal?  If so
than what Tim is implementing is on target. Considering we are trying 
to evaluate the use of all future glib and gtk+ users and not simply the 
current use of gtk+.  I can't say any of us have the proper foresight
to know for sure which is the right path.


> > >    One is saving space; by reusing closures, you can have
> > >    fewer. However I think it's clear that closures can only be reused
> > >    about 5% of the time, and if we can avoid multiple notifiers, we
> > >    can save more space the other 95% of the time.
> > 
> > What numbers to you have to support this?  
> 
> I see very few signal handlers reused in GTK+. It's very rare that
> someone connects the same function to more than one place.

There are few because it isn't possible, thus given your data set
how could you draw any sort of conclusion.  Further, you would
need to study gtk+ applications not gtk+ itself.  How would you
identify in the current gtk+ application code if they had to add
a proxy to handle these cases?  Each author would do it a different
way and it would likely be convoluted enough you couldn't tell.


> > This was not the only reason for reuse of closures.  A classic example
> > of reuse of closures is systems like MDI where an item is used over and
> > over repeatedly be the system.  Everytime the info is used to build a
> > menu the closure gets another connection.  The user may not know that 
> > the closure is being used more than once.  I believe having this
> > ability is likely going to be important.
> > 
> 
> You just create an abstraction such as the GtkAction I've proposed in
> the past, or the verb abstraction in Bonobo embedding. The abstraction
> is better than using closure anyway, because it's appropriate for the
> domain.

I still do not know if GtkAction will solve every case of reusable 
closures.  Further, with GtkAction you are going to have to hold
the closure in the GtkAction.  When you create an object from 
the Action you will need to connect.  This is a clear case of 
multiple objects. 

If closure are not copiable, you will need 2 closures to make the 
connection to one item.  One in the menu to call the GAction, and
the user supplied one in the GtkAction.  Result all GtkActions double
the number of closures.  All GtkActions count as one of the multiple
use cases.

If closures are copiable then each menu item or button you create
you simply pass the closure to their connect function.  One closure
memory is needed. 

Thus all use of GtkAction is going to fall into the case of
multiple senders as anything which can possibly have more
than one must assume it has more than one.



> Closures should be an abstraction of gtk_signal_connect_full(), not an
> attempt to solve all problems ever.
> 
> But even if I say "OK, you could use it in MDI," that's an instance of
> where it would be mildly convenient to do so; it's certainly not a
> necessity. When is it _needed_?

And I ask what would the user have to do in your system if they
need to copy a closure?


In libsigc++, the argument was simply this.  

   (1) A closure is an object.  
   (2) A closure should be act like a function pointer.
   (3) Function pointers can be copied.
   (4) Closures can be derived.  

   (R1) You can't copy a closure directly because that would be a 
        deep copy of a derived object.  ( 2 & 4)
   (R2) Therefore, it must be a ref counted object.
   (R3) Thus the process of binding it to an object must not be 
        destructive, because function pointers can be used 
        multiple times.  (3)
   (R4) If it has only one invalid function then the connecting 
        is destructive.
   =>  We need multiple parents.

Are there any assumptions here you feel are invalid?
I think that if closures don't act like function pointers they
will feel quite restricting and thus be likely require difficult
kludges for get around the limitation.   Therefore, closures will
be basically useless to C coders.   

[...]
> > How does this handle the case when 2 object have a peice of
> > data which points to the closure to tell the closure to 
> > invalidate
> > 
> >     A
> >      \ 
> >       C
> >      /
> >     B
> > 
> > How does the A going away remove B's invalid call?
> >
> 
> You can only have:
> 
>  C - A - B
> 
> or:
>            A
>           /
>  C - Proxy 
>           \ 
>            B
> 
> Because there's only one user data.

Then you missed the point.  (in psuedo code)

  struct MyData {
    GtkWidget *w1,*w2;
  };

  GClosure* make_closure(GtkWidget *w1, GtkWidget *w2)
    {  
      MyData *d=my_data_new(w1,w2)'
      GClosure *c=g_closure_new(&function,d,&g_closure_free_data);
      g_closure_add_dependency(closure,w1);
      g_closure_add_dependency(closure,w2);
    }

Here we have a fairly simple case in which the data is a structure which
holds two objects, if either of them are destroyed we want to 
invalidate the closure.

But you would be best to discuss this with Tim.  I am not
a very strong proponent of multiple data objects.  I believe
much more in multiple callers as that is more common situation.
I agree the case is possible in C.  But, in the next 
version of libsigc++ I am only implementing multiple callers
and not multiple callies because I don't think it is frequent enough.
However, that system is different in that the invalid not the
free does the dependency removal.  Thus I am allowing for 
N invalid and 1 dtor (which is chained automatically by C++).

  
> > Chaining is error prone.  It means that each time you connect you
> > must store the old routine.  Where do you plan to store it?  
> > In other words you just pushed to problem to the user to store
> > where before we stored it in our lists.  In other words, you
> > are going to have to add some sort of data list to store the chains.
> 
> Well, I'm not going to have to do this because I don't think GTK+
> needs to support multiple user data. libsigc++ and other language
> bindings could do it, but that's still in a library, not in user code.
> You might say we should share this between bindings; but I don't think
> we have evidence that other bindings than libsigc++ even want to use
> it, or that the abstraction we're considering will work for them. 

You are completely avoiding specifying what you would do in the 
specific cases I mentioned.  I really don't care whether you see
them as common or not.  If they are realistically possible and
your system doesn't cover them, then I want to know how you intend
to cover them.  Only then can we compare costs fairly.  We can
debate the frequency of occurance, take the cost under both systems
and see which one is best.

For each case give how the memory is freed and how much
runtime overhead is needed.

  1) 2 receivers who can invalidate
  2) 2 callers
  3) caller and then another caller added later 
  4) 2 receivers and 2 callers

The problem I see with chaining without copying or cloning is that
you end up with a stew of proxies all of which you must handle 
dependencies between.  


 
> Concrete examples? 

> Why did they want to reuse, 

This is a reasonably large pool of disjoint reasons.
Because they wanted to copy signals to make their code simpler.  
Because C++ STL requires that objects in lists be copiable.  Because
they needed to transfer ownership of a closure from one place to
another in an event queue.  Because it would allow simpler binding
of keys, menus, and buttons in a XML builder for gtk--. 

> what did it get them,

Flexiblity to use the system without needing to write their
own proxies on the reasonably common case.  Ablity to meet
STL specifications.  It just made a lot more sense to have
closures not be destroyed on connection.

> and how bad was the alternative way of coding it? 

Alternative was very bad, it meant the user needed to understand all
the interactions to write a proper proxy.  What happens if the closure 
goes first, what happens if the caller goes, what happens if the callie 
goes.  It was more than most users wanted to handle because the ended up 
looking at the goo of libsigc++ dependency code.


> I have to say I don't find this too much different:
> 
>   Slot s = slot (obj, &method);
>   connect (s);
>   connect (s);
> 
> vs. 
> 
>   connect (slot (obj, &method));
>   connect (slot (obj, &method));
>
> it's just not that bad a problem.

That is a vast oversimplification because you are assuming the 
user who created the closure also did the connect.  The whole
point of closures is to create the closure in one location and
then pass it to a districtly different location which it gets connected
such as the case of a builder such as menus and XML. 

The binding creates a closure.  The closure gets passed to a function.
The function wants to connect that closure to two things.  It can't
go back and ask for a second copy form the user.

A simple example of this is a XML file where you have a closure which
you want to connect to a menu item and a button symbolicly.
 

> > I think you are also missing a major point of a library.  The
> > point of a library is to solve hard problems so that the user
> > doesn't have to.  Thus no matter how complex the inter workings
> > of the implementation, so long as the interface is simple
> > it is good.
> 
> There are a number of problems with complexity in a library.
> 
>  - it locks you into a larger interface which is harder to
>    keep compatible, both because it's larger, and because
>    it has subtle interactions with the rest of the API
It is percisely interactions which we are trying to define.  If
you don't provide it and some user attempts to add it later the
interactions will be horrible.

>  - it takes maintainer time away from implementing other stuff
If you have some issue with Tim regarding his time, I suggest
talking to him directly.  I volunteered to produce and implementation
of what I suggest for review, but it was ahead of the time table. 
Unfortunately, since then I lost a maintainer and my time allocation
became smaller so I can't contribute now. 

>  - it's hard to maintain, and typically buggier
>  - it is more bloated
This is arguable.  Under your approach every problem case at
least doubles the cost.  Under his approach the uncommon cases
are at little addtional cost over the general case.   

>  - 95% of the time, implementation complexity leaks into the 
>    interface
This is a pointless generality.  95% of all statistics are made up.  :-)

These are very fuzzy arguments.  Maintainence time, implementation
complexity, etc are not quantitative.  Libsigc++ implmentation has 
been accused of being extremely complex to the point that no one
besides the author mucks with the insides.  At the same time, the
total maintainence of the package has amounted to almost zilch as
have the total number of bugs.  Asside from rather extravagant
feature requests and rare compilers, it needs no alterations at
all.

This system is very similar to the libsigc++ in this regard.
I think the idea is that this closure system is to be a largely
closed system.  It is extendable by deriving but for all C users
it should be a black box.   

If there are any reasonable numbers of cases where you need multiple
parents or multiple data objects then it is worth the complexity
to build the system with it now.  Otherwise, you will do just like
I am doing with libsigc++ and have to do it over again.   

 
> There are many tradeoffs which add up to "complexity bad." Here we are
> talking a major increase in complexity, with no demand _at all_ from
> people using the GTK+ C interfaces to write apps. I haven't seen a
> single post asking for any of these features. However there are many
> frequently-requested features we haven't gotten around to.

As to your "no demand" I would ask how could you tell.  There is
no current concept of a closure and thus there is no way that
someone would request copying one.  The closest thing to a closure
is the UIInfo action and it needed to be copied.   In short, your
assertion is flawed.  You are saying no one has yet requested
closures be copiable when closures don't exist.  


> For language bindings, I think we should provide what they _need_,
> it's OK if there's a bit of cruft involved in the language binding
> code.

The old system was workable but had a rather large pile of cruft and
thus many functions were not usable.  Since we don't plan to redo
it a third time we need to get it right now.  I gave my analysis
to Tim and Owen on what the current system did and how to 
generalize it.  I can't say for sure that signal copying will
get used in gtk+ nor can I say the multiple data objects will get
used either.  However, without asking the users I don't know how
you could either.  I can say that from what you describe if either 
of those features are missing it would mean a lot of kludging on the 
users part.   


You initially were arguing the memory aspect size.  If you want to do that
then I say estimate the cost of doing the action with Tim's system,
and with your system and then assign probablities.  Given the way
your system works you must add a new closure and special functions which
know how to chain and store the chain function probably in a derived
closure type.  This means every special case at least doubles the 
memory usage and requires a lot from the user, which likely overshadows
what little is saved. 

--Karl