Asynchronous update and recalculation rate-limiting

["Bill Rugolsky Jr." <brugolsky telemetry-investments com>]

Hi,

I'm attempting to move from Applix to Gnumeric, and in the process
I want to enhance Gnumeric to handle the following:

  1. Slow synchronous function evaluations (e.g., a Monte Carlo
     evaluation, or invocation of an external helper program via
     system() or popen()) shouldn't starve the user interface.  The
     function need not be CPU-bound, but might have high latency
     (e.g., RPC call).  In particular, a sheet might contain several
     high-latency function evaluations that are not (locally) CPU-bound.
     If split into a request and asynchronous notifier callback,
     this would have "one-shot" semantics.

  2. Fast asynchronous real-time data, necessitating recalculation
     throttling, particularly when invoking "slow" functions.  This
     would have "update" semantics.

Is there an existing plan or roadmap for this functionality?  If not, are
there strong preferences as to how such queuing ought to be implemented?

I could hide much of the asynchrony outside of Gnumeric proper using
plugins to talk to separate processes that have their own data cache,
etc., but it seems more useful to provide the mechanism directly in
Gnumeric so that, e.g., stale values can be highlighted, and CPU
consumption can be controlled manually or via a feedback mechanism.

I've yet to study the details of the control loop and recalculation
logic, but am encouraged by the overall design of the cell objects,
generation counters, and recalc queuing.

Concurrency is an important performance issue, since SMT and SMP are
increasingly common, but given the portability headaches that threading
entails, I'd prefer to take account of it in the design, but ignore
it in the implementation for now, by pushing responsibility onto
the plugins.

My goal is not to turn Gnumeric into an OLAP engine (!),  but to
provide a usable framework so that non-experts can use the plugin
scripting framework to inject a data feed, or perform high-latency
calculations.

Regards,

        Bill Rugolsky