Re: [BuildStream] Proposal: Moving UI into a subprocess

[Phil Dawson <phil dawson codethink com>]

Hi Tristan,

Sorry about the slow response, I was convinced I pressed send on this 
mail on Friday.


On 24/05/2019 08:37, Tristan Van Berkom wrote:
<snip>
> > > > # Interactive UI prompts
<much snip>
> > > > It looks like I was overcomplicating things somewhat here.
> > > >
> > > > Have I understood correctly, that only difference with how we're doing
> > > > things now would be that the callbacks to the front-end would be only
> > > > difference here via an IPC queue rather than a plain method call as they
> > > > are now?
> Approximately yes.
>
> Just to iron out some more specifics about this:
>
>   * The frontend would finally no longer know about the scheduler at
>     all, which has been desirable for a long time (the last part of the
>     deep core which the frontend unfortunately had knowledge of).
>
>   * The frontend would receive some of the callbacks from stream instead
>     of from the scheduler directly
>
>     Note that this part interestingly automatically buys us the Jobs
>     abstraction discussed in this email which I already referred to in
>     this thread:
>
>     https://mail.gnome.org/archives/buildstream-list/2019-April/msg00063.html
>
>     I.e. the Stream will still need to declare a data structure to hand
>     over to the frontend in order to describe an ongoing "job", and the
>     frontend app (logger and status bar) will no longer ever know
>     anything about a job or scheduler.
>
>     This would automatically be a nice step forward for improving
>     logging consistency as described in that thread.
>
>   * The frontend itself (as in the code living in the _frontend/
>     directory), doesn't ever deal with the IPC, that is all abstracted
>     away by Stream which provides convenient callbacks for the frontend.
*Nod*
> In addition to the above, there is the important detail that you
> pointed out about "what does the scheduling process do when a job
> fails".
>
> Interestingly, in a setup where the frontend and scheduling processes
> are separate, there is no implied blocking here.
>
> As I mentioned in the previous reply, probably the scheduler has to
> clearly advertise it's behavior in this case to the frontend, maybe it
> stops queuing more jobs until instructed what to do by the frontend.
Right, and the front-end will find out about this through the stream.
> This is because we would want the frontend to have a chance to ask the
> user what to do before spawning more jobs.
>
> [...]
> > > Besides this, I think that any "passing terminal control between
> > > processes" ideas need to be proven possible before we can consider this
> > > approach to be plausible.
> > >
> > > > Another approach would be to simply special case the `Stream.shell`
> > > > method, so that it does not run in a subprocess. The shell only
> > > > displays loading and staging messages and as so far as I can see will
> > > > never be rendering a large number of messages. While I'm not really a
> > > > fan of special casing one of the `Stream` methods, this would have the
> > > > advantage of reducing the complexity of the implementation.
> > > Right, this is the simple but impossible approach.
> > I'm not sure If I'm misunderstanding your point or if I didn't
> > adequately articulate mine.
> >
> > I'm struggling to see why this is impossible. If the scheduling process
> > is never forked, then it the shell will be being launched in the main
> > process and so will work almost exactly as it currently does, ignoring
> > any changes to the details message propagation from the rest of the
> > proposal.
> Ok so I may have been overestimating the challenges here, there are a
> few approaches which could work now that I think further on it.
>
> First, keep in mind that we are not talking about `bst shell` here, we
> are talking about the scenario where:
An important clarification. It's a feature I've not had much to do with, 
so this reminder is useful.
>    * A build fails
>    * The user is asked what to do
>    * The user decides to launch a shell to debug the failed build
>
> In this case, the frontend naturally needs to know the up to date build
> graph in order to launch that build shell.
>
> However, in this case it may be possible to obtain only the minimal
> information needed in order to launch a shell, which consists mostly
> of:
>
>    A) The resolved environment variables which are active for the
>       failed element
>
>    B) The elements resolved cache key
>
>    C) A directory where the element is already staged
>
> The thing about (C), as I understand it this doesn't exist anymore.
>
> We refactored this now that we cache failed builds, so instead we
> reconstruct that environment on demand instead of littering the
> filesystem with failed build directories.
>
> That said, maybe we could have something special in place for this,
> like maybe we could ask the scheduling process to setup the build
> directory for us, and then launch the shell in the frontend ?
>
> [...]
> > > Maybe:
> > >
> > > * We need to keep the initial loading process before the initial fork()
> > >
> > > * At the time of an interactive shell (or at any time the frontend may
> > >     ever *need* to know state for any other reason), we can have the
> > >     frontend download the state of an element recursively from the
> > >     the scheduling process.
> > >
> > > I think this might be the middle ground which allows us to move
> > > forward.
> > This also seems like an avenue worth exploring, especially if the above
> > is indeed impossible.
> Right, this is another way we could satisfy the frontend's need to
> launch a shell in mid-build, seems like either downloading all state,
> or getting the scheduling process to do the work of preparing a staged
> build sandbox on the frontend's behalf before shelling into it (as
> described above), are plausible avenues for this.

Indeed, we've discussed a few options now, so I think it's time for some 
experimentation :)

It sounds like we're in agreement now at a high level at least. I think 
we're at the point where it makes sense to start work on  a proof of 
concept, so we can validate that a) implementing the interactive shell 
in this approach is definitely possible and b) we actually see the 
performance improvements we're hoping for.

I'll report back on how this goes.

I've raised https://gitlab.com/BuildStream/buildstream/issues/1036 in 
which I plan to summarise the discussion here and will add extra detail 
as the experiments progress.

Cheers,

Phil

--
Phil Dawson, Software Engineer                          Codethink Ltd
https://www.codethink.co.uk/
We respect your privacy. See https://www.codethink.co.uk/privacy.html