Re: [BuildStream] Proposed removal of `workspace reset --soft`

[William Salmon <will salmon codethink co uk>]

Sorry for the very late reply. I hope some of these points will be novel 
/ `more detailed` and helpful. but the TL/DR is that i agree with Jürg.

On 19/11/2019 17:19, Darius Makovsky via buildstream-list wrote:
> After looking at the `--soft` option of `workspace reset` I'm still
> unclear on the exact use-case now that workspaces are being loaded via
> the workspace source plugin. It would simplify support for workspaces if
> this option were simply removed. As I understand it, originally this was
> intended as a way for a developer to execute the configure commands on
> an open workspace without affecting the contents. However, in most cases
> someone would want to do this due to some change in the workspace which
> would result in a different key for that source. In that case the
> workspace would have to be re-cached anyway.
>

As Jürg pointed out we dont want to re run config commands if we can 
avoid them, they can often be very slow. Jürg mentioned the autotools 
family but for big cmake projects they can take ~30min or so. So i would 
like to mirror Jürg's comments of saying that we really do want to cache 
them.

> Related to this is a question about the continued need for the
> `prepared` attribute of workspaces: This is really only used to prevent
> executing configure-commands in a sandbox and again ideally we'd hope
> that in any case where we don't skip this, the key of the workspace
> source has changed. It seems that it would be correct in those cases to
> execute the configure-commands. In my opinion, Buildstream can't
> accurately decide whether the result of executing configure-commands
> will be different in any case other than the source remaining unchanged.
>

I would like to mirror Jürg's point that up until now it has been the 
users responsibility, I would like to also mirror Darius's point that it 
can be confusing. I agree that we should keep this behavior but that we 
should make it clearer that you might want to soft reset.

A good example of the flexibility of workspace's with our current soft 
behavior is https://gitlab.com/BuildStream/buildstream/issues/919

>
> If the source has not changed then I don't think those jobs should be
> run at all.
>

I have mentioned before the case of something like a kernal which might 
take 4hrs but i have never got a kernal build right first time, i almost 
always mess up the install commands, getting the bsp files right always 
take me a few gose and having to wait 4hrs to rebuild something 4 or 5 
times would make this process untenable.

> Will someone please point out to me why these features have been
> retained? It would simplify things if we could just remove them. I
> understand there is some concern about duplication but I think in this
> case we might be asking Buildstream to decide more than can be known and
> things which should be deferred to the underlying build tool.

In sort if you rerun the config commands then the time stamp of the make 
files will be newer than the element retained eg .o files and so some 
build systems will just rebuild everything, added the time of the config 
files the results are that cached config commands are vital but if they 
are cached then sometimes they need to be reset and soft reset seems to 
work ok.


Will