Re: [BuildStream] Re-using top level elements

[Chandan Singh <chandan chandansingh net>]

Hi,

I must say I am feeling a bit lost on this thread. I am not really
sure whether we are discussing a problem, a proposal, or both.

I have some comments inline based on whatever little I do understand.
But, I think this thread can definitely benefit from stating the
problem statement more concisely and clearly.

> First I'm going to say that, while I have some context from private
> conversations to have a clue, I find the term "reusing elements" used
> here confusing.
>
> While artifacts and plugins are things to be reused and shared among
> projects, elements themselves are not, they are a project's mechanism
> of declaring an 'Element' (instance of a plugin).

+1

> I would not recommend trying to use includes to accomplish any
> templating, and I would not recommend using includes to define
> dependency lists (if using a cross junction include to define a
> dependency works at all, I would be quite surprised).

I generally agree with what Tristan said.

That said, another way to look at this could be via plugins. The
project can encapsulate the complex logic in a plugin so that the
downstream elements can just that plugin and fill in the blanks via
configuration options, if needed.

The story around using junctioned plugins is already getting nicer by
allowing one to use plugins declared in a subproject. We can make this
even nicer by implementing the default dependencies for plugins, which
can encapsulate the dependency lists for such a plugin.

> What I think you are driving at, in my own words, is that basically:
>
>   * There are some constructs which can be difficult or require
>     a lot of setup to express
>
>   * Sometimes these constructs require multiple elements, for instance:
>     - Build some special extra tools which allow one to deploy an image
>     - Compose the filesystem data which needs to be deployed into that
>       image
>     - Write a final script element, like x86image as an example, which
>       uses these tools to create the image
>
>   * It would be nice to be able to share these constructs across
>     projects in some way, such that the more complex multi-element
>     construct can be expressed with a simple declaration, safely.
>
>   * Especially, if I have a low level project which does provide
>     some artifacts which allow building the output, I should be
>     able to provide that construct to a project which junctions
>     my project.
>
>     Providing this from a lower level project appears to be something
>     safe, because I even provided builds of the required artifacts to
>     accomplish this task.
>
> Am I pretty much interpreting this correctly ?

I realize that this question wasn't for me, but I'm assuming this to
be correct for the remainder of my message.

> To bring back the x86image example, imagine a world where:
>
>   * Freedesktop SDK provided the x86image plugin, but this plugin
>     would define the whole construct, exposing some parameters of
>     how to apply `compose` configurations to the input, and
>     some parameters about filesystem image size, etc.
>
>   * Freedesktop SDK also provided artifacts allowing x86image
>     scripts to work (e.g. syslinux, coreutils, mkfs etc).
>
>   * Projects could then depend on Freedesktop SDK for the purpose
>     of rolling out bootable x86 images, regardless of whether they
>     even use Freedesktop SDK in the payload they are distributing.
>
>   * Now what if the plugin provided by Freedesktop SDK could be
>     "bound" to Freedesktop SDK in some way, such that it could
>     implicitly depend on elements from that project ?
>
>     This would allow the plugin to automatically depend on artifacts
>     from Freedesktop SDK for the purpose of preparing a sysroot in
>     which the commands it performs will succeed.
>
> This last point would also be quite powerful, as you point out, there
> is no way for a plugin to have knowledge about an element or even how
> that element can be addressed, because the plugin does not have a
> junction to the project from where it was provided from (the concept of
> a project where a plugin was provided from doesn't even exist).

I agree it'll be nice if a plugin was able to depend on elements in
the project where it was declared. For example, if we implement the
default dependencies for plugins, we can say that those dependencies
will always be resolved in the context of the project that defined the
plugins.

To continue this example, the plugin provided by the FSDK could list
all its requirements as default dependencies. Then the projects
consuming it can just use this plugin without worrying about how to
satisfy those dependencies.

Take this with a grain of salt as well, as it's not fully thought through :)

Cheers,
Chandan