Re: Package manager integration with BuildStream

[Jonathan Maw <jonathan maw codethink co uk>]

On 2018-06-18 17:03, Gökçen Nurlu wrote:
> Hello all,
>
> I pushed a MR that introduces SourceTransform plugin. Although it is
> WIP, I believe a review would help a lot to ensure that it is on the
> right track because it touches some parts of element.py.
>
> The MR also has a copy of an example plugin implementation for go/dep.
> I think it makes it easier to think about what SourceTransform should
> provide to implementors and how. It will be removed before merge (if
> we proceed with this MR :)
>
> Link to the MR: 
> https://gitlab.com/BuildStream/buildstream/merge_requests/505/
>
> Best regards,
> Gokcen
> _______________________________________________
> Buildstream-list mailing list
> Buildstream-list gnome org
> https://mail.gnome.org/mailman/listinfo/buildstream-list

Hi Gokcen,

I've been somewhat out of the loop for the package manager integration, 
so it's possible I'm raising things that have already been discussed, so 
I'll throw these questions out and complete this review later (I might 
have a long period of meetings before I can catch up on the E-mail 
discussion :) ).

I'm also going to avoid picking at specific details.

My understanding of the purpose of SourceTransform plugins is that they 
need to be able to access the previous sources so they can (in Gopkg's 
case) find the configuration file that lists which packages to download.

So, my questions are:

1. Is SourceTransform too specific? Could the ability to check previous 
sources be useful for other sources at some point? If so, we'd be able 
to avoid some complexity.
2. In __ensure_previous_sources() we're fetching and tracking a bunch of 
sources. I'm concerned that ought to be done in the pipeline, instead.
3. This has probably been discussed earlier, but would it be acceptable 
to embed the contents of the Gopkg.toml file into the source's config 
field, instead?
   If the files are too big, I believe there was a proposal for 
buildstream to be able to include loose files into the yaml.
   I suppose the main thing against sticking the package list in the 
project is that you wouldn't be able to track the package list and would 
have to include it manually :-/

Best regards,

Jonathan

--
Jonathan Maw, Software Engineer, Codethink Ltd.
Codethink privacy policy: https://www.codethink.co.uk/privacy.html