Re: Plan of action for source mirroring (client-side)

[Agustín Benito Bethencourt <agustin benito codethink co uk>]

Hi Jonathan,

On Tuesday, 15 May 2018 16:29:10 CEST Jonathan Maw wrote:
> Hi,
>
> This E-mail is a summary of what's planned for the source mirroring
> work. I feel it's important to have because Valentin and I are working
> on this in parallel and need to be sure we're working with the same
> assumptions.
>
> The main point of synchronisation is ensuring that the place I look for
> mirrors generated by buildstream is the same place that Valentin stores
> mirrors.
>
> To this end, I will do the following:
> * Create a new class of error, RefNotFoundError, which subclasses from
> SourceError.
> * When fetching from buildstream-generated mirrors, it will have the
> format {URL}/sources/{source_kind}/{URL_directory_name}/{mirror_number}
>    where {URL} is the URL of the buildstream mirror generated via config,
>    {source_kind} is the kind of the source (as specified in the yaml's
> "kind" field),
>    {URL_directory_name} is the URL specified for the source, mangled into
> a valid name for a directory (e.g. git://github.com becomes
> git___github_com),
>    {mirror_number} as a number starting from 0, and incrementing every
> time fetching/tracking returns a RefNotFoundError instead of a general
> SourceError.
> * To enforce part of this functionality / make it easier to change
> later, I will add a helper method to Source, e.g.
>    Source.get_normalised_mirror_path(upstream_url, *, prefix="",
> suffix="")
>
> Please let me know if there are any ambiguities / things I've forgotten
> to discuss.

I would only add that the outcome of this thread should be added to https://
gitlab.com/BuildStream/buildstream/issues/328

> Thanks,
>
> Jonathan
> _______________________________________________
> Buildstream-list mailing list
> Buildstream-list gnome org
> https://mail.gnome.org/mailman/listinfo/buildstream-list


-- 
Agustín Benito Bethencourt
Principal Consultant
Codethink Ltd