Re: Discussion on source mirroring (with counter proposal)

[Agustín Benito Bethencourt <agustin benito codethink co uk>]

Hi,

On Monday, 19 March 2018 05:47:30 GMT Tristan Van Berkom wrote:
> Hi again.
>
> After rereading this thread; I realize that I have left this in an
> ambiguous state, and have made some assumptions that people can read my
> mind. I want to rectify this and get the ball rolling properly.
>
> This is going to be a fairly long-ish email, but this is necessary,
> given that I've been too ambiguous and unclear.
>
> On Fri, 2018-03-16 at 15:30 +0900, Tristan Van Berkom wrote:
> > Thanks for the writeup !
> >
> > On Thu, 2018-03-15 at 16:30 +0000, Jonathan Maw wrote:
> > > I've been giving some thought on source mirroring, recently, after 
> > > reading the discussion at 
> > > https://gitlab.com/BuildStream/buildstream/issues/179.
> > >
> > > Source mirroring will be valuable for us because:
> > > * The canonical upstream may disappear without warning
> > > * The canonical upstream may be slow to access due to limited 
> > > infrastructure or geographical distance.
> >
> > * The organization may be mirroring everything in a local build farm
> >   - To be sure that their builds are repeatable in 10 years
> >   - To optimize fetch times on build machines
> >   - Without losing the information of what the original URL was
>
> So what are the use cases here ?
>
> This is the most important question to have answered before focusing on
> a single solution.
>
> Jonathan has stated that upstream sources can disappear, and
> downloading from some locations can be slow; my response doesn't really
> add much - so lets build on these two.
>
>
> What are we trying to achieve ?
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Before talking about how to achieve this, I want to pause and think
> about what exactly we want to achieve - I feel that we are not all on
> the same page about this.
>
> I can see two separate ideas of what "mirror" means here:
>
>   A.) Because a specific third party server proves to be unreliable,
>       we want to be able to have a fallback for that server, which we
>       can rollover to in the case that the upstream doesnt work.
>
>       So this is a quick fix / bandaid for a specific pain point that a
>       given organization experiences while using BuildStream; this
>       allows us to have a tarball server for unreliable tarballs, or
>       rollover to a github mirror for an unreliable upstream github.
>
>   B.) For the same reasons, an organization may just never want to
>       experience unreliable access to source code ever again.
>
>       The cost of hosting all sources which their BuildStream projects
>       require on a single server; or even mirrored in some
>       strategically placed locations (so that one can choose a mirror
>       that is geographically closer when building), is a relatively low
>       cost.
>
>       Instead of many points of failure on various servers scattered
>       across the globe, a single point of failure *that is under the
>       control of the organization in question*, is much more desirable.

As a distributor of GPL based code, you have to provide the source code on 
your device or:
* GPLv2 base code, provide a physical copy of the source code (like a CD or 
even paper if required). Offering the source code through a repo does not 
substitute the described requirement although it shows good will.
* For GPLv3 base code the physical format is no longer required if there is 
available the source code to be downloadable from a public network without 
restrictions.

Check this link for details: https://www.softwarefreedom.org/resources/2008/
compliance-guide.html It is from 2008 :-)

Other copyleft licenses has similar requirements.

So any company that ships Linux based systems has to mirror the sources for 
compliance reasons. It is not a choice.

Not building your system out of those mirrored sources makes little sense.

Best Regards

-- 
Agustín Benito Bethencourt
Principal Consultant
Codethink Ltd