GNOME FTP Restructure Issues

[Jeff Waugh <jdub perkypants org>]

Hi all,

So, this simple concept turns out to be a fairly painful problem. The
release team has looked at it about three times so far without a good, final
solution (which, according to some, means it's close to Bloody Impossible).


Here are some of the goals for the FTP changes we need to make (yes, we need
to make changes, please let the status quo go):

  - Move to hard version number directories:
  
    Currently, we have hard stablity directories ("stable", "unstable"),
    which should really be symlinks to proper version directories instead.
    This has resulted in the evil hack of "pre-gnome2" and "earthquake",
    which should have told us to fix FTP a long time ago.

  - Keep the minimal rsync requirements by using lots of symlinks:
  
    We've done this well by having big directories of symlinks for
    human-oriented download directories, and a big pool of tarballs for
    archiving and machine manipulation.

  - Continue archiving source as we've done:

    This is very useful, and works well, but we must also allow mirror
    operators to exclude old versions even if we want to keep them.
    Hopefully ftp.gnome.org in Sweden will continue to host these anyway.

  - Consider release sets (desktop, fifth-toe, office, etc):

    We have lots of software, so we've already separated the release
    strategies, now we should do the same on FTP.

  - Work out what to do with distribution packages:

    We're either crap at this, or it doesn't matter. Is it really important
    to have distribution packages on GNOME FTP? They're seldom updated, and
    the distros are better at handling updates, etc., than we are, so is it
    relevant anymore?

  - Continue to use automated tools for managment:

    Complexity of the tree / number of changes required for each release is
    less relevant with automated tools. I'm happy to continue work on
    install-module for whatever system we decide on - it's more important to
    get the structure right for users (mirror operators and proper
    end-users) than worry about how much work installing a tarball into the
    archive is.


Some of the ideas that various people on the release team have come up with
in the past are listed below. Please read these, but remember that the
important bit is solving the above problems.

  - Keep going the way we are, but rename stable/unstable/etc to their
    version numbers.
    
    That means that the relevant sources are kept in these directories, and
    rotated when there are new releases. Advantages: Little change, keeps
    source together within each version so mirrors can exclude versions
    easily. Disadvantages: doesn't take release sets into account, kind of
    confusing when it comes to pre-releases and their sources (especially
    before the pre-releases are actually released) - where do these go?

  - A small concept in a larger solution: Having a single sources directory
    for the tarball archive, and symlink basically everything else to it.

    Disadvantages: Hard for mirror operators to selectively exclude the bulk
    of the data (tarballs) for old versions, hard to keep module directories
    sane because every version *ever* would be kept in their dirs. One
    solution to that was to put them in subdirs named <major>-<minor> (based
    on the module's version not the release's), although that seems quite
    messy.

  - Use release set names as the root directories, with their versions
    underneath, like this:

    desktop/
      sources/
        gedit/
          (same problem here with the non-versioned source dirs)
      2.0.0/
        sources/
        redhat/

    I'm a bit unsure of making everything rely on the release sets like
    this, because it makes downloading stuff harder (might have to go to
    desktop and platform for a major release), and the sets might change in
    the future. But that means we have to question what we call and version
    the release sets. :-)


If anyone can think of a project that has similar release issues to GNOME
(different types of releases, and versions for each), please point them out
as I'd love to see how they arrange their FTP site. :-)

Comments and ideas encouraged. We will probably use a dirty hack to handle
the 2.0.1 releases, so don't be too worried about time pressures.

Thanks,

- Jeff

-- 
            ...                           *bounce*bounce*bounce*