[BuildStream] [Proposal] 1.4 release (and potential further 1.x releases)

[Abderrahim Kitouni <akitouni gnome org>]

Hi all,

As discussed in the meeting, I'd like to backport some features to the 
stable branch, which means a 1.4 release. As this was discussed before 
and led to an agreement of not releasing further 1.x versions, I'd like 
to add my arguments.

The main arguments against further 1.x releases were:
* Buildstream developers aren't interested in them,
* A release implies a freeze, which would slow down development,
* Buildstream is still unstable and we don't want incompatible minor 
versions.

For the first point, we (GNOME/freedesktop-sdk) need features which are 
only available in master. I'm ready to put in the effort needed for the 
release, and I convinced Tristan to help.

For the second and third point, we're planning to branch further 1.x 
releases from bst-1.2 and carefully select features to backport. I 
think we can have small (documented) incompatibilities, especially 
regarding cache key stability, but I'm open to discussing this.

With that out of the way, let me explain my vision for 1.4. I'm posting 
a list of MRs I'd like to backport. I'd appreciate any hints about 
prerequisites/followups to those. I split the list it in 5 categories.

As a plan, I think we should backport existing fixes for the first two 
categories, release a 1.3.1. And try to work on the rest on a 
best-effort basis. I'd like to have a final 1.4 before August for the 
freedesktop-sdk release, but Tristan might want to have it earlier.

* Features we want/need: These are either simple, non-intrusive 
features or features we really want. These are the priority for 1.4

- Allow source plugins to access previous sources 
https://gitlab.com/BuildStream/buildstream/merge_requests/568/
  - followup: Bst track can't track when multiple sources are on the 
same element without ref 
https://gitlab.com/BuildStream/buildstream/issues/1010
- Resolve "Make dependency type default to build" 
https://gitlab.com/BuildStream/buildstream/merge_requests/633
- loader: Allow dependencies to use ":" to refer to junctioned 
elements https://gitlab.com/BuildStream/buildstream/merge_requests/998

- git.py: Make `ref` human readable 
https://gitlab.com/BuildStream/buildstream/merge_requests/291
- Expose base class for Git source plugins 
https://gitlab.com/BuildStream/buildstream/merge_requests/1019
- Expose _GitMirror as part of plugin author facing API 
https://gitlab.com/BuildStream/buildstream/merge_requests/1022

- Allow customisation of max-jobs instead hardcore to a maximum of 8 
https://gitlab.com/BuildStream/buildstream/issues/1033

* Minor incopatibilities: These are features/fixes that bring in small 
incompatiblities (including breaking cache keys). I believe these are 
okay, but we should have them in as soon as possible to define the 
cache keys of 1.4.

- Execution environment reqs 
https://gitlab.com/BuildStream/buildstream/merge_requests/969
- followup: BuildStream doesn't hit the cache when building foreign 
arches https://gitlab.com/BuildStream/buildstream/issues/523
- Symlink fixes 
https://gitlab.com/BuildStream/buildstream/merge_requests/1138
- Do not resolve or mangle symlinks during staging 
https://gitlab.com/BuildStream/buildstream/merge_requests/1140
- Update cache keys to use JSON 
https://gitlab.com/BuildStream/buildstream/merge_requests/1151
- Allow absolute paths in whitelist 
https://gitlab.com/BuildStream/buildstream/merge_requests/968

* Nice to have: These are mostly UI additions, I'd like to have them in 
1.4 if they are believed to be stable, but they can wait for 1.6 if 
time doesn't permit.

- Patch for issue #191 support relative workspaces 
https://gitlab.com/BuildStream/buildstream/merge_requests/504
- Updated Workspace CLI 
https://gitlab.com/BuildStream/buildstream/merge_requests/897
- Support invoking buildstream from a workspace outside a project 
https://gitlab.com/BuildStream/buildstream/merge_requests/924
- Guess the element when running commands in a workspace 
https://gitlab.com/BuildStream/buildstream/merge_requests/954

- Add support for default targets 
https://gitlab.com/BuildStream/buildstream/merge_requests/925
- Add `bst source-checkout` command 
https://gitlab.com/BuildStream/buildstream/merge_requests/820
- Add artifact log command 
https://gitlab.com/BuildStream/buildstream/merge_requests/920
- Introduce new artifact subcommand `delete` 
https://gitlab.com/BuildStream/buildstream/merge_requests/1060 (merged 
manually as ac71ea61)
- Add `--deps build` option to `bst checkout` 
https://gitlab.com/BuildStream/buildstream/merge_requests/819
- Add --remote, -r option to bst build, inline with pull & push 
https://gitlab.com/BuildStream/buildstream/merge_requests/1119
- _frontend: Allow printing dependencies using `bst show` 
https://gitlab.com/BuildStream/buildstream/merge_requests/1121

* Cleanups, refactorings and performance improvements: I believe we 
want all of them, but not necessarily in 1.4. Backporting 
cleanups/refactorings makes it easier to backport features/fixes from 
master.

* Compatibility features for easy transition to Buildstream 2.0: 
Ideally, I'd like to be able to modify the project to be able to work 
with both 1.x and master. I'd also like to be able to use bst master 
UI/UX (all while keeping compatibility with 1.2).

I've left out the details for the last two categories as they can wait. 
I'd like to have at least virtual directories, and UI deprecation where 
we are sure the bst master UI is stable.


Cheers,
Abderrahim