[Notes] [Git][BuildStream/buildstream][jjardon/out-of-tree] 1042 commits: .gitlab-ci.yml: Use latest BuildStream Docker image

[Javier Jardón <gitlab mg gitlab com>]

Title: GitLab

Javier Jardón pushed to branch jjardon/out-of-tree at BuildStream / buildstream

Commits:

• 8c0aaa1c
  by Sam Thursfield at 2017-12-21T14:32:24Z

  .gitlab-ci.yml: Use latest BuildStream Docker image
  
  Integration tests were failing as the installed version of BuildStream
  didn't support the new --hardlinks option.
  

• dda1ebe4
  by Angelos Evripiotis at 2017-12-22T13:20:06Z

  utils._call: fix doc typo
  

• eae6d5c7
  by Angelos Evripiotis at 2017-12-22T13:24:07Z

  utils._call: fix race condition on 'process' var
  
  Tested by inflating the chance of hitting the race. First, insert a
  sleep between opening the terminator context and starting the process,
  then:
  
      python3 -c 'import buildstream.utils; import os; \
          buildstream.utils._call(["echo", "hello"], True); \
          print(os.getpid())' & sleep 1; kill $!
  

• ac2bcb2c
  by Angelos Evripiotis at 2017-12-22T13:29:00Z

  utils._call: rm unused assignments to exit_code
  

• 78fc8440
  by Angelos Evripiotis at 2017-12-22T19:45:51Z

  _signals.terminator_handler: wrap handlers in trys
  
  It's possible for the custom termination handlers to raise exceptions.
  Indeed this was actually the case for utils._call(). They're especially
  difficult to reach with tests, so make extra effort to insulate them.
  
  Print any exceptions encountered as a regular traceback before exiting.
  

• e5eb337a
  by Tristan Van Berkom at 2017-12-22T23:05:48Z

  .gitlab-ci.yml: Build documentation in every CI pipeline
  
  Build it in one stage before the `pages` target, and then only
  make the deployment step conditional on the branch.
  

• 3c75781b
  by Tristan Van Berkom at 2017-12-29T23:42:20Z

  Exceptions refactoring
  
  Outline of changes to exceptions:
  
    o Now BstError base class has a `domain` and `reason` member,
      reason members are optional.
  
    o All derived error classes now specify their `domain`
  
    o For LoadError, LoadErrorReason defines the error's `reason`
  
    o Now the scheduler `job` class takes care of sending the error
      reason and domain back to the main process where the last
      exception which caused a child task to fail can be discretely stored.
  

• 86b12964
  by Tristan Van Berkom at 2018-01-01T10:39:09Z

  _pipeline.py: Added context to some errors
  

• 913fcec6
  by Tristan Van Berkom at 2018-01-01T10:39:09Z

  plugins/sources/patch.py: Added context to some errors
  

• fbdb5118
  by Tristan Van Berkom at 2018-01-01T10:44:53Z

  tests/testutils/runcli.py: Augment the Result object to store task errors
  
  Before this, we only stored the last exception to have been raised in the main
  process, now we additionally provide some Result members allowing tests to
  inspect a machine readable error `domain` and `reason` describing why
  a task has "failed".
  
  This adds some new APIs to the Result() object for tests:
  
    assert_main_error() - asserts the nature of the main buildstream error
    assert_task_error() - asserts the nature of the error from a child task
  

• c6df92fc
  by Tristan Van Berkom at 2018-01-01T10:44:53Z

  tests/sources: Use new error checking tools for source tests
  
  Also added a test case for the `patch` plugin which checks for graceful
  failure when the specified patch file is not a regular file (but a block
  device or a named pipe instead).
  

• 6989b443
  by Tristan Van Berkom at 2018-01-01T10:44:53Z

  tests/cachekey/cachekey.py: Use new error checking
  

• c1b3a483
  by Tristan Van Berkom at 2018-01-01T10:44:53Z

  tests/format: Use new error assertions all around
  

• 54265b7a
  by Tristan Van Berkom at 2018-01-01T10:44:53Z

  tests/frontend: Updating tests to use new error checks
  
  This also fixes #177 - the problem here was solved simply
  by passing the project directory to `cli.run(project=project...)`
  

• 8b60c55e
  by Tristan Van Berkom at 2018-01-02T08:01:41Z

  source.py: Raise proper SourceError() when failing to create the staging directory
  
  This changes the UX to report a better human readable error, which
  is otherwise a BUG message with stack trace.
  

• 75c09207
  by Tristan Van Berkom at 2018-01-02T08:38:55Z

  plugins/sources/patch.py: Removing unneeded check
  
  The patch plugin was checking if the target directory exists, however
  this is automatically guaranteed by the Source abstract class and
  documented to be guaranteed as well.
  
  Since this error cannot be caught by the plugin (it will be caught
  in advance by the Source class), removing the check from patch.py.
  

• ed9a94a4
  by Tristan Van Berkom at 2018-01-02T08:38:55Z

  plugins/sources/local.py: pragma nocover on unreachable code
  
  The local plugin is always Consistency.CACHED, this means that
  fetch(), set_ref() and get_ref() methods will never be called.
  
  Instead of omitting them, just "pragma: nocover" on the `pass`
  statements, making our coverage report more realistic.
  

• 6f0b620f
  by Tristan Van Berkom at 2018-01-02T08:38:55Z

  tests/sources/local.py: Testing failures to create staging directories
  
  Seems that local.py is an appropriate plugin to use for testing
  errors which originate from the abstract Source class.
  
  This test checks that we raise the appropriate error in the case
  that we attempt to stage to a directory that is a regular file.
  

• 5084e662
  by Tristan Van Berkom at 2018-01-02T08:38:55Z

  tests/sources/local.py: Adding test for staging a symlink.
  
  This has remained without test coverage mostly due to upstream
  pytest-datafiles bug https://github.com/omarkohl/pytest-datafiles/issues/1
  

• c8b3b463
  by Tristan Van Berkom at 2018-01-02T08:56:54Z

  plugins/sources/_downloadablefilesource.py: Exclude a line from coverage
  
  When fetching a downloadable source, we make a defensive check
  to avoid redundant download at fetch() time by checking if it's
  already cached, but fetch() will never be called if the source
  is already cached.
  

• dd92c25b
  by Tristan Van Berkom at 2018-01-02T10:05:50Z

  .gitlab-ci.yml: Force colors in pytest for better output in the gitlab UI.
  

• f0d03c82
  by Tristan Van Berkom at 2018-01-03T08:35:32Z

  Added MAINTAINERS, NEWS and BuildStream.doap
  
  Some basic module metadata before publishing the 1.0 release.
  

• 07e4d543
  by Tristan Van Berkom at 2018-01-03T09:31:04Z

  setup.py: Removing explicitly set version, we use setuptools_scm for this.
  

• 2df05bd1
  by Tristan Van Berkom at 2018-01-04T09:46:57Z

  source.py & element.py: Fixing SourceError() and ElementError() constructors.
  
  Recently I added the `reason` member which can be used to set
  machine readable error reason strings for the purpose of testing.
  
  Forgot to add the necessary `*` argument, forcing `reason` to be
  a keyword-only argument.
  

• f54fe9a5
  by Tristan Van Berkom at 2018-01-04T09:48:31Z

  doc/source/install.rst: Silence a docs generation error
  
  Fixes the warning:
  
     WARNING: /path/to/buildstream/doc/source/install.rst:45: (WARNING/2) Title underline too short.
  

• 9870fcaa
  by Sam Thursfield at 2018-01-04T12:30:12Z

  utils.py: Add save_file_atomic() helper
  
  This is a context manager that can be used to divert file writes into a
  temporary file, which is then renamed into place once writing is
  complete. It is primarily intended for use by source plugins which
  download files, so they can ensure that their downloads appear atomic
  and there is no risk of leaving half-downloaded files in the cache.
  
  So far this is not used in the core, but it is needed by the Docker
  source plugin that is proposed for the bst-external plugins repo. See:
  https://gitlab.com/BuildStream/bst-external/merge_requests/9
  

• bb8f807b
  by Sam Thursfield at 2018-01-04T12:30:12Z

  Use save_file_atomic() when writing source-bundle scripts
  

• 3628cbbe
  by Sam Thursfield at 2018-01-04T12:30:12Z

  Move utils._generate_key() into a new 'cachekey' module
  
  This avoids a circular dependency between the 'utils' and '_yaml'
  modules.
  

• c666e70c
  by Sam Thursfield at 2018-01-04T12:30:12Z

  _yaml.py: Use save_file_atomic() when writing out YAML
  
  This makes writes done by `bst track` atomic, meaning a crash or power
  loss while overwriting a .bst file should never cause the the contents
  of that file to be lost or partially written.
  

• 4625bfb0
  by Tristan Van Berkom at 2018-01-04T15:25:34Z

  Documentation: Fix warnings, and make all warnings errors.
  
  Some of the warnings from sphinx-build are really just warnings,
  but a lot of the things we want to avoid and really break documentation,
  like broken internal references and some invalid rst directives should
  really be errors.
  
  Now we treat all warnings as errors, this should ensure that
  any commits landing upstream never break the docs.
  

• c9551eab
  by Tristan Van Berkom at 2018-01-04T15:25:34Z

  Documentation: Removing the sphinx-build3 script
  
  Instead add a comment about why this is really there,
  and invoke sphinx python modules with python3 directly.
  

• 390232ee
  by Javier Jardón at 2018-01-05T09:54:08Z

  doc/source/install.rst: You need python3-ruamel-yaml package in Fedora
  

• 4427d830
  by Tristan Van Berkom at 2018-01-06T08:39:48Z

  test/utils/savefile.py: Stringify the tmpdir so that tests work.
  
  It looks like this newly added test assumes the user has a very recent
  version of pytest, which supports treating the `tmpdir` fixture like
  a string.
  
  A reasonable alternative to this patch would be require at minimum
  a version of pytest which supports this newly introduced API.
  

• 328e9ab0
  by Tristan Van Berkom at 2018-01-06T09:16:01Z

  tests/pipeline/load.py: Removing unused tmpdir fixture from here
  

• 8a2f75ed
  by Tristan Van Berkom at 2018-01-06T10:26:43Z

  tests/testutils/runcli.py: Assert that buildstream actually exited.
  
  Enhanced the error checking Result() methods to always assert that
  the CLI actually exited, there are no cases worth testing for where
  buildstream would be expected to exit on an unhandled exception.
  

• 5905760a
  by Tristan Van Berkom at 2018-01-06T10:26:43Z

  local & patch source tests: Preflight errors now report PipelineError
  

• ee8cfafb
  by Tristan Van Berkom at 2018-01-06T10:26:43Z

  _frontend/main.py: Fix issue #180 - Fail gracefully when preflight fails
  
  This also makes main.py less redundant, for some reason there was a stray
  call to app.pipeline.initialize() beside app.initialize() in every function,
  instead move that into app.initialize() where we now handle the error
  and exit gracefully.
  

• f1183059
  by Tristan Van Berkom at 2018-01-06T10:26:43Z

  _pipeline.py: Raise detailed pipeline error at preflight time
  
  If a plugin raises an error, prepend the plugin identifier to the
  error message and raise PipelineError.
  

• d35896e1
  by Tristan Van Berkom at 2018-01-06T10:26:43Z

  tests/pipeline/preflight.py: Added test to ensure graceful exit at preflight error time
  

• 6b5ba80f
  by Tristan Van Berkom at 2018-01-08T07:39:58Z

  doc/source/install.rst: Add a note here about pip's `-e` developer option
  
  It's more suitable to add this note here than to have a separate section
  in the HACKING.rst for this.
  

• faf67314
  by Tristan Van Berkom at 2018-01-08T08:04:25Z

  HACKING.rst: Added a section on feature proposals and patch submission
  
  Also removed some things which are either of little importance, not
  worth mentioning, or covered already by the buildstream user documentation.
  

• 797135db
  by Tristan Van Berkom at 2018-01-08T13:01:28Z

  doc/source/projectconf.rst: Documenting plugin loading section.
  
  This fixes issue #183.
  
  Also move the `format-version` related documentation to the first
  section "Essentials", beside the project name and element path, since
  this is a quite global option it belongs here and not hidden away
  with the plugin loading documentation.
  
  Also adjust the main index.rst to include the plugins sub-section
  as an adjacent sub-point of the project configuration (consistent
  with other project configuration sections).
  

• 37562b10
  by Javier Jardón at 2018-01-08T16:52:09Z

  doc/source/install.rst: BuildStream depends on 'fuse' (for fusermount) and libfuse
  

• ab05f80d
  by Tristan Van Berkom at 2018-01-10T10:57:43Z

  utils.py: Make list_relative_paths() report sorted paths.
  
  This is needed because plugins make use of this function to
  generate a cache key which must be stable.
  
  In addition to the above, this patch also mitigates the performance
  hit of sorting, and allows _process_list() to function to iterate
  through the generator when additional sorting is not needed.
  
  This patch is an enhanced version of Sam Thursfield's patch on MR !216
  

• 8c8880bb
  by Tristan Van Berkom at 2018-01-10T13:33:52Z

  plugins/sources/local.py: Consider symbolic link target in cache key.
  
  Technically this breaks cache keys for the local source, but as
  this comes in a branch which fixes local source cache keys to be
  stable (they were random before this branch), we wont bother with
  considering this enhancement a separate API break, the cache key
  breakage was inescapable anyway.
  

• 4abd4382
  by Tristan Van Berkom at 2018-01-10T13:33:52Z

  tests/cachekey: Enhanced tests to consider local sources with multiple files.
  
  We had a cache key instability issue with local sources generating
  inconsistent cache keys due to iterating over their files in
  a random order. Tests did not catch this previously due to the local
  source sample only using one file to iterate over.
  

• 8d418c71
  by Tristan Van Berkom at 2018-01-10T13:33:52Z

  tests/cachekey/cachekey.py: Manually create a symlink for our tests
  
  This works around an inconsistent behavior with setuptools.
  
  Newer versions of setuptools fail to preserve symbolic links when
  creating a source distribution, meaning that tests run from the
  dist tarball will fail.
  

• fdb2aaf6
  by Jonathan Maw at 2018-01-10T15:16:48Z

  Fix inconsistency in debian stretch install instructions
  

• ab303aa6
  by Tristan Van Berkom at 2018-01-11T12:54:36Z

  plugin.py: Removing Plugin.error() while it's still early and nobody is using it.
  
  Errors are fatal, as such plugins must raise either SourceError() or ElementError().
  
  This method was never used in plugins so far, lets remove it even
  though we are technically breaking (unused) API for the overall
  betterment of the API.
  

• b07ca524
  by Tristan Van Berkom at 2018-01-11T12:56:01Z

  _scheduler/trackqueue.py: Dont use Plugin.error(), use Plugin.warn() for now.
  
  This is already problematic as this particular error is
  reported but fails to fail the session correctly.
  
  Documented this in issue #187
  

• 6af85a86
  by Tristan Van Berkom at 2018-01-11T12:58:02Z

  _message.py: Remove MessageType.ERROR, we have no need for this.
  

• c7ed4376
  by Tristan Van Berkom at 2018-01-11T13:40:15Z

  _pipeline.py: Only raise the error for inconsistent pipeline
  
  Dont make the ERROR message explicitly anymore, leave that
  to the frontend to sort out.
  

• 9e23c285
  by Tristan Van Berkom at 2018-01-11T13:40:25Z

  _exceptions.py: Added `detail` member to base BstError class.
  
  Now any BuildStream exception can potentially add detail to
  the errors they raise.
  
  Allow detail strings already in PluginError()
  

• a79cc0d0
  by Tristan Van Berkom at 2018-01-11T13:40:39Z

  element.py: Allow specifying detailed message in ElementError()
  

• 8efdbad3
  by Tristan Van Berkom at 2018-01-11T13:40:39Z

  source.py: Allow specifying detailed message in SourceError()
  

• 03ce25dc
  by Tristan Van Berkom at 2018-01-11T13:40:39Z

  _frontend/widget.py: Change how we format failure messages.
  
    o Now print the message itself on the same line in place of the abbreviated log file
  
    o When reading back the last lines of the log file, prefix that
      with an informative message about whence the log lines come,
      showing the full unabbreviated log file.
  
    o Fixed the behavior of read_last_lines() so that it works as
      intended, problems were:
  
      - Was prepending an additional newline when the log contains
        more lines than we want to print
      - Was not printing anything at all in the case we want to
        print more lines than exist in the log
  

• c4c9a48e
  by Tristan Van Berkom at 2018-01-11T13:40:39Z

  _frontend/main.py: Making error handling more consistent
  
  Added app.print_error() to consistently print an error while
  running the pipeline stuff.
  
  Also make bst push & pull commands more consistent with
  the rest, and print the summary at the end as expected.
  

• 7a9c1998
  by Sam Thursfield at 2018-01-11T17:07:34Z

  tests/frontend: Assert cached state in a single line
  
  Minor tweak to hopefully make the test cases a bit shorter and more
  readable, in preparation for adding more.
  

• d7959bd4
  by Sam Thursfield at 2018-01-11T17:13:10Z

  Add support for multiple remote caches
  
  This extends the 'artifacts' configuration block such that a list of
  `url` mappings can be given instead of a single entry. For example:
  
      artifacts:
        - url: http://example.com/artifacts1
        - url: ssh://ostree example com/artifacts2
  
  The OSTreeCache class is updated to set up multiple remotes and query
  remote refs from all of them.
  
  There are no automated tests for this yet.
  
  Empty URLs ('') now raise an exception. They cause breakages internally
  if we allow them through, and they can only occur if the user or our
  tests are misconfiguring things somehow.
  
  We report failure to fetch from the cache by printing a message to
  stderr for now. This is because BuildStream's actual logging
  functionality can't be used during frontend init -- see issue #168.
  

• 1490802c
  by Sam Thursfield at 2018-01-11T17:13:10Z

  Add `bst push --remote` and `bst pull --remote`
  
  This allows pushing and pulling from a specific cache, ignoring what is
  configured.
  
  If we choose to add a --remote option to `bst build` in future that
  would now be simple to do.
  

• 51c17e11
  by Sam Thursfield at 2018-01-11T17:26:51Z

  Allow push to multiple remotes, configurable on a per-remote basis
  
  The initial multiple cache support patch implemented a rather fragile
  logic where we would push to the first cache in the list that used the
  ssh:// protocol, if any. If we implement push-over-https:// in future
  then this will become totally unworkable.
  
  This patch alters the logic so that each remote has a 'push' option,
  and BuildStream will push to any remote that has 'push: true' set.
  

• 1f394776
  by Sam Thursfield at 2018-01-11T18:18:13Z

  tests: Exercise the new multiple cache support
  
  This adds a new test for parsing artifact cache configuration, which
  calls the helper function from the 'artifactcache' module directly
  rather than trying to assert based on blind push and pull commands
  whether or not we got the complex precedence rules exactly right.
  
  This means frontend push/pull tests no longer need to be so thorough
  about testing precedence but they are instead expanded to assert that
  multiple caches work correctly.
  

• 5f5ef4a3
  by Sam Thursfield at 2018-01-11T18:18:14Z

  doc: Document multiple cache support
  

• a59ebbf0
  by Sam Thursfield at 2018-01-11T18:18:14Z

  Improve error messages for SSH protocol handshake
  
  If you have an artifact remote cache in your config using the ssh://
  protocol and some SSH issue prevents us from connecting, we now show
  the user the stderr output from the `ssh` program, e.g.
  
      [--:--:--] WARNING Failed to fetch remote refs from ssh://artifacts@172.17.0.2:22200/artifacts/: BuildStream did not connect successfully to the shared cache ssh://artifacts@172.17.0.2:22200/artifacts/: Permission denied (publickey,keyboard-interactive).
  
  Previously we would just show the error message raised by the protocol
  code, which was less useful, e.g.:
  
      [--:--:--] WARNING Failed to fetch remote refs from ssh://artifacts@172.17.0.2:22200/artifacts/: BuildStream did not connect successfully to the shared cache ssh://artifacts@172.17.0.2:22200/artifacts/: Expected reply, got none
  

• 20d653de
  by Tristan Van Berkom at 2018-01-11T18:18:14Z

  _artifactcache/ostreecache.py: Handle ^C and shutdown child task when initializing cache
  
  This fixes issue #141
  

• 1787ff21
  by Jürg Billeter at 2018-01-11T18:18:14Z

  _artifactcache/pushreceive.py: Flush but do not close stdout
  
  Closing stdout will cause an exception when used with multiprocessing as
  that calls sys.stdout.flush() as well.
  

• ad08ad26
  by Jürg Billeter at 2018-01-11T18:18:14Z

  _artifactcache: Use pushreceive also for local artifact repositories
  
  This reduces the differences between local and remote artifact
  repositories, increasing code coverage of tests.
  

• a71bbed8
  by Sam Thursfield at 2018-01-11T18:18:14Z

  _artifactcache/ostreecache.py: Subprocesses should not return exceptions
  
  We have a policy that subprocesses can return error messages, but not
  actual exceptions.
  
  In particular this allows GLib.Error exceptions to be returned, which
  would otherwise be silently lost due to
  https://gitlab.gnome.org/GNOME/pygobject/issues/145
  

• 079f702f
  by Sam Thursfield at 2018-01-11T18:18:14Z

  Update copyright years of files I have touched
  

• 36026a72
  by Tristan Van Berkom at 2018-01-12T08:05:23Z

  NEWS: Added news entry for support for multiple artifact caches.
  

• 170a9d46
  by Tristan Van Berkom at 2018-01-12T08:33:59Z

  buildelement.py: Remove secret undocumented command lists
  
  Originally this was created with also `bootstrap-commands` and
  `test-commands` but these were never documented or used.
  
  For `bootstrap-commands`, these originated in baserock but are
  basically a part of the `configure-commands` stage, prepending
  commands to `configure-commands` is more suitable here since we
  now have the prepend/append list directives.
  
  For `test-commands`, these were never used and it's unclear at
  this time if it's the correct place for it. It would be interesting
  to implement `test-commands` as a separate operation which can run
  in parallel with reverse dependency builds (no need to block the
  build of a reverse dependency on failing tests, we can still fail
  the build as a whole based on a failing test without blocking
  builds).
  

• bc492fa8
  by Jürg Billeter at 2018-01-12T10:21:57Z

  Use explicit element state updates
  
  This adds the _update_state() method to the Element class to keep track
  of element state and avoid calculating the same cache key multiple
  times. This also consolidates the different get_cache_key
  methods into a single method that always returns the cache key
  calculated by _update_state(), if available.
  
  Fixes #149, #173
  

• 880ece19
  by Jürg Billeter at 2018-01-12T10:21:57Z

  element.py: Reuse dict for cache key calculations
  
  Do not recalculate the shared part of cache keys. In particular, do not
  call get_unique_key() multiple times for a single element or source.
  
  Fixes #167
  

• 571406d4
  by Jonathan Maw at 2018-01-12T17:50:53Z

  Make artifact cache tests work in python3.4
  
  A LocalPath was being passed into os.path.isabs, which failed when it
  tried to call the path's startswith method.
  
  Converting it to a string before using _yaml.dump seemed to solve the
  problem.
  

• fc48f087
  by Gökçen Nurlu at 2018-01-16T06:27:57Z

  Delay import of pkg_resources
  
  This does not introduce a performance gain alone, but importing pkg_resources
  is costly at the moment and can be avoided until it is actually needed.
  

• bc0f6cd5
  by Sam Thursfield at 2018-01-16T07:24:26Z

  Shorten the warnings raised when remote cache initialization fails
  
  Before:
  
      [--:--:--] WARNING Failed to fetch remote refs from ssh://ostree ostree baserock org:22000/cache: BuildStream did not connect successfully to the shared cache ssh://ostree ostree baserock org:22000/cache: SSH error: ssh: connect to host ostree.baserock.org port 22000: Connection refused
  
  After:
  
      [--:--:--] WARNING Failed to fetch remote refs from ssh://ostree ostree baserock org:22000/cache. ssh: connect to host ostree.baserock.org port 22000: Connection refused
  

• 633ef735
  by Tristan Van Berkom at 2018-01-16T11:03:38Z

  HACKING.rst: Added instructions to properly format commit messages.
  

• e311ebe5
  by Jürg Billeter at 2018-01-16T12:58:39Z

  _frontend/main.py: Fix `inconsistent` argument
  
  Do not mark target element as inconsistent for `bst source-bundle` and
  `bst workspace open` if tracking is disabled.
  

• d6ebd6ce
  by Jürg Billeter at 2018-01-16T12:58:39Z

  Rename _force_inconsistent() to _schedule_tracking()
  
  _force_inconsistent is too low level. Keep that detail contained in the
  Source class.
  

• 03412a9e
  by Jürg Billeter at 2018-01-16T12:58:39Z

  Use explicit source state updates
  
  This adds the _update_state() method to the Source class, similar to the
  corresponding method in the Element class.
  

• 88260c7a
  by Tristan Van Berkom at 2018-01-17T10:08:03Z

  Documentation: Restructured toplevel documentation
  
  This is intended to make things easier to find for the
  various types of people searching for stuff.
  
    o The installation instructions remain on the main page.
  
    o Three main separate pages have been created
      - Using BuildStream
      - Authoring BuildStream projects
      - Core API reference, for plugin authors
  
    o The "Authoring projects" section swallows the
      previous plugin index; so one can find the plugin one
      is looking for on the same page as the rest of the format
      docs
  
    o The plugin authoring section has been swallowed by the
      core API reference section, with a note that this is useful
      especially for plugin authors.
  

• d759994e
  by Jürg Billeter at 2018-01-18T11:24:26Z

  source.py: Properly handle Source.track() returning None
  
  Source.track() may return None when tracking is not available. Handle
  this identical to the case where track() returns the current ref.
  
  Fixes #201
  

• f490343d
  by Jürg Billeter at 2018-01-18T11:24:57Z

  element.py: Return all refs in _track()
  
  TrackQueue must call _set_ref() for all sources, otherwise the source
  will never get out of the inconsistent state. This fixes --track when
  using local sources.
  
  Fixes: 03412a9e ("Use explicit source state updates")
  

• af7ffc6d
  by Jürg Billeter at 2018-01-22T19:51:44Z

  element.py: Fix typo in _update_state()
  
  Fixes: bc492fa8 ("Use explicit element state updates")
  

• 4df5d035
  by Jürg Billeter at 2018-01-23T12:01:37Z

  element.py: Do not query caches too early or too late
  
  The strict cache key is preferred when pulling or extracting artifacts
  in non-strict mode, before falling back to the weak cache key. This
  means that the strict cache key must be available before the cache is
  queried.
  
  The strict cache key is also sufficient for querying caches using strong
  cache keys. Do not defer those cache queries.
  
  Fixes #202
  

• 68cef134
  by Jürg Billeter at 2018-01-23T12:03:37Z

  _scheduler: Replace skip() and ready() with status()
  
  Queue sometimes called ready() before skip() and sometimes it didn't.
  This change consolidates the two methods to ensure we always determine
  the correct status.
  

• a9f72b71
  by Jürg Billeter at 2018-01-23T12:03:37Z

  pullqueue.py: Wait for cache to become queryable
  
  Wait for cache to become queryable instead of simply skipping the
  element.
  

• bd336274
  by Jürg Billeter at 2018-01-23T12:03:37Z

  fetchqueue.py: Wait for cache to become queryable
  
  Wait for cache to become queryable instead of simply fetching the
  sources.
  

• 750a21ef
  by Jürg Billeter at 2018-01-23T12:16:08Z

  Add Element._pull_pending() method
  
  This moves the pull status logic from PullQueue to Element.
  As PullQueue.process() simply calls Element._pull(), it doesn't make
  sense to keep the status logic in PullQueue. This is also required to
  use the pull status in other places without code duplication.
  

• d98312b2
  by Jürg Billeter at 2018-01-23T12:21:08Z

  element.py: Use _pull_pending() in _update_state()
  
  This fixes strong cache key handling when the local cache contains an
  artifact that matches the weak cache key and the remote cache contains
  an artifact that matches the strict cache key.
  

• 0b2cbd19
  by Jürg Billeter at 2018-01-23T12:28:07Z

  Keep track of failed pull operations
  
  Do not consider pull as pending if pull failed. This is required to
  allow fallback to fetch and build after pull failure.
  

• 88187f12
  by Jürg Billeter at 2018-01-23T13:24:36Z

  tests/frontend/pull.py: Add pull test for non-strict build plan
  

• 043d56a6
  by Jürg Billeter at 2018-01-23T16:18:39Z

  tests/frontend/pull.py: Add track+pull test for non-strict build plan
  

• 0d69abed
  by Angelos Evripiotis at 2018-01-24T09:09:44Z

  buildstream/utils.py: remove unused import 'pickle'
  

• a950b86f
  by Jürg Billeter at 2018-01-25T10:17:59Z

  element.py: Call _update_state() in _schedule_tracking()
  
  Do not depend on caller calling it on its own.
  

• 5f1be604
  by Chandan Singh at 2018-01-25T11:52:48Z

  Add support for doing incremental builds
  
  This functionality is only supported for sources which have an open
  workspace. When such sources are present, the workspace directory will
  be mounted directly inside the sandbox. As opposed to the default
  behavior, which is to copy files inside the sandbox.
  
  This will save time when building large projects as only those files
  will need be re-compiled that have been modified during two consecutive
  builds (assuming the underlying build system supports such behavior).
  
  A few things to note regarding this behavior:
  
  - If there are any `configure-commands` present, they will run only once
    for each open workspace. If an element has multiple workspaces and any
    one of them is opened/closed, they will be executed again on the next
    run. But, modifying the contents of a workspace will not trigger the
    `configure-commands` to be executed on the next run.
  
  - Workspaced builds still leverage the cache. So, if no changes are made
    to the workspace, i.e. no files are modified, then it will not force a
    rebuild.
  
  Fixes #192.
  

• cf7e4357
  by Chandan Singh at 2018-01-25T11:52:48Z

  Raise warning if workspaced source's ref is updated
  
  In case a source with an open workspace is tracked and it's ref gets
  updated, BuildStream should inform the user that the new ref will not be
  picked up so long as the workspace is open. To start using the updated
  ref, the existing workspace will have to be closed.
  

• a0a26d7c
  by Jonathan Maw at 2018-01-25T13:27:38Z

  element: Fix 'detail' not being stored in the ElementError
  

• 2aa233da
  by Jonathan Maw at 2018-01-25T13:27:40Z

  element: Handle overlaps with a whitelist and option to raise errors
  
  * Adds the 'overlap-whitelist' field to elements' public data. This is a
    list of globs that match files that the element is allowed to overlap
    other elements with.
  * Adds the project-wide 'fail-on-overlaps' field. If set,
    non-whitelisted overlaps will raise an error instead of printing a
    warning.
  

• 9bf5278a
  by Jonathan Maw at 2018-01-25T13:27:40Z

  Add explicit error handling for ElementErrors that happen as a result of staging artifacts
  

• 24b730f8
  by Jonathan Maw at 2018-01-25T13:27:40Z

  tests: Remove stray printing during overlap test
  

• a921474d
  by Jonathan Maw at 2018-01-25T13:27:41Z

  tests: Move test_overlaps test files
  
  This was done so that the files can be re-used with different
  project.conf.
  

• efbf144e
  by Jonathan Maw at 2018-01-25T13:27:42Z

  tests: Add overlap tests
  

• b1bd7862
  by Jonathan Maw at 2018-01-25T13:27:42Z

  doc: Explain overlap-whitelist and fail-on-overlap
  

• 0a834156
  by Tristan Van Berkom at 2018-01-29T11:07:46Z

  doc/source/install.rst: Simplified as per recommendations of Allan Day
  

• 43046114
  by Gökçen Nurlu at 2018-01-31T10:23:49Z

  Modify the generated CLI script by monkey patching
  
  This change monkey patches setuptools' code generation functionality, so
  that pkg_resources won't be imported in the generated file.
  
  Fixes #172
  

• 88901421
  by Gökçen Nurlu at 2018-01-31T14:48:15Z

  Get version number w/o pkg_resources
  
  This removes most of the usages of pkg_resources from the codebase, helping
  the goal of getting rid of that completely.
  
  With this change, version number is generated during install and embedded into
  `__version__` which is then imported by root level `__init__`. From there,
  it is used by other parts of the codebase when needed.
  
  Generated `__version__` file is ignored and not tracked by git to prevent
  unnecessary 'changes' messages and accidental commits of that file.
  

• 4c116c6d
  by Angelos Evripiotis at 2018-01-31T15:42:04Z

  _frontend: more doc on parameters
  

• 315c4ef7
  by Angelos Evripiotis at 2018-01-31T15:42:04Z

  _frontend: remove unused 'ctx' param from override
  
  The 'ctx' parameter is carefully threaded through to 'complete_target',
  but it doesn't actually use it. Remove the parameter from unused places.
  

• 9177ec6f
  by Chandan Singh at 2018-02-05T07:45:43Z

  Make workspaces element-wide instead of source-specific
  
  At present, BuildStream supports source-specific workspaces. This patch
  makes workspaces element-wide. This will bring workspaces closer to what
  the build area looks like inside the sandbox when the build actually
  happens.
  
  As part of this change, format of `.bst/workspaces.yaml` file will also
  change. Previously, each element used to have a dict mapping each source
  to its workspace directory. Now, each element will directly map to its
  workspace directory.
  
  If users have existing workspaces open, this patch tries to cope with it
  but in some cases, it is not possible to reliably convert workspace
  config from old format to new format. When an element has workspace open
  for just one source, we assume that to be the workspace directory. But
  if there were more than one workspaces associated with an element,
  BuildStream will simply error out describing the issue.
  
  Note that the actual contents of the workspace directory are not changed
  in any case. So, if an element had multiple sources associated with it
  but only had workspace open for source, the other sources will not be
  automatically staged in the workspace unless the user runs `workspace
  reset` or something equivalent.
  
  Part of #209.
  

• c8a0d908
  by Chandan Singh at 2018-02-05T07:45:43Z

  Add tests for workspace config format conversion
  

• 2dd99bcc
  by Sam Thursfield at 2018-02-05T11:33:58Z

  plugins/elements/cmake.yaml: Remove unexplained FIXME comment
  

• 216c4b36
  by Sam Thursfield at 2018-02-05T11:33:58Z

  Improve documentation for autotools, cmake and meson elements
  
  The key change is to document the correct usage of the conf-local
  and conf-global style options, as it was previously impossible to
  figure that out from the documentation.
  

• aec3ccde
  by Sam Thursfield at 2018-02-05T11:33:59Z

  Update copyright years of changed files
  

• 1a678c13
  by Javier Jardón at 2018-02-05T13:43:41Z

  .gitlab-ci.yml: Update docker image so it contains jinja >= 2.10
  

• 945054df
  by Javier Jardón at 2018-02-05T13:43:42Z

  setup.py: Require jinja >= 2.10
  
  jinja2 2.10 import time has been divided by two compared to Jinja 2.9.6
  
  Fixes #194
  

• 0e0990c5
  by Jürg Billeter at 2018-02-06T09:53:54Z

  Increase the soft limit for open file descriptors
  
  SafeHardlinks FUSE needs to hold file descriptors for all processes in
  the sandbox. Avoid hitting the limit too quickly by increasing the soft
  limit to the maximum.
  
  Fixes #232
  

• 34a60189
  by Tristan Maat at 2018-02-06T16:59:19Z

  image_authoring.rst: Add image creation documentation
  

• fc384201
  by Jürg Billeter at 2018-02-07T10:24:34Z

  Revert "image_authoring.rst: Add image creation documentation"
  
  This was pushed accidentally.
  
  This reverts commit 34a60189be63dc86d333ca895ce44bd24adb43c3.
  

• 37f3c926
  by Tristan Van Berkom at 2018-02-07T10:26:15Z

  _pipeline.py: Silence some messages while interrogating source consistency
  

• 5c6cfe10
  by Tristan Van Berkom at 2018-02-07T10:26:15Z

  element.py: Consider execution environment in cache key
  
  Do this by default for every element, later we will have
  to handle it differently when supporting projects which
  build on specific arches with virtualization or networked workers.
  

• cd62e988
  by Tristan Van Berkom at 2018-02-07T10:26:15Z

  _project.py: Bumping artifact version for execution environment
  
  Now that we consider the arch and os for every artifact, lets
  bump the core artifact version to make this trackable.
  

• 8333af2f
  by Tristan Van Berkom at 2018-02-07T10:26:16Z

  tests/cachekey: Updating cache key test for new artifact version
  

• 07c84636
  by Abderrahim Kitouni at 2018-02-07T11:28:40Z

  element.py: Fix staging of runtime dependencies with overlaps
  
  This fixes `bst shell` crashes in the presence of overlaps.
  
  Fixes #225
  
  Fixes: 2aa233da ("element: Handle overlaps with a whitelist and...")
  

• 087f7c28
  by Jürg Billeter at 2018-02-07T11:28:40Z

  tests/frontend/overlaps.py: Add test for runtime dependency staging
  
  This test covers #225.
  

• ac33c1be
  by Tristan Maat at 2018-02-07T13:03:27Z

  .gitignore: Add rules for various testing directories
  

• b351de2a
  by Tristan Maat at 2018-02-07T13:03:28Z

  runcli.py: Allow setting individual config settings
  

• e4e713ee
  by Tristan Maat at 2018-02-07T13:03:28Z

  Add test utilities for integration tests
  

• cb71d0f6
  by Tristan Maat at 2018-02-07T13:03:28Z

  Migrate element test files
  

• 4d05c55d
  by Tristan Maat at 2018-02-07T13:03:28Z

  Add bzr test
  

• 11c8f386
  by Tristan Maat at 2018-02-07T13:03:28Z

  shell.py: Add shell test
  

• eeed9f62
  by Tristan Maat at 2018-02-07T16:14:11Z

  Merge integration tests into general tests
  

• 831a4dfc
  by Tristan Maat at 2018-02-07T16:14:15Z

  Remove old integration tests
  

• a32a06c4
  by Tristan Maat at 2018-02-07T16:14:15Z

  Add workspace mount integration test
  

• c178e80d
  by Jürg Billeter at 2018-02-07T17:26:20Z

  Remove old workspace mount integration test
  
  It has already been migrated.
  

• a83c94b7
  by Jürg Billeter at 2018-02-08T13:04:03Z

  element.py: Split errors and warnings for overlaps
  
  This ensures that overlap warnings from projects without
  fail-on-overlaps are not reported in fatal error messages.
  

• b5f101a2
  by Jürg Billeter at 2018-02-08T13:04:04Z

  _options: Support transform function for variable substitution
  

• 1d331b9b
  by Jürg Billeter at 2018-02-08T13:04:04Z

  Move cli_options from Context to Project
  
  cli_options are project-specific.
  

• 00c6e649
  by Jürg Billeter at 2018-02-08T13:04:04Z

  Move element and source factories from Pipeline to Project
  
  Element and source factories use a project-specific search path.
  

• b7491950
  by Jürg Billeter at 2018-02-08T13:04:04Z

  Pass Project instance to Loader
  
  Project access will be required for junctions.
  

• 6eeb2587
  by Jürg Billeter at 2018-02-08T13:04:04Z

  Add project attribute to MetaElement
  
  With junctions, elements may be loaded from multiple projects.
  

• 82913c77
  by Jürg Billeter at 2018-02-08T13:04:04Z

  _loader.py: Pass Loader instance to LoadElement
  
  Loader access will be required for junctions.
  

• 399f09b7
  by Jürg Billeter at 2018-02-08T13:04:38Z

  Add junction support for subprojects
  
  This introduces junctions as a new kind of elements to allow
  dependencies to cross project boundaries.
  

• 319af30c
  by Jürg Billeter at 2018-02-08T15:38:48Z

  Add junction tests
  

• ce1b5bf6
  by Jürg Billeter at 2018-02-08T15:38:48Z

  Add list of projects to Context
  

• 4b3d8310
  by Jürg Billeter at 2018-02-08T15:38:48Z

  _artifactcache: Use project-specific remotes for subprojects
  

• 4062a70d
  by Jürg Billeter at 2018-02-08T15:38:48Z

  Add test for project-specific artifact shares with junctions
  

• e2574285
  by Sam Thursfield at 2018-02-08T15:38:48Z

  Add a 'path' config option to junction elements
  
  This makes it possible to depend on a project which is in a subdirectory
  of a Git repository.
  
  The error message given when the expected project.conf file is not found
  has also been improved. Previously the error would look like this:
  
      Error loading pipeline: Could not find file at
      /home/sam/.cache/buildstream/build/freedesktop-sdk-junction-rvmn17s2/project.conf
  
  This is giving the path of an internal temporary directory where the foreign
  project's source is checked out. The new error gives more information:
  
      Error loading pipeline: Could not find the project.conf file for junction
      element at freedesktop-sdk-junction.bst [line 1 column 0]. Expecting a
      project at path '.' within git source at freedesktop-sdk-junction.bst [line
      4 column 2]
  

• d95c98bc
  by Tristan Maat at 2018-02-08T16:38:14Z

  Refactor CLI imports
  
  This reduces the number of unconditional imports, mainly to speed up
  bash completion.
  

• dbed35e6
  by Angelos Evripiotis at 2018-02-11T13:30:17Z

  projectconf.rst: clarify 'type option'
  
  Reword 'Flags type options' slightly for more clarity.
  
  We use 'option type' to refer to these thingies elsewhere, adjust
  'arch' to be the same way.
  

• 5e8dab02
  by Sam Thursfield at 2018-02-12T09:33:03Z

  Add integration tests for edge cases involving symlinks and overlaps
  

• 0f430622
  by Sam Thursfield at 2018-02-12T10:26:18Z

  utils.py: Wrap calls to os.path.realpath() in an LRU cache
  
  The os.path.realpath() function is expensive and we call it many times,
  to the point that os.path.realpath() calls make up around 40% of the
  total time spent in Element.stage_artifact().
  
  The cleanest way to fix this is with a `functools.lru_cache()` wrapper
  that caches recently used values. None of the code in question can be
  removed (as the tests added in the previous commit will demonstrate).
  
  I tested this by running `bst shell base/base-system.bst true` in
  the GNOME modulesets project.
  
    o Without this patch there are 240,019 calls os.path.realpath()
  
    o With this patch there are 10,379 calls to os.path.realpath()
  
    o If we increase the cache size to 128 items, there are 10,359 calls
      to os.path.realpath().
  
    o If we reduce the cache size to 32 items, there are 10,426 calls.
  
    o In all cases the number of *unique* calls is 10,327.
  
  This fixes issue #174.
  

• 73b6e1e5
  by Jonathan Maw at 2018-02-12T16:48:23Z

  bzr: Implement init_workspace
  
  This will checkout a branch from the cache, and switch its parent branch
  to the url and ref that the source came from
  

• 779f369b
  by Jonathan Maw at 2018-02-12T16:48:24Z

  test: Add tests that bzr workspace initialization acts as expected
  

• e52f2e42
  by Jürg Billeter at 2018-02-12T20:43:15Z

  .gitlab-ci.yml: Fix INTEGRATION_CACHE
  
  The working directory of the runner is /builds/BuildStream/buildstream,
  not /builds/BuildStream.
  

• 75f73ef7
  by Jürg Billeter at 2018-02-12T20:43:54Z

  scheduler.py: Do not prematurely terminate loop after skipping jobs
  
  It is possible that Queue.process_ready() skips jobs without starting a
  job. Pull the skipped jobs forward through the queues and process them
  instead of prematurely terminating the loop due to lack of active jobs.
  
  Fixes #236
  

• ecef1319
  by Jürg Billeter at 2018-02-13T04:49:49Z

  source.py: Fix typo in workspace warning
  
  Fixes #250
  

• 7002d25a
  by Angelos Evripiotis at 2018-02-13T04:56:49Z

  doc: consistent source staging dir doc
  
  All source plugins respond to the 'directory' key, so add the doc for it
  to those that didn't have it.
  

• b6e0dd36
  by Angelos Evripiotis at 2018-02-13T04:56:49Z

  source.py: fix comment typo
  

• b5f43ece
  by Phillip Smyth at 2018-02-13T05:07:43Z

  _frontend/cli.py: Improve closing of non-existing workspace
  
  Fixes #182
  

• 20b8d035
  by Phillip Smyth at 2018-02-13T05:25:35Z

  _frontend/cli.py: Check for invalid --track-except or --track-save
  
  Report error when --track-except or --track-save is specified without
  --track or --track-all.
  
  Fixes #181
  

• 955b184e
  by Jürg Billeter at 2018-02-13T17:47:37Z

  .gitlab-ci.yml: Support test execution in other repositories
  
  Do not make any assumptions about the working directory as that varies
  across repositories.
  
  Fixes #251
  

• 1b664f71
  by Sam Thursfield at 2018-02-14T12:28:25Z

  _plugincontext.py: Fix unhandled exception if plugin is not found
  
  This fixes an AttributeError triggered if we try to import a plugin from
  a package which is installed but does not contain that plugin.
  
  Before:
  
      [--:--:--] START   Resolving pipeline
      Traceback (most recent call last):
        ...
      AttributeError: 'NoneType' object has no attribute 'dist'
  
  After:
  
      [--:--:--] START   Resolving pipeline
      [00:00:00] FAILURE Resolving pipeline
      Error loading pipeline: Pip package buildstream-external does not contain a plugin named 'nonexistant-plugin'
  

• 57c8b888
  by Phillip Smyth at 2018-02-15T16:55:35Z

  _yaml.py: Prevent ruamel from removing underscores
  
  The ruamel parser interprets values such as 1_2_3 as numbers, ignoring
  the underscores. However, for `track` we need the value as a verbatim
  string. This change configures ruamel to parse all values as strings,
  leaving potential conversions to node_get() with the `expected_type`
  parameter.
  
  This fixes #166 and adds tests
  

• f275cf64
  by Phillip Smyth at 2018-02-15T17:08:13Z

  _yaml.py: Preserve quotes when run through yaml parsing
  
  This preserves the quoting style for string values in round trips.
  

• 6458d77a
  by Tristan Van Berkom at 2018-02-16T09:18:51Z

  element.py: Create real build shell for `bst shell --build`
  
  Now we infer that a build shell is desired if `bst shell --build`
  is specified, before there was no real build shell (only staged
  sources in the build style shell).
  
  Fixes issue #232
  

• 722045db
  by Chandan Singh at 2018-02-17T02:09:13Z

  bst-here: fix path of bst binary
  
  As shown in the snippet below, the `bst` binary is installed at
  `/usr/local/bin/bst` in the current Docker image
  `buildstream/buildstream-fedora`,
  
  ```
  $ docker run --rm buildstream/buildstream-fedora type -a bst
  bst is /usr/local/bin/bst
  ```
  

• e8de57f9
  by Chandan Singh at 2018-02-17T02:09:19Z

  bst-here: add usage instructions
  

• 9e383634
  by Tristan Van Berkom at 2018-02-19T07:09:53Z

  Updating NEWS for 1.1.0 release
  

• 2643b095
  by Tristan Van Berkom at 2018-02-19T07:10:39Z

  Regenerating man pages for 1.1.0 dev release
  

• c9a5a747
  by Jonathan Maw at 2018-02-19T15:17:00Z

  git.py: Add the ability to set whether to checkout submodules
  
  This can be set as broad as "No submodule in this element should be
  checked out" and as narrow as "only this submodule should be checked
  out"
  
  Note that this will change the cache-keys of every element that uses a
  git source. I could have structured the code to prevent the cache key
  changing, but that adds extra complexity in the long-term.
  

• d94125fe
  by Jonathan Maw at 2018-02-19T15:17:00Z

  tests: fix erroneous comment in git test
  

• 6498020e
  by Jonathan Maw at 2018-02-19T15:17:00Z

  testutils: Add support for git source controlling submodule checkout
  

• c8177624
  by Jonathan Maw at 2018-02-19T15:17:01Z

  tests: Add tests to verify git sources obey submodule checkout config
  

• 445372c4
  by Jonathan Maw at 2018-02-19T15:17:01Z

  Add a 'sources' field to project.conf to override defaults
  
  Equivalent to the 'elements' field, but slightly different because
  sources don't have accompanying yaml.
  

• 3d4d5688
  by Jonathan Maw at 2018-02-19T15:17:02Z

  tests: Add a git test that project overrides work
  

• a10ef1a4
  by Jonathan Maw at 2018-02-19T15:17:02Z

  doc: Update information on how plugin configuration is composed
  
  It previously neglected to describe how sources were composed, and we've
  changed how sources are composed, recently.
  
  In addition, it was slightly confusing to have the project conf be the
  title of two different stages of composition.
  

• 3937f24c
  by Chandan Singh at 2018-02-19T16:00:40Z

  Create workspace directory after checking for potential issues
  
  At present, in case that `bst workspace open` for reasons such as
  sources not being tracked or if tracking fails, BuildStream will leave
  behind an empty directort corresponding to the workspace. Restructure
  code so that workspace directory is only created after other checks
  have passed.
  
  Fixes #200.
  

• 3ce6b907
  by Sam Thursfield at 2018-02-20T13:15:01Z

  doc: Add 'sandboxing' section
  
  This attempts to document and specify our current sandboxing
  functionality.
  

• 260946ed
  by Sam Thursfield at 2018-02-21T10:26:41Z

  tests/integration: Use a simpler CMake test project
  
  The 'step7' tutorial was fine, but it requires a C++ compiler to build.
  I have replaced it with a simpler testcase which is based on the
  existing amhello testcase.
  
  This means that the integration test suite no longer requires a C++
  compiler, and in practice this can allow us to avoid depending on the
  GNU C/C++ compilers altogether and instead use the vastly smaller Tiny C
  Compiler.
  

• cea43cb2
  by Jürg Billeter at 2018-02-21T10:42:34Z

  tests/integration: Do not use C++ compiler in workspace-mount test
  
  This means that the integration test suite no longer requires a C++
  compiler, and in practice this can allow us to avoid depending on the
  GNU C/C++ compilers altogether and instead use the vastly smaller Tiny C
  Compiler.
  

• 5476f2a1
  by Sam Thursfield at 2018-02-21T10:42:34Z

  tests/integration: Use a minimal custom base sysroot
  
  We have been using the Freedesktop SDK binaries to provide a base
  system to run the integration tests. This works OK but it weighs in
  at 985MB of content, and there is no simple way to customize it so
  that we only download the bits we actually need.
  
  This commit changes the tests to use a custom sysroot based on the
  Alpine Linux distribution. The sysroot is 155MB unpacked, and packs down
  to a 27MB .tar.xz.
  
  This speeds up the integration tests significantly as we greatly reduce
  the amount of network traffic required and the amount of data that gets
  copied around when creating the staging area.
  

• 2e6566e2
  by Sam Thursfield at 2018-02-21T11:33:42Z

  tests/integration/base/generate-base.sh: Typo fix in comment
  

• 3d94bd25
  by Tristan Van Berkom at 2018-02-21T13:28:54Z

  doc/source/projectconf.rst: Moving docs for `fail-on-overlaps`
  
  This is not a part of the documentation on how external plugins
  are loaded and referenced; lets put it in the main general section
  instead.
  

• b2197bc2
  by Sam Thursfield at 2018-02-22T12:02:44Z

  _ostree.py: Disable OSTree's minimum disk space check in our repos
  
  This interacts badly with our builds, usually if the check does fire
  it's at the point that we are trying to commit a completed build (which
  may have taken a long time to complete) into the cache. Even though
  there might be enough space available to store the build, the error
  causes BuildStream to consider it "failed" and forces the user to start
  again.
  
  This aims to fix https://gitlab.com/BuildStream/buildstream/issues/216
  although there is some evidence that it does not completely do so.
  
  BuildStream currently has no minimum disk space check, and no way
  to automatically clean up the cache. There is an open issue for this:
  https://gitlab.com/BuildStream/buildstream/issues/135
  

• 11f2de77
  by Jürg Billeter at 2018-02-22T13:47:20Z

  Inherit user id and group id for bst shell
  
  This allows D-Bus access.
  
  Fixes #227
  

• b696d363
  by Tristan Van Berkom at 2018-02-22T16:52:50Z

  bst shell: Added --isolate option, disabled by default.
  
  This changes a few things:
  
    o How the shell environment and sandbox modes are setup is
      not decided by whether a `--build` sandbox was staged or not
  
    o In interactive build debugging mode, an isolated build
      sandbox is used
  
    o The bst shell command now has the option to control it
  
    o All invocations of bst shell now use a read-only rootfs,
      avoiding FUSE layer and file descriptor limits (this can
      be made optional later if needed)
  
  This is a part of issue #223
  

• bd05e2e6
  by Jürg Billeter at 2018-02-24T09:38:39Z

  sandbox/_sandboxbwrap.py: Unshare UTS namespace in build sandbox
  
  Set hostname to buildstream in build sandbox to reduce host
  contamination.
  

• cacb34a7
  by Jürg Billeter at 2018-02-24T09:38:39Z

  sandbox/_sandboxbwrap.py: Unshare IPC namespace in build sandbox
  
  Isolate sandbox processes from System V IPC.
  

• 34bf8a0d
  by Jürg Billeter at 2018-02-24T11:03:37Z

  _artifactcache/ostreecache.py: Make enable_push a keyword argument
  
  Boolean positional arguments can be confusing.
  

• 1476e436
  by Jürg Billeter at 2018-02-24T11:03:37Z

  sandbox/_sandboxbwrap.py: Use --die-with-parent
  
  This ensures subprocesses are cleaned up when the bwrap parent dies.
  
  This is available since bubblewrap 0.1.8. We skip the option if the host
  bwrap does not support it.
  

• 4ff02b9c
  by Tristan Van Berkom at 2018-02-24T14:08:50Z

  element.py: Element._shell() now allows a custom prompt for PS1
  

• 81acb103
  by Tristan Van Berkom at 2018-02-24T14:58:06Z

  _frontend: Added App.shell() to call instead of directly invoking the Element._shell()
  
  And use this place to format a custom prompt for PS1, here
  we have click and we use the ANSI colors in PS1 only if colors are
  enabled.
  

• f6ceefbb
  by Tristan Van Berkom at 2018-02-24T15:19:19Z

  _project.py: Parse the new 'shell' section
  
  This new section informs BuildStream how to launch interactive shells
  for the specific project.
  
  These new options are part of the effort to make applications work
  better inside a `bst shell` environment, issue #223
  
  Initial options include:
  
    o Default shell command to run for an interactive shell, this is
      used for interactive debugging sessions, and also if no command
      was specified in `bst shell`
  
    o List of environment variables to inherit from user environment,
      this is useful to propagate some host stuff into the environment
      for debugging; and only considered when it is not an "isolated"
      shell.
  
  These changes also bump the BST_FORMAT_VERSION
  

• 89f28086
  by Tristan Van Berkom at 2018-02-24T15:20:14Z

  _context.py: Added _get_toplevel_project() convenience accessor
  

• 306f82e2
  by Tristan Van Berkom at 2018-02-24T15:20:14Z

  element.py: Use project shell configuration when launching shells.
  
  Use the toplevel project configuration to decide:
    o What command to run for an interactive shell
    o Which environment variables to inherit from host environment
      when not running an isolated shell
  

• 647e28c3
  by Tristan Van Berkom at 2018-02-24T15:34:39Z

  doc/source/projectconf.rst: Added documentation about customizing the shell
  

• c5448d14
  by Tristan Van Berkom at 2018-02-24T15:34:39Z

  testutils/runcli.py: Print something helpful if BuildStream fails to exit properly
  
  When testing a recent patch, I had missed a return of the exit code
  in `bst shell` resulting in the CLI exiting with `None`; this patch
  makes the fixture more helpful, where previously it just printed `None`
  

• 0d5ba30e
  by Tristan Van Berkom at 2018-02-24T15:34:39Z

  tests/integration/shell.py: Dont use shlex, use command vectors
  
  Using shlex messes with the argv we want to pass to a shell, this
  was not working for the newly added test which passes "${var}" strings
  as arguments to the shell.
  
  Also, removed a redundant line to explicitly build something
  in the 'no_shell' test; that is taken care of by the helper.
  

• 9632eafb
  by Tristan Van Berkom at 2018-02-24T15:34:39Z

  tests/integration/shell.py: Added tests for inheriting environment variables
  

• f3f81cbb
  by Chandan Singh at 2018-02-24T18:24:01Z

  MANIFEST.in: remove files that are no longer in this repo
  
  We don't have `Dockerfile` and `Dockerfile-build.sh` anymore in this
  repository so remove them from `MANIFEST.in`.
  

• 855df10c
  by Jürg Billeter at 2018-02-25T20:02:54Z

  _downloadablefilesource.py: Add ETag support
  
  This avoids unnecessary downloads when tracking `tar` and `zip` sources.
  
  Fixes #62
  

• 59615b62
  by Tristan Van Berkom at 2018-02-26T12:43:45Z

  doc/source/format.rst: Document format version for junction dependency attribute
  
  The junction attribute was merged fairly recently in the 1.1 series without
  bumping the project format version, this documents the junctions addition
  to be added since the closest version bump where support exists.
  

• 2c0b69fb
  by Sam Thursfield at 2018-02-26T15:04:15Z

  _artifactcache/artifactcache.py: Fix name mismatch in doc comment
  

• c36a1825
  by Sam Thursfield at 2018-02-26T15:04:16Z

  _artifactcache/ostreecache.py: Rework so that all remotes for a ref are tracked
  
  This commit adds a couple of simple classes in order to reduce the
  number of different dictionaries tracking the state of the remotes
  within the OSTreeCache object.
  
  It also extends the internal ref map to remember all the remotes that
  store a given ref, not just the highest priority ref, and modifies the
  remote_contains_key() method to expose that data.
  

• eae43fa4
  by Sam Thursfield at 2018-02-26T15:04:17Z

  Don't push artifacts to remote caches that already contain them
  
  This adds two separate checks to avoid pushing artifacts unnecessarily.
  
  First, in the OSTreeCache.push() method we now first obtain the list of
  remotes that contain the refs that we are about to push. We then avoid
  pushing to any remote that already contains that ref.
  
  Note that the internal ref map is read once on process startup, so
  if multiple `bst` processes are pushing to a cache they might still both
  push the same artifact. There is an existing issue for this:
  
    https://gitlab.com/BuildStream/buildstream/issues/179
  
  Secondly the Element._skip_push() method now checks if all remote
  caches configured for pushing already have a given artifact, and will
  skip the push job altogether if they do. The first check would already
  mean that no pushes would actually happen, but without the second check
  the user would still see Push jobs being created for every artifact
  which would be quite misleading.
  

• ba4919a4
  by Sam Thursfield at 2018-02-26T15:04:17Z

  tests/frontend: Summarize the existing push and pull tests
  
  These tests are too long to understand at a glance so a summary is
  needed of each.
  

• 0c7bd18d
  by Sam Thursfield at 2018-02-26T15:04:18Z

  tests/frontend/push.py: Test that we don't push stuff that we just pulled
  

• b52ad07c
  by Tristan Van Berkom at 2018-02-27T10:00:44Z

  Integration tests: Removing some dead code
  
    o Removing format_files() helper in testutils
  
    o Removing imports of format_files()
  
    o Removing imports of walk_dir() where it's not used
  
    o Removing `element_path` variables that are unused
  

• 3bf19c0e
  by Jürg Billeter at 2018-02-27T11:43:14Z

  element.py: Remove unused strength parameter from _cached()
  
  Remove unused strength parameter from _cached() and _remotely_cached(),
  which makes them simple accessor functions as `__cached` is equivalent
  to `__strong_cached` with a strict build plan.
  

• cdcdc32c
  by Jürg Billeter at 2018-02-27T11:43:14Z

  _artifactcache/tarcache.py: Remove unused remove() method
  

• 8c1979dd
  by Jürg Billeter at 2018-02-27T11:43:14Z

  _artifactcache/artifactcache.py: Add push() to abstract base class
  

• 0c4f62ac
  by Jürg Billeter at 2018-02-27T11:43:15Z

  _artifactcache/artifactcache.py: Add pull() to abstract base class
  

• 7100e0db
  by Jürg Billeter at 2018-02-27T11:43:15Z

  _artifactcache: Add key parameter to contains() method
  
  Contain cache key logic in Element class.
  

• 9e7e9c28
  by Jürg Billeter at 2018-02-27T11:43:16Z

  _artifactcache: Add key parameter to extract() method
  
  Contain cache key logic in Element class.
  

• 623aa819
  by Jürg Billeter at 2018-02-27T11:43:16Z

  _ostree.py: Use list of refs as commit parameter
  

• e99e22c2
  by Jürg Billeter at 2018-02-27T11:43:16Z

  _artifactcache: Add keys parameter to commit() method
  
  Contain cache key logic in Element class.
  

• f439e26a
  by Jürg Billeter at 2018-02-27T11:43:17Z

  _artifactcache: Add key parameter to push_needed() method
  
  Contain cache key logic in Element class.
  

• 44bddc5f
  by Jürg Billeter at 2018-02-27T11:43:17Z

  _artifactcache: Add keys parameter to push() method
  
  Contain cache key logic in Element class.
  

• e6910600
  by Jürg Billeter at 2018-02-27T11:43:17Z

  _artifactcache: Add key parameter to pull() method
  
  Contain cache key logic in Element class.
  

• 14c81d99
  by Jürg Billeter at 2018-02-27T11:43:18Z

  Remove unused _KeyStrength imports
  

• a8a942b7
  by Jürg Billeter at 2018-02-27T11:43:18Z

  element.py: Add _can_query_cache() method
  
  Keep the knowledge about which cache key is required for cache queries
  within the Element class.
  

• 618f7d69
  by Jürg Billeter at 2018-02-27T11:43:18Z

  element.py: Remove unused _get_strict_cache_key() method
  
  Cache key handling is now contained within the Element class.
  

• f685ea68
  by Jürg Billeter at 2018-02-27T12:09:56Z

  tests/frontend/pull.py: Do not use a workspace for the non-strict test
  
  The cache status of reverse dependencies of workspaced elements cannot
  always be determined upfront.
  

• 33b14eb5
  by Jürg Billeter at 2018-02-27T12:09:56Z

  Determine as early as possible whether a build is pending
  
  Workspaced sources are considered unstable if a build is pending as the
  build will modify the contents of the workspace.  Determine as early as
  possible if a build is pending to be able to discard unstable cache
  keys.
  
  Fixes #273
  

• c9db2eb9
  by Jürg Billeter at 2018-02-27T12:09:56Z

  tests/frontend/workspace.py: Add cache key checks to test_build
  
  Regression test for #273
  

• 1b74422a
  by Tristan Van Berkom at 2018-02-27T15:03:04Z

  tests/integration/shell.py: Generate project.conf on the fly
  

• 63eb06bc
  by Tristan Van Berkom at 2018-02-27T15:05:53Z

  _project.py: Added 'host-files' shell configuration
  
  A shell configuration allowing one to bind mount files into the
  shell sandbox.
  
  This bumps the BST_FORMAT_VERSION to 2.
  
  This is related to #223, and solves #241 (name resolution problems)
  by making it possible to specify a bind mount for `/etc/resolv.conf`
  in the project configuration.
  

• 9e3a26aa
  by Tristan Van Berkom at 2018-02-27T15:07:04Z

  element.py: Inform the sandbox of host-files when running a shell
  
  Does not apply to isolated shells.
  

• ce6ea0e1
  by Tristan Van Berkom at 2018-02-27T15:07:10Z

  sandbox/_mount.py: Ensure file existence for bind mounts
  
  This logic existed but is now a bit more complex with
  the addition of allowing explicit bind mounts to be defined
  for the shell; the change is that now files can also be
  mounted into the sandbox instead of just directories.
  

• 0d5b0955
  by Tristan Van Berkom at 2018-02-27T15:07:10Z

  tests/integration/shell.py: Added tests for host-files
  
  Newly added tests:
  
    o Test that bind mounting a file works
  
    o Test that bind mounting a file into a non-existing directory works
  
    o Test that bind mounting is disabled in an isolated shell
  
    o Test that the shell still works, but a warning is printed and
      the mount is refused in case the specified file on the host
      was found to be a directory
  
    o Test that the shell still works, but a warning is printed
      in the case that the file specified on the host does not exist
  

• 78b4259a
  by Tristan Van Berkom at 2018-02-27T15:07:10Z

  doc/source/projectconf.rst: Documenting `host-files` shell configuration
  

• ad0369fd
  by Sam Thursfield at 2018-02-28T09:17:45Z

  Store integration tests cache inside the current directory by default
  
  Previously the code would default to a directory in `/tmp`, but this
  is often unsuitable as the Linux 'tmpfs' filesystem doesn't support
  extended file attributes and thus cannot store OSTree repositories.
  
  See: https://gitlab.com/BuildStream/buildstream/issues/267
  

• 5ebb7824
  by Sam Thursfield at 2018-02-28T09:17:45Z

  Modify how the INTEGRATION_CACHE environment var is interpreted
  
  When specifying a location for the integration tests cache directory,
  we would previously append '/integration-cache' to whatever path we
  were given. This is regarded as confusing.
  
  See: https://gitlab.com/BuildStream/buildstream/issues/267
  

• 7c290b43
  by James Ennis at 2018-02-28T16:46:05Z

  compose.py: key dictionary is now correctly initialised
  

• 543a0012
  by James Ennis at 2018-02-28T16:46:11Z

  Update expected cache keys in tests
  

• c88a8810
  by Jim MacArthur at 2018-03-01T14:36:08Z

  widget.py: Make the displayed text for LogLine messages a widget too
  
  Adds MessageOrLogFile widget, which contains the same functionality
  previously used to display messages.
  

• ad2781c2
  by Jim MacArthur at 2018-03-01T14:36:08Z

  widget.py: Modify TimeCode to take a microseconds argument
  

• 14234ccc
  by Jim MacArthur at 2018-03-01T14:36:08Z

  Add sequence ID and widget for it.
  
  The idea of a sequence number/ID is to tie together all messages
  associated with a given timed activity, to make performance
  analysis easier.
  
  _context.py: Add sequence number with atomic increment
  _widget.py: SequenceID widget, which displays this value
  _message.py: sequence_id variable storing the sequence for a message.
  

• c3f1a2e1
  by Jim MacArthur at 2018-03-01T14:36:08Z

  Add the FixedText and WallclockTime widgets
  
  WallclockTime is set at the time a message is created. Since messages
  can be placed in queues before being rendered, this is not always
  the same time as the wallclock time when rendering the message, so it
  needs to be stored in the message.
  
  widget.py: Add the two new widgets
  _message.py: record system time when a message is created.
  

• 03bdc5cd
  by Jim MacArthur at 2018-03-01T16:18:01Z

  Add logfile format tokeniser
  
  widget.py: Main body of tokenizer.
  _context.py: Read message-format from user config YAML.
  main.py: Pass in context message-format to LogLine constructor.
  

• 8c5a2c88
  by Jim MacArthur at 2018-03-01T16:18:08Z

  Remove brackets added inside widgets
  
  Brackets can now be added as plain text in a custom logging format
  string, so there's no need to add them in widgets.
  
  widget.py: Remove brackets.
  _status.py: Remove the 'brackets' optional argument to TimeCode.
  

• e566bc05
  by Jim MacArthur at 2018-03-01T16:18:31Z

  Add a test for default and custom logging
  

• c333adf4
  by Jonathan Maw at 2018-03-01T17:27:28Z

  Allow plugins to forbid runtime dependencies and sources
  
  The flags BST_FORBID_RDEPENDS and BST_FORBID_SOURCES can be set in the
  plugin's class declaration, and exceptions will be raised during
  pre-flight checks
  

• 2cbe1767
  by Jonathan Maw at 2018-03-01T17:27:28Z

  compose: Use built-in checks on runtime depends and sources
  

• 8a4fff9a
  by Jonathan Maw at 2018-03-01T17:27:28Z

  scriptelement: Use built-in checks on runtime depends and sources
  

• 42de997e
  by Jonathan Maw at 2018-03-01T17:27:28Z

  Pipeline: Give more helpful errors when opening a workspace for an element that has no sources
  
  When trying to look at the source code for an element, elements that
  modify artifacts (e.g. script and compose elements) don't have sources,
  so we suggest some other elements that they might try opening workspaces
  for.
  

• b884fe14
  by Jonathan Maw at 2018-03-01T17:27:28Z

  Add Filter element
  

• 09195b19
  by Jonathan Maw at 2018-03-01T17:27:28Z

  tests: Add filter element tests
  

• e0eb51b1
  by Tristan Van Berkom at 2018-03-02T07:11:54Z

  _frontend/cli.py: Better help output for the `--option` main option.
  
  Added metavar, making it more clear that you specify:
  
     bst --option OPTION VALUE
  

• 60e59a04
  by Tristan Van Berkom at 2018-03-02T07:11:54Z

  _frontend/cli.py: Adjust help output for `bst shell`
  
  Dont specify how we fallback to launching a shell with `sh -i`,
  since this is now configurable in project.conf.
  

• 5993c16c
  by Tristan Van Berkom at 2018-03-02T08:37:17Z

  NEWS: Adding missing entries for recently added features
  

• 85912562
  by Tristan Van Berkom at 2018-03-02T11:43:10Z

  _sandboxbwrap.py: Allow device nodes to be bind mounted
  

• a3f1a8ef
  by Tristan Van Berkom at 2018-03-02T11:51:23Z

  _sandboxbwrap.py: Force cleanup when in interactive mode
  
  When running a sandbox in interactive mode (implies `bst shell` was launched
  or an interactive shell for debugging), dont bail out when cleaning up
  directories which are not empty.
  
  We treat this as a bug, if any of the base sandbox directories (/dev,
  /tmp or /proc) are not empty when tearing down the sandbox, because
  it would indicate something is wrong with bwrap.
  
  When in interactive mode however, the user/project may have mounted
  additional directories inside these base directories; for which we
  need to create intermediate directories for the mount.
  
  Instead of keeping track of every intermediate directory, just force
  remove in interactive mode, as this is safe.
  
  Ideally, we should fix upstream bwrap to cleanup the debris it creates
  when exiting.
  

• 12ca9607
  by Tristan Van Berkom at 2018-03-04T11:56:07Z

  Enhanced bst shell configuration and cli options
  
  Some changes to the host-files configuration:
  
    o Dont require `host-files` to not be directories
  
      We need to specify directories to mount from `project.conf` after all.
  
    o Added possibility of specifying optional mounts, to avoid
      meaningless warnings where optional files don't exist on
      the host
  
  Added --mount CLI option to `bst shell`
  
  This allows users to explicitly mount whatever they want into the
  sandbox environment for `bst shell`.
  
  This closes issue #274
  

• bc0c4b71
  by Tristan Van Berkom at 2018-03-04T12:14:19Z

  tests/integration/shell.py: Updated shell tests
  
    o Added new test for `bst shell --mount`
  
    o Removed the test that a directory is refused in `host-files`,
      this is no longer a requirement
  
    o Updated names of the `host-files` mount specification members
  
    o Test the new `optional` feature of the mount specifications
  

• 83370f55
  by Tristan Van Berkom at 2018-03-04T12:15:04Z

  projectconf.rst: Amending documentation for `host-files`
  
  This is now enhanced and an API change was introduced.
  

• 2e13d61e
  by Tristan Van Berkom at 2018-03-06T07:00:03Z

  NEWS: Adding entry for new `--mount` options of `bst shell`
  

• 06ac03d4
  by Tristan Van Berkom at 2018-03-06T09:28:26Z

  integration tests: Refactoring for dynamic project configurations
  
    o CliIntegration.run() now takes a `project_config` option, this
      will be composited on top of the existing, substituted project.conf
  
    o Removing gnomesdk alias from integration tests project.conf
  
    o Using `sysroot` alias instead of `gnome7` alias
  
    o Make base-alpine.bst use the `sysroot` alias now
  
    o Removed `create_project_config()` from shell.py tests, now
      use the new built-in feature for this which is cleaner.
  

• c0be9295
  by Tristan Van Berkom at 2018-03-06T09:28:26Z

  _project.py: Support host environment expansion in `host-files` paths
  
  This allows one to mount paths based on host specific environment
  variables, this can be useful for mounting things under XDG_RUNTIME_DIR,
  which is needed for host pulseaudio connections.
  
  This is another part of addressing #223
  

• eef26583
  by Tristan Van Berkom at 2018-03-06T09:28:26Z

  tests/integration/shell.py: Adding tests for env var expansions in `host-files`
  

• ea74f326
  by Tristan Van Berkom at 2018-03-06T10:31:13Z

  Implement explicit environment assignments for `bst shell`
  
  This introduces a new `environment` section of the project `shell`
  configuration to set explicit env vars when running a non-isolated shell.
  
  This supports host environment variable expansion.
  
  This is a part of addressing #223
  

• 7ed0351a
  by Tristan Van Berkom at 2018-03-06T10:33:22Z

  tests/integration/shell.py: Testing new shell environment configuration
  

• f8a71cab
  by Tristan Van Berkom at 2018-03-06T11:22:02Z

  doc/source/projectconf.rst: Documenting new `environment` shell configuration
  
  And also ammend the documentation for `host-files` to note that
  the paths support host side environment variable expansion.
  
  Documenting both `host-files` and `environment` to require format version 4.
  

• be6b5564
  by Tristan Van Berkom at 2018-03-06T12:20:27Z

  Removing all traces of `environment-inherit` shell configuration.
  
  This is made redundant by the more complete `environment` configuration,
  so lets quickly remove the former in this new format version 4.
  

• 1f63d214
  by Tristan Van Berkom at 2018-03-06T12:41:36Z

  NEWS: Adding a few missing entries for the release
  

• ec5ee795
  by Tristan Van Berkom at 2018-03-06T12:42:41Z

  man: Updated man pages for the release
  

• 5a0f56af
  by Tristan Van Berkom at 2018-03-06T13:12:10Z

  doc/source/projectconfig.rst: Document `sources` overrides as since format version 1
  
  This was added slightly before format version 1, but was done
  without a version increment. Previous format versions cannot
  be relied upon for use of this feature.
  

• df9f696f
  by James Ennis at 2018-03-07T08:42:49Z

  _pipeline.py: Remove undefined variable source_index
  

• 141484fc
  by Sam Thursfield at 2018-03-07T09:07:55Z

  HACKING.rst: Mention benchmarking and profiling tools
  
  This adds a reference to the benchmarking tool generated as part of
  https://gitlab.com/BuildStream/buildstream/issues/205.
  
  It also documents recommended strategies for profiling, which fixes
  https://gitlab.com/BuildStream/buildstream/issues/206.
  

• 7fdddf30
  by Tristan Van Berkom at 2018-03-07T11:44:37Z

  Silence messages from Source cache interrogation
  
  Source interrogation usually involves calling out to host tools
  to quickly check if a given ref exists. This has however regressed
  over time when running `bst build --track`.
  
  This patch adds a new context manager to silence the messages,
  and silences messages while calling `Source.get_consistency()`
  
  Fixes #280
  

• 9f6ced45
  by Jim MacArthur at 2018-03-07T12:36:19Z

  status.py: Restore brackets to time in job display area
  

• 10fa1b4e
  by Tristan Van Berkom at 2018-03-10T09:53:36Z

  element.py: Report errors using `self.get_kind()`
  
  We should always show the user the "kind" string when referring
  to a plugin type, not the type name with `type(self).__name__`.
  

• 5e299889
  by Tristan Van Berkom at 2018-03-13T07:57:15Z

  _frontend/widget.py: Never abbreviate frontend messages
  
  Messages which come from the frontend do not have any backing log
  file where the remainder of the message can be reviewed. In the
  case of lengthly detailed messages originating from the frontend,
  we dont allow any abbreviation of the message.
  

• 9be5a16c
  by Jim MacArthur at 2018-03-13T12:03:13Z

  Remove other references to SequenceID from the code and tests
  
  widget.py: Remove SequenceID from the list of widgets.
  tests/frontend/logging.py: Remove %{sequence} check.
  

• ad8bd6f9
  by Jim MacArthur at 2018-03-13T12:03:13Z

  Revert "Add sequence ID and widget for it."
  
  This reverts commit 14234ccce3e7eec1153e810fab75afb2e90a2b22.
  

• 92b04688
  by Jim MacArthur at 2018-03-14T10:51:56Z

  doc/source/formatintro.rst: Better explanation of defaults
  
  Users shouldn't see any difference between values hardcoded in Python (if
  there are any) and the values in projectconfig.yaml, which were previously
  items 0 and 1 respectively. Combined these both into 1 and made it clear that
  not all of project.conf is applied right away.
  

• 76818202
  by James Ennis at 2018-03-14T14:10:26Z

  Add pylint to setup.cfg and setup.py
  

• 793780c6
  by James Ennis at 2018-03-14T14:10:26Z

  Generate pylint configuration file (pristine file):
  
  NOTE: pylint was installed using pip: `pip3 install pylint` and
  .pylintrc is the pylint configuration file which was generated with
  `pylint --generate-rcfile` from a fresh fedora-27 docker image.
  

• 7e79c4af
  by James Ennis at 2018-03-14T14:10:26Z

  .pylintrc: Amend which warning messages are disabled
  
  NOTE: We have categorised the warning messages into two groups.
  Those that are of no use to us in the buildstream project and
  those that should be enabled at some point.
  

• 6be20757
  by James Ennis at 2018-03-14T14:10:26Z

  .pylintrc: Alterations to the configuration file
  

• 388c1570
  by Tristan Maat at 2018-03-14T14:10:26Z

  fuse.py: skip file for linting
  

• 6eac2eda
  by Tristan Maat at 2018-03-14T14:10:26Z

  Add pylint to CI
  

• ef0efd70
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with anomalous-backslash-in-string warning
  

• 1ec691c7
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with bad-continuation warning
  

• cdd17bf3
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with broad-except and bare-except warnings
  

• a4a302bb
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with consider-merging-isinstance warnings
  

• 1ba6167b
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with dangerous-default-value warning
  

• 05fa346f
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with global-statement warning
  

• 4741d137
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with len-as-condition warning
  

• 0394d67d
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - disabled no-member, bad-exception-context and catching-non-exception warnings
  

• 81833add
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with superfluous-parens warning
  

• b11b48ea
  by James Ennis at 2018-03-14T14:10:26Z

  pylint - dealt with unidiomatic-typecheck warning
  

• 9b4f54e1
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with import warnings
  

• 432ddffd
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with unnecessary-pass warning
  

• 4cfa0bdf
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with redefined-argument-from-local warning
  

• dfa45f04
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with redefined-outer-name and redefined-built in warnings
  

• 3df8fb9f
  by James Ennis at 2018-03-14T17:31:06Z

  pushreceive.py: byteorder functions are now more sensibly named
  

• ccec158a
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with simplifiable-if-statement warning
  

• 16ff3374
  by James Ennis at 2018-03-14T17:31:06Z

  pylint - dealt with whitespace issues and disabled pep8 warnings
  

• 4b644aba
  by Jürg Billeter at 2018-03-15T15:48:07Z

  buildqueue.py: Do not mark assembly complete on build failure
  
  This fixes internal error when build is retried.
  
  Fixes #299
  

• 3f98a55d
  by Jürg Billeter at 2018-03-15T15:50:08Z

  element.py: Do not allow unplanned build jobs
  

• a92f9ddb
  by Jürg Billeter at 2018-03-15T17:43:05Z

  _sandboxchroot.py: Ensure the cwd exists
  
  This matches SandboxBwrap.
  

• 3861de9b
  by Jürg Billeter at 2018-03-15T17:43:05Z

  scriptelement.py: Mark install-root
  
  This matches BuildElement.
  

• 9543d078
  by Tristan Van Berkom at 2018-03-16T08:21:15Z

  _project.py and docs: Move defaults into the defaults yaml file
  
  Over time, the _project.py module has regressed into expressing
  some defaults only hard coded into the python file instead of
  properly exposing their default in the base configuration file
  in data/projectconfig.yaml, where the default values can be observed
  by users.
  
  This patch rectifies that, and also restructures the relevant
  surrounding documentation a bit.
  

• c187ab76
  by Tristan Van Berkom at 2018-03-16T08:39:37Z

  doc/source/format.rst: Fix misnomer in documentation
  
  The provided example shows the `tar` source ref being
  expressed with a `sha256sum` key name, lets make the example
  correct here and call it `ref`.
  

• 47ec4f52
  by James Ennis at 2018-03-16T11:44:45Z

  .pylintrc: Ensure pylint ignores gi.repository module as it dynamically adds classes
  

• 9cb30d35
  by James Ennis at 2018-03-16T11:44:45Z

  Make pylint and pep8 tests run by default
  

• 73993644
  by Javier Jardón at 2018-03-17T09:25:26Z

  docs/source/install.rst: Arch -> Arch Linux
  

• cf778d36
  by Javier Jardón at 2018-03-17T09:25:26Z

  docs/source/install.rst: fix terminal box for ArchLinux
  

• ca6dd66f
  by Javier Jardón at 2018-03-17T09:29:05Z

  source/install.rst: Divide in two sections
  
  - installing from distro packages
  - installing from source
  

• 5d7ee178
  by Rafael Fontenelle at 2018-03-17T09:29:05Z

  source/install.rst: Add instructions to install Arch packages
  

• 6b2f1727
  by Tristan Maat at 2018-03-19T14:56:45Z

  .gitlab-ci.yml: Fix lack of setuptools_scm causing CI failure
  

• 024a37c5
  by Tristan Maat at 2018-03-19T14:56:50Z

  .gitlab-ci.yml: Remove superfluous pytest-pylint install
  

• 810f6c33
  by Tristan Van Berkom at 2018-03-20T07:55:56Z

  Deprecate `--track-save` option from `bst build`
  
  It makes no sense to support this now that we are adding project.refs,
  instead of removing the option completely, we document it as
  deprecated and print a warning.
  
    o _frontend/cli.py: Mark the option as deprecated, warn if it's used
  
    o _scheduler/trackqueue.py: Remove save optionality
  
    o _pipeline.py: Remove save optionality
  
    o tests/frontend/buildtrack.py: Stop testing for no-save functionality,
  
      This test was actually broken, and only save functionality was being
      tested. This was due to using a list [True] or [False] in the
      @pytest.mark.parameterize() statement, both of which are truthy values.
  

• 7777d390
  by Tristan Van Berkom at 2018-03-20T07:55:56Z

  source.py: Document ref accessors to mandate support for None values.
  
  This is technically an API break, but will be transparant for the
  vast majority of the current hand full of source implementations
  which exist at this time. This is a lesser evil than bloating the
  API with new methods.
  

• 1fbedea8
  by Tristan Van Berkom at 2018-03-20T07:55:56Z

  _downloadablefilesource.py: Support None values in ref accessors.
  

• a39927a8
  by Tristan Van Berkom at 2018-03-20T08:46:41Z

  _projectrefs.py: Adding the ProjectRefs refs management object
  
  This object manages the project.refs file in a project directory.
  

• e2392ce7
  by Tristan Van Berkom at 2018-03-20T08:46:41Z

  _project.py: Load project.refs while loading the project.
  
  This also bumps the format version to signify a new format
  where storing refs centralized in project.refs is now supported.
  

• 19cad981
  by Tristan Van Berkom at 2018-03-20T08:46:42Z

  Fix #248 - Support project.refs in the core.
  
  This adds a new Source.load_ref() API which is technically optional to
  implement, projects which make use of a project.refs file must only
  use source plugins which implement the new load_ref() method.
  
    * source.py: Added load_ref() API to load a ref from a specified node.
  
      This also adds _load_ref() and _save_ref() wrappers which handle
      the logistics of when to load and save a ref to which location.
  
      This also fixes _set_ref() to apply the ref to the node unconditionally,
      this must be done independantly of whether the ref actually changed.
  
    o Modifications to the loading process such that Source now can have
      access to the element name and source index.
  
    o _pipeline.py: Delegate abstract loading of source refs to Source._load_ref()
  
      - Print a summarized warning about redundant source references
  
      - Assert that one cannot track cross-junction elements without project.refs.
  
    o _scheduler/trackqueue.py: Delegate saving refs to Source._save_ref()
  

• 93e29955
  by Tristan Van Berkom at 2018-03-20T08:46:42Z

  Source plugins: Implement load_ref() in all source plugins
  

• 6b63828c
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/frontend/buildtrack.py: Modified to test saving with project.refs
  
  Test that we get the same functionality from the complex build + track
  test regardless of whether we are saving refs inline or to a project.refs file.
  

• 50400e12
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/pipeline/load.py: Testing source without load_ref() implementation
  
  Test that we have no error when loading a project that doesnt use
  project.refs, and that we get the expected graceful failure when
  attempting to use project.refs with a Source which doesnt support it.
  

• 83a18f98
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/format/project.py: Test that conditional statements are supported in project.refs
  

• 0f6b75dc
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  testutils/repo/git.py: Added branch() method
  
  In order to test tracking on multiple branches, lets use
  the Git repo scaffolding.
  

• 150c7441
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/frontend/track.py: Testing that this works with project.refs
  
  Only added condition to the simplest case here, the other cases
  are mostly testing that track commands get the correct selection
  of elements when using `--deps all` and `--except` arguments.
  

• 2bcae91f
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/frontend/track.py: Test tracking with optionality
  
  Tests that `bst track` sets the ref in the expected node if the
  node containing the ref is conditionalized with a project option.
  
  This tests both the regular inline behavior, and also the project.refs behavior.
  

• da95c639
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  tests/frontend/track.py: Testing behavior of cross junction tracking
  
    o Test that this fails gracefully when the toplevel project uses
      inline ref-storage
  
    o Test that we successfully track cross junctioned elements when
      the project uses project.refs
  

• 208250cb
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  Documenting project.refs
  
    o Adding explanation of the `ref-storage` option in project.conf
  
    o Adjusting some adjacent relevant documentation
  
    o Adding a new projectrefs.rst section for the project.refs
      format itself, and linking it from the main authoring page.
  

• 3c3e9f79
  by Tristan Van Berkom at 2018-03-20T09:29:14Z

  NEWS: Adding news entry for new ref-storage options (project.refs)
  
  Also announce --track-save deprecation
  

• a3cb8199
  by Tristan Van Berkom at 2018-03-20T09:56:40Z

  source.py and loading: Refactoring of origin handles
  
  The Source object previously stored the __origin_node,
  __origin_toplevel and __origin_filename, this is from a time
  when we did not hold on to the plugin's Provenance object
  explicitly.
  
  Since this information comes from the same place, let's just
  use Plugin._get_provenance() to derive these values, instead
  of redundantly carrying them along separately.
  

• e8e88e5c
  by Jonathan Maw at 2018-03-20T13:18:00Z

  filter: Reword docstring to explain what the element does
  
  The docstring included instructions on what it *should* be used for,
  which is unnecessary.
  
  Further, it was vague on what the element actually did. The new
  docstring should be more explicit on what it does.
  

• 54827aa7
  by Javier Jardón at 2018-03-20T15:35:07Z

  Change theme to sphinx_rtd_theme
  
  The sphinx_rtd_theme is a sphinx theme designed to look modern and be mobile-friendly
  http://sphinx-rtd-theme.readthedocs.io/en/latest/index.html
  

• d97e9b95
  by Javier Jardón at 2018-03-20T15:35:07Z

  source/index.rst: Separate general and reference documentation
  
  Move artifacts from Installing section to general documentation
  
  This change is needed to make the TOC work correctly with the
  new theme
  

• e2564d21
  by Javier Jardón at 2018-03-20T16:31:53Z

  source/install.rst: put Debian version under the same subsection
  

• 9cf75154
  by Javier Jardón at 2018-03-20T16:31:53Z

  source/conf.py: Keep documentation version in sync
  

• 91f47004
  by Javier Jardón at 2018-03-20T16:45:34Z

  source/conf.py: Update copyright year to 2018
  

• 6a5e8446
  by James Ennis at 2018-03-20T17:05:02Z

  index.rst: Fixed typo in About BuildStream
  

• c36c545d
  by Tristan Van Berkom at 2018-03-21T13:53:18Z

  source.py: Source._save_ref() should return saved state
  
  So we can report proper processed status from tracking queues
  

• 66ff0245
  by Tristan Van Berkom at 2018-03-21T13:53:48Z

  _scheduler/trackqueue.py: Fixed to report proper processed status.
  
  This was a regression from the project.refs branch recently landed.
  

• f9edceed
  by Tristan Van Berkom at 2018-03-21T14:23:28Z

  .gitlab-ci.yml: Pin the sphinx version to 1.7.1
  
  The new version today 1.7.2 causes the following build error:
  
      Warning, treated as error:
      /builds/BuildStream/buildstream/dist/buildstream/doc/source/buildstream.sandbox.rst:document isn't included in any toctree
  

• 98b0d656
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  _message.py, _frontend/widget.py: Reviving MessageType.ERROR
  
  We need to distinguish between:
  
    o Errors which occurred in a task, and are related to a log file
      and an elapsed time.
  
    o Errors which occurred in the main process and are not related
      to any timed activity
  

• 28bff099
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  _scheduler/queue.py: Handle errors in task post processing gracefully.
  
  When errors occur after having processed an element, leaving these
  unhandled results in a stack trace that is partially covered up by
  the status area, and a hang is caused because we fail to update the
  queues properly with the failed element.
  
  This greatly improves the situation for issue #197
  

• 96c5c427
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  tests/frontend/track.py: Test that we fail gracefully for post tracking errors
  
  This tests that we handle errors from Source.get_consistency() in the
  post tracking state updates gracefully, one test added for a handled
  failure, and another test added for an unhandled/unexpected exception.
  

• cf43f8cf
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  tests/frontend/fetch.py: Test failure modes of Source.get_consistency()
  
  Added a test that handled errors are reported at load time as expected.
  
  Added another test that we get the expected exception. This needs to
  be fixed, test contains FIXME: comment explaining that we could be
  doing much better here.
  
  This should be fixed in the context of issue #197
  

• 21fe51fb
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  _context.py: Fix Context._silence() context manager.
  
  This needs to yield in a try: block, otherwise messages
  remain silenced after an exception passes through this
  context manager.
  

• 41274b2d
  by Tristan Van Berkom at 2018-03-21T14:38:07Z

  _scheduler/trackqueue.py: Remove unsafe calls to push/pop message depth
  
  These messages have been silenced more reliably already
  in Source._update_state(), using the Context._silence() context manager.
  

• d246403e
  by Tristan Van Berkom at 2018-03-21T15:58:19Z

  git.py source plugin: Ignore inconsistent submodules.
  
  If a submodule is listed in either the buildstream git source
  definition, or in the .gitmodules file; but is not a valid submodule
  which was added with `git submodule add`, we now emit a warning
  and avoid exploding in the user's face.
  
  This fixes issue #299
  

• 0f62dd05
  by Tristan Van Berkom at 2018-03-21T15:58:40Z

  tests/testutils/repo/git.py: Add new add_file() convenience
  
  To write tests which add and commit a file.
  

• 54ba7324
  by Tristan Van Berkom at 2018-03-21T15:58:40Z

  tests/sources/git.py: Test that we ignore inconsistent submodules.
  
  Test this at `bst track` time, when encountering a new ref which
  adds a .gitmodules file but does not actually add the submodule,
  we check that the expected warning is in the stderr and that BuildStream
  does not error out for this.
  

• dd67fecd
  by Tristan Van Berkom at 2018-03-22T08:27:38Z

  _artifactcache/ostreecache.py: Turn an ArtifactError() into an assert statement
  
  This should be an assertion as it will trigger a proper BUG
  message with a usable stack trace.
  
  An incident of this ArtifactError() was reported in issue #305.
  

• 9766da6e
  by Javier Jardón at 2018-03-22T08:40:24Z

  Add HACKING document to official docs
  

• 92509174
  by Javier Jardón at 2018-03-22T11:12:29Z

  .gitlab-ci.yml: No need to install python2 for docs job
  
  This also make the job faster: from ~1min to ~30s
  

• a52cc0dd
  by Javier Jardón at 2018-03-22T12:15:55Z

  source/conf.py: change copyright/author to "BuildStream Team"
  

• d1f25425
  by Richard Maw at 2018-03-22T15:15:39Z

  project.py: Add workspace.yml format versioning and version 1
  

• 357ac70e
  by Richard Maw at 2018-03-22T15:41:34Z

  frontend/workspace.py: Add tests for loading different versions
  

• 7d92ef0f
  by Phillip Smyth at 2018-03-23T12:01:20Z

  Created deb Source plugin for Issue #10
  
  added tests
  

• f4d3892e
  by Jim MacArthur at 2018-03-23T14:37:38Z

  Add 'sandbox' configuration key and build-uid/build-gid elements
  
  This only affects SandboxBWrap at the moment.
  
  buildstream/_loader.py: Add Symbol.SANDBOX and allow it in validation
  buildstream/_metaelement.py: Add 'sandbox' variable and store it in the object
  buildstream/_project.py: Add 'sandbox' configuration key and load it from
      project.conf.
  buildstream/data/projectconfig.yaml: Default build-uid/build-gid values of 0
      for 'sandbox'.
  buildstream/element.py: Add __extract_sandbox_config to find the final sandbox
      configuration. Pass this to the sandbox constructor.
  buildstream/sandbox/_sandboxbwrap.py: If sandbox configuration was supplied,
      use it for uid and gid instead of the default 0.
  buildstream/sandbox/_sandboxchroot.py: Throw exception if non-0 uid/gid were
      supplied.
  buildstream/sandbox/__init__.py: Import SandboxConfig.
  buildstream/sandbox/_private.py: New file, containing SandboxConfig. Made private
  to avoid documentation for this class.
  

• 7d61e8c1
  by Jim MacArthur at 2018-03-23T14:37:38Z

  Add simple tests of builder UID.
  
  tests/integration/build-uid: Add tests.
  tests/integration/project/elements/build-uid/build-uid-default.bst:
      A test for the default uid/gid of 0 in the sandbox.
  tests/integration/project/elements/build-uid/build-uid.bst:
      Tests for nonzero uid/gid values supplied in the element.
  tests/integration/project/elements/build-uid/build-uid-1023.bst:
      Tests for nonzero uid/gid values supplied in project.conf.
  

• cd7d0641
  by Jim MacArthur at 2018-03-23T14:37:38Z

  Add documentation on sandbox option
  

• e302baf5
  by Jim MacArthur at 2018-03-23T14:37:38Z

  Increase BST_FORMAT_VERSION to 6
  

• 3ea2297b
  by Jim MacArthur at 2018-03-23T14:37:38Z

  NEWS: Add mention of the new sandbox option
  

• 83620b38
  by Tristan Van Berkom at 2018-03-25T08:25:54Z

  sandbox/_config.py: Replace docstring with comment.
  
  For consistency, we document private things with comments,
  and public things with docstrings.
  

• c9b5692e
  by Tristan Van Berkom at 2018-03-25T08:30:55Z

  Consider new sandbox configuration in cache keys.
  
  This adds SandboxConfig.get_unique_key() to delegate
  the cache key contribution to the SandboxConfig.
  
  Further, this moves over the OS and Machine Architecture
  parts of cache key calculation into SandboxConfig.get_unique_key(),
  removing the comment which speaks of later delegating this
  part of cache key calculation to sandboxes.
  
  Cache key calculation algorithm is backward compatible,
  and so the artifact version remains unchanged.
  

• 094d878a
  by Tristan Van Berkom at 2018-03-25T09:09:41Z

  tests/cachekey: Added test to exercise new sandbox contributions to cache key
  

• d8ec3ecb
  by Jürg Billeter at 2018-03-25T09:20:07Z

  tests/testutils/site.py: Add HAVE_ARPY
  

• fa25a6a2
  by Jürg Billeter at 2018-03-25T09:20:17Z

  tests/sources/deb.py: Skip tests if arpy is not available
  
  Fixes #317
  

• c9701cdf
  by Jürg Billeter at 2018-03-25T10:12:02Z

  Remove unused imports
  

• 3845c3e0
  by Jürg Billeter at 2018-03-25T10:12:02Z

  .pylintrc: Enable unused-import checker
  

• a27d6620
  by Tristan Van Berkom at 2018-03-25T10:24:37Z

  NEWS: Added news entry for newly added `deb` source
  

• 65654ef4
  by Tristan Van Berkom at 2018-03-25T10:24:37Z

  Docs: Adding deb source plugin to the docs index
  

• e91dc833
  by Tristan Van Berkom at 2018-03-25T10:28:48Z

  man: Regenerating man pages for release
  
  This needed an update at least for the deprecation
  of `bst build --track-save`
  

• b0e6a45e
  by Phillip Smyth at 2018-03-26T15:27:47Z

  Generate unique subdirs for built elements
  
  Based on issue 89 (https://gitlab.com/BuildStream/buildstream/issues/89)
  Ensuring that elements are staged into unique subdirs while building.
  This patch supports that by doing the following:
  
  * Modify project config to add 2 new variables ("project-name" and "element-name")
  * Changed the default install-root from "/buildstream/install" to "/buildstream-install"
  * Update the tests to accommodate these changes
  * Update the expected cache keys in the tests
  
  This fixes #89
  

• b606fd14
  by Tristan Maat at 2018-03-27T09:58:18Z

  _ostree.py: Add diff_dirs function
  

• fb332267
  by Tristan Maat at 2018-03-27T09:58:18Z

  _artifactcache: Add diff() interface
  

• f761140f
  by Tristan Maat at 2018-03-27T13:32:52Z

  Make workspaces use objects instead of pipeline helper methods
  

• 6d5a7217
  by Tristan Maat at 2018-03-27T13:32:54Z

  Record last successful workspace build key
  

• dc9a4d39
  by Tristan Maat at 2018-03-27T13:32:54Z

  Allow stage_artifact to update mtimes
  

• 18896a9e
  by Tristan Maat at 2018-03-27T13:32:54Z

  element.py: Update mtimes of modified dependency files (#216)
  

• 1d9ea917
  by Tristan Maat at 2018-03-27T13:32:54Z

  Add list of running files to workspace metadata
  

• 989bb3fe
  by Tristan Maat at 2018-03-27T13:32:54Z

  element.py: Disallow incremental builds for caches that can't diff
  

• a7ae93ce
  by Tristan Maat at 2018-03-27T13:32:55Z

  integration/workspace.py: Add updated dependency tests
  

• be5a8638
  by Tristan Maat at 2018-03-27T13:32:55Z

  frontend/workspace.py: Adjust tests to new workspace format
  

• 1e2100d9
  by James Ennis at 2018-03-27T18:05:14Z

  install.rst: Inform the user to restart their terminal once their PATH is adjusted
  

• 0cb0be7a
  by James Ennis at 2018-03-27T18:05:14Z

  install.rst: Change ~ for $HOME just incase it expands to characters bash does not like
  

• 9de1c626
  by Javier Jardón at 2018-03-28T02:19:13Z

  source/main_core.rst: Use toctree instead normal list
  
  So index appear in left menu
  
  This is a part of the fix for #246
  

• 500bd0d2
  by Javier Jardón at 2018-03-28T02:19:43Z

  source/main_using.rst: Use toctree instead normal list
  
  So index appear in left menu
  
  This is a part of the fix for #246
  

• a32d15e3
  by Javier Jardón at 2018-03-28T02:19:47Z

  source/main_authoring.rst: Use toctree instead normal list
  
  So index appear in left menu
  
  This is a part of the fix for #246
  

• bb75d4a5
  by Javier Jardón at 2018-03-28T02:19:51Z

  docs: Remove orphan tag from all pages
  
  This is not needed anymore as all documented are part of the index
  
  This is a part of the fix for #246
  

• 6e2a342d
  by Tristan Van Berkom at 2018-03-28T03:44:02Z

  _project.py: Move format version assertion before any validation
  
  This fixes #314
  

• 95510ad6
  by Tristan Maat at 2018-03-28T09:25:48Z

  .gitlab-ci.yml: Use debian-8 for tests to test python3.4
  

• 2d7b721b
  by Tristan Maat at 2018-03-28T12:16:10Z

  _workspaces.py: Fix fallout from using python3.4
  

• 118bea08
  by James Ennis at 2018-03-28T16:42:50Z

  element.py: Make artifact lowercase in time_activity()
  
  NOTE: artifact was also made lowercase in the function
  get_pushed_elements() in runcli.py
  

• 9377212d
  by Javier Jardón at 2018-03-29T10:45:58Z

  Add basic resources page
  
  with info about git repo, IRC, mailing list and big tracker
  

• fbb0ca28
  by Javier Jardón at 2018-03-29T10:45:58Z

  source/index.rst: Add resources page to "contributing" section
  

• b9546d4d
  by Javier Jardón at 2018-03-29T11:25:08Z

  HACKING.rst: Be specific about the only pip packages required are the python3 ones
  

• 63d99463
  by Tristan Van Berkom at 2018-04-01T11:50:59Z

  Include initialization time in the total session time of the build log
  
  This patch refactors the frontend App object and touches a few internals.
  
    o The scheduler now takes a start time given to it at instantiation time,
      instead of considering the start time to commence in Scheduler.run()
  
    o The App.initialize() method has been swapped out for an
      App.initialized() context manager.
  
      This context manager now takes care of the main start/fail/success
      messages. For convenience and ensured consistency, this context manager
      is now responsible for:
  
        o Printing the startup heading
        o Printing the end of session summary
        o Exiting with a consistent error status in case of errors
  
    o The Pipeline() object no longer prints the start/fail/success messages,
      as they are now taken care of by the App()
  
    o The cli.py frontend code is much simplified by using the new context
      manager, also enforcing consistency in how we handle and report errors.
  
  This fixes issue #186
  

• aa8410d8
  by Tristan Van Berkom at 2018-04-01T12:59:01Z

  _workspaces.py: Dont unconditionally create workspace local state file at startup.
  
  Fixes issue #257
  

• 23cea695
  by Tristan Van Berkom at 2018-04-01T18:57:01Z

  _frontend/widget.py: Add "Strict Build Plan" setting to session heading
  
  Looks like this quite interesting attribute was not getting logged.
  

• 999d168a
  by Tristan Van Berkom at 2018-04-01T19:02:02Z

  _frontend/widget.py: Change heading separator to use '=' instead of '~'
  
  Seems that using `~` conflicts with some markdown when trying to
  paste build logs in, e.g. gitlab issues.
  

• 00ad7629
  by Tristan Van Berkom at 2018-04-02T07:01:54Z

  _frontend/app.py: Renamed from main.py
  
  Since the CLI and the App object was split up, we should name
  the file after the object which it implements.
  

• 6c9c2d42
  by Tristan Van Berkom at 2018-04-02T07:53:45Z

  _frontend/app.py: Split up initialization into two parts.
  
  Created partial_init() function to do a partial initialization
  which does not require loading the Pipeline, which is more expensive.
  
  This is useful for some bst commands which dont operate on a pipeline.
  

• 8b5742dd
  by Tristan Van Berkom at 2018-04-02T10:53:59Z

  _exceptions.py: Adding new AppError exception and error domain
  

• 19e31adb
  by Tristan Van Berkom at 2018-04-02T10:55:57Z

  Refactoring of highlevel workspace code
  
  Move all workspace related code out of Pipeline() and into the
  frontend App() object.
  
  Some changes in transition here include:
  
    o Workspaces() object methods for looking up and deleting workspaces
      now take an element name instead of an element.
  
    o Share code for partial App() initialization between the
      `workspace close` and `workspace list` commands
  
    o No longer require that an element exist in the project
      in order to close a workspace
  
  This fixes issue #249
  

• 202dd7cd
  by Tristan Van Berkom at 2018-04-02T11:00:06Z

  source.py: Remove unused method Source._del_workspace()
  
  This was never particularly useful, there is no circumstances
  under which a workspace needs to be deleted, and a cache key
  invalidated, in the course of a session.
  
  A workspace is deleted only atomically as a part of `bst workspace close`,
  which does not even load a pipeline anymore, so the pipeline state need not
  be adjusted in this case.
  

• 22c6ef17
  by Tristan Van Berkom at 2018-04-02T11:04:59Z

  tests/frontend/workspace.py: Fix inaccurate comments
  

• af719007
  by Tristan Van Berkom at 2018-04-02T11:11:11Z

  tests/frontend/workspace.py: Test that we can remove a workspace for a nonexisting element
  
  Guard against regressions of issue #249
  

• 18b60dbb
  by Tristan Van Berkom at 2018-04-02T11:17:54Z

  tests/frontend/workspace.py: Reduce unneeded coverage here.
  
  Here we are overly testing the same functionality which does
  not require source specific support, e.g. lets not test resetting
  of workspaces for every kind of repo, since we already test opening
  and closing of workspaces for every repo kind, this is redundant
  and slowing down tests.
  

• af08b16e
  by Tristan Van Berkom at 2018-04-03T13:49:10Z

  HACKING.rst: Documenting naming policy for private symbols
  
  This is a part of issue #285
  

• 70c73b93
  by Tristan Van Berkom at 2018-04-03T13:49:10Z

  _workspaces.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 500f4f33
  by Tristan Van Berkom at 2018-04-03T13:49:10Z

  _context.py: Adhere to policy on private symbols
  
  And adjust all surrounding sources for changed symbols.
  
  This is a part of issue #285
  

• 01db4072
  by Tristan Van Berkom at 2018-04-03T13:49:11Z

  _exceptions.py: Adhere to policy on private symbols
  
  And adjust all surrounding sources for changed symbols.
  
  Also, added new LoadErrorReason.UNSUPPORTED_PLUGIN, required
  for changes in how the project will report format version errors
  for plugins at creation time
  
  This is a part of issue #285
  

• af34d716
  by Tristan Van Berkom at 2018-04-03T13:49:11Z

  _project.py: Adhere to policy on private symbols
  
  And adjust all surrounding sources for changed symbols.
  
  Additional details:
  
    o Added Project.get_shell_config() to report the shell configuration,
      instead of making those members all public
  
    o Moved assertions about project specified format versions required
      of plugins out of Plugin.__init__, and into Project.create_element()
      and Project.create_source(), so we can keep more things private
  
  This is a part of issue #285
  

• 5c33a984
  by Tristan Van Berkom at 2018-04-03T14:48:56Z

  _pipeline.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 6c14a05a
  by Tristan Van Berkom at 2018-04-04T04:57:50Z

  _plugincontext.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• aff532d4
  by Tristan Van Berkom at 2018-04-04T05:11:50Z

  _variables.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 355b2263
  by Tristan Van Berkom at 2018-04-04T06:17:26Z

  _artifactcache modules: Adhere to policy on private symbols
  
  Additionally:
  
    o This shares more code by creating ArtifactCache.get_artifact_fullname(),
      which is used for the extract directory relative path in both backends,
      and for the ostree "ref" in the ostree backend.
  
    o Further, this removes some redundant documentation in derived abstract
      methods, and clarifies in both backends which methods are abstract methods,
      we should only be documenting abstract method semantics in one place, where
      they are defined.
  
  This is a part of issue #285
  

• 71f07656
  by Tristan Van Berkom at 2018-04-04T09:16:57Z

  Fix strict setting to be consistent throughout the pipeline.
  
  I originally changed the configuration of strict mode to be on
  a per project basis in the user configuration, because the user should
  be able to set their preference on a per project basis.
  
  I however made the mistake to make the strict mode be considered
  on a per project basis within a single pipeline, this commit corrects
  the behavior such that when you are building a project with junctions,
  only the toplevel project is considered when deciding strict mode
  for the whole pipeline.
  

• 9012a5ed
  by Tristan Van Berkom at 2018-04-05T06:16:15Z

  element.py: Fix _update_state() to consider whether an element is to be built properly
  
  This was previously using self._buildable() to determine whether
  an element is built locally and not to be downloaded, which misses
  out on elements which have already been built, like open workspaces
  which can only have their cache key calculated after the build.
  
  This fixes issue #316
  

• daffe498
  by Tristan Van Berkom at 2018-04-05T06:16:15Z

  tests/frontend/workspace.py: Exercise the build test in strict and non-strict mode
  
  This test fails without the previous patch fixing issue #316,
  so this should guard against regressions of building and caching
  workspace builds in non-strict mode.
  

• 638344e4
  by Tristan Van Berkom at 2018-04-05T06:16:15Z

  element.py: Updating comment for _assemble_done()
  
  This was misinforming that it is only called in a subprocess,
  it also needs to be called in the main process, also point out
  that updating element state is a side effect of this.
  

• cc43127e
  by Tristan Van Berkom at 2018-04-05T06:16:15Z

  _scheduler/buildqueue.py: Dont call Element._update_state() here
  
  This is implied by Element._assemble_done()
  

• 8c8b1c54
  by Tristan Van Berkom at 2018-04-05T06:35:20Z

  _options package: Adhere to policy on private symbols
  
  This also adds a couple of methods to OptionPool to avoid
  exposing some internals to other parts of the codebase, and
  calls those new methods from _project.py and _frontend/widget.py
  
  This is a part of issue #285
  

• b8c68284
  by Tristan Van Berkom at 2018-04-05T06:35:20Z

  _platform package: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• ab63b536
  by Tristan Van Berkom at 2018-04-05T08:13:35Z

  _frontend/widget.py: Defend against empty string message detail
  
  It appears we have some cases of empty detail strings that are
  not None, this was resulting in an IndexError when trying to strip
  the trailing newline from the message.detail.splitlines() result.
  

• 207b986f
  by Tristan Van Berkom at 2018-04-05T08:32:42Z

  _frontend/widget.py: Print the workspace directory for %{workspace-dirs}
  
  Instead of printing the list of workspace directory for each source.
  
  This fixes some fallout from the changes to make workspaces element-wide
  instead of being on a per-source basis, which was merged as merge request !257
  as a part of issue #209.
  

• a4eb8bd2
  by Tristan Van Berkom at 2018-04-05T08:35:03Z

  element.py: Removing Element._workspace_dirs()
  
  This private method is now unused and pointless since we
  made workspaces an element-wide concept as a part of issue #209.
  

• 34210d1e
  by Tristan Van Berkom at 2018-04-05T09:54:24Z

  _platform/linux.py: Fix fallout from context private symbols refactor
  
  This was still calling an outdated version of Context._message(),
  which is now Context.message(), causing stack traces on platforms
  without support for user namespaces.
  
  This was fallout from commit 500f4f330fde8b6001a2f8d8921bd5b8acb79960
  

• 22fe6b74
  by Tristan Van Berkom at 2018-04-05T13:11:58Z

  doc/source/projectrefs.rst: Ammended documentation for project.refs
  
  This was missing the main `projects` toplevel key
  

• 67338ef7
  by Tristan Van Berkom at 2018-04-05T13:13:49Z

  _frontend/app.py: More consistent error reporting
  
  Use App.print_error() in all error exit circumstances, this
  will automatically consider any detail strings reported in BstErrors.
  

• 1280b449
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  _exceptions.py: Added new LoadErrorReasons for junction load failures
  

• 81ec3635
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  _exceptions.py: Allow detail strings in LoadError exceptions
  

• 857751fb
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  _loader.py: Some fixes in how we load sources for junctions
  
    o Ensure that we call Source._load_ref(), and consider project.refs this way
  
    o Ensure that we report a warning in the case that project.refs is in use
      and the loaded junction source has a redundant ref which is going to
      be ignored
  
    o Report more distinct machine readable errors for failures to load
      junction element sources
  
    o Handle Consistency.INCONSISTENT and Consistency.RESOLVED separately:
  
      - The user should be told something different depending on whether
        they need to fetch or whether they need to track.
  
      - It is never possible to automatically fetch the source in the
        case that the source has no ref to begin with.
  
  This also adjusts the test/loader/junctions.py test to expect the new error
  

• 50af604e
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  tests/frontend: Share the configure_project() function
  
  Make buildtrack.py and track.py share the same configure_project() helper.
  

• abb9ef22
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  tests/frontend/show.py: Test behaviors of showing junctioned elements
  
    o Test error conditions for showing an unfetched junctioned project
  
    o Test error conditions for showing an untracked junctioned project
  
  Both tests check both modes of ref-storage.
  
  This adds a new shared helper function `generate_junction`
  

• 24ab5ce7
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  tests/frontend/fetch.py: Added tests for automatically fetching the junctions
  
    o This tests that a Consistency.RESOLVED junction will automatically be fetched
      when `bst fetch` is run on a pipeline which refers to a junction.
  
    o Further, it tests that a Consistency.INCONSISTENT junction will bail
      out with the expected error message
  
  Again testing with both ref-storage modes
  

• ee73a87b
  by Tristan Van Berkom at 2018-04-05T13:14:32Z

  tests/frontend/track.py: Added junction related tests
  
    o Test that we bail out with the expected errors when the
      junction element in question is Consistency.INCONSISTENT
  
    o Test that tracking the junction itself, causes a subsequent
      show of the pipeline to not bail out anymore (tests that
      tracking works and persists for a junction element).
  
  Again these use both ref-storage modes.
  

• 8704363a
  by Tristan Van Berkom at 2018-04-05T13:27:16Z

  tests/frontend/buildcheckout.py: Added junction related tests
  
    o Test that we get the expected error if the junction element
      the pipeline refers to is inconsistent
  
    o Test that the junction element is automatically fetched as
      a part of the build phase
  
  Both tests using both ref-storage modes
  

• 09e52016
  by Tristan Van Berkom at 2018-04-06T07:08:47Z

  _exceptions.py: Added LoadErrorReason.INVALID_SYMBOL_NAME
  

• 1b4307f1
  by Tristan Van Berkom at 2018-04-06T08:20:51Z

  _yaml.py: Added assert_symbol_name() helper function.
  
  This function asserts that a loaded symbol name is a valid one,
  and raises an appropriate and consistent LoadError if an invalid
  symbol name is encountered.
  

• 60f2a320
  by Tristan Van Berkom at 2018-04-06T08:36:13Z

  _project.py: Assert that the loaded project name is a valid symbol name
  
  Fixes issue #339
  

• 337075bb
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  _options/optionpool.py: Assert valid symbol names for option names.
  
  These must be alphanumeric and may not contain dashes.
  

• 550cb369
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  _options/option.py: Assert valid symbol names for variable exports
  

• 420114a0
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  tests/format/project.py: Converted to use CLI test harness
  
  Moved the old style project format tests into the new CLI based
  directory in tests/format/ and converted to use the CLI fixture
  for these tests.
  

• 50b694a3
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  tests/project/plugins.py: Removed this old style test
  
  This is sufficiently covered by other tests in tests/format/project.py
  

• c2500610
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  tests/format/project.py: Added tests for loading of invalid project names
  

• a2bc2a1c
  by Tristan Van Berkom at 2018-04-06T08:36:42Z

  tests/format/options.py: Added tests for invalid option name symbols
  

• 21050433
  by Tristan Van Berkom at 2018-04-06T08:51:16Z

  tests/format/options.py: Added tests for invalid variable names in options
  

• 06b1920b
  by Tristan Van Berkom at 2018-04-06T08:51:16Z

  doc/source/projectconf.rst: Properly document the project name.
  
  This is a part of issue #339
  

• 23edfe5d
  by Tristan Van Berkom at 2018-04-06T08:51:16Z

  doc/source/projectconf.rst: Document restrictions on option and option variable names.
  

• c55dad4b
  by Tristan Van Berkom at 2018-04-06T11:45:13Z

  data/userconfig.yaml: Change default logging to print full shas at init and bst show
  
  This may be obnoxious for `bst show` purposes, which will now also print
  the full artifact cache keys instead of nicely abbreviated ones, but it is a
  more appropriate default for the heading of a build session.
  
  This fixes issue #343
  

• 74c60178
  by Chandan Singh at 2018-04-07T09:42:25Z

  _frontend/cli.py: Add option to close multiple workspaces
  
  At present, it is only possible to close workspaces for elements one at
  a time. This can become slightly tedious process when you have multiple
  workspaces open and you want to close all of them, maybe because you
  just finished working on a set of related elements.
  
  Instead of accepting a single element, accept a list of elements as
  argument for `bst workspace close`. Additionally, add `-a`/`--all`
  option to close all workspaces.
  
  Fixes #337 - Add option to close all workspaces.
  

• 2ca11fde
  by Tristan Van Berkom at 2018-04-07T10:02:13Z

  _frontend/cli.py: Remove --no-checkout option for `bst workspace reset`
  
  This option in `bst workspace reset` is just pointless.
  

• 3ba544b8
  by Tristan Maat at 2018-04-07T10:05:10Z

  Allow 'None' as a default_value for _yaml.node_get
  

• 60dbf19f
  by Tristan Van Berkom at 2018-04-07T10:30:47Z

  Complete the work started to allow None as default in _yaml.node_get()
  
  This completes the work which was started in commit
  3ba544b80f9f268be8ffe62fc8589b30212ec4a2 which only went half
  way towards updating all the source code to be consistent and use
  the new semantic which allows None values.
  

• 59064042
  by Tristan Van Berkom at 2018-04-08T08:23:50Z

  _project.py: Remove BST_WORKSPACE_FORMAT_VERSION
  
  This is now defined in _workspaces.py
  

• 9be2dc5a
  by Tristan Van Berkom at 2018-04-08T08:25:21Z

  _versions.py: Added new file just to hold basic symbolic versions
  
  This is where we're going to store BST_CORE_ARTIFACT_VERSION
  and BST_FORMAT_VERSION from now on.
  
  This was a little bit weirdly defined before, and now we need to
  at least have BST_FORMAT_VERSION available from the frontend/cli
  without onerous imports (we need to import it there, but dont want
  to slow down load time for bash completion runs).
  

• b7ec278d
  by Tristan Van Berkom at 2018-04-08T10:59:31Z

  _yaml.py: Allow None for provenance argument in _yaml.assert_symbol_name()
  
  Since we now also use this to validate user input on the command line.
  

• 5de99191
  by Tristan Van Berkom at 2018-04-08T12:56:33Z

  _frontend/cli.py, _frontend/app.py: Implemented new `bst init` command.
  
  This comes with an interactive mode unless the project name is specified
  on the command line.
  
  This fixes issue #342
  

• 337b7a31
  by Tristan Van Berkom at 2018-04-08T12:57:01Z

  tests/frontend/init.py: New test to test edge cases for new `bst init` command.
  

• 96af6dd5
  by Tristan Van Berkom at 2018-04-08T12:57:01Z

  More specific exceptions when a project.conf is missing.
  
  Since we want to react and start an interactive session when the
  project.conf is missing, we need a more specific error to catch.
  

• 1f57e598
  by Tristan Van Berkom at 2018-04-08T12:57:01Z

  _frontend/app.py: Automatically launch interactive `bst init` when project.conf is absent
  
  When running a command where a project.conf does not exist, ask the
  user if they would like to interactively create a project in the
  said directory.
  
  This is a part of issue #342
  

• 6d7f86fe
  by Tristan Van Berkom at 2018-04-08T13:06:34Z

  NEWS: Adding news entry for new `bst init` command
  

• e0a8b9e5
  by Tristan Van Berkom at 2018-04-08T13:55:03Z

  _versions.py: Fixed copyright year on newly added file.
  
  Oops.
  

• 8bfc7371
  by Jim MacArthur at 2018-04-09T09:49:27Z

  _ostree.py: Mention the remote URL when we fail to fetch remote refs
  

• 68cee1c7
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  tests/frontend/workspace.py: Fixed test_build to make the right assertion
  
  This was removing a file from the workspace, building, checking out,
  and then asserting that the file is *still gone* in the workspace,
  ignoring the same file in the checkout.
  

• 66cf08fa
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  element.py: Cache source consistency logic and refactor workspace logic into Element
  
  Workspaces are now element wide, so consistency edge cases must
  be handled at the element level instead of the source level.
  

• 26f7f0b8
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  element.py: Factored out pointless extra Element._workspaced() method
  

• 62346702
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  element.py: Manage scheduled tracking state in Element
  
  This removes the scheduled state of tracking from Sources, as
  this is really an element wide thing.
  
  To be consistent with assembly, now this comes with:
  
    o Element._schedule_tracking()
    o Element._tracking_done()
    o Element.__tracking_scheduled
    o Element.__tracking_done
  
  Updated the TrackQueue() to call Element._tracking_done() similarly
  to how we have BuildQueue() call Element._assemble_done().
  

• 0fbb550e
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  element.py, source.py: Removing knowledge of assemble scheduling state from source
  
  And considering the workspace related edge cases in Element instead of Source.
  

• 67d22d76
  by Tristan Van Berkom at 2018-04-09T10:25:36Z

  element.py, source.py: Element consumes the rest of workspace logic.
  
  And Source no longer has any __workspace handle.
  

• 9e74988e
  by Tristan Van Berkom at 2018-04-09T11:24:34Z

  _workspaces.py: Fix cache key calculation regression
  
  This was regressed in the refactor done in commit f761140f
  

• b8b41724
  by Tristan Van Berkom at 2018-04-09T11:24:34Z

  tests/frontend/workspace.py: Test that we detect modifications made to a workspace
  
  This was previously only working for added or removed files and
  broken for modified files.
  

• 8da99651
  by Tristan Van Berkom at 2018-04-10T08:01:51Z

  element.py: Removed useless __workspace pointer
  
  I added this in the last refactor and forgot to remove it.
  

• be6b353d
  by Tristan Van Berkom at 2018-04-10T12:01:55Z

  _workspaces.py: Remove knowledge of Elements completely
  
  This makes workspaces more cleanly separated from everything else.
  
    o Removed some methods from Workspace()
  
    o Added Element._open_workspace() to initialize workspaces on
      sources
  
    o Cleanup some code in the pipeline
  
    o Have the App() call Element._open_workspace() instead of Workspace.open()
  

• 5a36d04e
  by Tristan Van Berkom at 2018-04-10T12:02:15Z

  _workspaces.py: Add generic serialization/deserialization
  
  Also streamline the instantiation code paths to layer the yaml
  parsing on top of simple dictionary deserialization.
  

• 888f0b9f
  by Tristan Van Berkom at 2018-04-10T12:02:15Z

  _workspaces.py: Added Workspaces.update_workspace()
  
  This takes a serialized workspace dictionary as understood
  by Workspace.from_dict() and created by Workspace.to_dict()
  
  Further, this adds a Workspace.differs() method to compare
  the state of two workspace instances.
  

• 0b5809c4
  by Tristan Van Berkom at 2018-04-10T12:11:54Z

  utils.py: Added _is_main_process()
  
  A helper function to identify if we are running in the main process
  or not.
  

• a9cbb0f7
  by Tristan Van Berkom at 2018-04-10T12:11:54Z

  Serialize workspace modifications in the main process.
  
  This patch makes the child process send any workspace state back
  to the main process and saves it there directly before processing
  the result of a build job, such that workspace modifications in
  child tasks work transparently and no special care needs to be taken
  to save them except when doing so in the main process.
  
  Also this removes the line where we update workspace data at staging
  time.
  
  This fixes issue #352
  

• a867f293
  by Tristan Van Berkom at 2018-04-10T12:11:54Z

  _workspaces.py: Assert that configuration is only saved in the main process.
  

• 37d97414
  by Jim MacArthur at 2018-04-10T12:50:53Z

  _frontend/widget.py: Correct log line if logdir is empty
  

• cb386651
  by Tristan Maat at 2018-04-11T06:49:22Z

  HACKING.rst: Add integration and pytest notes
  

• d46fbbcb
  by Tristan Van Berkom at 2018-04-11T09:42:56Z

  element.py: Preserve workspace state in failed build sandboxes.
  
  As reported in regression #346, since we started mounting the workspaces
  we now fail to debug build failures for workspaces, the shell command
  no longer sees the workspaces files to debug.
  
  As the failed sysroot represents a very particular build failure, it
  is incorrect to attempt to remount a workspace again after a failed
  build, as the workspace files may have changed since the build.
  
  Instead, in the case of a build failure on a workspaced element, we
  copy the files from the workspace in place (while automatically preserving
  the mtimes and everything, as utils.safe_copy() does).
  
  When we implement failed build artifacts, this case will have to be
  handled in a very similar way, too.
  
  This fixes issue #346
  

• a1564c4a
  by Tristan Van Berkom at 2018-04-11T09:42:56Z

  tests/integration/shell.py: Test that workspaced files are visible in a bst shell
  
  Test both in regular `bst shell --build`, and on a failed build sysroot
  with `bst shell --sysroot`, the latter being the semantic used when debugging
  a failed build.
  
  This guards against regressions of issue #346
  

• 5217359f
  by Javier Jardón at 2018-04-11T13:58:10Z

  doc/source/install.rst: Update list of base requirements
  

• 274e1831
  by Javier Jardón at 2018-04-11T13:58:10Z

  source/install.rst: Update ArchLinux installation instructions
  

• 6ad88ecb
  by Tristan Van Berkom at 2018-04-11T14:10:54Z

  element.py: Restructure artifact metadata
  
   o Store the artifact.yaml into 4 separate files now
   o Only load on demand what is needed at a given time
   o Add local caching for what was loaded, ensuring we never redundantly load metadata
  
  This breaks artifact format and will require an artifact revision
  
  This should improve performance for issue #294
  

• c94f1264
  by Tristan Van Berkom at 2018-04-11T14:10:55Z

  element.py, source.py: Cleanup how Source cache keys are calculated.
  
  Recently after a refactor we kept the Source adding workspace keys
  to the source keys because, now clean this up to have the workspace
  key added directly in the Element cache key calculation.
  
  This breaks cache keys.
  

• acfe2494
  by Tristan Van Berkom at 2018-04-11T14:10:55Z

  buildstream/_versions.py: Increment BST_CORE_ARTIFACT_VERSION
  
  Now that we've restructured the artifact metadata into
  separate files, we need to rev the artifact version.
  

• be10c324
  by Tristan Van Berkom at 2018-04-12T12:18:13Z

  _frontend/status.py: Fixed status widgets to show full element names.
  

• 95d1dafc
  by Tristan Van Berkom at 2018-04-12T12:37:24Z

  _frontend/widget.py: Fixed regression in logfile abbreviations.
  
  Added required size_request() method to new MessageOrLogFile() widget.
  
  This came up in merge request !377
  

• fece8cc8
  by Tristan Van Berkom at 2018-04-12T12:39:06Z

  _frontend/cli.py, _pipeline.py: Add options for cross junction tracking.
  
  This patch makes cross junction tracking disabled by default, which
  was the initial intention when landing project.refs but never got around
  to doing this (intended to get addressing of junctioned elements via
  command line sorted first, but didnt happen).
  
  This adds the following options to enable cross-junction tracking:
  
    o bst build -J / --track-cross-junctions
  
    o bst fetch -J / --track-cross-junctions
  
    o bst track -J / --cross-junctions
  
  This also fixes `bst fetch --track` which had a bug, it was avoiding
  to track and fetch elements which are in a `cached` consistency state,
  which is wrong when `--track` is specified.
  
  This also updates some test cases which were broken by
  this change.
  
  This fixes issue #354
  

• 337fc965
  by Tristan Van Berkom at 2018-04-12T12:39:06Z

  NEWS: Added news entry detailing that cross-junction tracking is disabled by default
  

• e3e72616
  by Richard Maw at 2018-04-12T13:25:25Z

  mount: Wrap yields in context managers with try-finally
  
  The terminator context will only clean up on a signal,
  so if another exception causes context manager cleanup
  then unmount won't be called unless it's part of another context manager
  or is wrapped in a try block's except or finally.
  
  Everywhere else's unmounts are handled by delegating to another context manager
  but these were what needed to be fixed.
  
  The change in buildstream/_fuse/mount.py would cause lockups
  since the build worker process still having a subprocess
  blocked its termination from completing
  which in turn caused the pipeline manager process to block indefinitely on it.
  

• 4aa482ae
  by Tristan Van Berkom at 2018-04-12T15:01:36Z

  _pipeline.py: Fix inconsistent element error to print element fullnames.
  

• 39c4cd17
  by Tristan Van Berkom at 2018-04-12T15:01:36Z

  Fix disaster while making cross junction tracking optional.
  
  This disaster was introduced in fece8cc81e8d8412e32c6667682a33e7d2f9dafe
  
  When doing `bst build --track`, we were:
  
    o first scheduling every element to be tracked
  
    o later filtering out the cross junction elements
  
    o finally asserting consistency state, which would trigger
      an error because we previously scheduled for tracking.
  
  Fixed this by moving all code which resolves elements to track
  into Pipeline.initialize(), and removing special element list handling
  from the individual build/fetch/track commands.
  

• 522d6538
  by Tristan Van Berkom at 2018-04-12T15:01:36Z

  _scheduler/trackqueue.py: Mark skipped elements as tracked.
  
  This ensures their state will be updated, it is especially important
  for sourceless elements which are scheduled to be tracked, like stack elements.
  
  This fixes a regression from my previous refactor or Sources
  which added Element._tracking_done()
  

• 7c9713ad
  by Richard Maw at 2018-04-12T18:03:09Z

  frontend: capture errors for later use
  
  This will be used to provide an error summary before the pipeline summary.
  

• 30af44e8
  by Richard Maw at 2018-04-12T18:03:09Z

  frontend: Use textwrap.indent to indent
  
  The intent is more obvious by using an existing module.
  

• 7b0a9656
  by Richard Maw at 2018-04-12T18:03:09Z

  frontend: Summarise build log output in --no-verbose mode
  

• bea028a8
  by Richard Maw at 2018-04-12T18:03:09Z

  frontend: Control indentations of multi-line values
  

• fc6d192d
  by Richard Maw at 2018-04-12T18:03:09Z

  frontend: Redisplay failure messages in result summary
  
  It's more convenient than having to search back through all the output
  to find out what the cause of failure was.
  

• eb522a4f
  by Richard Maw at 2018-04-12T18:03:09Z

  tests: Check that output includes log files of failed builds
  

• d94d47c5
  by Richard Maw at 2018-04-12T18:03:09Z

  NEWS: Mention failed job summary
  

• f1567355
  by Javier Jardón at 2018-04-12T22:09:00Z

  .gitlab-ci.yml: Create template for linux-tests job
  

• 9bc2bb04
  by Javier Jardón at 2018-04-12T22:09:00Z

  .gitlab-ci.yml: Run tests in Fedora:27 as well
  

• 1af6f442
  by Javier Jardón at 2018-04-12T22:09:00Z

  .gitlab-ci.yml: Change jobs from <distro>-tests to tests-<distro>
  
  So they show in order nicely in the gitlab CI UI
  

• 8378f5dc
  by Javier Jardón at 2018-04-12T22:15:33Z

  .gitlab-ci.yml: Fix coverage job
  

• 63b7c821
  by Tristan Van Berkom at 2018-04-13T06:01:37Z

  .gitlab-ci.yml: Show which coverage reports are getting combined in CI
  
  This extra line shows which reports will be combined, which helps us
  to more quickly debug problems in the coverage job of the .gitlab-ci.yml
  

• 3b8dd43e
  by Tristan Van Berkom at 2018-04-13T10:32:53Z

  plugin.py: Documentation improvements.
  
  Adding "Abstract Methods" section indicating what implementors
  have to implement.
  
  Also fixed some inconsistencies, and moved the abstract methods
  to the top of the class.
  

• 1ea74544
  by Tristan Van Berkom at 2018-04-13T10:32:53Z

  element.py: Documentation improvements
  
  Added heading section describing the abstract methods, and
  moving the abstract methods to the top of the class.
  

• 03bead6c
  by Tristan Van Berkom at 2018-04-13T10:32:53Z

  source.py: Enhanced documentation
  
  Added new Abstract Methods section here, and rearranged sources
  such that abstract methods are on top.
  

• 596264d1
  by Tristan Van Berkom at 2018-04-13T10:36:27Z

  buildelement.py: Documentation improvements
  
  Document here in terms of what the BuildElement does to implement
  the abstract Element methods.
  

• 299df233
  by Tristan Maat at 2018-04-13T11:51:17Z

  Add element.prepare method
  
  This is one of the tasks of #209
  

• ccb6acc1
  by Tristan Maat at 2018-04-13T11:51:17Z

  integration/workspace.py: Test that configure commands run only once
  

• d1b14086
  by Tristan Maat at 2018-04-13T11:51:17Z

  tests/frontend/workspace.py: Automatically update version
  

• 5632d291
  by Tristan Maat at 2018-04-13T11:51:17Z

  tests/frontend/workspace.py: Test new workspace format version
  

• 2cec9c27
  by Tristan Maat at 2018-04-13T11:54:10Z

  Update plugin documentation
  

• cb702588
  by Tristan Maat at 2018-04-13T12:02:51Z

  Add NEWS entry
  

• a4927ad3
  by Tristan Van Berkom at 2018-04-14T08:32:09Z

  tests/frontend/show.py: Removing some unused variables
  

• ae8543c3
  by Tristan Van Berkom at 2018-04-14T08:32:09Z

  tests/format/variables.py: Ported from old style tests
  

• 188d819d
  by Tristan Van Berkom at 2018-04-14T08:32:09Z

  buildelement.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• d45e12f7
  by Tristan Van Berkom at 2018-04-14T09:13:27Z

  source.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 3ef81d7f
  by Tristan Van Berkom at 2018-04-14T09:14:12Z

  buildelement.py: Use Element.get_variable() instead of poking at private thing
  
  This is the only place where Element._get_variables() was used, which we'll
  now happily remove - also Element implementations in general should not
  have to use private API.
  

• 7db09e2c
  by Tristan Van Berkom at 2018-04-14T10:19:51Z

  element.py: Adhere to policy on private symbols
  
  This involves a lot of reordering of function in the Element class.
  
  This is a part of issue #285
  

• 476526fc
  by Tristan Van Berkom at 2018-04-14T10:23:00Z

  element.py: Rename _get_full_display_key() -> _get_display_key()
  
  The extra "full" was a bit redundant here, the Element does not
  export any "less than full" display key accessor, so lets use
  a more simple name for this.
  

• 1094d62b
  by Tristan Van Berkom at 2018-04-14T10:45:09Z

  _artifactcache/ostreecache.py: Extracting a missing artifact is a bug
  
  Raise an assertion here, not an ArtifactError.
  

• d603051e
  by Tristan Van Berkom at 2018-04-14T10:45:40Z

  _artifactcache/tarcache.py: Extracting a missing artifact is a bug
  
  Raise an assertion here, not an ArtifactError.
  

• d41940f5
  by Abderrahim Kitouni at 2018-04-14T13:53:36Z

  plugins/elements/cmake.yaml: allow using ninja instead of make (#279)
  
  This uses the build tool mode of cmake to have a single command that can call
  either make or ninja.
  
  I've also modified the tests to take the new commands into account
  

• f6f4738c
  by Tristan Van Berkom at 2018-04-15T10:04:07Z

  _frontend/app.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 06da7a02
  by Tristan Van Berkom at 2018-04-15T11:10:01Z

  _frontend/profile.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• 219c7d29
  by Tristan Van Berkom at 2018-04-15T11:10:01Z

  _frontend/status.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• e769beca
  by Tristan Van Berkom at 2018-04-15T11:10:01Z

  _frontend/widget.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• f2edff31
  by Tristan Van Berkom at 2018-04-15T11:48:44Z

  buildstream/plugin.py, doc/source/projectconf.rst: Changed policy for core plugin format revisioning
  
  The policy for format revisioning of core plugins maintained in BuildStream
  now changes such that we revision all plugins under the same core format version.
  
  This means that format extensions in plugins should be revisioned in
  the global _versions.BST_FORMAT_VERSION along with the core format, this
  simplifies parameterization of project.conf for usage of core plugins.
  
  This did not really require any code support as our plugins have never
  been revisioned before.
  

• 43e09ccf
  by Tristan Van Berkom at 2018-04-15T11:50:01Z

  _versions.py: Bump BST_FORMAT_VERSION for new cmake format extension
  
  With commit d41940f516498f827967b8e5d311ce6accb88f56, the cmake plugin
  now supports a new `generator` option which allows one to specify whether
  to use `Unix Makefiles` (default) or `Ninja` (optional).
  
  This requires a format version bump if people want to be sure that
  the feature is supported.
  

• 075b9819
  by Tristan Van Berkom at 2018-04-15T11:52:56Z

  NEWS: Adding news entry for cmake plugin enhancement.
  

• 938ec1f4
  by Tristan Van Berkom at 2018-04-16T06:15:30Z

  doc/source/format.rst: Some enhancements in how we document dependencies
  
   * Added link target in the project.conf documentation for the element-path
  
   * List elements in examples without the leading `elements/` path, this
     is misleading and predates the existence of the element-path
  
   * Specify "element path relative" instead of "project relative", this
     was also outdated. Use these opportunities to link back to the element
     path project.conf docs section
  
   * Document dependency attributes differently
  
     - Use a list of attributes
  
     - Link back to the first example when speaking of shorthand, instead
       of duplication the example in two places
  
     - Enhanced description of how junction attributes are used
  
     - Note that ordering in dependency declaration lists is not meaningful
  
       This is an incorporation of Javier Jardón's patch for issue #358
  
  This fixes issue #358.
  

• 5cedd894
  by Tristan Van Berkom at 2018-04-16T06:50:56Z

  element.py: Document new Element.prepare() method as available since 1.2
  

• 7c2a43d1
  by Tristan Van Berkom at 2018-04-16T06:53:17Z

  element.py: Document new BST_FORBID_SOURCES and BST_FORBID_RDEPENDS as available since 1.2
  
  Also use any() for the check on the generators, instead of expanding the
  whole list exhaustively and checking if the list is not empty.
  

• ca2331c3
  by Tristan Van Berkom at 2018-04-16T07:45:04Z

  source.py, element.py, _pipeline.py: Streamling preflighting.
  
  Instead of having the pipeline preflight all sources separately
  from elements, have the element preflight it's sources.
  
  This is in order to simplify the shared code path for the pipeline
  and the loader to use for instantiating elements.
  
  Also updated tests to expect the new ElementError and SourceError
  instead of the PipelineError which was raised for preflighting before.
  

• cd90fbde
  by Tristan Van Berkom at 2018-04-16T09:26:05Z

  Clean up element/source instantiation code paths.
  
  This removes the extra `kind` parameter from all of the related
  codepaths, it is redundant since the `kind` attribute is already
  stored on the MetaElement and MetaSource objects.
  

• b4f0a52a
  by Tristan Van Berkom at 2018-04-16T10:59:47Z

  element.py, _pipeline.py: Moved instantiation codepath to Element class methods.
  
  This will allow the instantiation codepath to be shared by the Loader
  which also needs to instantiate elements for junctions.
  

• d02a1b4e
  by Tristan Van Berkom at 2018-04-16T11:34:40Z

  _loader.py: Use shared element instantiation codepaths for junction loading.
  
  The consequently fixes issue #292, and you can now use workspaces
  to work on junctioned projects.
  

• e48f2f82
  by Tristan Van Berkom at 2018-04-16T12:14:37Z

  tests/frontend/buildcheckout.py: Added regression tests for workspaced junctions
  
  This guards against regressions of issue #292
  

• 827a1294
  by Tristan Van Berkom at 2018-04-16T12:52:04Z

  _pipeline.py: Stop printing warning when a build plan does not use some existing workspaces
  
  Fixes issue #360
  

• eb0ad4d4
  by Tristan Van Berkom at 2018-04-16T13:30:54Z

  .gitlab-ci.yml: Perform some python code analysis with radon
  
  This adds a new job in the prepare stage which can be viewed in
  the gitlab CI, and also produces an artifacts with the json metrics.
  

• a33ac81e
  by Tristan Van Berkom at 2018-04-16T13:58:26Z

  .gitlab-ci.yml: Add coverage regex
  

• 86587bf9
  by Javier Jardón at 2018-04-16T13:58:26Z

  README.rst: Add coverage badge
  

• 9cf8d35f
  by Tristan Van Berkom at 2018-04-17T07:02:09Z

  _frontend: Fix error messaging for closing workspaces
  
  This fixes issue #182 again
  

• c9b3aad7
  by Tristan Van Berkom at 2018-04-17T07:24:40Z

  _frontend: Earlier assertion for `bst workspace open` when directory is not empty
  
  This just raises the error to the user that the checkout directory
  where they intend to create a workspace in is not empty, a little
  bit earlier, without bothering to initialize the pipeline first.
  

• 548c03f5
  by devcurmudgeon at 2018-04-17T07:47:16Z

  Update README.rst with more detail
  

• 4fc2da9a
  by Tristan Van Berkom at 2018-04-17T08:31:48Z

  element.py: Fix force opening of workspaces.
  
  This was broken because we let the Source objects initialize workspaces
  directly in the target workspace directory, where files might already
  exist.
  
  Instead, initialize the workspace in a tempdir and hardlink the files
  from there.
  
  This fixes issue #364
  

• 098ee3cf
  by Tristan Van Berkom at 2018-04-17T08:31:48Z

  tests/frontend/workspace.py: Added tests for force opening workspaces
  
  Guard against regressions of issue #364
  

• 226880ee
  by Javier Jardón at 2018-04-17T08:49:59Z

  README.rst: Point to new location of x86image example
  

• 880e1913
  by Paul Sherwood at 2018-04-17T12:36:46Z

  Fix typo, add inspiration projects
  

• 6c23e337
  by Chandan Singh at 2018-04-17T13:01:22Z

  refactor: Do not assume there is only one workspace target in app.py
  
  While working with workspaces in _frontend/app.py, we currently assume
  that there is only workspace in the pippeline. This limits our ability
  to support multiple workspaces in a given command.
  
  Remove this assumption from _frontend/app.py which will allow for the
  possibility of supporting multiple workspaces from _frontend/cli.py in
  future. This commit does not change the behavior of these commands
  though as the target is instead passed as an argument.
  

• b264430d
  by Chandan Singh at 2018-04-17T13:01:22Z

  _frontend/cli.py: Add option to reset multiple workspaces
  
  !357 added support for closing multiple workspaces. Similarly, also
  allow `bst workspace reset` to work on multiple workspaces, with `--all`
  as a helper to reset all open workspaces.
  

• 78a86f60
  by Chandan Singh at 2018-04-17T14:53:17Z

  NEWS: workspace close/reset support multiple elements
  

• fb6687b6
  by Tristan Van Berkom at 2018-04-17T15:55:01Z

  _loader package: Creating a new sub package to hold the loader related things.
  

• 5f3da731
  by Tristan Van Berkom at 2018-04-17T15:55:01Z

  _loader package: Splitting up the loader code into separate pieces
  
  This also refactors the loader code to honor private symbol
  naming policy.
  
  This is a part of issue #285
  

• 1159bbc8
  by Tristan Van Berkom at 2018-04-17T15:55:01Z

  _context.py: Removing unused _platform private variable
  

• ea7d6ed2
  by Tristan Van Berkom at 2018-04-17T15:55:01Z

  _context.py: Make `fetch_subprojects` public.
  
  This is actually used by the Loader and is not private.
  
  This is a part of #285
  

• 8027b30d
  by James Ennis at 2018-04-17T16:52:27Z

  index.rst: remove captions
  

• f0b350f1
  by James Ennis at 2018-04-17T16:52:27Z

  main_install.rst: New file with install, docker and artifacts
  
  We now have a main installing page with three sections:
  1. How to install buildstream onto your machine.
  2. How to run buildstream inside docker image.
  3. How to install an artifact server
  

• 4a49cedb
  by James Ennis at 2018-04-17T16:52:27Z

  main_authoring.rst: move to using buildstream
  

• 5d2f6ec6
  by James Ennis at 2018-04-17T16:52:27Z

  Add a general documentation section
  

• 02119f19
  by James Ennis at 2018-04-17T16:52:27Z

  index.rst: Remove unnecessary toctree blocks
  

• 58f8e6bc
  by James Ennis at 2018-04-17T16:52:27Z

  Consistent titling
  

• 516daa5c
  by James Ennis at 2018-04-17T16:52:27Z

  invoking.rst: renamed to commands.rst (anchors also changed)
  

• 5b5c87bf
  by James Ennis at 2018-04-17T16:52:27Z

  config.rst: Renamed to user_config.rst and anchors changed
  

• e4cea3da
  by James Ennis at 2018-04-17T16:52:27Z

  core_framework.rst: New file containing content that was in main_core.rst
  

• 9a3f008e
  by James Ennis at 2018-04-17T16:52:27Z

  main_core.rst: Change title and moved authoring, sandboxing and cachekeys documentation here
  

• 974bd246
  by Tristan Van Berkom at 2018-04-18T03:44:50Z

  _scheduler: Move job.shutdown() calling responsibility inside of Job.
  
  This was being called by Queue if Job succeeds, or inside Job if
  Job fails and retries, lets just keep this inside of Job.
  

• 835e5475
  by Tristan Van Berkom at 2018-04-18T04:49:40Z

  _scheduler: Slightly changing the Job / Queue API contracts
  
  For the Job:
  
     o The action_cb, complete_cb and max_retries are now passed
       into the constructor, and the Job.spawn() API has no arguments.
  
     o The complete_cb() signature has changed to take a success
       boolean instead of a returncode integer, and also now receives
       the result directly.
  
  For the Queue:
  
     o Adapted to new Job API contract changes
  
     o Changed the Queue.done() method to now take a success boolean
       instead of a returncode integer
  
  Updated Queue implementations to handle the success boolean instead
  of the returncode integer.
  

• 6a4949c2
  by Tristan Van Berkom at 2018-04-18T06:04:32Z

  _scheduler/job.py: Adhere to policy on private symbols
  
  This also removes an illegal peek into element._Element__environment
  and uses the public Element.get_environment() accessor instead, removing
  the need to also sanitize the returned dictionary.
  
  Further, this reduces the amount of data passed between processes
  in asyncio callbacks.
  
  This is a part of issue #285
  

• 494fdd65
  by Tristan Van Berkom at 2018-04-18T06:18:43Z

  _scheduler: Adding Queue.dequeue_ready() to avoid peeking into private queue data
  
  And micro-optimizing the scheduler use a generator _expression_ with any()
  to check the result instead of creating a list.
  

• 3b447405
  by Tristan Van Berkom at 2018-04-18T06:44:48Z

  _scheduler/queue.py: Adhere to policy on private symbols
  
  And change the API contract a bit more, now the Queue object
  takes the scheduler in the constructor and the Queue.attach()
  method is removed.
  
  This is a part of issue #285
  

• 853c54eb
  by Tristan Van Berkom at 2018-04-18T09:38:40Z

  _scheduler/job.py: Don't mess with scheduler loop when forking
  
  Instead of unregistering the mainloop handling of SIGTSTP, block
  it (along with SIGTERM) while forking, clear the signal handlers
  and then unblock them in the child process; let the parent handle
  SIGTSTP after unblocking in the main process.
  

• 268340a5
  by Tristan Van Berkom at 2018-04-18T11:02:25Z

  _scheduler/scheduler.py: Adhere to policy on private symbols
  
  Also added Scheduler.jobs_suspended() context manager to
  use in the App instead of exposing Scheduler.suspend_jobs()
  and Scheduler.resume_jobs() along with signal callback connect
  and disconnect APIs, this keeps things more private in the
  scheduler.
  
  This is a part of issue #285
  

• e18b7e5b
  by Tristan Van Berkom at 2018-04-18T11:02:25Z

  _scheduler/fetchqueue.py: Adhere to policy on private symbols
  
  This is a part of issue #285
  

• ee7e89d2
  by Tristan Van Berkom at 2018-04-18T14:13:21Z

  HACKING.rst: Clarify policy that related issues must be mentioned in commit messages.
  
  This was somehow less than obvious.
  

• fd9ee860
  by James Ennis at 2018-04-18T14:16:02Z

  HACKING.rst: Include Documentation Formatting Policy
  

• 149d0859
  by James Ennis at 2018-04-18T14:16:02Z

  HACKING.rst: move location of useful links
  

• 39a7c1f9
  by Tristan Van Berkom at 2018-04-18T14:16:55Z

  HACKING.rst: Fixing profiling section to follow rst formatting policy
  
  Two empty lines before a section title, no empty line after the section title
  

• ae1d5093
  by Javier Jardón at 2018-04-18T14:19:00Z

  README.rst: Add pipeline status badge
  

• 499c6f9d
  by Richard Maw at 2018-04-18T14:50:04Z

  frontend: Log which elements resolved new keys
  
  It can be difficult to pick out important information in all the messages,
  so it's convenient to have classes of them collected into one place.
  
  One such class of important information is when the cache key of an element changes,
  such as when element's source is tracked or cached from a workspace,
  since it can be important to know what the cache key is for that artifact.
  
  Fixes issue #252
  

• a850b8ed
  by Richard Maw at 2018-04-18T14:50:33Z

  Update NEWS
  

• cd75869f
  by Javier Jardón at 2018-04-18T15:25:10Z

  Add about page
  

• 899c3ce0
  by Javier Jardón at 2018-04-18T15:25:10Z

  doc/source/index.rst: Replace welcome paragraph
  

• 64069c8c
  by Tristan Van Berkom at 2018-04-19T06:11:47Z

  _pipeline.py: Added PipelineSelection
  
  Part of a slow, ongoing refactor to unmangle the pipeline into
  something which just creates pipelines of elements for a centerpiece
  to process.
  
  This also renames Pipeline.deps_elements() -> Pipeline.get_selection()
  

• 67437803
  by Tristan Van Berkom at 2018-04-19T06:11:47Z

  _pipeline.py, _frontend/app.py: Added track_selection initialization argument
  
  This informs the pipeline what PipelineSelection mode to use
  for constructing the list of elements to track.
  

• d73d75c4
  by Tristan Van Berkom at 2018-04-19T06:11:47Z

  _frontend/cli.py: Set the track_selection argument to the deps parameter
  
  Tell the pipeline to only track either 'none' or 'all'
  PipelineSelection modes.
  
  This fixes issue #367
  

• 670487be
  by Tristan Van Berkom at 2018-04-19T06:11:47Z

  tests/frontend/track.py: Added test for `bst track --deps none`
  
  This guards against regressions of issue #367
  

• 6fbc81e7
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_artifactcache/pushreceive.py: Fixing unused variables
  

• 480876af
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_context.py: Fixing unused variables
  

• 6004943e
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_frontend/app.py: Fixing unused variables
  

• dc30af8a
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_frontend/status.py: Fixing unused variables
  

• ec2648cb
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_loader/loader.py: Fixing unused variables
  

• cb3a2149
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/_options/optionpool.py: Fixing unused variables
  

• a8c31ff3
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/element.py: Fixing unused variables
  

• f5b99559
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/plugin.py: Fixing unused variables
  

• 849d66e0
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/plugins/sources/_downloadablefilesource.py: Fixing unused variables
  

• 0a718d86
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/sandbox/_mount.py: Fixing unused variables
  

• 0830323d
  by Tristan Van Berkom at 2018-04-19T06:38:09Z

  buildstream/utils.py: Fixing unused variables
  

• c1c09e36
  by Tristan Van Berkom at 2018-04-19T06:38:20Z

  .pylintrc: Enable linting for unused variables
  

• 78bd7f56
  by devcurmudgeon at 2018-04-19T11:11:09Z

  Add example formats
  

• 61889ed5
  by Tristan Van Berkom at 2018-04-20T05:32:05Z

  plugins: Disable attribute-defined-outside-init pylint error
  
  Plugins set their attributes in configure(), because the
  constructor is not public API.
  

• 671a7412
  by Tristan Van Berkom at 2018-04-20T05:34:04Z

  _context.py: Properly declare and document log_message_format in the constructor
  

• be5b26c3
  by Tristan Van Berkom at 2018-04-20T05:35:43Z

  _pipeline.py: Declare tracking related members in the constructor
  

• 1fcfd000
  by Tristan Van Berkom at 2018-04-20T05:37:39Z

  _artifactcache/pushreceive.py: Fixed ProcessWithPipes to declare attributes in constructor
  

• 2282af27
  by Tristan Van Berkom at 2018-04-20T05:40:46Z

  _scheduler/job.py: Disable attribute-defined-outside-init lint error
  
  This is in a derived Process from the standard lib, we add this
  here because we are unsure about the constructor signature.
  

• 5f378fea
  by Tristan Van Berkom at 2018-04-20T05:41:04Z

  .pylintrc: Enable attribute-defined-outside-init messages
  
  This will give us an error if an instance attribute is ever
  declared outside of the constructor, which usually constitutes
  either a bug or a violation of the coding standard, rendering
  code more difficult to read.
  

• 1b7b51ac
  by milloni at 2018-04-20T10:50:06Z

  setup.py: Assert Bubblewrap version
  
  The required version is 0.1.2 based on the fact that we're using
  --hostname and --remount-ro.
  

• 7854fc50
  by milloni at 2018-04-20T10:52:38Z

  doc/source/install.rst: Mention bwrap version requirements
  

• aab0229b
  by Tristan Maat at 2018-04-20T11:00:46Z

  deb.py: Make pylint stop complaining about arpy
  

• 0213102c
  by Tristan Maat at 2018-04-20T11:03:30Z

  HACKING.rst: Add a note about arpy being required for docs
  

• 1598d35e
  by Tristan Van Berkom at 2018-04-23T06:59:16Z

  doc/source/index.rst: Make main documentation title sober again.
  
  Drop the happy welcome phrase with exclamation point.
  

• f7713400
  by Tristan Van Berkom at 2018-04-23T07:37:02Z

  tests/pipeline/preflight-error: Fixing typo in error string formatting
  

• 3199d357
  by Tristan Van Berkom at 2018-04-23T08:19:58Z

  Overwrite sys.excepthook to handle exceptions in the main application.
  
  This uses the existing messaging system to report errors in the main
  application as a BUG type. It requires the use of a global_app variable
  which isn't ideal; this may be replaced in future. This partially addresses
  issue #197.
  
  Theoretically, an exception could occur before Scheduler.loop is set up,
  hence the check for it when terminating all jobs.
  
  NOTE: This was originally submitted by Jim MacArthur, and manually
        reapplied after some refactoring took place.
  

• 4dd72d6c
  by Javier Jardón at 2018-04-23T11:13:35Z

  Revert "doc/source/install.rst: Mention bwrap version requirements"
  
  Seems this was not propperly rebased; libostree and bubblewrap requirements
  are already there.
  Also the paragraph about docker was removed in
  commit f0b350f158ed14c97613dc22b83ed89759b8d255
  
  This reverts commit 7854fc5037204474d16618c657a1cd1ca12c888e.
  

• c939d5f9
  by Javier Jardón at 2018-04-23T11:13:35Z

  doc/source/install.rst: minimum bubblewrap version is 0.1.2
  

• 2fe1010a
  by Tristan Van Berkom at 2018-04-23T13:00:28Z

  doc/source/install.rst: Removing blank line which appeared below title
  
  Sections have 2 blank lines before them, and no blank line after them.
  

• a18fcf0c
  by Jim MacArthur at 2018-04-23T13:03:27Z

  Add artifact cache receive profiling domain
  
  _artifactcache/pushreceive.py: Wrap OSTreeReceiver.do_run in a
    profiling domain.
  _profile.py: Add 'ARTIFACT_RECEIVE' domain.
  

• 1cd3728c
  by Jim MacArthur at 2018-04-23T13:03:27Z

  HACKING.rst: Add a section on BST_PROFILE and the artifact cache.
  

• 15789951
  by Tristan Van Berkom at 2018-04-23T13:04:17Z

  HACKING.rst: Fixing formatting of titles.
  
  Titles have 2 blank lines before them, and no blank lines after them.
  

• 75ebcf56
  by Tristan Maat at 2018-04-23T13:10:18Z

  bst-here: Add '-p' flag to upgrade buildstream (Issue #291)
  

• 4759e2ad
  by Sam Thursfield at 2018-04-23T13:37:06Z

  plugins/elements/compose.py: Use sets instead of lists where appropriate
  
  There is no significant order for the lists of added, removed and
  modified files, so use an unordered set() to store the data.
  

• d423732f
  by Sam Thursfield at 2018-04-23T13:37:06Z

  plugins/elements/compose.py: Only calculate snapshot when needed
  
  This is an optimisation for the case where the compose element doesn't
  do any splitting, and also brings 2 related code fragments closer to
  each other.
  

• cb10e2bd
  by Sam Thursfield at 2018-04-23T13:37:06Z

  plugins/elements/compose.py: Avoid losing files inside directory symlinks
  
  The logic for determining which files were removed by integration
  commands was broken when dealing with files staged within symlink
  directories.
  
  This rather weird scenario is only possible because of the way
  BuildStream layers artifacts. If artifact 1 contains a symlink from
  `/sbin` to `/usr/sbin`, and artifact 2 is staged on top and contains
  a file `/sbin/init`, then the resulting filesystem contains a file at
  `/usr/sbin/init`.
  
  The manifest used by the compose element is generated from the contents
  of the individual artifacts, so it lists the original paths such as
  `/sbin/init`, but would would not contain `/usr/sbin/init` as nothing
  has processed the symlinks.
  
  The path `/sbin/init` is valid inside the composed tree, but filesystem
  traversals that don't follow symlinks will not report that path in their
  results. The compose plugin would look for `/sbin/init` in the results
  of `utils.list_relative_paths()`, find it missing, and would act as if
  some integration command had removed the file. This meant it would not
  end up in the results.
  
  To fix this, I have inverted the logic that processes the results of the
  integration commands. We now work through every path in the manifest
  and check it against the results of the integration commands, rather
  than the other way around, and if any path from the manifest doesn't
  appear in the snapshot we assume that it has staged in a different
  location due to symlinks.
  
  See: https://gitlab.com/BuildStream/buildstream/issues/270
  

• 3ec1cab1
  by Sam Thursfield at 2018-04-23T13:37:06Z

  Add compose-symlinks testcase
  
  This is a regression test for the issue raised here:
  https://gitlab.com/BuildStream/buildstream/issues/270
  

• caa52411
  by Sam Thursfield at 2018-04-23T13:37:06Z

  tests/testutils/integration.py: Avoid inconsistent symlink handling
  
  The output of walk_dir() seemed to be inconsistent in how it traversed
  symlinks. Presumably this is to do with differences in how the filesystem
  return files. If we do an in-place sort of the list of files and directories
  that we get, os.walk() will honour that order which should make the output
  stable.
  

• 84862708
  by Javier Jardón at 2018-04-24T01:45:08Z

  doc/source/install.rst: Update install instruction for Debian strech
  

• dabaf601
  by Javier Jardón at 2018-04-24T01:45:08Z

  doc/source/install.rst: Update build instructions for Debian Testing/Sid
  

• fbcf4ad4
  by Javier Jardón at 2018-04-24T01:45:08Z

  doc/source/install.rst: Update build instructions for Fedora
  

• 4be5f1f6
  by Javier Jardón at 2018-04-24T01:45:08Z

  doc/source/install.rst: Add fuse to the list of dependencies
  
  See #358
  

• 07ac1bfa
  by Javier Jardón at 2018-04-24T01:45:08Z

  doc/source/install.rst: Add requirements to use the bst cache
  
  See #358
  

• b6a64313
  by Javier Jardón at 2018-04-25T08:22:53Z

  Move curent before_script: commands to the only jobs where they are actually needed
  

• ae927b81
  by Javier Jardón at 2018-04-25T08:29:55Z

  .gitlab-ci.yml: Add job to analyze project code quality with Code Climate CLI
  
  See #431
  

• 60029b31
  by Javier Jardón at 2018-04-25T08:29:55Z

  .gitlab-ci.yml: Move radon job to post stage
  

• 38ac4992
  by Tristan Van Berkom at 2018-04-25T13:29:05Z

  _projectrefs.py: Adhere to policy on private symbols
  
  A late fix for issue #285
  

• 7caecc68
  by Tristan Van Berkom at 2018-04-25T14:26:42Z

  _loader: Give the loaded MetaSource objects the kind of the owning element
  
  This is needed so that Sources can derive whether they belong
  to a junction or not, which is needed for separating where
  junction refs are stored.
  

• 719a3fc3
  by Tristan Van Berkom at 2018-04-25T14:26:42Z

  Load and save junctioned source refs from/to junction.refs
  
    o _projectrefs.py: Additional constructor option to choose the base name
  
    o _project.py: Load two ProjectRefs objects, one for the junctions
  
    o source.py: Load and save junctioned source refs with the appropriate ProjectRefs object
  
    o tests: Updated some tests to expect junctions to be stored in junction.refs
  
  This fixes issue #361
  

• 52da8751
  by Tristan Van Berkom at 2018-04-25T14:32:47Z

  doc: Clarify that junction sources are stored separately from project.refs
  

• 54633e92
  by Javier Jardón at 2018-04-25T16:16:18Z

  .gitlab-ci.yml: Run test in current Debian stable (stretch)
  

• 4be462d3
  by Tristan Van Berkom at 2018-04-26T05:17:44Z

  Use versioneer instead of setuptools_scm
  
  Using setuptools_scm had a couple of bad problems:
  
    o Unexpected versioning semantics, setuptools_scm would
      increment the micro version by itself in the case that
      we derive a version number from something that is not a tag,
      making the assumption that we are "leading up to" the next
      micro version.
  
      People mostly dont expect this.
  
    o When installing in developer mode, i.e. with `pip3 install --user -e .`,
      then we were always picking the generated version at install time
      and never again dynamically resolving it.
  
      Many of our users install this way and update through git, so it's
      important that we report more precise versions all the time.
  
  This commit needs to make a series of changes at the same time:
  
    o Adds versioneer.py to the toplevel, this is used by setup.py
      for various activities.
  
      This is modified only to inform the linter to skip
  
    o Adds buildstream/_version.py, which is generated by versioneer
      and gives us the machinery to automatically derive the correct version
  
      This is modified only to inform the linter to skip
  
    o Adds a .gitattributes file which informs git to substitute
      the buildstream/_version.py file, this is just to ensure that
      the versioning output would work if ever we used `git archive`
      to release a tarball.
  
    o Modifies setup.py and setup.cfg for versioneer
  
    o Modifies utils.py and _frontend/cli.py such as to avoid importing
      the derived version when running bash completion mode, we dont
      derive the version at completion time because this can result
      in running a subprocess (when running in developer install mode)
      and is an undesirable overhead.
  
    o Updates tests/frontend/version.py to expect changed version output
  

• 16e719a9
  by Tristan Van Berkom at 2018-04-26T05:23:45Z

  NEWS: Added news entry for versioning fixes
  

• b5c59295
  by Tristan Van Berkom at 2018-04-26T05:47:17Z

  element.py: Added BST_FORBID_BDEPENDS to compliment BST_FORBID_RDEPENDS
  

• 14aa2590
  by Tristan Van Berkom at 2018-04-26T05:47:17Z

  plugins/elements/junction.py: Forbid dependencies
  
  Move this logic into the junction element itself, instead
  of special case erroring for this in the loader.
  

• edfcd0f6
  by Tristan Van Berkom at 2018-04-26T05:47:17Z

  _loader.py: Removing special case error for junctions depending on elements
  
  Leave this error to be handled by preflight.
  
  Updated test case to expect the new ElementError instead of a LoadError
  

• 0b35585b
  by Tristan Van Berkom at 2018-04-26T05:55:17Z

  _versions.py: Bumping format version for changes project.refs behavior
  
  If we want to depend on being able to revision junction.refs and
  project.refs separately, then we are better off just considering the
  project.refs feature as available since the new version 8.
  
  This will not harm projects which depended on it since version 5 instead.
  

• 78ad3b77
  by Tristan Van Berkom at 2018-04-26T05:57:57Z

  doc/source/projectconf.rst: Documenting ref-storage as available since format version 8
  

• d3a991a5
  by Tristan Van Berkom at 2018-04-26T06:11:43Z

  setup.py: Stop installing test cases.
  
  As recommended by Valentin David in issue #332.
  

• 24028499
  by Tristan Van Berkom at 2018-04-26T06:40:13Z

  man: Updated man pages for 1.1.3 release
  

• 7bb59282
  by Tristan Van Berkom at 2018-04-26T06:42:27Z

  README.rst: Removed 'covers many programming languages' from the About BuildStream
  
  This is rather innacurate, you can build software in any programming languange
  given that you have provided the required resources for this in the runtime
  you are using.
  

• 06ae434a
  by Tristan Van Berkom at 2018-04-26T06:43:52Z

  README.rst: Fixed formatting
  
  No empty lines after a section title, 2 empty lines before a section title.
  

• d1c3d705
  by Jürg Billeter at 2018-04-30T09:03:43Z

  _artifactcache/pushreceive.py: Fix commit object check in send_info()
  
  Fixes #325.
  

• 6f1d1552
  by Jürg Billeter at 2018-04-30T09:03:43Z

  _artifactcache/pushreceive.py: Do not return True for skipped push
  

• cb9f318b
  by Jürg Billeter at 2018-04-30T09:06:10Z

  element.py: Print info message when artifact was actually pushed
  
  This matches the pull code path.
  

• 0916d81e
  by Jürg Billeter at 2018-04-30T09:06:10Z

  tests/testutils/runcli.py: Improve detection of pulled/pushed elements
  

• f00a435c
  by Phillip Smyth at 2018-05-01T11:34:19Z

  _artifactcache/pushreceive.py: Modified incorrect error message
  
  changed encode_message push exception from
  'Command must by GLib.Variant'
  to
  'Command must be PushCommand'
  

• eceb7c98
  by Jürg Billeter at 2018-05-03T14:06:55Z

  element.py: Fix buildable check in non-strict mode
  
  Ensure that the strong cache key of each build dependency is available
  before an element is built. Otherwise the strong cache key of the
  element cannot be calculated and caching the artifact produces an
  AssertionError.
  
  In non-strict mode an element's strong cache key may not be available
  yet even though an artifact is available in the local cache. This can
  happen if the pull job is still pending as the remote cache may have an
  artifact that matches the strict cache key, which is preferred over a
  locally cached artifact with a weak cache key match.
  
  Fixes #383.
  

• 494f18fe
  by James Ennis at 2018-05-04T09:24:41Z

  commands.rst: Split up commands to easily allow for further description
  
  An introduction for this section was also added
  

• 451b05fc
  by Valentin David at 2018-05-06T06:47:17Z

  Fix provenance in error message for undefined variables.
  

• 745d0647
  by Valentin David at 2018-05-06T06:47:17Z

  tests/format/variables.py: Add a test for undefined variable error message
  

• 53bff7d6
  by James Ennis at 2018-05-07T10:25:52Z

  app.py: Ensure an elements sub-dir is created upon project initialisation
  

• 2e252e9d
  by Chandan Singh at 2018-05-07T11:21:32Z

  bst-here: Fix typo in help text
  

• 6ffca067
  by Chandan Singh at 2018-05-07T11:21:32Z

  bst-here: Allow users to specify Docker image
  
  Currently, the name of the Docker image is hardcoded in bst-here script.
  This makes it harder for users to override it, which may be needed for
  various reasons (custom plugins, private Docker registries etc.)
  
  Add `-i` option to allow users to specify the base image. When `-i` is
  not provided, default to `BST_HERE_IMAGE` environment variable if it's
  set and otherwise to the current image -
  `buildstream/buildstream-fedora`.
  
  Also, re-order the command-line options in help text and source code in
  alphabetical order to maintain sanity as the number of options is slowly
  growing.
  
  Part of #378.
  

• 49176039
  by Chandan Singh at 2018-05-07T11:21:32Z

  bst-here: Allow users to specify additional volumes to mount
  
  One may want to mount additional volumes to preserve certain directories
  or to share some data between the host and the container. Allow users to
  do so by providing a `-v` option that passes its arguments to
  corresponding `-v`/`--volume` option for `docker run` command.
  
  Part of #378.
  

• e6049105
  by Javier Jardón at 2018-05-07T11:40:54Z

  .gitlab-ci.yml: Remove redundant declaration of dependencies:
  
  "artifacts from all previous stages are passed by default."
  https://docs.gitlab.com/ee/ci/yaml/#dependencies
  

• 03823d12
  by milloni at 2018-05-07T13:51:16Z

  _site.py: Add check_bwrap_version() function
  
  Lazily parse the version of bwrap the first time the function is called.
  On subsequent calls, used cached version info.
  
  See: #373
  

• f8952d6b
  by milloni at 2018-05-07T13:51:16Z

  Replace bwrap checks with calls to check_bwrap_version
  
  See: #373
  

• bbb894bd
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  tests/format/variables.py: Removed testing of stderr
  
  We have machine readable errors for this purpose, and the
  strings happen to change causing tests to break if we test the specific
  UI (reported error strings are UI).
  

• 2390c814
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _stream.py: New Stream object, main calling interface for BuildStream core
  
  This is the first part of the pipeline refactor, at this stage
  all calling interfaces remain the same, except that invocation
  of the scheduler has been moved from Pipline to Stream.
  

• 11dffaef
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _stream.py: Add StreamError exception
  
  Use Stream error for Stream errors.
  

• 76c260b0
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _stream.py: Absorb workspace functionality from App.
  

• c9f67f1e
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _frontend: Early logging initialization
  
    o This makes logging independent from the Pipeline()
  
    o Removed size_request Widget() method, add context to Widget() initializer
  
    o Make the Status() widget derive anything it needs through the Context()
  

• 541cd760
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _stream.py, _frontend: Call Stream() APIs with CLI arguments directly.
  
  This shifts the whole responsibility of interpreting command line
  targets etc to the Stream() object itself. With this commit, the
  Pipeline() truly becomes slaved to the Stream().
  

• c81fb2da
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  Moving 'fetch_subprojects' configuration to stream <--> loader
  
  This was previously decided in CLI, but knowledge of what to initialize
  has been moved to Stream().
  
  Now there is no more point to store this configuration in the Context,
  we just have the Stream() decide it when asking the Pipeline() to
  invoke the Loader().
  

• a542c818
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _frontend/app.py: Remove partially_initialized()
  
  There is no more need for this distinction.
  

• b8e15706
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _artifactcache: Added ArtifactCache.setup_remotes()
  
  This removes some additional initialization code from Pipeline().
  
  Some symbols have changed here, the initialization is now called
  from Stream(), and a test case was also adjusted for this.
  

• 5bdc0a79
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _stream.py, _pipeline.py: Refactoring of the pipeline itself
  
  Here the pipeline becomes essentially stateless, some dangling
  state remains to be factored out because of frontend accesses
  which will be changed in a later commit.
  
  Essentially, the Pipeline.load() method no longer has any knowledge
  of the specific purposes of the loaded targets, and now takes
  a list of target groups and returns a corresponding list of element
  groups.
  
  The Stream() business logic methods now use other pipeline helper
  methods to create and filter lists from the loaded target elements.
  
  The Stream() also finally absorbs the Scheduler frontend facing
  APIs. However Queues are still exposed on the Stream object for
  logging purposes and through callbacks such that the frontend can
  retry elements.
  

• 1ec80edc
  by Tristan Van Berkom at 2018-05-07T18:59:38Z

  _frontend/cli.py: Fetch all elements when tracking a build plan
  
  It makes not sense to type `bst fetch --track --deps all <targets>`,
  because tracking will inevitably modify the build plan.
  
  Stream initialization will not cope with this either, instead of
  silently doing something which does not make any sense, we add an
  assertion that this should not happen.
  
  Unfortunately since `plan` is the default deps type for `bst fetch`,
  this is likely to happen so it's important to correct.
  
  This patch adds a warning in the case tracking of the build plan
  elements is requested, and converts the request to track all elements
  instead.
  

• 32c30766
  by Tristan Van Berkom at 2018-05-10T06:59:47Z

  _frontend/app.py: Fixed ordering of initialization sequence.
  
  We need the logging handler setup before creating the Stream(),
  as the Stream() will expect logging to be already functional.
  

• 8985da07
  by Valentin David at 2018-05-10T11:33:12Z

  buildstream/plugins/sources/_downloadablefilesource.py: Store etag along with cache.
  
  Fixes #377.
  

• edf3e029
  by Tristan Van Berkom at 2018-05-10T12:20:06Z

  Add soft reset functionality for workspaces
  
  Add `--soft` option to `bst workspace reset` which would allow uses to
  reset workspace-related state without affecting its contents. This will
  be useful in case when an user wants to re-run configure-commands for a
  workspaced element.
  
  Patch originally by Chandan Singh, rebased against recent refactor.
  
  Fixes #375.
  

• b2b365fb
  by Chandan Singh at 2018-05-11T01:01:46Z

  _project.py: Remove unused internal function _extract_plugin_paths()
  
  For reference, this method was added in
  commit c41f1093d83eb32606f81005227542699f756b0a, but since
  commit 4912ed5f8922b47c1a16c3752eb92a3f1dea76b2 it has been unused.
  

• e1420437
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _artifactcache/ostreecache.py: Fix global remotes
  
  Counter-intuitively, Python list += mutates the list. Use explicit
  copy() and extend() instead of += to avoid adding project-specific
  remotes to the global remote list.
  

• 6d71826e
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _pipeline.py: Remove unused ignore_cache parameter from _Planner
  

• 37e5362e
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _pipeline.py: Allow pulling strict artifacts in non-strict mode
  
  Remotely cached artifacts matching the strict cache key take precedence
  over locally cached artifacts matching only the weak cache key. However,
  locally cached artifacts were excluded from the build plan, which means
  that BuildStream never even checked whether the strict artifact is
  available in the remote artifact cache.
  
  This changes planning to keep cached elements in the build plan in
  non-strict mode if a remote artifact cache is used.
  

• df17f602
  by Jürg Billeter at 2018-05-11T05:57:37Z

  tests: Do not rely on 'downloadable' status
  
  This is in preparation for moving away from summary files.
  

• da4af798
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _frontend/cli.py: Remove bst show --downloadable option
  
  This is in preparation for moving away from summary files.
  

• 4e512529
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _stream.py: Remove unused downloadable parameter from load_selection()
  

• db503a42
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _artifactcache: Add bool return value to pull()
  
  Calling pull() for a potentially unavailable artifact is no longer
  considered an error.
  

• e2c150f3
  by Jürg Billeter at 2018-05-11T05:57:37Z

  _artifactcache: Add element paramater to has_fetch_remotes()
  

• 17fd0b24
  by Jürg Billeter at 2018-05-11T06:01:12Z

  element.py: Do not query remote artifact caches ahead of pull/push
  

• 095ce5a5
  by Jürg Billeter at 2018-05-11T06:01:12Z

  _artifactcache/artifactcache.py: Remove unused methods
  

• 31ae2f55
  by Jürg Billeter at 2018-05-11T06:01:12Z

  _artifactcache/ostreecache.py: Do not use summary file
  

• 6ba62c57
  by Jürg Billeter at 2018-05-11T06:01:12Z

  doc/source/artifacts.rst: Drop section about summary file updates
  
  Summary files are no longer used.
  

• b78df1e2
  by Jürg Billeter at 2018-05-11T06:01:12Z

  _ostree.py: Remove unused function list_remote_refs()
  

• 270da060
  by Jürg Billeter at 2018-05-11T06:01:12Z

  Do not pull/fetch/build elements that are not required
  
  For `bst build --deps plan`, do not process elements in pull/fetch/build
  queues until they are requested by a reverse dependency.
  

• 3d9491dd
  by Jürg Billeter at 2018-05-11T08:19:54Z

  _stream.py: Do not force `PipelineSelection.ALL` for build with tracking
  
  With the dynamic plan using `_set_required()` we can avoid
  pulling/fetching/building unneeded elements even if elements are being
  tracked.
  

• 01a02a9a
  by Tristan Van Berkom at 2018-05-11T08:45:09Z

  _workspaces.py: Use file mtime for workspace cache keys instead of checksumming
  
  The rationale for this is basically:
  
    o A workspace is completely local
  
    o A cache key for a workspace is used to determine if the
      workspace has changed or not
  
    o Within reason, one should not expect buildstream to detect
      a change without a file's mtime having been modified
  
  On my machine, it takes about 1 minute to checksum 11GB of WebKit checkout,
  that is annoyingly a long time. With this change, it takes only 5 seconds.
  
  Not perfect, but it's a start.
  
  This is related to issues #294, #295 and #392
  

• b9dea666
  by Chandan Singh at 2018-05-11T11:29:45Z

  bst-init: Ensure --element-path is respected by the command
  
  Previously "elements" was hard-coded as the path for the elements
  directory whereas it was supposed to be configurable via the
  `element_path` option to `init_project()`. This led to incorrect
  behavior when `bst init` was run `--element-path` option.
  
  Also, extend tests to test the creation of elements directory.
  
  Fixes #398.
  

• cb24a1d8
  by Tristan Van Berkom at 2018-05-16T06:18:07Z

  doc/source/projectconf.rst: Added link anchors for project option types
  

• cb076de8
  by Tristan Van Berkom at 2018-05-16T06:18:07Z

  doc/source/projectconf.rst: Added link anchor for source aliases
  

• bb8bd1cf
  by albfan at 2018-05-16T07:51:31Z

  doc: Add first example, building on a flatpak SDK
  
  This adds:
    o A ToC area for adding examples
    o The instructive example page for the first example
    o The example project under doc/examples
    o The corresponding integration test in tests/examples
  

• b7e75da3
  by Tristan Van Berkom at 2018-05-16T07:51:31Z

  tests/examples/flatpak-autotools.py: Workaround setuptools bug
  
  Until the setuptools bug is fixed, symlinks are not included
  in source distributions - this works around the problem until which
  point we can use a setuptools version without the bug.
  

• b98791cb
  by Tristan Van Berkom at 2018-05-16T07:51:31Z

  doc/source/examples.rst: Rearranged ToC for examples
  
  We want one example per file, not a huge file full of different examples.
  

• e6f59b63
  by Tristan Van Berkom at 2018-05-16T07:51:31Z

  doc/examples/flatpak-autotools/README.md: Removing
  
  This is redundant with doc/source/examples_flatpak_base.rst, lets
  please only document things once.
  

• c205991e
  by Tristan Van Berkom at 2018-05-16T07:51:31Z

  doc/source/examples_flatpak_autotools.rst: Enhanced example
  
  This has now changed to:
  
    * Have explanations of the project.conf and each element
      in the example, explaining what they are for
  
    * Have links into the reference for the specific features
      this example uses, such that the reader can get familiar
      with the reference manual from example windows
  

• e023beb5
  by Tristan Van Berkom at 2018-05-17T05:46:46Z

  _frontend: Notify failures when on linux
  
  Opens the door to platform specific frontend features,
  and implements a platform specific "notify" method.
  
  This is based on Valentin David's patch on merge request 447,
  and this fixes issue #385.
  

• 84a57b2f
  by Tristan Van Berkom at 2018-05-17T10:56:04Z

  _frontend/linuxapp.py: Fixed crash in previous commit
  
  Commit e023beb56b180316a58f409d48f158e9dc1ece95 introduced
  a crash with an invalid access to os.environ, this fixes it.
  

• 314c25d8
  by Chandan Singh at 2018-05-17T13:21:09Z

  element.py: Fix consistency of workspaced elements when ref is missing
  
  Fixes #393.
  

• ec16561f
  by Jonathan Maw at 2018-05-17T15:45:50Z

  tests: Add workspace tests for filters
  

• 8316a1fe
  by Jonathan Maw at 2018-05-17T15:45:50Z

  tests: Add track tests for the filter element
  

• 38dc2dda
  by Jonathan Maw at 2018-05-17T15:45:50Z

  element.py: Add _get_source_element() method
  

• 6e2e976c
  by Jonathan Maw at 2018-05-17T15:45:50Z

  filter.py: Redirect _get_source_element() calls to its build depend
  
  We do this because trying to open/close/reset a workspace on a filter
  element doesn't make sense, but is probably meant for the element that
  the filter element filters.
  

• c09be483
  by Jonathan Maw at 2018-05-17T15:45:50Z

  Redirect elements when performing tracking and workspace commands
  
  i.e. when an element is specified, it may be replaced with its source element.
  

• 206a7ee8
  by Jonathan Maw at 2018-05-21T09:59:45Z

  NEWS: Add item for element redirection
  

• 77da5945
  by toscalix at 2018-05-22T14:19:19Z

  bst bug template
  

• 3749e44e
  by toscalix at 2018-05-22T14:21:31Z

  bst merge request template
  

• b69e6482
  by toscalix at 2018-05-22T14:25:22Z

  correction :
  

• 4d92c106
  by toscalix at 2018-05-22T14:27:29Z

  correction :
  

• a06778aa
  by James Ennis at 2018-05-23T14:45:48Z

  junction.py: Include note regarding explicit tracking
  

• 9b261f56
  by toscalix at 2018-05-24T19:07:24Z

  bst_task.md: default template also as option
  

• a4cd9d7c
  by toscalix at 2018-05-24T19:11:27Z

  Merge branch 'master' of https://gitlab.com/BuildStream/buildstream
  
  bst_task.md : default template also as option.
  
  This is the default template. It is neccessary to include it in this directory to have it as option which is required in special cases.
  
  This fix issue https://gitlab.com/BuildStream/nosoftware/alignment/issues/12
  

• c0de75e2
  by albfan at 2018-05-25T14:35:18Z

  .gitignore: Ignore autogenerated docs
  

• 4708ee6b
  by Antoine Wacheux at 2018-05-31T12:57:51Z

  _artifactcache/ostreecache.py: Fix artifact cache initialization result tuple
  
  In case of failure, the tuple contained 4 elements instead of 3, causing
  BuildStream to crash.
  

• 065f5ac7
  by Tristan Maat at 2018-06-05T08:47:33Z

  _context.py: Normalize user-defined paths
  

• 279dcb40
  by Tristan Maat at 2018-06-05T08:47:33Z

  tests/context/context.py: Test setting XDG_CACHE_HOME
  

• fe24fc3b
  by Gökçen Nurlu at 2018-06-05T16:23:22Z

  Add test to verify cleanup behaviour after bwrap
  
  This adds `sandbox-bwrap/test-cleanup.bst` to test the cleanup behaviour in
  `sandbox/_sandboxbwrap.py`: There already exists a non-empty `/tmp` folder and
  BuildStream should not try to remove it after a command is executed with bwrap.
  

• 75fe8037
  by Gökçen Nurlu at 2018-06-05T16:23:22Z

  _sandboxbwrap.py: Fix post-bwrap cleanup behaviour
  
  The cleanup was supposed not to remove folders (`/dev`, `/tmp`, `/proc`) if
  they already existed before bwrap but it did the opposite: it tried to remove
  them if they existed before, and didn't remove them if they were created during
  bwrap. This was caused by a `not` clause, and this removes it.
  
  Fixes #379
  

• 8d88b52a
  by Phillip Smyth at 2018-06-05T18:00:03Z

  utils.py: Correcting a typo in safe_remove's comment
  

• 1d694b28
  by Chandan Singh at 2018-06-05T19:34:21Z

  _artifactcache/pushreceive.py: Add Click type for CLI argument 'repo'
  
  The CLI for `bst-artifact-receive` expects a `repo` argument, which is
  supposed to be a directory, but Click currently expects it to be just
  any string. This results in stack traces like the one below when the
  argument provided is not a directory:
  
          $ ~/.local/bin/bst-artifact-receive --pull-url http://foo foobaz
          Traceback (most recent call last):
            File "/root/.local/bin/bst-artifact-receive", line 8, in <module>
              sys.exit(receive_main())
            ...
            File "/src/buildstream/buildstream/_artifactcache/pushreceive.py", line 581, in __init__
              self.repo.open(None)
          GLib.Error: g-io-error-quark: /src/buildstream/43fref: opendir(/src/buildstream/foobaz): No such file or directory (1)
  
  Add types for this argument such that it throws better error messages
  when it receives bad arguments. With the Click types added, it will
  instead fail with messages like these:
  
          $ ~/.local/bin/bst-artifact-receive --pull-url http://foo foobaz
          Usage: bst-artifact-receive [OPTIONS] REPO
  
          Error: Invalid value for "repo": Directory "foobaz" does not exist.
  
          $ ~/.local/bin/bst-artifact-receive --pull-url http://foo setup.py
          Usage: bst-artifact-receive [OPTIONS] REPO
  
          Error: Invalid value for "repo": Directory "setup.py" is a file.
  
  Fixes #409.
  

• 7f9216b3
  by Tristan Van Berkom at 2018-06-06T17:20:24Z

  _frontend/app.py: Remove unused variable at global scope
  

• 5dc8ab98
  by Ed Baunton at 2018-06-06T17:56:59Z

  Add a kind for Make
  

• b36db258
  by Tristan Van Berkom at 2018-06-06T18:50:54Z

  doc/source/authoring.rst: Adding `make` element to the docs
  

• 5ddec56f
  by Tristan Van Berkom at 2018-06-06T18:54:03Z

  _versions.py: Bump format version to 9
  
  For addition of new `make` plugin
  

• 17fa7ba4
  by Tristan Van Berkom at 2018-06-06T18:54:27Z

  make plugin: Ammended documentation to note it's since version.
  

• 0f2fd8c8
  by Tristan Van Berkom at 2018-06-06T18:55:56Z

  NEWS: Mentioning new `make` build element
  

• 55dc3b27
  by Tristan Van Berkom at 2018-06-06T18:59:00Z

  make build element: Add support for PREFIX
  

• 7df95654
  by Chandan Singh at 2018-06-06T21:27:30Z

  _loader/loader.py: Report element-path when failing to load elements
  
  It can be confusing, especially to new BuildStream users, that the CLI
  expects targets to be specified relative to element-path and not the
  current directory. Previously, the CLI would give a generic message
  stating that the file could not be found but it was not obvious that it
  was looking in the `element-path` directory.
  
  Explicitly print the element-path in the summary. Also, try to check if
  the specified element exists in the elements directory and print a hint
  to use the element-path relative paths if that's the case.
  
  Fixes #396.
  
  This is is also related to #341. This commit aims to tackle that issue
  by trying to educate users about element-path.
  

• a5ff465d
  by Chandan Singh at 2018-06-06T22:10:49Z

  _project.py: Allow running bst commands from subdirectories of project root
  
  When initializing the project, BuildStream will continue searching for
  project.conf in parent directories in case it is not found in the
  current directory.
  
  Fixes #368.
  

• 69e59443
  by Chandan Singh at 2018-06-06T22:10:49Z

  _frontend/cli.py: Try to autocomplete element paths when running from a subdirectory
  
  The previous commit added support for running bst commands form
  subdirectories of the project root. Make autocomplete also work in a
  similar way.
  

• e36001f9
  by Ed Baunton at 2018-06-07T13:43:27Z

  Make `bst help` work
  

• b25c31ea
  by Ed Baunton at 2018-06-07T17:34:27Z

  Implement bst help
  

• 8234e9c7
  by Tristan Van Berkom at 2018-06-07T20:14:11Z

  _frontend/cli.py: Allow specifying commands in `bst help`
  
    o This supports deeply nested commands as well as shallow commands
  
    o Automated support for bash completions included
  

• 9c65b908
  by Tristan Van Berkom at 2018-06-07T20:14:11Z

  tests/completions: Added test case for custom `bst help` completions
  

• 4632cb03
  by Tristan Maat at 2018-06-07T20:35:51Z

  _ostree.py: Reintroduce remove()
  

• 60a29f68
  by Tristan Maat at 2018-06-07T20:35:51Z

  _ostree.py: Introduce _list_all_refs() and list_artifacts()
  
  The unused list_remote_refs() function has also been removed as
  part of this commit.
  

• b7191fda
  by James Ennis at 2018-06-07T20:35:51Z

  pushreceive.py: Ensure there is a repopath
  

• 58adbe0b
  by James Ennis at 2018-06-07T20:41:03Z

  pushreceive.py: Remove LRP artifacts from cache, introduce clean_up_cache()
  
  This fixes #136
  

• 8a0c0a9a
  by James Ennis at 2018-06-07T20:41:33Z

  element_generators.py: Add a utility function to create arbitrary size elements
  
  New file consisting of functions that allow us to generate elements
  on the fly
  __init__.py now includes create_element_size()
  

• ed0df6b5
  by James Ennis at 2018-06-07T20:41:33Z

  push.py: Add tests to test remote cache expiry
  
  The tests include a new environment variable: BST_CACHE_QUOTA.
  This variable is used in receive_main() in pushreceive.py.
  Test names: test_artifact_expires, test_large_artifact_is_not_cached_remotely
  test_recently_pulled_artifact_does_not_expire: marked as xfail until
  we implement LRU expiry in remote share
  

• b8a52cbd
  by James Ennis at 2018-06-07T20:41:33Z

  pushreceive.py: Add comments to code and name variables more sensibly
  

• b823a616
  by James Ennis at 2018-06-07T20:41:33Z

  pushreceive.py: Ensure huge artifacts are not pushed
  

• a231d41b
  by James Ennis at 2018-06-07T20:41:33Z

  pushreceive.py: Abstract the buffer_
  

• de194dad
  by Tristan Van Berkom at 2018-06-07T20:41:33Z

  _artifactcache/pushreceive.py: Cleanup reported error when receiving oversized artifacts
  
  This user facing string was redundantly declared in two places, only
  the message when catching the error was ever printed.
  

• 8f2bf4e6
  by Tristan Van Berkom at 2018-06-08T19:48:11Z

  autotools plugin: Dont regenerate existing configure scripts
  
  This closes #256
  

• af10c1ba
  by Valentin David at 2018-06-08T21:07:22Z

  Interpret names as colon separated junction path in loader.
  
  'a.bst:b.bst' gets interpreted as 'b.bst' from junction 'a.bst'.
  
  Part of #359.
  

• ccec163b
  by Valentin David at 2018-06-08T21:07:22Z

  Reword uses of project in Workspaces.
  
  Make it clear we expect the top-level project here as we use it to
  resolve paths relative to project directory.
  

• 130bfbb8
  by Valentin David at 2018-06-08T21:07:22Z

  Handle cross junction elements in workspaces.
  
  Workspaces are now index by colon separated junction path. This
  now allows to create workspaces for elements in external projects.
  
  Workspaces are owned by context instead of root project. However
  it is initialized once top-level project is registered as we need
  to resolve paths relatively to this top-level project.
  
  Part of #359.
  

• acde3ba8
  by Valentin David at 2018-06-08T21:07:22Z

  Allow tracking dependencies within sub-projects.
  
  --track-cross-junctions now concerns crossing junctions rather than
  forbidding elements in sub-project to be tracked.
  
  Part of #359.
  

• 71b78794
  by Tristan Van Berkom at 2018-06-09T19:48:42Z

  doc/source/index.rst: Use only one level depth ToC on main page
  
  The main page has too much information on it otherwise, we want
  a friendly, not overwhelming first page to our docs.
  

• 055b77e8
  by Tristan Van Berkom at 2018-06-09T21:14:10Z

  doc: Added plugins as ToC elements instead of orphaned links
  
    o Now the page titles are declared in plugins, allowing for
      a more descriptive ToC
  
    o Makefile and plugin.rsttemplate updated to not produce the title,
      to no longer use `:orphan:` for plugin pages, and to ignore any
      private modules in the plugin directories.
  
    o Interestingly, now the docs will fail to build if you add
      a new plugin and forget to add it to the documentation.
  

• 5f794829
  by Tristan Van Berkom at 2018-06-09T21:23:23Z

  doc/source/main_core.rst: Use a hidden toctree to hide away some things
  
  Sphinx generates some library style module index, we just include
  it in a hidden toctree and avoid using it altogether.
  

• 96c219d0
  by Tristan Van Berkom at 2018-06-09T21:36:07Z

  doc/source/core_framework.rst: Use a toctree instead of links
  

• 1e98ecb2
  by Tristan Van Berkom at 2018-06-09T22:00:24Z

  doc/source/index.rst: Add resources directly at the bottom of the main page
  
  This is nice to have on the main page, and is only a few links, dont
  like having a whole toplevel ToC entry for this.
  

• 77dd1091
  by Tristan Van Berkom at 2018-06-09T22:08:49Z

  doc: Split out cache keys and sandboxing writings into a sub section
  

• 53653d29
  by Tristan Van Berkom at 2018-06-09T22:09:50Z

  doc/source/authoring.rst: Removed unneeded link anchor from here
  

• 52412db4
  by Tristan Van Berkom at 2018-06-09T22:18:28Z

  .gitlab-ci.yml: Stop depending on specific version of sphinx
  
  This branch fixes #312 by using a hidden toctree to include
  the buildstream package and reducing the amount of allowed :orphan:
  pages.
  

• 8e6f1a9b
  by Tristan Van Berkom at 2018-06-09T23:44:25Z

  doc: Stop generating the modules index
  
  We still have a few unused artifacts in the docs generation,
  this is just one less.
  

• 99fb2d53
  by Tristan Van Berkom at 2018-06-09T23:45:41Z

  doc/examples/flatpak-autotools: Use consistent project name
  
  This was named autotools-flatpak, changed to flatpak-autotools.
  

• 710e1fd4
  by Tristan Van Berkom at 2018-06-09T23:48:09Z

  doc/source/examples_flatpak_autotools.rst: Added some links to the example project on gitlab.
  

• 4af8112d
  by Tristan Van Berkom at 2018-06-10T00:11:07Z

  doc/source/examples.rst: Use a flat ToC here
  
  Here we're really listing examples, a ToC with depth is
  not great here.
  

• d0dc9e64
  by Tristan Van Berkom at 2018-06-10T00:21:47Z

  doc/source/main_using.rst: Reversed ToC here
  
  Seems that the "commands" is taking a lot of space such that
  we can't see the other sections here easily, that is alright
  if "commands" remains at the end.
  

• ea775fa3
  by Tristan Van Berkom at 2018-06-10T01:12:37Z

  doc: Removed some unused link targets
  

• 9f77cb66
  by Tristan Maat at 2018-06-11T18:16:33Z

  Handle missing tags in git repositories correctly
  
  Fixes issue #380
  

• e112a88f
  by Tristan Van Berkom at 2018-06-12T21:38:25Z

  doc/bst2html.py: Added bst2html.py
  
  This baby runs bst and captures the output with colors enabled
  and then generates some html we can include in documentation.
  
  These can be generated in CI continuously and used in the documentation.
  

• 52f70a0d
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  doc/Makefile: Run bst2html.py to generate .html files from .run files
  
  If you need an example output of bst to put in the documentation,
  just add a .run file to the doc/examples directory and it will result
  in a similarly named .html file in doc/source/examples.
  

• 050cf2eb
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  doc/sessions: Added run files describing scenarios to build
  
  The Makefile uses these to run some scenarios which are
  later included by the documentation directly.
  

• c900d42e
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  doc/source/examples_flatpak_autotools.rst: Using generated examples
  
  Show the commands at work in this example.
  

• 4dfc8607
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  doc/source/examples.rst: Restructuring individual examples into an examples subdirectory
  

• 19dddf82
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  doc/source/sessions: Committing generated session files.
  
  This is only to make it easier for people who just want to
  build docs locally and not regenerate the session files.
  
  The session snapshot html files are always generated in CI every time.
  

• f3b19f2c
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  HACKING.rst: Added guidelines for adding examples
  
  And some instructions about generating `bst` output for documentation
  

• d9da078d
  by Tristan Van Berkom at 2018-06-12T21:38:26Z

  .gitlab-ci.yml: Speed up docs build by setting up BST_SOURCE_CACHE
  
  Use the gitlab cache for caching sources needed for building projects
  in order to collect sample output for documentation pages.
  

• 53bc7272
  by Tristan Van Berkom at 2018-06-13T02:41:15Z

  doc: Don't allow setting a height on the generated session html
  
  This was a bad idea and doesnt play well with mobile UIs, better
  off to just include the whole thing even if it's long, and let
  the backing page handle vertical scrolling.
  

• 9d7296eb
  by Tristan Van Berkom at 2018-06-13T04:24:27Z

  doc: Overhaul of page names and titles
  
   o Giving main pages simple word titles
  
     This makes the main page:
  
     * About
     * Installing
     * Using
     * Reference
     * Contributing
  
   o Now named all rst files with their parent page name as a prefix.
  
   o Also changed some titles to make overall consistent titles.
  

• ba018efe
  by Chandan Singh at 2018-06-13T20:24:15Z

  contrib/bst-here: Fix command/arg in usage instructions
  
  As we can only accept arguments when a command is specified, make it
  clear in the usage instructions.
  

• fa73abe4
  by Chandan Singh at 2018-06-13T20:24:15Z

  contrib/bst-here: Remove redundant create_volume_if_not_exists() method
  
  This method doesn't really do anything as this is already the default
  behavior of `docker volume create` so remove it and always call
  `docker volume create` directly. This command will always print the name
  of the volume on STDOUT which is not very interesting so silence that.
  (If it errors out for some reason, that output will go to STDERR so the
  user should still be able to see that.)
  

• 6da05577
  by Tristan Van Berkom at 2018-06-14T03:12:53Z

  doc: Improve the new console output html generator
  
  Before we were creating one description file for each output,
  making it easier to declare a make rule for it - but the result
  was that we would have to build things more and it takes a
  long time.
  
  Instead, now we have session files which describe a series of commands
  to run in a session, and each command optionally produces an output file.
  

• 5ba5415e
  by Tristan Van Berkom at 2018-06-14T22:22:03Z

  doc/Makefile: Fixed loop over session files
  
  This list needs to not be quoted.
  

• a01a5cc8
  by Tristan Van Berkom at 2018-06-16T17:27:49Z

  doc/bst2html.py: Add support for `fake-output` when running commands.
  
  When specifying a fake-output string, we don't really run the command
  or assume it was a `bst` command, and we pretend that `fake-output`
  was the output of the command.
  
  Specifying an empty string explicitly enables the behavior too
  for faking a command that has no stdout/stderr.
  
  This also adds the "remove-files" hack allowing the session scripts
  to remove files before executing commands (kind of unsure if we're
  gonna keep this...)
  

• b592a80b
  by Tristan Van Berkom at 2018-06-16T19:44:56Z

  doc/bst2html.py, doc/Makefile: Added --force option
  
  If --force is not specified, then we'll skip session files in
  the case that all of the outputs exist.
  
  Now setting BST_FORCE_SESSION_REBUILD when building the docs
  will cause the session files to be rebuilt regardless of whether
  they exist or not.
  
  The .gitlab-ci.yml was also changed to use this and force rebuilds.
  

• 1b88e2c6
  by Tristan Van Berkom at 2018-06-16T20:14:04Z

  doc: Adding part 1 of the getting started tutorial
  
    o doc/Makefile: Added new directory to collect rst files from
  
    o doc/examples/first-project: Added the "first-project" example
      project.
  
    o doc/source/sessions/first-project-*.html: Added the generated
      snippets
  
    o doc/source/using_tutorial.rst: Added the new main tutorial page
  
    o doc/source/tutorial/first-project.rst: Added part 1 of the tutorial here
  
    o tests/examples/first-project.py: Added test for the example project
  
  This is largely based on an example by Javier Jardón, which was
  submitted at https://gitlab.com/BuildStream/buildstream/merge_requests/323
  
  Fixes #103
  

• 6fa3f922
  by Tristan Van Berkom at 2018-06-16T20:14:04Z

  doc/Makefile: Stage stored sessions from session-stored
  
  Separate the revisioned provisional session html files such
  that the git tree does not become dirty as a result of a
  documentation build process - which messes up the docs version
  number and the version number printed in some command line output.
  

• 3f560378
  by Tristan Van Berkom at 2018-06-18T00:58:20Z

  doc/source/tutorial/first-project.rst: Adding links
  
  Linking out to the relevant invoking pages for the command line
  reference, and adding a link anchor here for use by the next chapter.
  

• 73b93002
  by Tristan Van Berkom at 2018-06-18T01:37:22Z

  doc: Adding part 2 of the getting started tutorial
  
    o doc/examples/running-commands: New example project of a `manual` build element
  
    o doc/sessions/running-commands.run: New session file to capture bst output
  
    o doc/source/sessions-stored: Added new generated sessions
  
    o doc/source/tutorial/running-commands.rst: New tutorial entry describing how
                                                commands are run in the sandbox
  
    o tests/examples/running-commands.py: Test case validating the tutorial's assertions
  

• 72fbaa1c
  by Tristan Van Berkom at 2018-06-18T19:01:14Z

  doc/source/tutorial/running-commands.rst: Adding summary
  
  And adding some link anchors needed by the incomming chapter.
  

• b4105e8d
  by Tristan Van Berkom at 2018-06-18T19:17:52Z

  doc: Adding part 3 of the getting started tutorial: autotools element
  
  This part of the tutorial uses a lot of the work from Phil Dawson
  and James Ennis, and uses their example submitted on merge request
  499 as a basis to introduce the user to yaml composition and variable
  resolution.
  
  This is a part of issue #103
  

• 9b81f958
  by Tristan Van Berkom at 2018-06-18T19:18:19Z

  doc/source/using_tutorial.rst: Use a numbered ToC for the tutorial
  

• 48b4cb5c
  by Tristan Van Berkom at 2018-06-19T00:19:09Z

  doc/examples/autotools/project.conf: Correcting project name here
  

• 5b6441fa
  by Tristan Van Berkom at 2018-06-19T02:41:50Z

  doc: Use consistent titles when referring to files in an example project
  
  Use the following form across the board:
  
    ``elements/foo.bst``
    ~~~~~~~~~~~~~~~~~~~~
  
    .. literalinclude:: ../path/to/foo.bst
       :language: yaml
  
  Always use an example project relative path, too.
  

• bb223156
  by Tristan Van Berkom at 2018-06-19T02:47:46Z

  doc/source/examples/flatpak-autotools.rst: Reordering of titles.
  
  Place the titles of literally included `bst` files directly before
  the includes, and moved all related text to start below the included
  file for each section.
  

• 4016bec1
  by Gökçen Nurlu at 2018-06-19T04:41:24Z

  Remove shebangs from python files
  
  Fixes #424
  

• 4f168b9b
  by Daniel Playle at 2018-06-20T22:05:19Z

  Restrict version of pylint
  
  pylint >2 is not compatible with pytest_pylint in its current form. As
  such, allowing any version of pylint for testing results in a failure.
  
  This commit restricts down the allowable versions of pylint to those
  that are both compatible with pytest_pylint, and also offer the feature
  set that we require.
  
  See https://gitlab.com/BuildStream/buildstream/issues/427 for further
  details.
  

• 49fb3636
  by Phil Dawson at 2018-06-21T23:32:46Z

  MANIFEST.in: Fix typo when including BuildStream.doap
  
  Correct includsion of buildstream.doap which does not exist
  to 'include BuildStream.doap'.
  
  This commit resolves issue #430
  

• 411dd100
  by Patrick Martin at 2018-06-22T15:41:34Z

  doc/source/index.rst: fix tiny, but impactful typo
  
  Fixes #435
  

• d9a433a7
  by Javier Jardón at 2018-06-22T17:03:44Z

  BuildStream depends on host tools for Source plugins
  

• b5d91794
  by Javier Jardón at 2018-06-22T17:03:44Z

  Source plugin tar depends on host's lzip
  
  Issue #353
  

• 7d97c6d3
  by Javier Jardón at 2018-06-22T17:03:44Z

  Source plugin bzr depends on host's bzr
  
  Issue #353
  

• 3b1e869b
  by Javier Jardón at 2018-06-22T17:03:44Z

  Source plugin git depends on host's git
  
  Issue #353
  

• be92cc8d
  by Javier Jardón at 2018-06-22T17:03:44Z

  Source plugin patch depends on host's patch
  
  Issue #353
  

• e90098e0
  by Javier Jardón at 2018-06-22T17:03:44Z

  Source plugin deb depends on host's arpy python package
  
  Issue #353
  

• 4f1d5a1c
  by Tristan Van Berkom at 2018-06-22T17:05:39Z

  doc/source/core_plugins.rst, doc/source/install_main.rst: Fixing link anchor name
  
  In core_plugins.rst, we are already using _plugins, _plugins_build_elements,
  so lets call this one _plugins_sources to be consistent.
  

• 10d21ff0
  by Valentin David at 2018-06-23T01:17:40Z

  Fix element check for BST_FORBID_BDEPENDS
  

• 6641bcd6
  by Tristan Van Berkom at 2018-06-25T07:04:58Z

  doc/source/main_using.rst: Updating headline for this page.
  
  The purpose of this page has changed with time, better to clarify
  this in the headline of the page.
  

• 6d4cbe0c
  by Tristan Van Berkom at 2018-06-25T07:04:58Z

  doc/source/format_declaring.rst: Improving linkage to other parts of the manual
  
  Adding a couple of important links to relevant material.
  

• 4b499ba3
  by Tristan Van Berkom at 2018-06-25T07:04:58Z

  doc/source/tutorial/autotools.rst: Consistent wording of "chapter"
  
  We've been calling these tutorials "chapters", let's not start
  also calling them "sections".
  

• 1d84bb80
  by Tristan Van Berkom at 2018-06-25T07:04:58Z

  doc/source/tutorial/running-commands.rst: Added tip about using `--` in `bst shell`
  

• a25d683e
  by Tristan Van Berkom at 2018-06-25T07:04:58Z

  doc: Adding part 4 of the getting started tutorial: integration commands
  
  This new section talks about how integration commands work
  and shows them at work.
  

• 91d87e3c
  by Tristan Van Berkom at 2018-06-26T23:40:28Z

  _scheduler/job.py: Added long comment
  
  This explains the nature of a complicated asyncio callback,
  which I've looked up on multiple occasions and is not available
  in python online docs (only in the source can you follow it).
  

• 463698ec
  by Francisco Redondo Marchena at 2018-06-28T14:18:10Z

  Add error message when running commands on directories
  
  Issue #446
  

• 48848f4e
  by James Ennis at 2018-06-29T08:45:12Z

  loader.py: Check whether the dir specified is also a <dir>.bst file
  
    - Slight modification to the wording of existing error message.
    - Closes #446
  

• aaf52d03
  by Tristan Maat at 2018-06-29T15:56:20Z

  cli.py: Make buildstream check element paths instead of click
  
  This is to avoid inconsistencies when dealing with paths inside an
  elements directory
  

• a85aaee6
  by Tiago Gomes at 2018-07-02T14:45:04Z

  Fix documentation typo
  

• 9067e269
  by Tiago Gomes at 2018-07-02T14:45:04Z

  Provide better error message on missing commands
  
  Before running a command in the sandbox, check its existence and fail
  early if it does not.
  
  This fixes issue #289.
  

• c5fe9465
  by Jürg Billeter at 2018-07-08T12:33:46Z

  _signals.py: Guard against use in non-main thread
  

• 3d1e23d6
  by Jürg Billeter at 2018-07-09T13:44:49Z

  tests: Remove unneeded ArtifactShare.update_summary() method
  
  The OSTree summary file is no longer used.
  

• 5f4b9377
  by Jürg Billeter at 2018-07-09T13:44:49Z

  _artifactcache/artifactcache.py: Update remote init error message
  
  initialize_remotes() no longer fetches remote refs.
  

• 9defbb62
  by Jürg Billeter at 2018-07-09T13:44:49Z

  tests/frontend/push.py: Add missing skip annotation for non-Linux
  

• bc5a40e3
  by Phillip Smyth at 2018-07-11T11:06:40Z

  Move _list_dir_contents to __init__.py
  

• ddeac63a
  by Jonathan Maw at 2018-07-16T12:19:14Z

  tests: Fix filter tests not checking whether files should be missing.
  
  They weren't actually catching it if you checked-out the entire depended
  element, instead of just the specified split domains
  

• 4c6512d6
  by Jonathan Maw at 2018-07-16T16:12:08Z

  Only redirect elements for workspace and track operations
  
  It was redirecting for checkout operations
  

• fa2eaba2
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests: Use context manager for ArtifactShare
  

• ba4581f8
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/testutils/artifactshare.py: Add support for statvfs mocking
  

• 8cd81636
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/frontend/push.py: Use ArtifactShare statvfs mocking
  
  This makes it unnecessary to update the free space in the mock object in
  the middle of tests.
  

• 9b63f6f2
  by Jürg Billeter at 2018-07-17T05:56:40Z

  .gitlab-ci.yml: Update images to include protobuf and grpcio
  

• 1a2ee26e
  by Jürg Billeter at 2018-07-17T05:56:40Z

  setup.py: Add grpcio dependency and support for code generation
  
  This allows code generation with ./setup.py build_grpc
  

• b585388e
  by Jürg Billeter at 2018-07-17T05:56:40Z

  HACKING.rst: Add note about protocol buffers and code generation
  

• 383e0586
  by Jürg Billeter at 2018-07-17T05:56:40Z

  Import Google proto dependencies for the Remote Execution API
  
  This imports protos from https://github.com/googleapis/googleapis/ and
  the modules generated with ./setup.py build_grpc.
  

• 73fe9af1
  by Jürg Billeter at 2018-07-17T05:56:40Z

  Import protos for the Bazel Remote Execution API
  
  This imports protos from https://github.com/bazelbuild/remote-apis/ and
  the modules generated with ./setup.py build_grpc.
  

• df9a4949
  by Jürg Billeter at 2018-07-17T05:56:40Z

  remote_execution.proto: Add proposed symlink support
  
  https://github.com/bazelbuild/remote-apis/pull/18/
  

• f13bef56
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache: Add CAS artifact cache
  
  Content Addressable Storage specified in the Remote Execution API.
  
  Fixes #387.
  

• a3bdfc18
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/cascache.py: Add prune() method
  

• 687b9a8b
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/cascache.py: Add remove() method
  

• bed6c800
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/cascache.py: Add list_artifacts() method
  

• 8a0fe273
  by Jürg Billeter at 2018-07-17T05:56:40Z

  Add proto for BuildStream reference storage service
  

• 72284b59
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/cascache.py: Add remote cache support
  

• ff8703c9
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache: Add CAS artifact server
  

• 971606ae
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/casserver.py: Add cache cleanup based on pushreceive
  

• 4b4c5268
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/casserver.py: Add update_mtime parameter to resolve_ref()
  

• 9b879015
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache/casserver.py: Update artifact ref mtime on access
  
  This makes cache cleanup LRU.
  

• 8de8ef1f
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _platform: Use CAS artifact cache
  

• a161c746
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/testutils/artifactshare.py: Use CAS artifact server
  

• d0346e89
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/cachekey: Update expected cache keys after switch to CAS
  

• 05a1a25d
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests: Test push/pull on all platforms
  

• 6fd3f523
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/frontend/push.py: Remove xfail mark from LRU cache test
  
  The CAS server now supports LRU.
  

• b87c696f
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/frontend/pull.py: Add pull test for missing blobs on the server
  

• 99846137
  by Jürg Billeter at 2018-07-17T05:56:40Z

  tests/integration/workspace.py: Test incremental build on all platforms
  

• e35098d3
  by Jürg Billeter at 2018-07-17T05:56:40Z

  doc/source/artifacts.rst: Update documentation for CAS artifact server
  

• 5a7d49dc
  by Jürg Billeter at 2018-07-17T05:56:40Z

  Remove tar artifact cache
  
  No longer used.
  

• 1f8b4aa2
  by Jürg Billeter at 2018-07-17T05:56:40Z

  Remove OSTree artifact cache
  
  No longer used.
  
  Fixes #134, #138, #148, #217, #268, #276, #443, #460.
  

• d87df668
  by Jürg Billeter at 2018-07-17T05:56:40Z

  NEWS: Add news entry for CAS-based artifact cache
  

• 7acc64e4
  by Jürg Billeter at 2018-07-17T05:56:40Z

  element.py: Do not use ArtifactCache.can_diff()
  
  The only existing artifact cache implementation always returns True.
  

• f2484fe4
  by Jürg Billeter at 2018-07-17T05:56:40Z

  _artifactcache: Remove unused method can_diff()
  

• f770ab1c
  by Jürg Billeter at 2018-07-17T08:40:16Z

  element.py: Include fail-on-overlap setting in cache key
  
  Fixes #473.
  

• 7e9282c9
  by Phillip Smyth at 2018-07-17T12:53:55Z

  buildstream/_frontend/cli.py: Added a `--deps` flag to `bst checkout`
  
  buildstream/_stream.py: Added deps param to _prepare_sandbox function call
  buildstream/element.py: Added deps param and logic to _prepare_sandbox function
  

• 2797d1fd
  by Phillip Smyth at 2018-07-17T12:53:55Z

  tests/frontend/buildcheckout.py: Added tests for "Add a `--deps` flag to `bst checkout`"
  

• 70a83ef8
  by William Salmon at 2018-07-17T14:04:16Z

  Adding a helpful link to the example
  

• e4f8fb07
  by devcurmudgeon at 2018-07-17T15:07:05Z

  Fix 'main install' to be explicit that it is for Linux distros only
  

• 87c103d8
  by Jürg Billeter at 2018-07-17T15:07:56Z

  _frontend/cli.py: Fix help text for `bst checkout --deps`
  

• 4c09b686
  by Jürg Billeter at 2018-07-17T17:31:32Z

  _artifactcache/cascache.py: Fix prune()
  
  _reachable_refs_dir() failed to add directory digests to the set of
  reachable objects.
  

• adedd867
  by Tristan Maat at 2018-07-18T05:11:03Z

  Bump required python version to 3.5
  

• b8543f8a
  by Tristan Maat at 2018-07-18T05:11:03Z

  _exceptions.py: Add `detail` to ArtifactErrors
  

• fc79ad2c
  by Tristan Maat at 2018-07-18T05:11:03Z

  utils.py: Allow `list_relative_paths` to list directories
  

• 4f9da15d
  by Tristan Maat at 2018-07-18T05:11:03Z

  _scheduler/*queue.py: Move queues to a subdirectory
  

• 24925634
  by Tristan Maat at 2018-07-18T05:45:59Z

  Make Jobs abstract and element-independent
  

• 33984448
  by Tristan Maat at 2018-07-18T05:45:59Z

  Add cache_quota to user config
  

• 1ec5c7b1
  by Tristan Maat at 2018-07-18T05:45:59Z

  Make elements keep track of their built artifact size
  

• 7229d2e5
  by Tristan Maat at 2018-07-18T05:45:59Z

  Compute the artifact cache size after each build/pull
  

• dc17de38
  by Tristan Maat at 2018-07-18T06:07:24Z

  Clean the artifact cache when we hit the cache quota
  
  When the cache quota is hit, we will remove any elements not required
  for the current build, until our cache is only filled halfway.
  

• 6ff12e5e
  by Tristan Maat at 2018-07-18T06:07:24Z

  tests/artifactcache/expiry.py: Add expiry tests
  

• 308d59be
  by Tristan Maat at 2018-07-18T06:07:24Z

  NEWS: Add news about cache expiry
  

• a950a985
  by Tristan Van Berkom at 2018-07-18T09:59:03Z

  _context.py: Adding apis for message recording and task logging.
  
  Added the new Context.recorded_messages() context manager, this
  causes messages to be logged in a dedicated log file before being
  propagated to the log handler, and also provides an open log handle
  for the capturing and logging of stdout/stderr from child processes.
  
  This comes with the additional accessors:
  
     Context.get_log_handle()
     Context.get_log_filename()
  

• d835c37f
  by Tristan Van Berkom at 2018-07-18T09:59:03Z

  Refactor: Use new logging mechanism from Context
  
    o element.py / plugin.py: Removed supporting logging code, and derive
                              the log handle from Context.
  
    o _scheduler/scheduler.py, _scheduler/queues/queue.py: Adapt to new Job initializer API for the logfile
  
    o _scheduler/jobs/job.py: Run job activities within the new context manager
                              which turns on logging for a job. Also removed a lot
  			    of custom logging abstract methods which are unneeded.
  
    o _scheduler/jobs: Job implementations need not implement so much custom logging.
  

• 26b9f6fa
  by Tristan Van Berkom at 2018-07-18T11:52:43Z

  _stream.py: Clarifying ambiguous FIXME comment about required artifacts
  

• 3814905a
  by Daniel Playle at 2018-07-18T12:08:24Z

  Reorder app.py imports
  
  When testing locally on some systems, it appears that pylint fails on
  the order of imports for `buildstream/_frontend/app.py`. This commit
  changes the order of these imports.
  
  This commit addresses issue !453.
  

• 48f66a3c
  by Chandan Singh at 2018-07-18T18:03:31Z

  .gitlab-ci.yml: Use testsuite images for running tests
  
  Instead of re-using the user-facing `buildstream/buildstream-fedora`
  image for tests (that comes with BuildStream pre-installed), use
  `buildstream/testsuite-fedora` image that is supposed to be used
  exclusively for these tests.
  
  Similarly, use the dedicated `buildstream/testsuite-debian` image
  instead of `buildstream/buildstream-debian`.
  
  Fixes #458.
  
  For related discussion, see
  https://gitlab.com/BuildStream/buildstream-docker-images/issues/8.
  

• 1016dcb4
  by Josh Smith at 2018-07-19T12:52:21Z

  scheduler.py: Correct some anomalies within the docstrings
  

• 709b9bca
  by Francisco Redondo Marchena at 2018-07-19T14:03:12Z

  source-bundle: Enable --except option
  
  Before this option was listed in help but was ignored when
  creating the source-bundle.
  
  Issue #468
  

• 96d07153
  by Jordan Petridis at 2018-07-19T23:54:39Z

  Docs: Update the required build packages for fedora based systems.
  
  `ruamel.yaml` seems to require `Python.h` header file to build.
  
  `python3-devel` is what provides it for Fedora.
  

• 93a77c1d
  by Javier Jardón at 2018-07-19T23:58:28Z

  Docs: Update the required build packages for the rest of the systems
  
  Also remove the recomendation to install psutil as we need to build
  other python modules anyway (like ruamel)
  
  Completes 96d07153b7817cdaeda57dd163eed52b2b1b31e8
  

• 57e0d1c5
  by William Salmon at 2018-07-20T00:03:06Z

  Added dependency to the Docs
  
  These dependencies are needed to use the plugins that are installed
  as part of this guild.
  
  As the dependencies added are not dependencies of the core package
  they have been added to a separate section.
  
  The lzip package is needed to build gnome.
  https://wiki.gnome.org/Newcomers/BuildSystemComponent
  

• 1b6688eb
  by Javier Jardón at 2018-07-20T00:07:23Z

  install_linux_distro.rst: document plugins installed by default
  

• fc63b972
  by Javier Jardón at 2018-07-20T00:12:50Z

  install_linux_distro.rst: Document plugins deps for the rest of the systems
  

• 790fb40b
  by Tiago Gomes at 2018-07-20T09:07:01Z

  Update checkout man page for new --deps option
  

• 9c1f024b
  by Tiago Gomes at 2018-07-20T09:07:01Z

  Add support for creating a tarball on bst checkout
  
  One of the tests added is configured to be skipped for now, as dumping
  binary data is causing a bad descriptor exception when using the pytest
  capture module.
  
  Closes #263.
  

• 0ddbf0e0
  by Tristan Van Berkom at 2018-07-20T11:52:00Z

  setup.py: Specify minimum required version of pytest-cov plugin
  
  This causes the new artifact tests to pass (unless you happened
  to already have a recent enough version of pytest-cov, in which case
  you didn't notice the breakage).
  

• 6158ee5a
  by Sam Thursfield at 2018-07-20T13:17:14Z

  Fix crash when --debug is passed
  
  I hit the following backtrace running `bst --debug push` and `bst
  --debug build`:
  
      pid:16736 id:000[--:--:--][][] START   Push
      pid:16736 id:000[--:--:--][][] START   Loading pipeline
      pid:16736 id:000[00:00:00][][] SUCCESS Loading pipeline
      pid:16736 id:000[--:--:--][][] START   Resolving pipeline
      pid:16736 id:000[--:--:--][][] BUG     'MesonElement' object has no attribute '_Element__cache_key'
  
          Traceback (most recent call last):
            File "/home/sam/.local/bin/bst", line 8, in <module>
              sys.exit(cli())
            File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
              return self.main(*args, **kwargs)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 162, in override_main
              standalone_mode=standalone_mode, **extra)
            File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 697, in main
              rv = self.invoke(ctx)
            File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
              return _process_result(sub_ctx.command.invoke(sub_ctx))
            File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
              return ctx.invoke(self.callback, **ctx.params)
            File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
              return callback(*args, **kwargs)
            File "/home/sam/.local/lib/python3.6/site-packages/click/decorators.py", line 27, in new_func
              return f(get_current_context().obj, *args, **kwargs)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 471, in push
              app.stream.push(elements, selection=deps, remote=remote)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 341, in push
              fetch_subprojects=True)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 814, in _load
              fetch_subprojects=fetch_subprojects)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in load
              for meta in meta_elements
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in <listcomp>
              for meta in meta_elements
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 894, in _new_from_meta
              element = project.create_element(artifacts, meta)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_project.py", line 184, in create_element
              element = self._element_factory.create(self._context, self, artifacts, meta)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_elementfactory.py", line 57, in create
              return element_type(context, project, artifacts, meta, default_config)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 196, in __init__
              super().__init__(meta.name, context, project, meta.provenance, "element")
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 171, in __init__
              self.debug("Created: {}".format(self))
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 407, in debug
              self.__message(MessageType.DEBUG, brief, detail=detail)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 652, in __message
              self.__context.message(message)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_context.py", line 338, in message
              self._message_handler(message, context=self)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/app.py", line 665, in _message_handler
              text = self.logger.render(message)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 585, in render
              return self._render(message)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 620, in _render
              text += widget.render(message)
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 228, in render
              _, key, missing = plugin._get_display_key()
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 1138, in _get_display_key
              cache_key = self._get_cache_key()
            File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 991, in _get_cache_key
              return self.__cache_key
          AttributeError: 'MesonElement' object has no attribute '_Element__cache_key'
  
  The issue here is that when `--debug` is enabled, Plugin.__init__()
  tries to log a message like this:
  
          pid:16929 id:001[--:--:--][????????][ main:tracker.bst                   ] DEBUG   Created: meson element at tracker.bst [line 1 column 0]
  
  The log formatter is trying to get the cache key of the element in
  order to show it in the log, but the attribute hasn't yet been set
  because we've not got to the Element() constructor.
  

• f484ab4f
  by Phillip Smyth at 2018-07-20T14:29:44Z

  Updating .bzr plugin to omit the .bzr dir
  
  Fixes issue #455
  

• b662f175
  by Tristan Van Berkom at 2018-07-20T14:29:44Z

  git source plugin: Omit the .git directory completely.
  
  This will break builds which use `git describe` to determine
  their version number until a more elegant solution is implemented,
  and will unblock work on caching of build trees in the artifacts.
  
  Fixes issue #455
  

• 80effb5c
  by Tristan Van Berkom at 2018-07-22T08:30:40Z

  HACKING.rst: No longer requiring issue numbers in commit messages.
  
  Instead, we are enabling the merge commit feature in gitlab and
  hopefully reducing some of the friction in review this way.
  

• c7124c3d
  by Tristan Van Berkom at 2018-07-22T09:46:05Z

  HACKING.rst: Added convention for naming of branches in merge requests.
  

• 282b1092
  by Tristan Van Berkom at 2018-07-23T08:47:36Z

  plugins/sources/git.py: Use --shared instead of --hardlinks
  
  This improves staging performance by avoiding making any
  copies of the objects in the cloned repository in the staging
  area before removing the .git directory.
  
  Fixes #488
  

• fe0df616
  by Tristan Van Berkom at 2018-07-23T09:20:44Z

  Merge branch 'tristan/git-stage-with-shared-clone' into 'master'
  
  plugins/sources/git.py: Use --shared instead of --hardlinks
  
  Closes #488
  
  See merge request BuildStream/buildstream!557

• 6559db4c
  by Josh Smith at 2018-07-23T09:22:39Z

  element.py: Make Element._get_brief_display_key() available to Internal API
  
  This change will allow for more detailed info messages throughout
  the CASCache.
  

• 6b82e476
  by Josh Smith at 2018-07-23T09:22:39Z

  cascache.py: Add push info detailing which remote is being pushed to
  
  Work towards fixing issue #275
  

• 1d36df25
  by Josh Smith at 2018-07-23T09:22:39Z

  cascache.py: Add pull info detailing which remote is being pulled from
  
  Work towards fixing issue #275
  

• 364706f3
  by Tristan Van Berkom at 2018-07-23T09:57:02Z

  Merge branch 'Qinusty/275' into 'master'
  
  Indicate where artifacts are going to and coming from in the log
  
  Closes #275
  
  See merge request BuildStream/buildstream!553

• e70237a5
  by Phillip Smyth at 2018-07-23T13:26:47Z

  Adding caching build trees
  
  buildstream/element.py: Adding build tree to cache
  buildstream/_versions.py: Bumping BST_CORE_ARTIFACT_VERSION
  

• 335afb5b
  by Tristan Van Berkom at 2018-07-23T14:07:07Z

  Merge branch 'caching_build_trees_limited' into 'master'
  
  Adding caching build trees (limited)
  
  See merge request BuildStream/buildstream!560

• 60290f31
  by Javier Jardón at 2018-07-23T15:29:37Z

  doc/source/install_linux_distro.rst: buildstream doesn't depend on ostree or pygobject anymore
  

• de162e43
  by Javier Jardón at 2018-07-24T11:58:16Z

  Merge branch 'jjardon/ostree_repo' into 'master'
  
  doc/source/install_linux_distro.rst: buildstream doesn't hard depend on ostree or pygobject anymore
  
  Closes #492
  
  See merge request BuildStream/buildstream!558

• 6106d657
  by Josh Smith at 2018-07-25T10:50:14Z

  cascache.py: Remove misleading info message
  
  This addresses issue #501.
  

• 4a637d63
  by Jonathan Maw at 2018-07-25T12:05:19Z

  Merge branch 'Qinusty/501' into 'master'
  
  Remove misleading info message
  
  See merge request BuildStream/buildstream!567

• a0ad985f
  by Phil Dawson at 2018-07-25T13:03:23Z

  tests/frontend/show.py: Add test case for maximum recursion depth being exceeded
  
  Add test to ensure gracefull handling of exception thrown while loading
  a pipeline due the python's max recursion depth being exceeded.
  
  This is part of the work for issue #203
  

• 95b4eae4
  by Phil Dawson at 2018-07-25T13:03:23Z

  app.py: Handle exception thrown when recursion limit is exceeded
  
  Because the RecursionError exception was introduced in Python 3.5, until we
  drop support for for Python 3.4, we must use RuntimeError.
  

• 889bf238
  by Jim MacArthur at 2018-07-25T13:43:21Z

  Merge branch 'phil/203-BuildStream-crashes-when-dependency-tree-too-deep' into 'master'
  
  Phil/203 BuildStream crashes when dependency tree too deep
  
  See merge request BuildStream/buildstream!512

• 54fee6c8
  by Phillip Smyth at 2018-07-25T14:15:36Z

  _stream.py: Added functionality for workspace open -f
  
  tests/frontend/workspace.py: Added tests
  

• f62b6cb7
  by Jonathan Maw at 2018-07-25T15:01:33Z

  Merge branch 'bst_workspace_open_force_does_nothing' into 'master'
  
  _stream.py: Added functionality for workspace open -f
  
  See merge request BuildStream/buildstream!549

• c049d021
  by Ed Baunton at 2018-07-25T19:43:51Z

  Add remote source plugin
  
  Add a plugin that supports downloading files verbatim from a source with
  an optional overridable filename and destination directory. Bumps bst
  format version to 10.
  
  Fixes #163
  

• bd1196ef
  by Ed Baunton at 2018-07-26T13:22:52Z

  Provide explicit reason for SourceError exception on path
  
  When the user provides a path for the filename parameter, provide a reason
  

• 9a46c16f
  by Ed Baunton at 2018-07-26T14:06:52Z

  Merge branch 'edbaunton/remote-source' into 'master'
  
  Add remote source plugin
  
  Closes #163
  
  See merge request BuildStream/buildstream!541

• 7ceacfea
  by William Salmon at 2018-07-26T14:32:49Z

  Documentation typos and fixes
  
  Daniel went through the Doc's etc and made a list of suggestions.
  These are the result of the most easily patched suggestions and
  one or two thing that I noticed as well.
  

• 8c9fd9e4
  by Javier Jardón at 2018-07-26T15:32:16Z

  Merge branch 'willsalmon/documentation_form_notes' into 'master'
  
  Documentation typos and fixes
  
  See merge request BuildStream/buildstream!569

• 9ebd7fe1
  by Josh Smith at 2018-07-27T04:41:26Z

  _exceptions.py: Modify BstError API to allow optional retry
  
  job.py: Changes to the logic surrounding retry attempts and child process return codes
  element.py, source.py: ElementError and SourceError also implement this change.
  
  These exceptions now have an optional parameter of temporary which defaults to false. This will potentially break
  backwards compatibility where exceptions were previously raised and a retry was intended.
  
  To trigger a retry, one must now raise their SourceError or ElementError with temporary=True.
  
  This aims to fix #397.
  

• ca224425
  by Josh Smith at 2018-07-27T04:41:26Z

  _downloadablefilesource.py: Update network errors to be temporary
  
  This follows the change in 67ecd97a05279a3b7570ad59f05bf0a5973ef04c.
  

• 0a7cfcdf
  by Josh Smith at 2018-07-27T04:41:26Z

  tests/sources: deb, tar and zip all test for retry behaviour
  
  This is following 5d6418a06a3a0262df5f6b9e6e1a36578515fd4a where
  DownloadableFileSource was modified to raise 'temporary'
  errors when relating to file downloading.
  

• 9b2bc78f
  by Josh Smith at 2018-07-27T04:41:26Z

  cascache.py: Update cache push/pull errors to be temporary
  
  Further work needs to be done for the current grpc exceptions which are reraised.
  

• 4986527f
  by Josh Smith at 2018-07-27T04:41:26Z

  plugin.py: Extend Plugin.call() API
  
  Plugin.call() now takes fail_temporarily as an optional parameter,
  when supplied it will cause subsequent failures to trigger temporary
  errors as opposed to permanent errors.
  
  This also extends Plugin.check_output() which makes use of Plugin.call()
  

• a9a1048e
  by Josh Smith at 2018-07-27T04:41:26Z

  git.py: Update git plugin to fail temporarily on clone failures
  
  The git plugin will now make use of the fail_temporarily parameter
  to Plugin.call(), allowing failures to trigger a retry.
  

• aa071cb6
  by Tristan Van Berkom at 2018-07-27T06:13:53Z

  Merge branch 'tristan/temporary-failures' into 'master'
  
  Temorary failures
  
  Closes #397
  
  See merge request BuildStream/buildstream!574

• 4d86e7c5
  by Tristan Van Berkom at 2018-07-27T06:43:48Z

  element.py, source.py, plugin.py: Adjusting API version since annotations.
  
  Since we backported the temporary/permanent failures patch for #397
  into the `bst-1.2` branch, we need to adjust the since versions in master
  down to 1.2.
  

• fb222ba1
  by Josh Smith at 2018-07-27T08:50:45Z

  cascache.py: Raise ArtifactError on grpc error
  
  This aims to fix issue #502.
  

• 48916b8a
  by Tristan Maat at 2018-07-27T10:22:41Z

  Merge branch 'Qinusty/502' into 'master'
  
  cascache.py: Raise ArtifactError on grpc error
  
  Closes #502
  
  See merge request BuildStream/buildstream!576

• 23e080b9
  by James Ennis at 2018-07-27T11:13:50Z

  README.md: Minor corrections
  

• 196cfffc
  by James Ennis at 2018-07-27T11:13:50Z

  README.md: Direct users towards the tutorial before existing projects:
  
    This patch also removes displaying the projects that live within
    the buildstream-examples repository, as we wish to deprecate this
    repo and migrate the two examples.
    - #438 and #439
  
    In addition to this, both examples do not 'build' in their current
    state.
  

• 2b93574e
  by James Ennis at 2018-07-27T11:13:50Z

  main_install.rst: Add note to refer Windows and macOS users to docker
  

• 63e2320e
  by Josh Smith at 2018-07-27T11:16:55Z

  .gitignore: Ignore .pytest_cache/
  

• 6ea97b17
  by Javier Jardón at 2018-07-27T11:54:20Z

  Merge branch 'jennis/doc_improvements' into 'master'
  
  Refer readers to our tutorial before referring them to existing bst projects
  
  See merge request BuildStream/buildstream!578

• 8b46e874
  by Jonathan Maw at 2018-07-27T12:24:56Z

  project: Parse and store mirrors
  

• a2e9c62a
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Move _ALIAS_SEPARATOR into utils
  
  The separator is useful in source files other than _project.py
  

• 7c993ac0
  by Jonathan Maw at 2018-07-27T12:24:56Z

  source: Store the url aliases or use an override
  
  This is part of a later plan to implement mirroring without forcing
  everyone to update their sources. We use the expected calls to
  Source.translate_url() when running Source.configure() to extract the
  aliases from the URL. Multiple aliases must be extracted because
  sources exist that may fetch from multiple aliases (for example, git
  submodules)
  
  Later, we want to substitute another URI where the alias normally reads
  from the project - We accomplish this by re-instantiating the Source
  with the alias overrides passed as an argument to the constructor.
  

• 2889003c
  by Jonathan Maw at 2018-07-27T12:24:56Z

  source: When fetching, try to fetch from mirrors first
  
  **KLUDGE WARNING**: This involves making the source store its "meta"
  object so that it's possible to create a copy of the source inside the
  fetch queue, instead of back when the pipeline was being loaded.
  
  This adds the SourceFetcher class, which is intended for sources that
  fetch from multiple URLs (e.g. the git source and its submodules)
  
  Fix when fetching
  

• f81e8e7b
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Set default mirror via command-line or user config
  
  In user config (buildstream.conf), it is set with the "default-mirror"
  field.
  
  On the command-line, it is set with "--default-mirror"
  

• 755ed898
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Add mirrored source tracking
  

• 19c01a56
  by Jonathan Maw at 2018-07-27T12:24:56Z

  bzr.py: Improve mirror support
  
  This fixes:
  * Bzr repositories pulling from the branch they were created with.
  * Bzr's _ensure_mirror() not actually checking that it successfully
    mirrored the ref.
  

• 84872141
  by Jonathan Maw at 2018-07-27T12:24:56Z

  git.py: Improve mirror support
  
  When implementing fetching from mirrors, I encountered some problems
  with the git source:
  
  1. The mirror URL was using translate_url()'s output, so if a different
     alias was used, then fetching from the mirror would go to a different
     directory, and be inaccessible.
  2. After fixing that, fetching was unable to pull from a URL other than
     the one used at repository creation, meaning it wouldn't actually
     pull from the mirror.
  

• bd51a0b2
  by Jonathan Maw at 2018-07-27T12:24:56Z

  testutils: Add a helper to copy a testutils repo
  
  This is helpful if you want to test what happens when you have one repo
  that has diverged from another. By copying the repo you're sure they
  start with shared history.
  
  This is especially useful when mirroring.
  

• 909120ab
  by Jonathan Maw at 2018-07-27T12:24:56Z

  tests: Add mirrored fetching and tracking tests
  

• 1cbc2e17
  by Jonathan Maw at 2018-07-27T12:24:56Z

  doc: Add documentation for source mirroring
  

• 202d9d26
  by Jonathan Maw at 2018-07-27T12:24:56Z

  NEWS: Add item for source mirroring
  

• 2b23898d
  by Jonathan Maw at 2018-07-27T12:24:56Z

  doc: Add tutorials for setting up git and tar mirrors
  

• bd4d0355
  by Jonathan Maw at 2018-07-27T13:08:00Z

  Merge branch '328-support-for-downloading-sources-from-mirrors' into 'master'
  
  Resolve "Support for downloading sources from mirrors"
  
  Closes #328
  
  See merge request BuildStream/buildstream!404

• cb4693b2
  by Josh Smith at 2018-07-27T14:00:26Z

  Merge branch 'Qinusty/pytest_cache_gitignore'
  

• 8a96679a
  by Josh Smith at 2018-07-27T14:01:07Z

  Merge branch 'master' of gitlab.com:BuildStream/buildstream
  

• f5c8ff61
  by Josh Smith at 2018-07-27T14:10:45Z

  _context.py: Cache size is now restricted to available disk space
  
  This address issue #491.
  
  When attempting to run buildstream with a configuration specifying
  a cache quota larger than your available disk space, buildstream
  will alert the user and exit.
  
  Note:
  This takes into consideration your current cache usage and
  therefore restricts the overall size of your artifact cache folder.
  

• 32ddb544
  by Qinusty at 2018-07-27T14:57:30Z

  Merge branch 'Qinusty/491' into 'master'
  
  Cache quota is now restricted to available disk space
  
  See merge request BuildStream/buildstream!563

• cb8c99e5
  by Tristan Maat at 2018-07-29T07:52:06Z

  buildelement.py: Only display one activity message (#507)
  

• 08b7562b
  by Tristan Van Berkom at 2018-07-29T10:23:43Z

  Merge branch '507-some-log-lines-appear-to-be-duplicates' into 'master'
  
  Resolve "Some log lines appear to be duplicates"
  
  Closes #507
  
  See merge request BuildStream/buildstream!573

• 3b7158b3
  by Tristan Van Berkom at 2018-07-29T10:25:02Z

  source.py: Bring down since version for source mirroring APIs
  
  Since we have now backported this to `bst-1.2`, the APIs have
  been introduced in 1.2 and not 1.4
  

• e84f8b24
  by Tristan Van Berkom at 2018-07-29T10:25:02Z

  NEWS: Bring down since version of backported features to 1.1.5
  
  These features will be released in 1.1.5 since they have been
  backported, lets reflect this in the NEWS file in master.
  
  Also this adds the backported `remote` plugin to the NEWS
  

• df18d38c
  by Tristan Van Berkom at 2018-07-29T10:26:08Z

  git.py: Handle concurrent download completions properly
  
  Use os.rename() to rename the cloned temporary repository into
  place in the source cache, and issue a STATUS message when discarding
  a duplicate clone, in the case where the same repository is cloned
  twice in parallel.
  
  The problem with using shutil.move() is that it will create the source
  directory in a subdirectory of the destination when the destination
  exists, so it's behavior depends on whether the destination exists.
  
  This shutil.move() behavior has so far hidden the race condition
  where a duplicate repo is created in a subdirectory, as you need
  to have three concurrent downloads of the same repo in order to
  trigger the error.
  
  This fixes issue #503
  

• 92714b03
  by Tristan Van Berkom at 2018-07-29T11:18:52Z

  Merge branch 'tristan/fix-spurious-errors' into 'master'
  
  git.py: Handle concurrent download completions properly
  
  Closes #503
  
  See merge request BuildStream/buildstream!583

• 8568d272
  by Phil Dawson at 2018-07-30T07:58:07Z

  examples: add project demonstrating a junction element
  
  This is part of the work towards #437.
  

• efa24e7e
  by Phil Dawson at 2018-07-30T07:58:07Z

  junctions.run: Add terminal session output for junctions walkthrough.
  
  This is part of the work towards #437.
  

• bf175bf7
  by Phil Dawson at 2018-07-30T07:58:07Z

  junction-elements.rst: Add junctions walkthrough
  
  This is part of the work towards #437.
  

• e8931048
  by James Ennis at 2018-07-30T07:58:07Z

  using_advanced_features.rst: Added file
  

• b4f25190
  by Phil Dawson at 2018-07-30T07:58:07Z

  tests/examples/junctions.py: Add tests for junctions walkthrough
  
  This is part of the work towards issue #437.
  

• aff76ae9
  by Phil Dawson at 2018-07-30T09:15:49Z

  Merge branch 'phil/437-junction-tutorial' into 'master'
  
  Phil/437 junction tutorial
  
  See merge request BuildStream/buildstream!550

• c313ad17
  by Tom Pollard at 2018-07-30T10:57:49Z

  widget.py: Limit failure summary to currently failing elements
  
  Change widget.py print_summary() to only print the failure
  messages of elements in the Failure Summary that failed on the
  current try.
  

• 4a3def5c
  by Tristan Maat at 2018-07-30T11:47:21Z

  Merge branch 'tpollard/386' into 'master'
  
  widget.py: Limit failure summary to currently failing elements
  
  See merge request BuildStream/buildstream!561

• 34015a26
  by Tristan Maat at 2018-07-30T12:19:45Z

  HACKING.rst: Add note about asking for dev permissions
  

• ae47a72b
  by Tristan Van Berkom at 2018-07-30T13:11:34Z

  Merge branch 'tlater/ask-for-dev-permissions' into 'master'
  
  HACKING.rst: Add note about asking for dev permissions
  
  See merge request BuildStream/buildstream!587

• 695c0cb0
  by Phil Dawson at 2018-07-30T16:07:36Z

  junction-elements.rst: Add missing language specifier to literalinclude
  
  A missing language specifier on a literal include directive results in yaml
  being rendered as python code. This commit adds the :language: tag to that
  directive.
  

• fcb87b0a
  by Phil Dawson at 2018-07-30T16:53:37Z

  Merge branch 'phil/fixup-junctions-tutorial' into 'master'
  
  junction-elements.rst: Add missing language specifier to literalinclude
  
  See merge request BuildStream/buildstream!591

• 3d308894
  by Richard Maw at 2018-07-31T16:31:36Z

  element.py: Cache the result of checking whether an artifact is cached weakly
  
  Normally we'd only need it in the case of scheduling a weakly cached build,
  but to allow caching of failed builds we need to be able to distinguish
  between cached successes and cached failures
  for both strong and weak cache keys.
  
  To allow other cache lookup codepaths to look up via the weak key
  requires changes through the call stack to consult which key to use,
  and cache invalidation of the saved state when it changes.
  

• b3a68e28
  by Richard Maw at 2018-07-31T16:31:36Z

  element.py: Add metadata to distinguish between successful and failed builds
  
  This just puts the metadata in place,
  we're adding code paths to add failed builds later.
  

• f4573df3
  by Richard Maw at 2018-07-31T16:31:36Z

  Convert call-sites of Element._cached() that assume success
  
  When we later add cached failures it needs to not treat them as successes.
  

• 553df108
  by Richard Maw at 2018-07-31T16:31:36Z

  _frontend/widget.py: Render cached failures differently to successes
  

• d14d8ee2
  by Richard Maw at 2018-07-31T16:32:35Z

  Cache failed builds
  
  This creates an artifact when element assembly fails too,
  and if it's the right kind of exception uses the now-included install directory
  similarly to if it had returned successfully.
  
  If there's a failure during install the artifact contains any installed files,
  but may contain nothing at all.
  

• d83122bb
  by Richard Maw at 2018-07-31T16:32:35Z

  _scheduler/queues/buildqueue.py: Skip rebuilding cached failures
  
  This flags up a failure and if run in an interactive prompt
  permits the user to attempt a rebuild.
  

• 78944e9a
  by Richard Maw at 2018-07-31T16:32:35Z

  tests: Add tests for cached behaviours
  

• 3fa79d8d
  by Richard Maw at 2018-07-31T16:32:35Z

  _scheduler/queues: Add failed builds to "done" queue
  
  This allows the scheduler to move jobs from the current queue to the next.
  
  As a result of this change later queues than the build queue
  mustn't skip a cached failure, so the logic is specialised to build queues only.
  

• e8cd43dc
  by Richard Maw at 2018-07-31T16:32:58Z

  NEWS: Describe caching of failures
  
  Closes #76.
  

• 88cd61ea
  by knownexus at 2018-07-31T16:32:58Z

  Added the missing `_get_build_log` function
  

• 4fc1f5d1
  by Phillip Smyth at 2018-07-31T17:17:34Z

  Merge branch 'richardmaw/cache-fail' into 'master'
  
  Store failed builds in the cache
  
  Closes #76
  
  See merge request BuildStream/buildstream!475

• 6cd3df09
  by Phil Dawson at 2018-08-01T08:05:45Z

  doc/sessions: Add terminal sessions for use in workspaces tutorial
  
  This is part of the the work towards issue #437
  

• 8c0522ad
  by Phil Dawson at 2018-08-01T08:05:45Z

  Add project for workspaces tutorial
  
  This is part of the work towards issue #437
  

• dc60b713
  by Phil Dawson at 2018-08-01T08:05:45Z

  tutorials: Add workspaces tutorial
  
  This is part of the work towards issue #437
  

• bf8c3dfb
  by Phil Dawson at 2018-08-01T08:05:45Z

  testutils/patch.py: Add methods for applying and removing patches
  

• b69783d4
  by Phil Dawson at 2018-08-01T08:05:45Z

  tests/examples/developing.py: Add tests for workspaces walkthrough
  

• 480565c2
  by Phil Dawson at 2018-08-01T09:04:26Z

  Merge branch 'phil/437-workspaces-tutorial' into 'master'
  
  Phil/437 workspaces tutorial
  
  Closes #437
  
  See merge request BuildStream/buildstream!519

• 87edf67e
  by Martin Blanchard at 2018-08-01T11:07:05Z

  complete.py: Ensure paths get completed from 'element-path'
  
  Element paths should always be completed from the root element folder
  defined by the element-path key in project.conf. Fix complete_path() to
  always search into its given base_directory argument.
  
  See issue BuildStream/buildstream#448
  

• d10e4668
  by Phillip Smyth at 2018-08-01T12:05:40Z

  Merge branch 'mablanch/448-autocompletion-broken-defaults' into 'master'
  
  Fix autocompletion for elements in sub-folders
  
  See merge request BuildStream/buildstream!592

• 6aedc57b
  by Jim MacArthur at 2018-08-01T12:07:32Z

  Add BST_VIRTUAL_DIRECTORY flag for element plugins
  

• 1d3af84d
  by Jim MacArthur at 2018-08-01T12:07:32Z

  utils.py: add getmtime() and magic_timestamp
  
  magic_timestamp is moved into file scope so other classes
  can use it.
  

• 4397e45a
  by Jim MacArthur at 2018-08-01T12:07:32Z

  Sandbox.py: Rename __root to _root.
  
  This is to allow to allow its use by subclasses.
  
  Since access to get_directories is now blocked for some plugins,
  and the subclasses of Sandbox do not have configuration defined
  by YAML files, they need another way to get at the root directory.
  

• 9bf343cf
  by Jim MacArthur at 2018-08-01T12:07:32Z

  Add the virtual directory class 'Directory' and one implementation.
  
  buildstream/storage/Directory.py: New file.
  buildstream/storage/_filebaseddirectory.py: New file.
  buildstream/_exceptions.py: New VIRTUAL_FS exception source.
  

• f124dfec
  by Jim MacArthur at 2018-08-01T12:07:32Z

  storage: Add __init__.py
  

• 42bdce0b
  by Jim MacArthur at 2018-08-01T12:07:32Z

  sandbox.py: Add function to get virtual directory.
  

• fe1a3428
  by Jim MacArthur at 2018-08-01T12:07:33Z

  _stream.py: Convert to virtual directories.
  
  This removes _add_directory_to_tarfile since it is now implemented in
  _filebaseddirectory.py.
  

• 849724ce
  by Jim MacArthur at 2018-08-01T12:07:33Z

  element.py: Conversion to use virtual directories
  

• 270325b4
  by Jim MacArthur at 2018-08-01T12:07:33Z

  plugins/elements/compose.py: Convert to virtual directories
  

• a8c71ba4
  by Jim MacArthur at 2018-08-01T12:07:33Z

  plugins/elements/import.py: Convert to virtual directories
  

• 4e6a9049
  by Jim MacArthur at 2018-08-01T12:07:33Z

  sandbox: Remove instances of get_directory
  
  sandbox/_mount.py, sandbox/_sandboxbwrap.py:
    Remove instances of get_directory
  

• e69f2b23
  by Jim MacArthur at 2018-08-01T12:07:33Z

  plugins/elements/stack.py: Convert to virtual directories
  

• 6877b6fc
  by Jim MacArthur at 2018-08-01T12:07:33Z

  scriptelement.py: Convert to virtual directories
  

• 8edf1f80
  by Jim MacArthur at 2018-08-01T12:07:33Z

  compose.py: Remove unused getmtime()
  

• 02c6c846
  by Jim MacArthur at 2018-08-01T12:07:33Z

  sandbox.py: Use _root in _has_command
  
  This will need more attention when we bring in another virtual
  directory backend, however, we've said it is acceptable for the
  sandbox itself to access the underlying directory, and this is
  the best fix in the meantime.
  

• eabc3899
  by Jim MacArthur at 2018-08-01T13:16:04Z

  Merge branch 'jmac/virtual_directories' into 'master'
  
  Abstract directory class and filesystem-backed implementation
  
  See merge request BuildStream/buildstream!445

• 60ddb193
  by Josh Smith at 2018-08-01T14:17:17Z

  message.py: MessagesType.SKIPPED added
  

• 0756c615
  by Josh Smith at 2018-08-01T14:17:17Z

  cascache.py: Display SKIPPED message for each remote skipped
  

• 62a250fe
  by Qinusty at 2018-08-01T15:01:25Z

  Merge branch 'Qinusty/481' into 'master'
  
  Add SKIPPED message type for actions being skipped
  
  See merge request BuildStream/buildstream!562

• 00ddcbd7
  by Martin Blanchard at 2018-08-01T15:16:33Z

  element.py: Raise an exception on unbuilt element checkout
  
  Trying to checkout an element that has no cached artifacts should be and
  handled failure. See BuildStream/buildstream#447.
  

• 4c739e2a
  by Martin Blanchard at 2018-08-01T15:16:33Z

  _stream.py: Print details on checkout failure
  

• 4a4fed7a
  by Qinusty at 2018-08-01T16:14:50Z

  Merge branch 'mablanch/447-stack-trace-checkout' into 'master'
  
  Handle checkout failure for unbuilt elements
  
  See merge request BuildStream/buildstream!590

• 547d20b2
  by Phil Dawson at 2018-08-01T16:31:08Z

  install_linux_distro.rst: Add install instructons for Ubuntu
  
  This commit is part of the work required to resolve issue #436
  

• 592c9061
  by Javier Jardón at 2018-08-01T17:28:05Z

  Merge branch 'phil/436-add-ubuntu-install-intructions' into 'master'
  
  Add Ubuntu install intructions
  
  Closes #436
  
  See merge request BuildStream/buildstream!525

• 4d2e8a14
  by Josh Smith at 2018-08-02T09:12:16Z

  HACKING.rst: Add running a single test example
  

• 039d43e4
  by Tom Pollard at 2018-08-02T10:02:08Z

  Merge branch 'Qinusty/docs_changes' into 'master'
  
  HACKING.rst: Add running a single test example
  
  See merge request BuildStream/buildstream!597

• e788bda4
  by Tiago Gomes at 2018-08-02T11:24:43Z

  plugin: bake API to get and validate a project path
  
  A project path is a path relative to a project directory.
  
  A project path can not also refer to the parent directory in the first
  path component, or point to symbolic links, fifos, sockets and
  block/character devices.
  

• 33292be4
  by Tiago Gomes at 2018-08-02T11:24:43Z

  local plugin: validate project paths
  
  The autotools example had to be copied over inside of the junction
  example, as referring to a path outside of the project directory is
  now disallowed.
  

• 00f170b8
  by Tiago Gomes at 2018-08-02T11:24:43Z

  patch plugin: validate project paths
  

• 4f0f4171
  by Tiago Gomes at 2018-08-02T11:24:43Z

  ostree plugin: validate project paths
  

• c4c0bb1e
  by Tiago Gomes at 2018-08-02T11:24:43Z

  project: validate project paths (element-path)
  

• 385aa592
  by Tiago Gomes at 2018-08-02T11:24:43Z

  project: validate project paths (local plugin paths)
  

• 731328ad
  by Tiago Gomes at 2018-08-02T11:24:43Z

  tests: add generate_file_types testing function
  

• b7362a32
  by Tiago Gomes at 2018-08-02T11:24:43Z

  tests: add additional tests to local plugin
  

• 7e3944dc
  by Tiago Gomes at 2018-08-02T11:24:43Z

  tests: add additional tests to patch plugin
  

• 9f0c8fcf
  by Tiago Gomes at 2018-08-02T11:24:43Z

  tests: test local plugins and element-path paths
  

• 8656a65d
  by Tiago Gomes at 2018-08-02T12:10:29Z

  Merge branch 'tiagogomes/issue-195' into 'master'
  
  Add validation for project paths
  
  See merge request BuildStream/buildstream!593

• 170f33a1
  by Valentin David at 2018-08-02T14:17:01Z

  Move tests.frontend.generate_junction to test.testutils
  

• 3bae4aeb
  by Valentin David at 2018-08-02T14:17:01Z

  Pass targets to Loader.load() instead of its constructor.
  
  This is required to be able to build a Loader before the list
  of targets is known.
  

• 483b0223
  by Valentin David at 2018-08-02T14:17:01Z

  Make Project owner of Loader.
  

• 19cd954b
  by Valentin David at 2018-08-02T14:17:01Z

  Refactor plugin factory creation
  

• 171e803f
  by Valentin David at 2018-08-02T14:18:10Z

  Add support for include '(@)' in project.conf and .bst files
  
  Fixes #331.
  

• 4e1160a9
  by Valentin David at 2018-08-02T15:17:03Z

  Merge branch 'valentindavid/331_include' into 'master'
  
  Add support for include in project.conf
  
  Closes #331
  
  See merge request BuildStream/buildstream!471

• 3782c15f
  by Ed Baunton at 2018-08-04T08:54:58Z

  element.py (docs): dashes not underscores for build and install root
  
  Fix small error in docs in relation to `get_variable` documentation.
  

• f918205a
  by Tristan Van Berkom at 2018-08-04T10:08:21Z

  Merge branch 'edbaunton/doc-typo' into 'master'
  
  element.py (docs): dashes not underscores for build and install root
  
  See merge request BuildStream/buildstream!577

• 9efe1202
  by Javier Jardón at 2018-08-04T12:28:16Z

  buildstream/plugins/elements/autotools.yaml: Make builds out of tree by default
  
  Similar to what cmake and meson do
  

20 changed files:

• .coveragerc
• + .gitattributes
• .gitignore
• .gitlab-ci.yml
• + .gitlab/issue_templates/bst_bug.md
• + .gitlab/issue_templates/bst_task.md
• + .gitlab/merge_request_templates/bst_merge_request.md
• + .pylintrc
• + BuildStream.doap
• HACKING.rst
• + MAINTAINERS
• MANIFEST.in
• + NEWS
• README.rst
• buildstream/__init__.py
• buildstream/_artifactcache/__init__.py
• buildstream/_artifactcache/artifactcache.py
• + buildstream/_artifactcache/cascache.py
• + buildstream/_artifactcache/casserver.py
• − buildstream/_artifactcache/ostreecache.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.