[Notes] [Git][BuildStream/buildstream][phil/848-plugin-deprecation-warnings] 271 commits: _profile.py: Add timestamp to the logs

[Phil Dawson <gitlab mg gitlab com>]

Title: GitLab

Phil Dawson pushed to branch phil/848-plugin-deprecation-warnings at BuildStream / buildstream

Commits:

• 63b02f7c
  by James Ennis at 2019-01-17T15:13:27Z

  _profile.py: Add timestamp to the logs
  

• f64ee46f
  by James Ennis at 2019-01-17T17:02:28Z

  _profile.py: Write binaries as well as logs
  
  Private class methods which write the logs and write the binaries
  have been added to Profile. The binaries are able to be used
  by various visualisation tools.
  

• 2cb37a7e
  by James Ennis at 2019-01-18T09:31:24Z

  Merge branch 'jennis/profiling_outputs_binaries' into 'master'
  
  Make the profiler output binaries (which can be used to visualise the data)
  
  See merge request BuildStream/buildstream!1082

• 56ec33cc
  by Valentin David at 2019-01-18T11:14:41Z

  .gitlab-ci.yml: Add overnight tests logs to artifacts.
  

• ad2df651
  by Javier Jardón at 2019-01-18T12:01:56Z

  Merge branch 'valentindavid/overnight-tests-logs-artifacts' into 'master'
  
  .gitlab-ci.yml: Add overnight tests logs to artifacts.
  
  See merge request BuildStream/buildstream!1084

• f874295f
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  bzr source plugin: Use lock files to avoid corrupting the source cache
  
  This patch by itself fixes #868
  

• 42a2fe3c
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  bzr source plugins: Remove all of the atomic backup directory business logic
  
  Follow up of last commit which uses exclusive locking to
  protect bzr operations instead.
  

• a8713ed2
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  testutils/runcli.py: Added cli.get_element_states()
  
  With get_element_state(), you need to invoke BuildStream once
  for every element state you want to observe in a pipeline.
  
  The new get_element_states() reports a dictionary with
  the element state hashed by element name and is better to use
  if you have more than one element to observe the state of.
  

• a895cb2a
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  tests/frontend/track.py: Use test_track_recurse() to stress test Sources
  
  This causes multiple source instances to interact with the same
  backing data store at the same time, increasing the likelyhood
  of triggering issues around concurrent access.
  
  This more reliably triggers issue #868
  

• 4236bcc7
  by Tristan Van Berkom at 2019-01-18T16:35:23Z

  Merge branch 'tristan/fix-bzr-race' into 'master'
  
  Fix bzr race conditions
  
  Closes #868
  
  See merge request BuildStream/buildstream!1083

• 951a8df1
  by Angelos Evripiotis at 2019-01-18T18:02:22Z

  cli.py: add a hint about '--' to 'bst shell' help

• 9911023f
  by Angelos Evripiotis at 2019-01-18T18:56:21Z

  Merge branch 'aevri/shell_separator_hint' into 'master'
  
  cli.py: add a hint about '--' to 'bst shell' help
  
  See merge request BuildStream/buildstream!1078

• c9ce89d2
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  _cas/cascache.py: Cleanup directories when removing refs
  
  With out this, empty directories in the refs/heads directory just
  grow unconditionally.
  

• c536ab6a
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  tests/artifactcache/expiry.py: Test refs directory cleanup
  
  Enhance the test which checks removal of the extract directories
  with an additional check that the ref directories are cleaned up
  when removing artifacts.
  

• 99699ffc
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  utils.py: Added _tempnamedfile()
  
  When used in a child process in BuildStream, this should be
  used instead of tempfile.NamedTemporaryFile() directly, otherwise
  we fail to cleanup the file on SIGTERM.
  

• 8ce483d4
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  _cas/cascache.py: Use utils._tempdir() and utils._tempnamedfile()
  
  The direct usage of tempfile.TemporaryDirectory() and
  tempfile.NamedTemporaryFile() here causes leakage of any temporary
  data when the process operating on temporary data is terminated
  with SIGTERM.
  
  Using the utilities ensures that trash is not left behind in
  ~/.cache/buildstream/artifacts/tmp when the user terminates
  BuildStream with ^C.
  

• 73c7252d
  by Tristan Van Berkom at 2019-01-18T20:57:42Z

  Merge branch 'tristan/cas-cleanup-improve' into 'master'
  
  CASCache cleanup improvements
  
  See merge request BuildStream/buildstream!1087

• adfb9291
  by Tristan Van Berkom at 2019-01-18T21:23:05Z

  tests/testutils/python_repo.py: Use subprocess to run sdist
  
  The current approach using setuptools.sandbox.run_setup() was
  causing a spurious (but highly frequent) failure where setuptools
  gets mixed up with it's manipulation of sys.modules and hits a
  RuntimeError as a dictionary changes size while being iterated over.
  
  For instance: https://gitlab.com/BuildStream/buildstream/-/jobs/147967307
  
  Since this already happens in an isolated virtual environment created
  by tox, we should not need additional sandboxing here from setuptools,
  and launching this as a subprocess will be safer.
  

• d114a6bd
  by Tristan Van Berkom at 2019-01-18T22:07:22Z

  Merge branch 'tristan/fix-pip-source-test' into 'master'
  
  tests/testutils/python_repo.py: Use subprocess to run sdist
  
  See merge request BuildStream/buildstream!1090

• bf591ade
  by Chandan Singh at 2019-01-18T22:09:04Z

  Derive import plugin from Element instead of BuildElement
  
  The `import` element is not really a build element. The main purpose of
  the `BuildElement` class is to allow users to run `build-commands`,
  `install-commands` etc. But, `import` does not run such commands.
  Moreover, we already override all the methods provided by
  `BuildElement`. So it only makes it confusing to have it derived from
  `BuildElement` class when it is not a build element.
  
  So, derive it from the base `Element` class instead.
  

• 2233a532
  by Chandan Singh at 2019-01-18T22:46:41Z

  Merge branch 'chandan/import-is-not-buildelement' into 'master'
  
  Derive import plugin from Element instead of BuildElement
  
  See merge request BuildStream/buildstream!1089

• aa3411f9
  by Tristan Maat at 2019-01-18T22:47:46Z

  testutils/runcli.py: Allow removing artifacts from arbitrary dirs
  
  `remove_artifact_from_cache` used a hard-coded path to remove
  artifacts, which wasn't sufficient for integration tests.
  

• fff882fe
  by Tristan Maat at 2019-01-18T22:47:46Z

  widget.py: Avoid "showing 0 lines" messages when there are no lines
  
  This happened when bst is invoked with --message-lines 0 or
  --error-lines 0, and was arguably a little too verbose (the user
  explicitly asked us not to show them any lines, after all).
  
  Fixes #779
  

• e230dedb
  by Tristan Van Berkom at 2019-01-18T23:23:35Z

  Merge branch 'tlater/message-lines' into 'master'
  
  Avoid "showing 0 lines" messages when we're asked to show no lines
  
  Closes #779
  
  See merge request BuildStream/buildstream!1031

• 43797617
  by Richard Maw at 2019-01-21T10:41:37Z

  _gitsourcebase.py: Fetch with explicit refspecs
  
  Old versions of git lack --force and --tags,
  but the same effect can be had by specifying refspecs.
  

• ce8dab0f
  by Richard Maw at 2019-01-21T10:41:37Z

  tests/frontend/workspace.py: Skip test_open_multi_unwritable when root
  
  The test assumes that a directory with write permission removed isn't writable,
  this isn't the case if the process running the tests has CAP_DAC_OVERRIDE
  which is common when running as root.
  

• 50165081
  by Richard Maw at 2019-01-21T10:41:37Z

  tests/sources/git.py: Skip tests that assume too new a git
  
  test_track_invalid_submodule depends on being able to remove a submodule
  by `git rm $submoduledir`, but old versions of git don't update .gitmodules
  so BuildStream still thinks there's a submodule present.
  
  For expediency the test is skipped rather than changed to manually remove
  the entry from .gitmodules if git hasn't done it,
  since in the common case git is new enough to do that itself.
  
  test_git_describe expects --first-parent to find another tag,
  but `bst track` will gracefully degrade if the option doesn't work
  so a different history will be retained with old versions of git.
  
  It's of marginal benefit to add additional cruft
  to test for different output on old versions of git that won't persist forever.
  

• 8677a256
  by Benjamin Schubert at 2019-01-21T10:41:37Z

  Add a Centos runner
  
  Centos is apparently different enough from fedora when running tests.
  

• 33782865
  by Valentin David at 2019-01-21T11:47:18Z

  Merge branch 'richardmaw/centos-oldgit-test-fixes' into 'master'
  
  Fix CentOS
  
  Closes #833
  
  See merge request BuildStream/buildstream!1085

• c00b3782
  by Chandan Singh at 2019-01-21T19:37:53Z

  CONTRIBUTING.rst: Fix formatting of link to pip docs
  
  The link explaining python's requirements files was formatted
  incorrectly (missing underscore at the end). This resulted in the text
  being rendered literally, as opposed to being converted into a
  hyperlink.
  

• 638ceb8a
  by Chandan Singh at 2019-01-21T20:16:38Z

  Merge branch 'chandan/fix-contrib-link' into 'master'
  
  CONTRIBUTING.rst: Fix formatting of link to pip docs
  
  See merge request BuildStream/buildstream!1094

• 383142e3
  by James Ennis at 2019-01-22T12:32:43Z

  using_commands.rst: Split out top level commands and subcommand groups
  

• ac135333
  by James Ennis at 2019-01-22T12:32:43Z

  using_commands.rst: Add the source checkout command
  

• a3fc350f
  by James Ennis at 2019-01-22T12:32:43Z

  Move push and pull to the new artifact subcommand group
  
  This commit also ensures that if we try to use the 'old' commands,
  BuildStream will fail and instruct the user to use the new command.
  

• 9eefe863
  by James Ennis at 2019-01-22T12:32:43Z

  cli: Add artifact checkout subcommand
  
  'artifact checkout' has slightly different behaviour from 'checkout',
  that is, either '--directory' or '--tar' are now required options.
  This is a step towards allowing checkout to take multiple args.
  

• fbd15939
  by James Ennis at 2019-01-22T12:32:43Z

  Mark 'old' checkout command as obsolete
  
  This commit marks 'bst checkout' as a 'hidden' command. If used,
  the user will be prompted to use the new 'bst artifact checkout'
  command.
  
  All tests which used 'bst checkout' have been modified to use
  the new artifact sub-command.
  
  This partially solves #822.
  

• 9e8034e9
  by James Ennis at 2019-01-22T12:32:43Z

  Add NEWS entry describing the deprecation of checkout, pull and push
  

• 1bccf1d2
  by James Ennis at 2019-01-22T12:38:40Z

  using_commands.rst: Split out the artifact subcommands
  

• d9072371
  by James Ennis at 2019-01-22T12:38:40Z

  using_commands.rst: Add artifact log subcommand
  

• 077d5a96
  by James Ennis at 2019-01-22T12:41:29Z

  Change bst checkout/pull/push references to bst artifact checkout/pull/push
  

• ea2fbe4d
  by James Ennis at 2019-01-22T13:02:14Z

  man: Regenerate all of our man pages
  
  Due to an upstream click_man issue:
  https://github.com/click-contrib/click-man/issues/10
  generating the man pages with our current setup.py only generates
  a man page for bst-artifact-server, our first entry point.
  I then had to remove this entry point from setup.py and regenerate to
  obtain man pages for the commands in cli.py
  

• 9c2a6b93
  by James Ennis at 2019-01-22T15:08:17Z

  Merge branch 'jennis/migrate_pull_push_commands' into 'master'
  
  Move push/pull/checkout to the artifact subcommand group
  
  See merge request BuildStream/buildstream!1045

• 35377522
  by James Ennis at 2019-01-22T15:28:13Z

  cli.py: Add an obsoletion note to our deprecated commands.
  
  Click 7.0 allows us to declare commands as 'hidden'.
  However, sphinx-click and click-man still generate these hidden
  commands in the documentation and man pages, respectively.
  
  This is a stop gap solution until the upstream issues (tracked in
  issues #879 and #881) have been addressed.
  

• 76148785
  by James Ennis at 2019-01-22T16:15:20Z

  Merge branch 'jennis/add_obsoletion_note' into 'master'
  
  Add an obsoletion note to our deprecated commands.
  
  See merge request BuildStream/buildstream!1097

• 1443c542
  by Tristan Van Berkom at 2019-01-22T16:26:56Z

  _scheduler: Fix dont display a failure for terminated jobs
  
  This fixes a recent regression introduced in c2fc2a5ea
  

• 6e5c9987
  by Tristan Van Berkom at 2019-01-22T17:14:50Z

  Merge branch 'tristan/fix-terminated-status' into 'master'
  
  _scheduler: Fix dont display a failure for terminated jobs
  
  See merge request BuildStream/buildstream!1096

• 3315b9a1
  by Tristan Van Berkom at 2019-01-22T18:46:24Z

  tests/integration/pullbuildtrees.py: Fix the non-integration case.
  
  This test has one test case which is marked as an integration test,
  and the other is not an integration test, but was using the integration
  cli. The integration cli does not work correctly if not run in
  integration mode.
  
  This was causing an error locally in conftest.py when trying to
  create a tmpdir inside a nonexisting integration cache directory.
  

• 56e857f4
  by Tristan Van Berkom at 2019-01-22T19:59:57Z

  Merge branch 'tristan/fix-pullbuildtrees-test' into 'master'
  
  tests/integration/pullbuildtrees.py: Fix the non-integration case.
  
  See merge request BuildStream/buildstream!1098

• 3895571a
  by Chandan Singh at 2019-01-23T00:45:47Z

  tox.ini: Unpin sphinx dependency
  
  https://github.com/rtfd/sphinx_rtd_theme/pull/672 has been fixed
  upstream, and the newer versions of `sphinx_rtd_theme` do not break
  search functionality with Sphinx >= 1.8.
  

• 528e8ed7
  by Chandan Singh at 2019-01-23T01:31:49Z

  Merge branch 'chandan/unpin-sphinx' into 'master'
  
  tox.ini: Unpin sphinx dependency
  
  See merge request BuildStream/buildstream!1093

• 0098a900
  by Tristan Van Berkom at 2019-01-23T13:57:53Z

  tests: Migrated cache quota test into artifactcache/cache_size.py
  
  Instead of sitting mysteriously alone in internals/utils.py
  

• 9be31641
  by Tristan Van Berkom at 2019-01-23T13:57:53Z

  _artifactcache.py: Raise ArtifactError() when quota size exceeds disk space.
  
  This is not an error related to loading data, like a parse error
  in the quota specification is, but a problem raised by the artifact
  cache - this allows us to assert more specific machine readable
  errors in test cases (instead of checking the string in stderr, which
  this patch also fixes).
  
  This also removes a typo from the error message in the said error.
  
    * tests/artifactcache/cache_size.py
  
      Updated test case to expect the artifact error, which consequently
      changes the test case to properly assert a machine readable error
      instead of asserting text in the stderr (which is the real, secret
      motivation behind this patch).
  
    * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()
  
      Now expect the artifact error for the tests which check configurations
      which create caches too large to fit on the disk.
  

• 3a6e953f
  by Tristan Van Berkom at 2019-01-24T01:59:13Z

  Merge branch 'tristan/insufficient-storage-error' into 'master'
  
  Tristan/insufficient storage error
  
  See merge request BuildStream/buildstream!1102

• 68339b19
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/testutils/runcli.py: Make get_element_states() take a list of targets
  
  Instead of a single target, we can always provide a single target
  in a list.
  

• 6ecc2b0a
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/artifactcache/expiry.py: Refactored to use get_element_states()
  

• 1140aed5
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/elements/filter.py: Refactored to use get_element_states()
  

• b7ea8b74
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/sources/remote.py: Refactored to use get_element_states()
  

• c2c004f9
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/frontend/pull.py: Refactored to use get_element_states()
  

• 21b2958b
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/frontend/push.py: Refactored to use get_element_states()
  

• 83fcaa9f
  by Tristan Van Berkom at 2019-01-24T05:01:56Z

  tests/frontend/track.py: Refactored to use get_element_states()
  

• 46eb3018
  by Tristan Van Berkom at 2019-01-24T05:01:56Z

  tests/frontend/workspace.py: Refactored to use get_element_states()
  

• 341b131b
  by Tristan Van Berkom at 2019-01-24T06:13:57Z

  Merge branch 'tristan/test-element-states' into 'master'
  
  Reduce number of calls to `bst show` in tests
  
  See merge request BuildStream/buildstream!1103

• ef2b4648
  by Tristan Van Berkom at 2019-01-24T06:14:20Z

  tests/frontend/track.py: test_track_error_cannot_write_file() fixup
  
  This tests how BuildStream reacts when it fails to write the tracking
  results to the element files or project.refs file, which is an operation
  that plugins do not play a part in.
  
  As such, removing the per repo kind parameterization from this test
  as multiple runs are redundant here.
  

• 40c18174
  by Tristan Van Berkom at 2019-01-24T07:00:50Z

  Merge branch 'tristan/track-test-reduce' into 'master'
  
  test_track_error_cannot_write_file() fixup
  
  See merge request BuildStream/buildstream!1104

• 5b0bba85
  by Jonathan Maw at 2019-01-24T10:15:06Z

  Test that tracking in workspaces actually works
  
  Previously, it merely tested that buildstream did not fall other,
  rather than whether it did anything useful.
  

• 58d7d722
  by Jonathan Maw at 2019-01-24T10:18:01Z

  tests: Test that fetching an open workspace will fetch its dependencies
  
  Previously, there was no way of detecting whether fetching happened, as
  an element with an open workspace will not be fetched.
  

• 24bd8994
  by Jürg Billeter at 2019-01-24T13:36:33Z

  Merge branch 'jonathan/test-missing-workspace-guessing' into 'master'
  
  Add tests to cover reinstated support for guessing targets
  
  See merge request BuildStream/buildstream!1042

• e03dd5fc
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _frontend/cli.py: Guess element also for bst build --all
  
  There is no reason to disallow guess_element() for bst build --all.
  

• 3850274b
  by Phillip Smyth at 2019-01-24T13:37:28Z

  projectconfig.yaml: Add key for default targets
  
  _versions.py: Bump format version
  

• ab72d384
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _project.py: Add get_default_target() and get_default_targets() methods
  
  _frontend/cli.py: Use new methods.
  
  Based on patches by Phillip Smyth.
  

• ee2d8434
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _stream.py: Add ignore_junction_targets parameter
  
  This filters out junctions from the list of targets.
  

• 2f175f0a
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _frontend/cli.py: Ignore junctions in default targets where appropriate
  
  Junctions cannot be built, pulled, or pushed. Specifying a junction on
  the command line for these commands will result in an error. However,
  junctions may be in the list of default targets, so they need to be
  ignored for build, pull, and push commands.
  

• 5a351dee
  by Jürg Billeter at 2019-01-24T13:38:50Z

  format_project.rst: Add documentation for default targets
  

• 77414518
  by Phillip Smyth at 2019-01-24T13:38:50Z

  tests/frontend: Add default target tests for bst show and build
  

• ce1c10ec
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/fetch.py: Add default target test for bst source fetch
  

• 99b5c0af
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/pull.py: Add default target test for bst push/pull
  

• 3642c8e7
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/buildcheckout.py: Add default target test with junction
  
  Test that `bst build` does not fail in a project where the list of
  default targets includes a junction (junctions cannot be built).
  

• 50c5159f
  by Phillip Smyth at 2019-01-24T13:38:50Z

  NEWS: Add entry for default target feature
  

• 05587f22
  by Jürg Billeter at 2019-01-24T15:10:08Z

  Merge branch 'issue-638-validate-all-files' into 'master'
  
  Add support for default targets
  
  See merge request BuildStream/buildstream!925

• 3e36e363
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _scheduler/resources.py: Dont error out in unregister_exclusive_interest()
  
  Don't require the interest to be registered, just discard any interest,
  this function just sets a bit in a mask, and is not intended to maintain
  a balance like the reserve() function is.
  

• ce01f87e
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _scheduler/scheduler.py: Run cache size exclusively at startup
  
  When running any session that has Queues which require Resource.CACHE,
  check if our loaded estimated size exceeds the quota, and if so;
  lock the Resource.CACHE resource exclusively right away and run
  an exclusive initial cache size job.
  
  This ensures we cleanup first before doing anything which might
  add to the cache at startup time, if deemed needed.
  
  This is a partial fix for issue #737
  

• 2479e8df
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _frontend/widget.py: Render core messages more like other messages
  
  In order to test when core activities occur by parsing the stderr
  in tests, we should make the messages conform more.
  
  At the same time, this restores alignment of columns in core
  messages with the element processing related messages.
  
  Also, _scheduler/scheduler.py is updated to make it's activity names
  conform to the (current) 5 character limit for the sake of alignment.
  
  The tests/frontend/logging.py test gets it's regexes updated for
  the log lines it checks for in stderr.
  

• fdb8ff65
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  tests/artifactcache/expiry.py: Test that expiry happens first
  

• acd0bf22
  by Tristan Van Berkom at 2019-01-24T18:01:41Z

  Merge branch 'tristan/cache-management' into 'master'
  
  Cache management fixes
  
  See merge request BuildStream/buildstream!1091

• 41f03296
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/app.py: Initialize logging before preflighting the artifact cache
  
  The artifact cache emits messages, and we want to allow that in preflight.
  

• 24ca2f46
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  utils.py: Add _get_volume_size()
  
  We can streamline this call to os.statvfs() in a few places.
  

• 9fd9fbb9
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Refactored to use utils._get_volume_size()
  
  This will benefit from a better UtilError being raised, and
  and turns the artifact cache's local function into a one liner.
  
  The loop which finds the first existing directory in the
  given path has been removed, being meaningless due to the
  call to os.makedirs() in ArtifactCache.__init__().
  
  The local function was renamed to _get_cache_volume_size() and
  no longer takes any arguments, which is more suitable for the
  function as it serves as a testing override surface for
  unittest.mock().
  
  The following test cases which use the function to override
  the ArtifactCache behavior have been updated to use the new
  overridable function name:
  
     tests/artifactcache/cache_size.py
     tests/artifactcache/expiry.py
  

• 7ee0c579
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Added ArtifactCacheUsage()
  
  A simple object which creates a snapshot of current
  usage statistics for easy reporting in the frontend.
  

• 353293b6
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _context.py: Added get_artifact_cache_usage()
  
  A frontend facing API for obtaining usage statistics.
  
  I would have put this on Stream instead, but the Context
  seems to be the de facto place for looking up the artifact cache
  in general so let's put it here.
  

• 51ed36de
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/widget.py: Added cache usage entry in the startup heading
  

• 5797238b
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/status.py: Added Cache size usage indicator to status bar
  
  This also adds some comments around the main status bar heading
  rendering function.
  

• 8074ebf4
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Add status messages in cache management operations
  
  Added some useful status messages when:
  
    * Calculating a new artifact cache usage size
  
    * Starting a cleanup
  
    * Finishing a cleanup
  
  Also enhanced messaging about what was cleaned up so far when
  aborting a cleanup.
  

• 49c11bc8
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Added client progress callback to ArtifactCache.clean()
  

• 3616e939
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _scheduler/jobs/job.py: Allow subclasses to message the frontend
  

• bcd19266
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _scheduler/jobs/cleanupjob.py: Update cache size while processing
  
  Updates the known cache size in the main process while the cleanup
  process is ongoing, so that the status indicators update live
  while the cleanup happens.
  

• 9c33107f
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Correcting API documenting comment for remove()
  
  This seems to have been copy/pasted from cascache, and
  documents the function to possibly return None if defer_prune
  was specified, but this function does not expose defer_prune.
  

• a2140d74
  by Tristan Van Berkom at 2019-01-24T19:44:52Z

  Merge branch 'tristan/cache-management-logging' into 'master'
  
  Cache management logging enhancements
  
  See merge request BuildStream/buildstream!1105

• d5847462
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Added a new profiling topic, scheduler
  
  profile_start() and profile_end() calls have been incorporated into
  Scheduler.run()
  

• 56a3954c
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Added a new profiling topic, load-selection
  
  profile_start() and profile_end() calls have been added to
  Stream.load_selection()
  

• 116da6d7
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Update copyright statement
  

• 137d31cd
  by James Ennis at 2019-01-25T17:59:22Z

  Merge branch 'jennis/add_new_profile_topic' into 'master'
  
  Add new 'scheduler' and 'load-selection' profiling topics
  
  See merge request BuildStream/buildstream!1088

• 22b3a0c1
  by Tristan Van Berkom at 2019-01-25T18:35:36Z

  _artifactcache.py: Don't require the quota to be available on disk.
  
  Instead only rely on the headroom to be enough to protect against
  out of space conditions. The headroom can become configurable as
  a separate step is required.
  
  The changes to achieve this are:
  
    * Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().
  
    * ArtifactCache.full() now also reports True if the available
      space on the artifact cache volume is smaller than the headroom.
  
      This ensures jobs get triggered to cleanup the cache when
      reaching the end of the disk.
  
    * When loading the artifact quota, it is now only an error if
      the quota exceeds the overall disk space, not if it does not
      fit in the available space.
  
      It is still a warning if the quota does not fit in the
      available space on the artifact cache volume.
  
    * Updated scheduler.py and buildqueue.py for the API rename
  
    * tests: Updated the artifactcache/expiry.py test for its
      expectations in this regard.
  
      Added a new test to test an error when quota was specified to
      exceed total disk space, and adjusted the existing tests to
      expect a warning when the quota does not fit in the available
      space.
  
  This fixes issue #733 and #869.
  

• 27ca6cc7
  by Tristan Van Berkom at 2019-01-25T21:18:15Z

  Merge branch 'tristan/cache-quota-max-only' into 'master'
  
  _artifactcache.py: Don't require the quota to be available on disk.
  
  Closes #869 and #733
  
  See merge request BuildStream/buildstream!1106

• 89ba2abe
  by Chandan Singh at 2019-01-25T23:50:49Z

  tox.ini: Add environment to update man pages
  
  Previously, one had to manually install `click-man` package, and
  remember to run the correct command. Now, we can simply run `tox -e man`
  to update the man pages.
  

• d7438688
  by Chandan Singh at 2019-01-25T23:50:49Z

  man: Refresh man pages
  
  Notable changes:
  
  * New `source` and `artifact` command groups
  * Man pages corresponding to obsolete commands, that were marked as
    hidden in Click, have now been removed.
  
  Fixes #881.
  

• 4edbbd27
  by Chandan Singh at 2019-01-25T23:50:49Z

  setup.py, CONTRIBUTING.rst: Recommend using tox to generate man pages
  
  Simplify our docs, by requesting users to run `tox -e man` to update man
  pages, instead of manually installing `click-man` and running the
  command manually.
  

• a3e7aee8
  by Chandan Singh at 2019-01-25T23:50:49Z

  setup.py: Do not error out when man directory is empty/missing
  
  If the `man` directory is empty, then it won't be copied in the source
  distribution, and `list_man_pages()` will throw an exception when trying
  to list files in a non-existent directory. This prevents us from
  installing the BuildStream package when the man pages are not there.
  
  The most common use-case for this is when we want to re-generate the man
  pages but want to install the package before re-generating them.
  

• 222753ac
  by Tristan Van Berkom at 2019-01-26T05:10:36Z

  Merge branch 'chandan/toxic-man' into 'master'
  
  Generate man pages using tox & update them
  
  Closes #880 and #881
  
  See merge request BuildStream/buildstream!1107

• 5df4105a
  by Angelos Evripiotis at 2019-01-28T10:13:40Z

  news: fix 'osbolete' spelling
  

• 9c981eff
  by Angelos Evripiotis at 2019-01-28T10:17:57Z

  Fixup refs to 'bst track'
  
  Now that 'bst track' is obsolete, change guidance to refer to the
  replacement 'bst source track' instead.
  

• bef80291
  by Angelos Evripiotis at 2019-01-28T10:19:17Z

  Fixup refs to 'bst fetch'
  
  Now that 'bst fetch' is obsolete, change guidance to refer to the
  replacement 'bst source fetch' instead.
  

• 564cb245
  by Angelos Evripiotis at 2019-01-28T11:36:16Z

  Merge branch 'aevri/bst_track_guidance' into 'master'
  
  Fixup refs to 'bst track' and 'bst fetch'
  
  See merge request BuildStream/buildstream!1086

• a3e2cdd2
  by Tom Pollard at 2019-01-28T12:14:40Z

  _stream.py: Add use_artifact_config opt arg for load_selection()
  
  use_artifact_config added as an optional default arg, allowing
  for loading of given elements artifact remote config.
  

• 805baf7d
  by Tom Pollard at 2019-01-28T12:14:40Z

  Download buildtrees on demand for bst shell --use-buildtree
  
  Provide bst shell --use-buildtree the ability to attempt to
  acquire missing buildtrees, given respective option, user
  pull-buildtree context and remote availability.
  
  _frontend/cli.py: Refactor logic for determining --use-buildtree
  option with given opportunity to attempt pulling a non-local
  buildtree. Element loaded with artifact_config to allow remote
  querying.
  
  _stream.py: With given user option and element state, construct
  PullQueue to fetch remote buildtree. Continue or Error without
  buildtree if cannot be attained.
  
  tests/integration/build-tree.py: Update to support new usecases
  

• 38356932
  by Tom Pollard at 2019-01-28T13:44:37Z

  Merge branch 'tpollard/829' into 'master'
  
  Download buildtrees on demand for bst shell --use-buildtree
  
  Closes #829
  
  See merge request BuildStream/buildstream!1050

• 3590ca8c
  by Abderrahim Kitouni at 2019-01-28T14:59:19Z

  requirements/requirements.in: require protobuf >= 3.6
  
  This is needed since 0f2bc3754
  

• 39b952dc
  by Abderrahim Kitouni at 2019-01-28T15:19:04Z

  requirements/requirements.in: require Click >= 7.0
  
  This is needed since 629a6e524, and was lost in the conversion to requirements.in
  

• 80b36d0c
  by Javier Jardón at 2019-01-28T17:54:52Z

  Merge branch 'abderrahim/protobuf-version' into 'master'
  
  requirements/requirements.in: update minimum versions
  
  Closes #884
  
  See merge request BuildStream/buildstream!1114

• a1ab48da
  by Valentin David at 2019-01-28T21:30:26Z

  Fix crash when spawned job completes very fast
  
  Job can complete before we return from `Job.span()` to
  `Scheduler._spawn_job()`, so that `_active_jobs` would not yet contain
  the job.
  
  This would print a stack on the console and try to run a second time
  the job which can have unexpected effects.
  
  In order to reproduce the issue, in
  `buildstream/_scheduler/jobs/job.py`, in `Job.spawn`,
  add a call to `time.sleep()` right before call to
  `asyncio.get_child_watcher()`.
  
  This fixes issue #857.
  

• 2fcb4491
  by Jürg Billeter at 2019-01-28T22:36:22Z

  Merge branch 'valentindavid/crash_in_scheduler_857' into 'master'
  
  Fix crash when spawned job completes very fast
  
  Closes #857
  
  See merge request BuildStream/buildstream!1095

• 1c05a092
  by Valentin David at 2019-01-29T05:58:17Z

  Fix type of error codes in CAS server
  
  Fixes #882.
  

• 785da59c
  by Jürg Billeter at 2019-01-29T06:49:10Z

  Merge branch 'valentindavid/wrong_type_in_status_code' into 'master'
  
  Fix type of error codes in CAS server
  
  Closes #882
  
  See merge request BuildStream/buildstream!1099

• ddef91ea
  by Valentin David at 2019-01-29T07:23:35Z

  Make sure testing cache directory exists
  
  Fixes #873
  

• 6a4c8611
  by Jürg Billeter at 2019-01-29T08:18:45Z

  Merge branch 'valentindavid/make_cache_dir' into 'master'
  
  Make sure testing cache directory exists
  
  Closes #873
  
  See merge request BuildStream/buildstream!1092

• 86c8e414
  by Angelos Evripiotis at 2019-01-29T10:39:29Z

  BREAK:remove unconditional 'are you sure?' prompts
  
  This is a breaking change, as it affects behaviour that people might be
  relying on. An entry has been added to NEWS.
  
  As proposed on the mailing list, this change removes the unconditional
  prompts on:
  
      o: bst workspace reset
      o: bst workspace close --remove-dir
  
  If interactive, these commands would always interrupt you with a prompt
  like this:
  
      This will remove all your changes, are you sure?
  
  This seems like it may just save someone's work some time. It may also
  condition folks to hit 'y' quickly without thinking.
  
  This change also makes the non-interactive behaviour consistent with the
  interactive behaviour in the default case. There is also the case of the
  prompt configured by 'really-workspace-close-project-inaccessible',
  which may be tackled in later work.
  
  This change also removes the new config options to suppress those
  prompts, and their associated news entry.
  
  The relevant bit of the mailing list conversation is here:
  https://mail.gnome.org/archives/buildstream-list/2018-December/msg00106.html
  
  The issue to make interactive and non-interactive behaviour consistent
  is here:
  https://gitlab.com/BuildStream/buildstream/issues/744
  

• aae35e13
  by Angelos Evripiotis at 2019-01-29T12:08:07Z

  Merge branch 'aevri/are_you_sure' into 'master'
  
  BREAK:remove unconditional 'are you sure?' prompts
  
  See merge request BuildStream/buildstream!1061

• 03111d39
  by Dor Askayo at 2019-01-30T10:35:06Z

  filter.py: don't recurse when staging dependencies
  
  Also bump the element's version so cached artifacts would be
  invalidated.
  
  Fixes #883
  

• 7256bb0c
  by James Ennis at 2019-01-30T11:34:04Z

  Merge branch 'doraskayo/filter-indirect-deps-fix' into 'master'
  
  filter.py: don't recurse when staging dependencies
  
  Closes #883
  
  See merge request BuildStream/buildstream!1110

• 36746730
  by Chandan Singh at 2019-01-31T10:50:05Z

  tox.ini: Specify minimum version of click-man
  
  `click-man` versions < 0.3.0 do not properly support multiple
  entrypoints. Since this was added to `tox` after `0.3.0` was released,
  `tox` should never be pulling older versions. But, let's add it here for
  documentation purposes.
  
  See
  https://gitlab.com/BuildStream/buildstream/merge_requests/1107#note_135187046
  for some background on this.
  

• fa4a21ce
  by Chandan Singh at 2019-01-31T12:15:43Z

  Merge branch 'chandan/min-version-click-man' into 'master'
  
  tox.ini: Specify minimum version of click-man
  
  See merge request BuildStream/buildstream!1120

• dd791373
  by Chandan Singh at 2019-01-31T14:32:44Z

  testutils/site.py: Support parsing more exotic git versions
  
  We use output of `git --version` to determine if we can run some tests
  that rely on features from newer git versions. Usually, we expect the
  output to be like:
  
      git version 2.17.2
  
  On some platforms, like MacOS, there could be a suffix after the version
  string, so that it looks something like:
  
      git version 2.17.2 (Apple Git-113)
  
  This causes things to fail like so:
  
      ValueError: invalid literal for int() with base 10: '2 (Apple Git-113)\n'
  
  Fix logic around `HAVE_OLD_GIT` such that we split the output of
  `git --version` without limit on how many times we split. Previously we
  used to split only twice so the suffixes like `(Apple Git-113)` are not
  part of the parsed version.
  

• 96c0fbd6
  by Chandan Singh at 2019-01-31T15:39:19Z

  Merge branch 'chandan/fix-git-version-mac' into 'master'
  
  testutils/site.py: Support parsing more exotic git versions
  
  See merge request BuildStream/buildstream!1118

• d25e2795
  by Benjamin Schubert at 2019-01-31T17:06:23Z

  Add LoadElement in dependency list for LoadElement idrectly
  
  This removes the need for the 'Dependency' list to then be matched
  with the corresponding LoadElement and will allow iterating the
  graph more easily
  

• 2d0eebbf
  by Benjamin Schubert at 2019-01-31T17:06:23Z

  Pass element directly to collect_element
  
  This simplifies the loading
  

• 583bd97d
  by Benjamin Schubert at 2019-02-01T10:26:37Z

  Merge branch 'bschubert/loader' into 'master'
  
  Cleanup loader by linking LoadElements sooner
  
  See merge request BuildStream/buildstream!1122

• 51cec3da
  by Phil Dawson at 2019-02-01T14:25:44Z

  tests/cachekey: Test cache keys are independent of target elements
  

• 2b38aabe
  by Phil Dawson at 2019-02-01T15:33:00Z

  Merge branch 'phil/cache-key-stability-test' into 'master'
  
  tests/cachekey: Test cache keys are independent of target elements
  
  See merge request BuildStream/buildstream!1123

• dbb3d232
  by James Ennis at 2019-02-01T15:51:32Z

  filter.py/filter.yaml: Documentation improvements
  

• 7e4205cb
  by James Ennis at 2019-02-01T15:51:32Z

  filter.py: Add an example to the documentation
  

• 4109a34a
  by James Ennis at 2019-02-01T17:11:29Z

  Merge branch 'jennis/filter-docs' into 'master'
  
  Improve our filter documentation
  
  Closes #278
  
  See merge request BuildStream/buildstream!1112

• c9345014
  by James Ennis at 2019-02-04T13:53:42Z

  filter.py: Fail if declared domains do not exist in the parent element
  
  This patch also uncovered the fact that our test_filter_deps_ok() test
  has been inaccurate. Thus the element built in this test
  (deps-permitted.bst) has been modified so that it build depends on the
  input.bst element, as it should.
  
  tests/filter.py: Ensure deps_ok test passes
  

• 3ab09651
  by James Ennis at 2019-02-04T14:47:43Z

  Merge branch 'jennis/warn_for_nonexistent_domains' into 'master'
  
  Fail when we explictly try to include/exclude non-existent domains in a filter element
  
  See merge request BuildStream/buildstream!1117

• cd8e5e27
  by Tom Pollard at 2019-02-05T11:22:01Z

  Add --remote, -r option to bst build, inline with pull & push
  
  Providing a remote will limit build's pull/push remote actions to
  the given remote specifically, ignoring those defined via user or
  project configuration.
  

• a46650d0
  by Tom Pollard at 2019-02-05T12:41:30Z

  Merge branch 'tpollard/buildremote' into 'master'
  
  Add --remote, -r option to bst build, inline with pull & push
  
  See merge request BuildStream/buildstream!1119

• 9c94e8e5
  by Daniel Silverstone at 2019-02-05T13:15:24Z

  requirements: Add roaringbitmap to the requirements
  
  In order to support use of roaring bitmaps in the loader, we need to
  depend on it here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7682ef49
  by Daniel Silverstone at 2019-02-05T13:15:24Z

  loadelement.py: Use roaring bitmaps in dep cache
  
  The dependency cache in LoadElement causes the peak RAM consumption of
  the loader to be exceedingly large, upwards of 25GB for a test of a
  Debian stack.  By switching from the old dict cache to a roaring bitmap
  based cache, we reduce the cost of the loader cache in RAM terms to
  around 5.5GB for the same stack, which makes it plausible for the short
  term.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 0e471144
  by Benjamin Schubert at 2019-02-05T15:45:37Z

  Merge branch 'danielsilverstone-ct/roaring-bitmaps' into 'master'
  
  Switch to roaring bitmaps for the loader dependency caches
  
  See merge request BuildStream/buildstream!1128

• f17ea6a6
  by Jonathan Maw at 2019-02-06T10:32:05Z

  gitlab-ci: Add commands to be run by WSL runners
  

• d4d2f897
  by Jonathan Maw at 2019-02-06T10:45:34Z

  tests: Skip tests that use sandboxes on WSL
  

• f4c8a0ec
  by Jonathan Maw at 2019-02-06T10:45:34Z

  tests: Fix test failures caused by unix sockets' path length limit
  
  The unix specification for unix sockets only allocates ~100 characters
  for the length of the path. This may be longer than the path for a CI
  runner, or the path to a user's buildstream directory.
  

• 256bcde5
  by Jonathan Maw at 2019-02-06T15:29:18Z

  Merge branch 'jonathan/wsl-tests' into 'master'
  
  Add pre-merge tests that use a WSL runner
  
  Closes #852
  
  See merge request BuildStream/buildstream!1108

• 335ef2b8
  by Daniel Silverstone at 2019-02-06T17:30:13Z

  loadelement.py: Switch from roaringbitmap to pyroaring
  
  The licence for roaringbitmap was not compatible.  pyroaring is
  a similar project which is MIT licensed and is slightly better on
  memory too.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 8f90be91
  by Benjamin Schubert at 2019-02-06T18:36:17Z

  Update testing images to include c++ compiler
  

• 1a75b252
  by Benjamin Schubert at 2019-02-06T19:34:09Z

  Merge branch 'danielsilverstone-ct/other-roaring' into 'master'
  
  loadelement.py: Switch from roaringbitmap to pyroaring
  
  Closes #899
  
  See merge request BuildStream/buildstream!1133

• 7285e87e
  by William Salmon at 2019-02-07T09:42:36Z

  Add more log formatting options
  
  The 'wallclock-us' is a option that we thought we already had.
  

• e61f4713
  by Will Salmon at 2019-02-07T10:58:54Z

  Merge branch 'willsalmon/log_formating' into 'master'
  
  Add more log formatting options
  
  See merge request BuildStream/buildstream!1125

• b4d4c4f5
  by Phil Dawson at 2019-02-08T14:27:54Z

  Expose basic api for testing external plugins.
  
  We want external plugins to be able to make use of the core testing utils.
  This commit exposes the basic utilities which are currently in use in
  bst-external plugins. If necessary, more utilities could be exposed in the
  future.
  
  Moves the following files from tests/testutils/ to
  buildstream/plugintestingutils/:
  o runcli.py
  o integration.py
  
  As part of this, this commit makes the following changes to runcli.py
  and integration.py:
  o runcli.py: Fix linting errors
  o runcli.py: Add user facing documentation
  o Integration.py: Add user facing documentation
  

• 77ce0c16
  by Phil Dawson at 2019-02-08T16:28:05Z

  Merge branch 'phil/plugin-testing-api' into 'master'
  
  Expose basic api for testing external plugins.
  
  Closes #847
  
  See merge request BuildStream/buildstream!1075

• 1ee4a4ba
  by Abderrahim Kitouni at 2019-02-08T16:42:23Z

  _artifactcache.py: don't leak the project specific remote caches
  
  the code for initializing remotes added the project specific remote
  caches to the global list instead of making a copy.
  
  Fixes #618
  

• 24c0de16
  by Abderrahim Kitouni at 2019-02-08T16:42:23Z

  _project.py: use artifact caches from the parent project for junctions
  
  This makes a junction use the artifact cache of the parent project
  before the ones defined for the junction
  
  Fixes #401
  

• a937f99a
  by Javier Jardón at 2019-02-08T20:29:05Z

  Merge branch 'abderrahim/artifact-cache-junction' into 'master'
  
  Use artifact cache specs from the parent project before those defined in junctions
  
  Closes #618 and #401
  
  See merge request BuildStream/buildstream!1113

• da1560e4
  by Javier Jardón at 2019-02-09T09:10:01Z

  .gitlab-ci.yml: Remove testing on Fedora 27
  
  Fedora 27 is EOL since 2018-11-30
  
  See https://fedoraproject.org/wiki/End_of_life
  

• 79fbab61
  by Jürg Billeter at 2019-02-09T10:30:20Z

  Merge branch 'jjardon/fedora_27' into 'master'
  
  .gitlab-ci.yml: Remove testing on Fedora 27
  
  See merge request BuildStream/buildstream!1136

• 354c563d
  by Jürg Billeter at 2019-02-10T20:03:46Z

  import.py: Validate config node
  
  Fixes #662.
  

• 5e1be71f
  by Jürg Billeter at 2019-02-11T05:10:56Z

  Merge branch 'juerg/import' into 'master'
  
  import.py: Validate config node
  
  Closes #662
  
  See merge request BuildStream/buildstream!1141

• 99e1be45
  by Jürg Billeter at 2019-02-11T05:12:25Z

  local.py: Do not follow symlinks in local directories
  
  isdir() follows symlinks on the host, resulting in potential host
  contamination. This change reorders the file checks to avoid this issue.
  

• f95e222e
  by Jürg Billeter at 2019-02-11T05:12:25Z

  sandbox/sandbox.py: Do not follow symlinks in _has_command()
  
  This is required to ensure symlinks are not resolved on the host.
  

• 89973fb3
  by Jürg Billeter at 2019-02-11T05:12:25Z

  utils.py: Remove list_dirs parameter from list_relative_paths()
  
  list_dirs was always True in the BuildStream code base. There was also a
  bug in the list_dirs=False code path as it did not return symlinks in
  `dirnames`.
  
  This is an API break, however, there are no known external callers.
  

• d1da3fb0
  by Jürg Billeter at 2019-02-11T05:12:25Z

  utils.py: Fix sorting of symlinks to directories
  
  os.walk() resolves symlinks to check whether they point to a directory
  even when followlinks is set to False. We already work around that
  broken behavior by extracting symlinks from `dirnames`. However, the
  sort order was still incorrect as we returned symlinks in dirnames
  before files and other symlinks. This change fixes this, sorting all
  files and symlinks in a single list.
  

• 7cf67ed3
  by Jürg Billeter at 2019-02-11T05:12:25Z

  _casbaseddirectory.py: Do not mimic os.walk() in list_relative_paths()
  
  This matches the change in utils.list_relative_paths() that now sorts
  all symlinks as files, instead of following the broken behavior of
  os.walk().
  

• 7ec0bb5e
  by Jürg Billeter at 2019-02-11T05:44:20Z

  tests/sources/local.py: Add directory symlink test
  

• c07cc967
  by Jürg Billeter at 2019-02-11T07:13:28Z

  Merge branch 'juerg/symlinks' into 'master'
  
  Symlink fixes
  
  See merge request BuildStream/buildstream!1138

• 9e3d3e05
  by Angelos Evripiotis at 2019-02-11T07:14:50Z

  contributing: snakeviz replaces pyflame+flamegraph
  
  Replace the instructions for pyflame+flamegraph with simpler ones for
  snakeviz. For our general use-case this seems to be easier and better.
  Usage of this tool was demonstrated at the 2019 BuildStream Gathering in
  January by Daniel Silverstone, when presenting the aggregate results of
  profiling on many target environments.
  
  Here is the relevant mailing list thread:
  
  "Profiling before the gathering"
  https://mail.gnome.org/archives/buildstream-list/2019-January/msg00057.html
  

• bb6a692d
  by Jürg Billeter at 2019-02-11T09:20:34Z

  Merge branch 'snakeviz' into 'master'
  
  contributing: snakeviz replaces pyflame+flamegraph
  
  See merge request BuildStream/buildstream!1129

• 1ed63e54
  by Angelos Evripiotis at 2019-02-11T09:24:48Z

  _includes: better provenance on recursive include
  
  Use the provenance of the include block, instead of the whole node.
  

• 02e48209
  by Angelos Evripiotis at 2019-02-11T09:24:48Z

  _includes: better error on missing include
  
  Previously, a missing include would result in an error like this:
  
      Could not find file at not-a-file.include
  
  Note that the file containing the include was not mentioned.
  
  Now we get an error like this instead:
  
      element.bst [line 7 column 5]: Include block references a file that
      could not be found: 'not-a-file.include'.
  

• 4336e3bf
  by Angelos Evripiotis at 2019-02-11T09:24:48Z

  _includes: better error on including directory
  
  Previously, include a directory result in an error like this:
  
      mydir is a directory. bst command expects a .bst file.
  
  Note that the file containing the include was not mentioned.
  
  Now we get an error like this instead:
  
      element.bst [line 12 column 0]: Include block references a
      directory instead of a file: 'mydir'.
  

• 3f6c5000
  by Angelos Evripiotis at 2019-02-11T09:24:48Z

  _includes: re-use file_path variable
  
  Avoid an unnecessary call to os.path.join().
  

• adde0c94
  by Angelos Evripiotis at 2019-02-11T09:24:48Z

  tests/format/include: remove unused tmpdir's
  
  Don't create and remove temp dirs unnecessarily when they are not used,
  looks like these were just copy-pastes without intended side-effects.
  

• a66f8379
  by Jürg Billeter at 2019-02-11T13:52:54Z

  Merge branch 'aevri/include-error' into 'master'
  
  More user-friendly reporting on include errors
  
  See merge request BuildStream/buildstream!891

• a482b008
  by Javier Jardón at 2019-02-11T19:23:02Z

  .gitlab-ci.yml: Do not automatically run "test-wsl" until runner problems are sorted out
  

• 0816b8b1
  by Jürg Billeter at 2019-02-11T21:29:29Z

  Merge branch 'jjardon/allow_fail_WSL' into 'master'
  
  .gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out
  
  See merge request BuildStream/buildstream!1145

• a7aed65a
  by Jürg Billeter at 2019-02-12T05:52:07Z

  utils.py: Increase buffer size in sha256sum()
  
  Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
  factor 4, according to a very simple benchmark.
  

• 8b9e1d24
  by Jürg Billeter at 2019-02-12T05:52:07Z

  _cas/cascache.py: Increase buffer size in add_object()
  
  Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
  factor 4, according to a very simple benchmark.
  

• 86a9048a
  by Jürg Billeter at 2019-02-12T07:08:18Z

  Merge branch 'juerg/buffer-size' into 'master'
  
  Increase read buffer size to improve performance
  
  See merge request BuildStream/buildstream!1142

• 9b6c18e4
  by Chandan Singh at 2019-02-12T10:20:52Z

  _frontend: Allow printing dependencies using `bst show`
  
  At present, there isn't an easy way to print anything from `bst show`
  that would give the users an idea of what the dependency graph looks
  like. One could use things like `--deps build`, but that will just print
  a list, without any information about the dependency edges.
  
  Add `%{deps}`, `%{build-deps}` and `%{runtime-deps}` format strings to
  `bst show` that would simply print the list of all dependencies, build
  dependencies and runtime dependencies respectively.
  
  Summary of changes:
  
    * buildstream/_frontend/cli.py: Add help for new format symbols.
    * buildstream/_frontend/widget.py: Add support for new format symbols
                                       for dependencies.
    * tests/frontend/show.py: Add tests for new format symbols.
  

• 126bc006
  by Chandan Singh at 2019-02-12T10:23:44Z

  Re-generate man pages
  

• dbce3434
  by Chandan Singh at 2019-02-12T12:19:57Z

  Merge branch 'chandan/deps' into 'master'
  
  _frontend: Allow printing dependencies using `bst show`
  
  Closes #890
  
  See merge request BuildStream/buildstream!1121

• 24d29a17
  by Javier Jardón at 2019-02-12T12:27:42Z

  .gitlab-ci.yml: Add job to run test in current Fedora:29
  

• 80dcf65d
  by Javier Jardón at 2019-02-12T12:27:42Z

  .gitlab-ci.yml: run all generic fedora test in current stable version:29
  

• 58b84905
  by Javier Jardón at 2019-02-12T12:27:42Z

  .gitlab-ci.yml: Remove "tests-python-3.7-stretch" job
  
  Fedora 29 includes python 3.7 so this is not needed anymore
  

• 95d9b9ae
  by Javier Jardón at 2019-02-12T13:31:56Z

  Merge branch 'jjardon/fedora_29' into 'master'
  
  .gitlab-ci.yml: Test with current fedora release: 29
  
  See merge request BuildStream/buildstream!1137

• a39c4767
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Only retrieve provenance in node_get() when needed
  
  We were indiscriminately retrieving the node's provenance data in the
  `node_get()` function which was accounting for approximately a third of
  the total runtime of `node_get()` which dominates pre-scheduler time in
  `bst build`.  This change ameliorates that situation by only retrieving
  the provenance data when it's actually needed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• cda03313
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Reduce use of `isinstance()` in `node_sanitize()`
  
  We know that nodes are typically one of:
  str, list, dict, bool, tuple, NoneType or our ChainMap
  Of these, dict and ChainMap are Mapping, only list is list
  and the rest are returned unchanged.  We can reduce/defer our use
  of isinstance here, dramatically, improving performance.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b6f08e1b
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Remove use of isinstance() in `{node,list}_{chain_,}copy`
  
  A non-trivial proportion of time pre-scheduler in `bst build` is spent
  copying (or chain-copying) nodes.  Approximately a quarter of the time
  spent in that effort is in `isinstance()`.  This removes that CPU load.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 69ee11c6
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Reduce cost of node_final_assertions
  
  By re-using the isinstance replacements from earlier commits and
  using a tuple of the string constants for checking for composition
  markers, we reduce the cost of node_final_assertions by two thirds
  in basic testing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 0928e570
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _context.py: Cache result of get_strict()
  
  While get_strict() doesn't look expensive per-se, it is called so many
  times that it is valuable to cache the result once computed.  Since I
  don't think it can change once it is computable, cache it immediately
  that becomes possible and we save 20s in my test case.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 022a59f0
  by Benjamin Schubert at 2019-02-12T16:14:05Z

  Merge branch 'danielsilverstone-ct/further-optimisations' into 'master'
  
  Further optimisations
  
  See merge request BuildStream/buildstream!1131

• 352f4ad9
  by Jonathan Maw at 2019-02-12T16:44:35Z

  loader.py: Stage junctions into .bst instead of a tmpdir
  

• 62396af9
  by Jonathan Maw at 2019-02-12T16:44:35Z

  loader.py: Load junction from workspace if one's open
  

• f1e9cb66
  by Jonathan Maw at 2019-02-12T16:44:35Z

  Remove cleanup functionality from loader
  
  Since junction check-outs are now persistent, there is no need to clean
  up temporary directories once finished.
  Now, junction checkouts are stored within the top-level project dir and
  will be cleaned-up when that directory is removed.
  
  This involves changes in:
  * loader.py: Remove cleanup logic and passing-around of tmpdir.
  * _project.py: Remove passing around tmpdir and calling loader's
                 cleanup.
  

• e51116d5
  by Jonathan Maw at 2019-02-12T17:59:25Z

  Merge branch 'jonathan/junction-no-tmpdir' into 'master'
  
  Stage junctions into .bst instead of a tmpdir
  
  Closes #895
  
  See merge request BuildStream/buildstream!1134

• 478e5c47
  by Valentin David at 2019-02-12T18:06:50Z

  buildstream/_cas/cascache.py: Set 0644 rights to pulled files
  
  This was broken by 5ef19a0b31df84caed1e41719ef7ea5c6bd8a8bc.
  

• 6de65306
  by Valentin David at 2019-02-12T19:13:43Z

  Merge branch 'valentindavid/pull-chmod-bug' into 'master'
  
  buildstream/_cas/cascache.py: Set 0644 rights to pulled files
  
  See merge request BuildStream/buildstream!1144

• 9b140fa0
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Lift ArtifactCache.get_artifact_fullname() to here
  
  This commit removes the method ArtifactCache.get_artifact_fullname()
  and replaces it with Element.get_artifact_name()
  
  Given a key, we are now able to construct the full name of any of an
  element's artifacts.
  

• 039d7c1d
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Make calculate_cache_key() API private
  

• 7cb6be74
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Return early in __init_defaults if no plugin config
  

• ddf642e1
  by James Ennis at 2019-02-13T09:35:45Z

  metaelement.py: Provide constructor with default keyword arguments
  

• 55c15f8f
  by James Ennis at 2019-02-13T09:35:45Z

  _artifactelement.py: New ArtifactElement object (derived from Element)
  
  This object should be used when we want to handle artifact refs directly
  from the command line.
  An ArtifactElementError has also been added to _exceptions.py
  

• 6951cfc8
  by James Ennis at 2019-02-13T09:35:45Z

  _project.py: Add create_artifact_element() method
  

• bf1933b6
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Add the _classify_artifacts() helper
  

• ea6ff6da
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Modify behaviour of _classify_artifacts()
  
  * There is no need for this method to use a cas object.
  * Search for artifact globs in the project's element path
  * An artifact key is always 64 chars long
  

• fcc79917
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Allow loading to handle artifact refs
  

• 77345317
  by James Ennis at 2019-02-13T09:35:45Z

  _artifactcache.py: Add get_artifacts_log() method
  
  A CasBasedDirectory object of an artifacts logs can be obtained
  with ArtifactCache.get_artifacts_log(). This ultimately calls
  CASCache.get_top_level_dir() to obtain a CasBasedDirectory
  of an artifact's subdirectory (or subdirectories).
  

• 775ac472
  by James Ennis at 2019-02-13T09:35:45Z

  cli.py: Move artifact ref handling logic to stream
  
  The loading of elements and the handling of artifacts does not belong
  in this module. Such logic should be invoked using the Stream API
  

• f95d6ee8
  by James Ennis at 2019-02-13T09:35:45Z

  cli.py: Remove _classify_artifacts() method
  
  This method has been moved to Stream, where is it used there exclusively.
  

• 491937d7
  by James Ennis at 2019-02-13T09:35:45Z

  cascache.py: Allow CASCache.list_refs() to handle globs
  
  This commit ensures that CASCache.list_refs(), and
  ArtifactCache.list_artifacts(), can both handle glob expressions.
  

• 6ae41474
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: _classify_artifacts() should be able to handle globs
  
  _classify_artifacts() no longer filters glob expressions by first
  obtaining a list of ALL refs locally cached. We now only obtain refs
  specified by the glob _expression_.
  
  Furthermore, the Project.element_path is used to start searching for
  globbed elements as opposed to the Project.directory.
  

• 8c6baf23
  by James Ennis at 2019-02-13T10:48:21Z

  Merge branch 'jennis/refactor_artifact_log' into 'master'
  
  Refactor artifact log command
  
  See merge request BuildStream/buildstream!1101

• 14db3287
  by Jürg Billeter at 2019-02-13T10:48:45Z

  projectconfig.yaml: Consistently include directories in split rules
  
  Most split rules already included the relevant directories themselves in
  addition to the directory contents.  Add the missing bin, sbin, and
  libexec directories.
  
  This is required to fix tests with the following commit that changes
  list_relative_paths() to return all directories.
  

• c7625cdf
  by Jürg Billeter at 2019-02-13T10:48:45Z

  tests/integration/project: Add tests directory to split rule
  
  This is required to fix tests with the following commit that changes
  list_relative_paths() to return all directories.
  

• 925429ea
  by Jürg Billeter at 2019-02-13T10:48:45Z

  utils.py: Return all directories in list_relative_paths()
  
  Returning only empty directories leads to inconsistencies when computing
  a manifest by combining results from multiple list_relative_paths()
  calls as done by the compose plugin.
  
  I.e., the same directory may be empty in one dependency and non-empty in
  another dependency.  The merged file list will still contain that
  directory even though it's no longer empty.
  
  This inconsistency causes problems when calculating differences between
  manifests. Returning all directories fixes these inconsistencies.
  
  This is a change in API behavior.
  

• 353745a4
  by Jürg Billeter at 2019-02-13T10:48:45Z

  _casbaseddirectory.py: Return all directories in list_relative_paths()
  
  This matches the change in utils.list_relative_paths().
  

• ae0ee361
  by Jürg Billeter at 2019-02-13T12:42:42Z

  Merge branch 'juerg/list-all-directories' into 'master'
  
  Return all directories in list_relative_paths()
  
  See merge request BuildStream/buildstream!1139

• 118644b2
  by Tom Pollard at 2019-02-13T13:37:32Z

  Add cli main & user conf option for 'cache-buildtrees' context
  
  _context.py: Add cache_buildtrees global user context, the default
  of which is set to by default to 'always' via the addition of
  cache-buildtrees to userconfig.yaml cache group. 'failure' & 'never'
  can be given as valid options.
  
  app.py & cli.py: Add --cache-buildtrees as a bst main option, which
  when passed with a valid option can override the default or user
  defined context for cache_buildtrees.
  
  tests/completions/completions.py: Update for the added flag.
  

• b546bac1
  by Tom Pollard at 2019-02-13T13:37:32Z

  _frontend/cli.py: Ensure failed buildtree warning is correct
  
  not _cached_sucess() could resolve to true if the element wasn't
  cached at all. switch to _cached_failure() to ensure condition
  reflects expected artifact state
  

• d2029444
  by Tom Pollard at 2019-02-13T13:37:32Z

  Provide configuration for the optional creation of buildtrees
  
  Artifacts can be cached explicitly with an empty `build tree` when
  built via the cli main options or user config for all or only
  successful build artifacts. Default behaviour is to still create
  and cache all expected buildtrees.
  
  element.py: _cache_artifact() Check if context for cache_buildtrees
  has been set to always or failure with a corresponding build
  result, if not skip attempting to export the build-root. Element
  types without a build-root are cached with an empty buildtree
  regardless. Update _stage_sources_at() to warn the user that the
  buildtree import is empty.
  
  tests/integration: Add test to artifact.py for the optional caching
  of buildtree content from bst build. Rename build-tree.py to
  shellbuildtrees.py to reflect included test cases, add test for
  empty buildtree warning and failure option.
  
  NEWS: Add entry for new option.
  

• 54ec032a
  by Jürg Billeter at 2019-02-13T14:50:15Z

  Merge branch 'tpollard/896' into 'master'
  
  Optional creation of buildtrees
  
  Closes #896
  
  See merge request BuildStream/buildstream!1135

• ec4bbf35
  by Benjamin Schubert at 2019-02-13T14:59:47Z

  Cleanup internal Loader cache after loading elements
  

• b41a82d3
  by Benjamin Schubert at 2019-02-13T14:59:47Z

  Don't register exceptions when not running the testsuite
  
  This fix a problem with the garbage collector not being able to
  clean the MetaElements that are loaded.
  
  On small projects this is not a problem, but in bigger projects, this
  can save a few hundred of MBs at runtime
  
  The reason behind this is, whenever we have a "stack" element, which
  has no stack.yaml configuration, since it doesn't need it, we would
  get an exception thrown when initiating the first one, as loading the
  yaml file would fail.
  
  This would capture the frame in which this command was executed, which
  references meta_elements. Therefore, as long as another exception is not
  thrown, the garbage collector would not be able to clean all the
  MetaElements.
  

• 9db7f489
  by Benjamin Schubert at 2019-02-13T16:13:16Z

  Merge branch 'bschubert/cleanup-local-state' into 'master'
  
  Cleanup MetaElement local state
  
  See merge request BuildStream/buildstream!1147

• c2c10d68
  by Daniel Silverstone at 2019-02-14T13:02:07Z

  Cache Keys: Update to use JSON rather than pickle
  
  This affects the cache key version (updated to 7) and introduces
  a dependency on `ujson` which is BSD licenced as of the version
  locked in `requirements.txt`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 868c057e
  by Tristan Van Berkom at 2019-02-14T13:58:39Z

  Merge branch 'danielsilverstone-ct/json-cache-key' into 'master'
  
  Update cache keys to use JSON
  
  See merge request BuildStream/buildstream!1151

• ace15fa6
  by Benjamin Schubert at 2019-02-14T14:04:45Z

  Don't keep MetaSource around in Source
  
  We can easily rebiuld MetaSource from any source so there is no
  reason to keep them around.
  
  This will slightly improve memory usage.
  

• 174f5394
  by Tristan Van Berkom at 2019-02-14T15:04:27Z

  Merge branch 'bschubert/dont-keep-metasource' into 'master'
  
  Don't keep MetaSource around in Source
  
  See merge request BuildStream/buildstream!1150

• e7e4086e
  by Jürg Billeter at 2019-02-14T15:05:38Z

  utils.py: Change _ensure_real_directory() to not resolve symlinks
  
  Resolving symlinks during staging causes various issues:
  
  * Split rules may not work properly as the resolved paths will differ
    depending on whether another artifact with a directory symlink has
    been staged in the same root directory or not, e.g., as part of
    compose.
  
  * The order of symlinks in file lists is difficult to get right to
    guarantee consistent and predictable behavior as paths in a file list
    might rely on symlinks in the same file list. See #647 and #817.
  
  * Staging order differences can lead to surprising results. See #390.
  
  * Difficult to properly support absolute symlinks. Absolute symlinks are
    currently converted to relative symlinks, however, this doesn't always
    work. See #606 and #830.
  
  This will require changes in projects that rely on the current behavior.
  However, the changes are expected to be small and are often a sign of
  buggy element files. E.g., elements that don't fully obey `bindir` or
  `sbindir` variables.
  

• c0570871
  by Jürg Billeter at 2019-02-14T15:05:38Z

  _casbaseddirectory.py: Do not resolve symlinks
  
  This matches the change in utils._process_list().
  
  This also removes the _Resolver class as it is now unused. We may want
  to support controlled symlink resolution in the future, in which case
  the _Resolver class can be resurrected from this commit.
  

• 55bccf63
  by Jürg Billeter at 2019-02-14T15:05:38Z

  utils.py: Do not mangle absolute symlinks
  
  Copy symlinks as they are, absolute or relative. We no longer resolve
  symlinks when copying files, which makes this safe.
  

• 94c6d59b
  by Jürg Billeter at 2019-02-14T15:05:38Z

  Bump artifact version for changes in symlink handling
  

• 949962f8
  by Jürg Billeter at 2019-02-14T15:05:38Z

  NEWS: Add entry for change in symlink handling
  

• 56c07baa
  by Jürg Billeter at 2019-02-14T16:08:42Z

  Merge branch 'juerg/symlinks2' into 'master'
  
  Do not resolve or mangle symlinks during staging
  
  See merge request BuildStream/buildstream!1140

• 69520011
  by Chandan Singh at 2019-02-15T06:58:27Z

  contrib/bst-graph: Add script to print graph in DOT format
  
  This script leverages the recently added format strings
  (`%{build-deps}`, `%{runtime-deps}`) to `bst show` to print a graph in
  DOT format. This requires users to have the `graphviz` python package
  installed.
  
  Additionally, users can also render the graph using the `--format`
  option if they have the `graphviz` command line tool installed.
  

• 51dae747
  by Chandan Singh at 2019-02-15T08:50:21Z

  Merge branch 'chandan/dot-graph' into 'master'
  
  contrib/bst-graph: Add script to print graph in DOT format
  
  Closes #705
  
  See merge request BuildStream/buildstream!1148

• 0f78a47d
  by Daniel Silverstone at 2019-02-15T10:03:46Z

  Variables: Rework how expansion strings work
  
  Rather than constantly using regular expressions and retrieval from YAML
  nodes, pre-parse expansion strings into a list representation cached
  for reuse, and then expand them as simple string concatenation.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 14176e51
  by Benjamin Schubert at 2019-02-15T11:04:10Z

  Merge branch 'danielsilverstone-ct/variables-rework' into 'master'
  
  Variables: Rework how expansion strings work
  
  See merge request BuildStream/buildstream!1152

• 39febfda
  by Angelos Evripiotis at 2019-02-15T11:10:51Z

  storage.Directory.export_to_tar: mtime=_magic...
  
  Change the default value of mtime when doing export_to_tar() from `0` to
  `_utils._magic_timestamp`. This avoids problems in other software, which
  assume that an mtime of `0` means the file does not exist.
  
  There is more than one example where files with an mtime of zero are
  treated as non-existant by other software. e.g. [ninja][1] and [Template
  Toolkit][2].
  
  The OSTree project also [express an intention][3] to move from an mtime
  of 0 to an mtime of 1:
  
  > For this reason, OSTree acts as though all timestamps are set to
  > time_t 0, so that comparisons will be considered up-to-date. Note that
  > for a few releases, OSTree used 1 to fix warnings such as GNU Tar
  > emitting "implausibly old time stamp" with 0; however, until we have a
  > mechanism to transition cleanly to 1, for compatibilty OSTree is
  > reverted to use zero again.
  
  >From the comments on export_to_tar(), the motivation for having an mtime
  of 0 was to have reproducible results, rather than it specifically being
  the value 0.
  
  Additionally, the reproducible builds project has a [page on
  archives][4]; it mentions the benefits of setting all the files to have
  the same mtime, or clamping the mtime. It makes no mention of a
  motivation for the mtime to be specifically 0.
  
  Fixes #914
  
  [1]: https://github.com/ninja-build/ninja/issues/1120
  [2]: https://github.com/abw/Template2/blob/8d7d37200af436f1ad43628278d3caad257c8e27/lib/Template/Provider.pm#L635
  [3]: https://ostree.readthedocs.io/en/latest/manual/repo/
  [4]: https://reproducible-builds.org/docs/archives/
  

• 8b34e356
  by Angelos Evripiotis at 2019-02-15T12:25:21Z

  Merge branch 'aevri/mtime1' into 'master'
  
  storage.Directory.export_to_tar: default mtime=utils._magic_timestamp
  
  Closes #914
  
  See merge request BuildStream/buildstream!1149

• 3bbb90e7
  by Benjamin Schubert at 2019-02-15T12:34:36Z

  Use sets when checking for existence of an element
  

• 0921ccf4
  by Benjamin Schubert at 2019-02-15T13:39:52Z

  Merge branch 'bschubert/set-as-set' into 'master'
  
  Use sets when checking for existence of an element
  
  See merge request BuildStream/buildstream!1154

• c24f2971
  by Angelos Evripiotis at 2019-02-15T13:56:48Z

  userconfig: rm really-workspace-close-project-inaccessible
  
  Remove the need for the 'really-workspace-close-project-inaccessible'
  config option, as well as the option itself.
  
  As agreed on the mailing list [1], all the 'are you sure?' prompts on
  workspace reset and close were removed. While that discussion was going
  on, this new prompt and option was added. At the 2019 BuildStream
  Gathering, it was verbally agreed between myself and Tristan VB that we
  would also remove this instance.
  
  It was also agreed that we should have a notice to let the user know
  what they'd done, this was already in place if interactive. Moved it to
  be unconditional so that there's no difference in non-interactive
  behaviour. Made it output to stderr, as it's diagnostic meant for the
  user. Made it the last thing echo'd so it's next to the prompt - it's
  very relevant to what they type next. Added a test to make sure the text
  makes it to stderr in the appropriate case, and not in an inappropriate
  one.
  
  This is the last instance of any prompt configuration, so BuildStream
  can also forget all of that machinery.
  
  [1] https://mail.gnome.org/archives/buildstream-list/2018-December/msg00111.html
  

• 91eb29a6
  by Angelos Evripiotis at 2019-02-15T14:55:06Z

  Merge branch 'are_you_sure2' into 'master'
  
  userconfig: rm really-workspace-close-project-inaccessible
  
  Closes #726 and #744
  
  See merge request BuildStream/buildstream!1130

• b8561fb1
  by Chandan Singh at 2019-02-15T14:58:22Z

  contrib/bst-here: Change default image to buildstream/buildstream
  
  See https://gitlab.com/BuildStream/buildstream-docker-images/issues/26
  for detailed discussion around this.
  
  `buildstream/buildstream-fedora` is now considered deprecated. Switch to
  `buildstream/buildstream` image. This image also offers more tags that
  will provide users more flexibility.
  

• 2699c77c
  by Chandan Singh at 2019-02-15T14:58:31Z

  contrib/bst-here: Allow users to specify image variant
  
  Now that the `buildstream/buildstream` image has 9 variants, let's make
  it easier to choose the desired tag, using a command-line option.
  
  This is otherwise possible by specifying the full image name `image:tag`
  using the `-i` option. But, this will make it easier to specify just the
  tag using `-j`.
  
  The following two invocations of `bst-here` are now equivalent:
  
      bst-here -i buildstream/buildstream:dev
      bst-here -j dev
  

• 677fc6c5
  by Chandan Singh at 2019-02-15T16:09:21Z

  Merge branch 'chandan/bst-here-tags' into 'master'
  
  contrib/bst-here: Allow users to specify image variant
  
  See merge request BuildStream/buildstream!1153

• 3b889eb8
  by Javier Jardón at 2019-02-15T20:22:08Z

  README.rst: Add table with distros with packaged buildstream
  
  [ci skip]
  

• 232906b1
  by Javier Jardón at 2019-02-15T23:08:50Z

  Merge branch 'jjardon/distros' into 'master'
  
  README.rst: Add table with distros with packaged buildstream
  
  See merge request BuildStream/buildstream!1143

• ee2296ab
  by Tristan Van Berkom at 2019-02-18T08:41:57Z

  CONTRIBUTING.rst: Documenting the release process
  
  So that other people can also make releases.
  

• c79696fe
  by Tristan Van Berkom at 2019-02-18T09:43:22Z

  Merge branch 'tristan/document-release-process' into 'master'
  
  CONTRIBUTING.rst: Documenting the release process
  
  See merge request BuildStream/buildstream!1155

• fb65af6c
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Fix order in _recalculate_recursing_up()
  
  The directory needs to be serialized after the Digest for the
  subdirectory `caller` has been updated.
  

• f5660fa0
  by Jim MacArthur at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Correct typo for _get_underlying_directory
  

• cdcf0dc5
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Remove unused find_root() method
  

• f9dd6ea2
  by Jim MacArthur at 2019-02-18T10:29:25Z

  storage: Add Directory.get_size()
  

• 050249bb
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Add _get_digest() method
  

• 3b881efc
  by Jürg Billeter at 2019-02-18T10:29:25Z

  element.py: Expand scope of timed_activity in _cache_artifact()
  
  ArtifactCache.commit() is no longer the only expensive part and even
  more so with the upcoming change to use CASBasedDirectory.
  

• 3832c0d1
  by Jim MacArthur at 2019-02-18T10:29:25Z

  element.py: Alter _cache_artifact to use CasBasedDirectory
  

• ef85e3b2
  by Jürg Billeter at 2019-02-18T11:18:10Z

  Merge branch 'jmac/cache_artifacts_with_vdir' into 'master'
  
  Cache artifacts with virtual directories instead of filesystem.
  
  Closes #787
  
  See merge request BuildStream/buildstream!991

• e0f0a01d
  by Chandan Singh at 2019-02-18T12:41:31Z

  requirements/Makefile: exclude pkg-resources
  
  Having `pkg-resources` in requirements files is never the right thing as
  it is not really a package that one can install. This is basically an
  artifact of Debian-based systems providing incorrect metadata to pip.
  See [this pip issue](https://github.com/pypa/pip/issues/4022) and
  [this Ubuntu issue](https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1635463)
  for more background.
  

• f149fed2
  by Tristan Van Berkom at 2019-02-18T13:43:28Z

  Merge branch 'chandan/remove-pkg-resources' into 'master'
  
  requirements/Makefile: exclude pkg-resources
  
  Closes #918
  
  See merge request BuildStream/buildstream!1156

• 4022234e
  by Tristan Van Berkom at 2019-02-18T13:44:29Z

  .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.
  
  The test jobs which create coverage reports need to be listed
  in the dependencies of the coverage job, as highlighted in
  issue #356.
  

• 86466e7e
  by Tristan Van Berkom at 2019-02-18T14:27:08Z

  Merge branch 'tristan/collect-missing-coverage' into 'master'
  
  .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.
  
  See merge request BuildStream/buildstream!1157

• 506a8f75
  by Phil Dawson at 2019-02-19T15:27:08Z

  plugin.py: Add API to allow plugins to raise deprecation warnings
  
  A plugin's deprecation warning may be silenced by a project by adding
  the plugin to the list 'supress-deprecation-warnings' in the project's
  project.conf
  

28 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• NEWS
• README.rst
• buildstream/_artifactcache.py
• + buildstream/_artifactelement.py
• buildstream/_cachekey.py
• buildstream/_cas/cascache.py
• buildstream/_cas/casserver.py
• buildstream/_context.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/status.py
• buildstream/_frontend/widget.py
• buildstream/_gitsourcebase.py
• buildstream/_includes.py
• buildstream/_loader/loadelement.py
• buildstream/_loader/loader.py
• buildstream/_loader/metaelement.py
• buildstream/_pipeline.py
• buildstream/_profile.py
• buildstream/_project.py
• buildstream/_scheduler/jobs/cleanupjob.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/buildqueue.py
• buildstream/_scheduler/queues/queue.py
• buildstream/_scheduler/resources.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.