[Notes] [Git][BuildStream/buildstream][chandan/bst-checkout-build] 310 commits: README: use correct links for documentation website

[Benjamin Schubert <gitlab mg gitlab com>]

Title: GitLab

Benjamin Schubert pushed to branch chandan/bst-checkout-build at BuildStream / buildstream

Commits:

• edc07536
  by Tiago Gomes at 2018-10-17T13:49:15Z

  README: use correct links for documentation website
  

• 155296a7
  by Tiago Gomes at 2018-10-17T13:49:15Z

  setup.py: use correct link for documentation website
  

• 131f95eb
  by Tiago Gomes at 2018-10-17T14:15:38Z

  Merge branch 'tiagogones/update-links-to-documentation' into 'master'
  
  Update links to documentation website
  
  See merge request BuildStream/buildstream!876

• c327d1f3
  by Valentin David at 2018-10-17T14:33:40Z

  Change URL to the Alpine tarball
  

• eeb334f2
  by Javier Jardón at 2018-10-17T15:03:17Z

  Merge branch 'valentindavid/move_alpine_tarball' into 'master'
  
  Change URL to the Alpine tarball
  
  See merge request BuildStream/buildstream!880

• b8f69946
  by Daniel Silverstone at 2018-10-18T09:32:09Z

  _loader/loader.py: Be more explanatory in CircDep exception
  
  When a circular dependency is detected, the reported error hilights
  only one part of the loop in question.  This change ensures that
  the entire loop is listed so that the user has a greater chance of
  easily determining the correct fix.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 892866e1
  by richardmaw-codethink at 2018-10-18T10:04:32Z

  Merge branch 'danielsilverstone-ct/report-loop' into 'master'
  
  _loader/loader.py: Be more explanatory in CircDep exception
  
  See merge request BuildStream/buildstream!878

• f1aa0974
  by Javier Jardón at 2018-10-18T14:13:35Z

  Fix python warnings: Use collections.abc instead collections
  
  Since python 3.3, collections has been moved to collections.abc module.
  For backwards compatibility, they continue to be visible in this module through Python 3.7.
  Subsequently, they will be removed entirely.
  
  See https://docs.python.org/3/library/collections.html
  

• f1550c8e
  by Valentin David at 2018-10-18T16:02:45Z

  Merge branch 'jjardon/warnings_collections' into 'master'
  
  Fix python warnings: Use collections.abc instead collections
  
  See merge request BuildStream/buildstream!883

• 1c24979a
  by Angelos Evripiotis at 2018-10-19T21:17:01Z

  Spelling fixes: 'affect', not 'effect' as verb
  
  Fix most instances of 'effect' being used as a verb instead of a noun.
  Skipped generated instance in doc/source/conf.py.
  

• 43a2eee3
  by Angelos Evripiotis at 2018-10-19T21:48:59Z

  Merge branch 'aevri/affect' into 'master'
  
  Spelling fixes: 'affect', not 'effect' as verb
  
  See merge request BuildStream/buildstream!885

• 12719f0d
  by Jürg Billeter at 2018-10-22T17:05:41Z

  _artifactcache: Rename update_atime() to update_mtime()
  
  os.utime() updates both, however, list_artifacts() sorts refs by mtime,
  i.e., atime is irrelevant.
  

• a7a28d14
  by Jürg Billeter at 2018-10-22T17:05:41Z

  _artifactcache: Fix ref in update_mtime()
  
  get_artifact_fullname() is required to construct the ref. The cache key
  alone does not suffice.
  

• be29e0f5
  by Jürg Billeter at 2018-10-22T17:53:26Z

  Merge branch 'juerg/cas-mtime' into 'master'
  
  _artifactcache: Fix ref in update_mtime()
  
  See merge request BuildStream/buildstream!882

• b74aca1a
  by Jürg Billeter at 2018-10-23T09:22:19Z

  tests/testutils/artifactshare.py: Fix has_artifact() to match core code
  
  Replace path separator with '-' to match Element.normal_name.
  

• c7dda150
  by Jürg Billeter at 2018-10-23T09:48:00Z

  Merge branch 'juerg/test-artifactshare' into 'master'
  
  tests/testutils/artifactshare.py: Fix has_artifact() to match core code
  
  See merge request BuildStream/buildstream!890

• 74c115b9
  by Angelos Evripiotis at 2018-10-23T10:07:31Z

  plugins/sources: refactor, rm needless ExitStack-s
  
  Remove two uses of ExitStack that aren't necessary and make the code
  a little harder to follow.
  

• ecb58b42
  by Phil Dawson at 2018-10-23T10:33:47Z

  Merge branch 'aevri/rm-exitstack' into 'master'
  
  plugins/sources: refactor, rm needless ExitStack-s
  
  See merge request BuildStream/buildstream!888

• aa0cbf5d
  by Martin Blanchard at 2018-10-23T10:54:40Z

  _sandboxremote.py: Try to reopen operation steam on failure
  
  The REAPI allows a client to reconnect to an ongoing operation stream by
  providing a WaitExecution(). If implemented on server side, BuildStream
  will try to recover from connection errors using it.
  
  https://gitlab.com/BuildStream/buildstream/issues/630
  

• 552f5fc6
  by Jim MacArthur at 2018-10-23T11:19:48Z

  Merge branch 'mablanch/630-remote-execution-reconn' into 'master'
  
  Handle connection losses during remote build execution
  
  See merge request BuildStream/buildstream!806

• 276d2195
  by Angelos Evripiotis at 2018-10-25T10:17:49Z

  _downloadablefilesource: handle ValueError-s
  

• 2d012a21
  by Angelos Evripiotis at 2018-10-25T10:17:49Z

  Merge branch 'aevri/unknown-uri-452' into 'master'
  
  _downloadablefilesource: handle ValueError-s
  
  Closes #452
  
  See merge request BuildStream/buildstream!893

• cf4741ce
  by James Ennis at 2018-10-25T13:05:43Z

  docs: Add helpful notes to artifact server configuration instructions
  

• 5737bb06
  by James Ennis at 2018-10-25T13:05:43Z

  using_configuring_artifact_server.rst: Fix incorrect reference to bst push (with link)
  

• 28317622
  by James Ennis at 2018-10-25T13:35:22Z

  Merge branch 'jennis/patch_remote_cache_docs' into 'master'
  
  Improve our documentation for configuring an artifact server
  
  See merge request BuildStream/buildstream!899

• d2743c52
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _context.py: Remove useless `return` from `Context.message`
  
  This pointless bare `return` was causing modern pylint to raise an
  error.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 95adf09e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _variables.py: Fix lint issue in `find_recursive_variable`
  
  The python `for`/`else` construct only makes sense if the `for` loop body
  has a `break` statement.  Otherwise the use of `else` causes pylint
  to raise an error.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 9359c067
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _workspaces.py: Simplify chained comparison
  
  In `_parse_workspace_config` there was a chained version comparison
  which is simplified in this commit to quieten a pylint issue.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 796bd8c9
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yaml.py: use `in (a,b)` to simply boolean checks
  
  Where we use a construct `val == foo or val == bar` we can instead use
  `val in (foo, bar)` which pylint prefers.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5a3a537f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yaml.py: Quieten a lint in Python < 3.6
  
  For Python before 3.6, `path.resolve()` could not take the `strict`
  keyword argument.  Linting on such Python versions will raise an unnecessary
  issue given the check present.  As such, quieten that lint.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b279c730
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Correct bug in `YamlCache._get_filepath()`
  
  The correct file path was computed but never returned.  This was
  never caught because of a test missing consumption of one of its
  parameters.  This commit resolves the bug mentioned in #722 leaving
  the test to be amended by a future commit.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 527f3b32
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Correct braino in variable name
  
  This braino would mean that we'd fail to raise the right exception
  if we failed to find the right project object when unpickling the
  yaml cache.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 6d0c697d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Remove unused imports
  
  The imports of _cachekey and utils were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 32323695
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  buildelement.py: Quieten a lint about `__commands`
  
  Unfortunately elements don't override `__init__` in general, so
  we have to quieten pylint here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• c5c0654d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  element.py: Simplify some conditions with `in (foo, bar)`
  
  Where we have conditions of the form `var == foo or var == bar` it
  can be simplified to `var in (foo, bar)` which pylint prefers.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 4f84e2f6
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  element.py: Silence pylint warning about abstract method
  
  Unfortunately pylint can't understand that we'll always use subclasses
  of Element and as such complains of a no-return function (`assemble()`)
  having its return value used.  This quietens that warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b4645745
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  plugin.py: use dict.get() instead of an if statement
  
  Pylint recommends the use of dict.get() rather than a multi-line if
  statement for handling optional keyword arguments.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 68ec9bcf
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Quiet several pylint issues with abstract methods
  
  A number of times, abstract methods are called which pylint cannot
  possibly understand will only ever happen in subclasses which implement
  the abstract methods.  This silences those specific warnings
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1bf0a031
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Remove unused local variable
  
  Remove `source_kind` as it was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e9c6db1e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Remove unused variable
  
  Remove the `context` variable which was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5a187a1e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  utils.py: Disable lint warning about popen
  
  We use `preexec_fn` in our use of Popen.  Since there are threads
  in the codebase, this is not necessarily safe.  However we go to
  great lengths to ensure that the main process doesn't spawn threads
  (they're used in gRPC a lot) and so it should be safe for our use.
  As such, we disable the lint here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b368f57f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  utils.py: Group stat imports
  
  The imports of `stat` and `S_ISDIR` from `stat` should be grouped to
  satisfy pylint.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b45fea19
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/artifactcache.py: Remove superfluous parens
  
  The pylint report dislikes the superfluous parens which used to
  be here in this assert.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 3e4a6507
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/artifactcache.py: Silence lint issues
  
  The pylint tool is unable to understand that the abstract methods
  in the artifact cache will only be called when there's concrete
  implementations behind them.  Silence the lint errors for these
  specific calls.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 447780b0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _frontend/app.py: Remove unused import of `resource`
  
  The `resource` import was unused.  Removed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 247b9fa8
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _frontend/status.py: Reorder imports
  
  The `curses` import is "standard" whereas `click` is not.  As such, despite
  alphabetical order normally being the rule, `curses` must come before `click`.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 522867c8
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaults
  
  Since default values in arguments to functions and methods are created once
  at the compilation of the code, it is dangerous to include list and dict
  literals in them.  This changes the use of {} to None.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 31a7bce5
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _fuse/mount.py: Deal with abstract method lint issue
  
  Unfortunately pylint can't tell this is an abstract method which will
  only ever end up invoked from a class which concretises it.  As such
  we need to silence this warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• eb805c6c
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _options/optionbool.py: Refactor conditions
  
  The pylint tool prefers that conditions of the form "var == foo or var == bar"
  be refactored into "var in (foo, bar)".
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• aca9a612
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/darwin.py: Remove useless __init__()
  
  An otherwise empty __init__ passing no arguments to super.__init__
  is worthless.  Remove it.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 77b1506f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/darwin.py: Remove unused imports
  
  The import of `resource` was unused.
  The import of `PlatformError` was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 70000b0d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/linux.py: Remove unused imports
  
  The imports of `Message` and `MessageType` were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• bab617d0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/linux.py: Simplify return statement
  
  The linter dislikes an if/return/else/return where it can be
  simplified to a return of the if condition.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 45a1d15f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/jobs/job.py: Correct lint issue around abstract method
  
  The linter cannot detect abstract methods the way we use them, so
  this silences the warning which ensues.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e9154b4d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/queue/pullqueue.py: Remove unused return value
  
  The `done()` method does not return anything anywhere else.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7b082bb7
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/queue/trackqueue.py: Remove unused import
  
  The `SourceError` import was never used
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• f1cfc0b7
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  plugins/source/git.py: Rationalised unused return values
  
  The `assert_ref_in_track()` method's return value was never used so
  rationalise it to never return a value.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7c72d25b
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_mount.py: Do not use dict literals in argument defaults
  
  The use of dictionary literals in argument defaults is disrecommended
  due to the way that they are static and thus potentially very confusing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 4e7408ee
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_mounter.py: Remove useless inheritance on `object`
  
  In Python 3, inheriting from `object` explicitly is pointless.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• df495d16
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxbwrap.py: Disable lint about dict.get
  
  Sometimes `dict.get()` is preferable, in this instance it's less clear
  so we'll disable that lint
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 63c6e6b1
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about unused variable
  
  The variable was unused, remove it.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 3394741a
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about preexec_fn
  
  We are super-careful to not use threads in places where we might use
  Popen and as such this warning is save to quash.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• cd20e7a1
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about dict.get
  
  Sometimes `dict.get()` is preferable to an `if` statement, but this time
  it's clearer if we keep the structure.  As such, silence the warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5087f5c5
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Refactor slightly for indentation
  
  Correct some multi-line conditional indentation by slight refactor
  if necessary.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• d73d965b
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Silence len(SEQUENCE) as condition warnings
  
  The lint tool dislikes the use of `if len(SEQ) == 0` type statements
  because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• fa740402
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Prepare attributes in __init__
  
  Python linters prefer that attributes are defined in __init__ rather
  than left to later routines.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• ad8271f0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Remove unused imports
  
  The imports of `Message` and `MessageType` were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1445a901
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Cleanup lint warning about indexing
  
  In order to quieten the linter, and to make a more correct comparison,
  we switch to using the `Code.OK` enumeration from `google.rpc.Code`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 19df07ea
  by Tristan Maat at 2018-10-25T14:43:52Z

  Bump CI image tags
  
  We need to do this so that the new linting errors pop up
  

• 64dbe9ae
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  MANIFEST: Include a lot more missing stuff in the manifest
  
  Versioneer needs to be in the MANIFEST.in and its .pyc needs to be ignored
  in the .gitignore.  Also much docs were not being included, nor conftest.py.
  Much of the test suite data files needed including, so changed that to a
  raw include of everything in the tests/ tree.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• f6bc25dd
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  .gitlab-ci.yml: Change how coverage is recovered
  
  We need to alter how coverage data is recovered in order to work with
  newer coverage tooling.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1c1d14e7
  by Chandan Singh at 2018-10-25T15:04:00Z

  Merge branch 'danielsilverstone-ct/fix-lint-issues' into 'master'
  
  Fix lint issues
  
  See merge request BuildStream/buildstream!892

• 23140b6b
  by Chandan Singh at 2018-10-25T15:25:22Z

  sources: Don't redundantly document directory configuration
  
  Instead of redundantly documenting "directory" configuration option in
  each source plugin, refer to the built-in functionality documentation of
  the Source class.
  
    * source.py: Add anchor for built-in functionality.
    * sources/plugins/*.py: Remove directory config option, and add reference to
      Source's built-in functionality.
  
  Fixes #713.
  

• db2a676e
  by Chandan Singh at 2018-10-25T15:26:39Z

  build elements: Add reference to BuildElement's built-in config
  
  For each element plugin derived from BuildElement, add reference to the
  built-in functionality documentation of BuildElement.
  
    * buildelement.py: Add anchor for built-in functionality.
    * plugins/elements/*.py: Add refence to BuildElement's built-in
      functionality for each build element.
  
  Related to #713.
  

• 356d84cd
  by Chandan Singh at 2018-10-25T16:59:13Z

  Merge branch 'chandan/source-doc-directory' into 'master'
  
  plugins: Don't redundantly document common configurations
  
  Closes #713
  
  See merge request BuildStream/buildstream!896

• 8c41b5b8
  by Daniel Silverstone at 2018-10-26T09:02:30Z

  _yaml.py: Do not insert into cache if retrieved from cache
  
  We were unconditionally inserting into the yaml cache anything we had
  retrieved from the yaml cache.  This had the side-effect of being expensive
  on re-runs of commands where yaml hadn't changed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 494d7018
  by Jonathan Maw at 2018-10-26T09:26:36Z

  Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'
  
  _yaml.py: Do not insert into cache if retrieved from cache
  
  See merge request BuildStream/buildstream!902

• 80762ecb
  by Benjamin Schubert at 2018-10-26T09:45:39Z

  Check if command is a str and replace by list before checking existence
  
  Currently we would make sure the sandbox had a command before
  converting it to a list if it was given as a string. That meant that a string
  command would never exist and the check be invalid.
  
  This also adds the same logic in the dummy sandbox for consistency.
  

• f131c407
  by Benjamin Schubert at 2018-10-26T10:09:51Z

  Merge branch 'bschubert/fix-command-sandbox' into 'master'
  
  Check is command is a str and replace by list before checking existence
  
  Closes #728
  
  See merge request BuildStream/buildstream!898

• 54d89aa1
  by Daniel Silverstone at 2018-10-26T10:20:36Z

  _loader/loadelement.py: Use enumerate() in dependency extraction
  
  In cases where elements end up with large numbers of dependencies,
  the n-squared performance in `_extract_depends_from_node` can produce
  significant delays during project load.  This patch corrects that.
  In addition we were double-retrieving provenance which was unnecessary.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e4c18ba4
  by Phil Dawson at 2018-10-26T10:43:55Z

  Merge branch 'danielsilverstone-ct/use-enumerate' into 'master'
  
  _loader/loadelement.py: Use enumerate() in dependency extraction
  
  See merge request BuildStream/buildstream!901

• 20e5c58a
  by James Ennis at 2018-10-26T11:14:58Z

  using_configuring_artifact_server.rst: Fix lines that are too long
  

• 320dae02
  by James Ennis at 2018-10-26T11:37:42Z

  docs: Add more information to "managing the cache with systemmd" section
  

• 1eb993bf
  by James Ennis at 2018-10-26T12:02:49Z

  Merge branch 'jennis/another_docs_patch' into 'master'
  
  More information regarding the use of systemd to manage the cache
  
  See merge request BuildStream/buildstream!904

• f069d82f
  by Daniel Silverstone at 2018-10-26T12:19:58Z

  various: Move _sentinel from utils.py to _yaml.py
  
  The `_sentinel` in `utils.py` was used only for yaml node tracking.
  As such, simplify matters by removing it from `utils.py` and move it
  to `_yaml.py` which means that we no longer need to try and avoid a
  circular import issue by means of runtime importing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 283887a5
  by James Ennis at 2018-10-26T13:12:12Z

  Merge branch 'danielsilverstone-ct/yaml-sentinel-rework' into 'master'
  
  Move _sentinel from utils.py to _yaml.py
  
  See merge request BuildStream/buildstream!903

• 2c6cb230
  by Valentin David at 2018-10-26T14:21:18Z

  Fix some ruamel warnings during tests
  

• a8250ca4
  by Valentin David at 2018-10-26T15:04:25Z

  Merge branch 'valentindavid/ruamel_warnings' into 'master'
  
  Fix some ruamel warnings during tests
  
  See merge request BuildStream/buildstream!889

• 1b308fe3
  by Daniel Silverstone at 2018-10-26T15:50:42Z

  _stream.py: Preserve stdout FD across checkout-to-stdout
  
  In order to support things which might need to use the stdout FD
  after checkout has completed writing a tarball to it, preserve it
  by means of `os.dup()` and `os.dup2()`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 65d90c31
  by Daniel Silverstone at 2018-10-26T15:50:54Z

  tests/testutils/runcli.py: Support binary-mode capture of stdout
  
  In order to test things which write tarballs to stdout correctly,
  we need to capture the binary output cleanly.  This ensures we're not
  potentially mismatching encodings in and out.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• ed733f3e
  by Daniel Silverstone at 2018-10-26T15:51:40Z

  tests/frontend/buildcheckout.py: Enable test_build_checkout_tarball_stdout
  
  With the new support for saving the stdout FDs cleanly, and binary mode
  capture in runcli, enable the tarball-to-stdout test.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e97d2a75
  by Benjamin Schubert at 2018-10-29T10:17:07Z

  Merge branch 'bschubert/fix-binary-output-capture' into 'master'
  
  Don't close stdout when running 'checkout' to stdout
  
  Closes #486
  
  See merge request BuildStream/buildstream!905

• 1625de0e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/conf.py: Added `images` directory for statically included files
  

• 38507b4e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/format_intro.rst: Adding link anchor for include directives
  

• 26164bcd
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  MANIFEST.in: Include SVG and ODG files in source distributions
  

• 3a23c8f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document with initial "Overview of modules" section
  

• 41d370f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the data model
  

• bed7f7ac
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the dependency model
  

• 9bdf7f40
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about highlevel program flow
  

• 6b8bdf35
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about how the scheduler works.
  

• b1fadaa5
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Moved cachekey and sandboxing writings into the architecture section
  

• baf6b578
  by Tristan Van Berkom at 2018-10-29T14:28:41Z

  Merge branch 'tristan/architecture-docs' into 'master'
  
  Tristan/architecture docs
  
  See merge request BuildStream/buildstream!907

• 09ef8b25
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element.py: Always clean up the rootdir
  
  We shouldn't need it to persist now that we cache failed build dirs.
  
  This change breaks the test
  `tests/integration/shell.py::test_sysroot_workspace_visible`.
  I can no longer see a use-case for this test.
  AIUI, it tested that the failed build sysroot stored in the builddir has
  the workspace's files in, despite the workspace being unmounted.
  I believe this behaviour is made redundant by cached buildtrees.
  
  This fixes part of #539
  

• 37afde0a
  by Jonathan Maw at 2018-10-30T15:42:05Z

  Element: Use cached buildtree in build shells and failure shells
  
  This includes changes in app.py:
  * Interactive failure shell no longer uses the failed build sysroot,
    defaulting to the cached build tree.
  
  Changes in element.py are:
  * Errors caused by building don't store the failed build sysroot,
    instead storing that a sandbox can be created to debug the error.
  * When staging sources, will stage the element's cached build tree if it
    exists.
  
  Changes in _exceptions.py:
  * BstError.sandbox is now a flag of whether a sandbox can be opened up
    to debug the error.
  
  Changes in widget.py:
  * Don't try to print any information about the sandbox.
  
  Changes in _message.py:
  * Fix documentation so Message.sandbox is not a directory any more.
  
  This is part of #539
  

• 4bc71773
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for cached buildtree behaviour
  

• 9b89564f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add test that cached build trees are staged in build shells
  
  This is related to #539
  

• ec76cbe1
  by Jonathan Maw at 2018-10-30T15:42:05Z

  sandbox.py: Remove redundant Sandbox.__directory
  

• 825963b5
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element: Make "--sysroot" take a bare directory
  
  i.e. instead of taking a directory that must contain "root" and
  "scratch", and treating "root" as the root, use the directory directly.
  
  In element.py:
  * __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor
  
  In sandbox.py:
  * If bare_sandbox, `_root` is the passed-in directory, and `__scratch`
    is None.
  * Trying to use `__scratch` when bare_sandbox is True is a bug.
  
  In _mount.py:
  * Don't get the value of `__scratch` if it's not needed.
  
  This is part of #539
  

• cf721838
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for bst shell --sysroot changes
  

• 79d1bb7f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add tests for 'bst shell --sysroot'
  
  This is related to #539
  

• c7ac7e7d
  by Jonathan Maw at 2018-10-30T16:19:17Z

  Merge branch 'jonathan/debug-remote-failed-builds' into 'master'
  
  Jonathan/debug remote failed builds
  
  See merge request BuildStream/buildstream!869

• 516e990e
  by ctolentino8 at 2018-10-31T11:36:46Z

  _frontend/app.py: Set correct element-path in interactive bst-init
  

• b8a37a63
  by Tristan Van Berkom at 2018-11-01T10:16:25Z

  Merge branch 'chiaratolentino/bst-init-interactive-elementpath' into 'master'
  
  _frontend/app.py: Set correct element-path in interactive bst-init
  
  See merge request BuildStream/buildstream!912

• b27b592a
  by Benjamin Schubert at 2018-11-01T10:49:57Z

  Remove dependency on pytest-runner
  
  This includes a new command mimicking pytest-runner so that we
  can drop this dependency
  
  This was the only setup_requires dependency that we had and
  will make like easier for people behind proxies
  

• 89ace5d7
  by Benjamin Schubert at 2018-11-01T11:16:36Z

  Merge branch 'bschubert/remove-pytest-runner' into 'master'
  
  Remove dependency on pytest-runner
  
  See merge request BuildStream/buildstream!886

• 4cfabce8
  by Angelos Evripiotis at 2018-11-01T11:35:02Z

  BREAK: manual.yaml: don't set any default env vars
  
  Remove the 'MAKEFLAGS' and 'V' defaults.
  
  Now that there is a specialised 'make' element, these make-specific
  defaults don't make sense here. This element is meant to be a blank
  slate for folks to build on.
  
  Note that this is a breaking change, that will require some users to
  make changes to their .bst files if they are expecting these environment
  variable to be set.
  
  _versions.py: BST_FORMAT_VERSION bumped to 18.
  
  This fixes issue #718
  

• 48860aac
  by Tristan Van Berkom at 2018-11-01T12:01:04Z

  Merge branch 'tristan/empty-manual-element' into 'master'
  
  BREAK: manual.yaml: don't set any default env vars
  
  Closes #718
  
  See merge request BuildStream/buildstream!916

• d868b409
  by Daniel Silverstone at 2018-11-01T13:40:24Z

  _yaml.py: Implement `get()` for `ChainMap`
  
  Since the core Python `ChainMap.get()` implements with:
  
      self[key] if key in self else default
  
  The double-chain-lookup is expensive.  This simple change solves that
  for our ChainMap derived structure.  As such it improves matters
  for #466 somewhat.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7f79b9ce
  by Tristan Van Berkom at 2018-11-01T14:25:57Z

  Merge branch 'danielsilverstone-ct/improve-chainmap-get' into 'master'
  
  _yaml.py: Implement `get()` for `ChainMap`
  
  See merge request BuildStream/buildstream!919

• de59ebdb
  by ctolentino8 at 2018-11-02T16:41:54Z

  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  

• 8d7cf806
  by ctolentino8 at 2018-11-02T16:41:54Z

  tests/integration/project/files: Remove pypi-repo
  

• 9c2f9bf7
  by Chandan Singh at 2018-11-02T17:09:46Z

  Merge branch 'chiaratolentino/fix-pip-source-regex' into 'master'
  
  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  
  See merge request BuildStream/buildstream!914

• 3788e701
  by Jürg Billeter at 2018-11-03T11:52:00Z

  element.py: Pass bare_directory also to SandboxRemote()
  
  Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
  

• 82e971ef
  by Jürg Billeter at 2018-11-05T11:33:20Z

  Merge branch 'juerg/remote-execution' into 'master'
  
  element.py: Pass bare_directory also to SandboxRemote()
  
  See merge request BuildStream/buildstream!928

• 62942bfd
  by Valentin David at 2018-11-05T12:14:20Z

  Use less fetchers in overnight build to work-around limitations from servers.
  
  Issue happens with sourceware.org server.
  
  Fixes #742
  

• 442da2f9
  by Javier Jardón at 2018-11-05T12:41:54Z

  Merge branch 'valentindavid/limit_fetchers_in_overnight_build' into 'master'
  
  Use less fetchers in overnight build to work-around limitations from servers.
  
  Closes #742
  
  See merge request BuildStream/buildstream!929

• 0993de53
  by Richard Maw at 2018-11-05T16:41:05Z

  dev-requirements: Update pytest dependency to 3.8
  
  "_frontend/app.py: Set correct element-path in interactive bst-init"
  added a dependency on the tmp_path test fixture, which was added by 3.8.
  

• be8f0a54
  by richardmaw-codethink at 2018-11-05T17:08:43Z

  Merge branch 'richardmaw/pytest-dep-update' into 'master'
  
  dev-requirements: Update pytest dependency to 3.8
  
  See merge request BuildStream/buildstream!930

• bfb639bf
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove artifacts parameter from constructor
  
  Get the artifact cache from the context instead.
  

• ca855f91
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _elementfactory.py: Remove unused artifacts parameter from create()
  

• 15fed21c
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from create_element()
  

• 0085d2aa
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove unused artifacts parameter from _new_from_meta()
  

• f69b1117
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from load_elements()
  

• e398f877
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Do not include type name of artifact cache in cache key
  
  The artifact cache backend does not affect build outputs and we anyway
  no longer have pluggable artifact cache backends. This hardcodes
  CASCache instead of removing the entry completely to avoid cache key
  changes.
  

• 626d20ae
  by Jürg Billeter at 2018-11-05T17:18:12Z

  Split up artifact cache and CAS cache
  
  This changes CASCache from a subclass to a delegate object of
  ArtifactCache. As the lower layer, CASCache no longer deals with
  elements or projects.
  
  Fixes #659.
  

• ec04446b
  by Jürg Billeter at 2018-11-05T17:51:39Z

  Merge branch 'juerg/cas' into 'master'
  
  Split up artifact cache and CAS cache
  
  Closes #659
  
  See merge request BuildStream/buildstream!922

• 9a045080
  by Josh Smith at 2018-11-06T13:23:19Z

  utils: Fix _pretty_size() for sizes > 1024T
  

• d3a07e6b
  by Josh Smith at 2018-11-06T13:23:19Z

  Add regression test for _pretty_size
  

• 0c09fb9c
  by richardmaw-codethink at 2018-11-06T14:12:20Z

  Merge branch 'Qinusty/unit-test-utils' into 'master'
  
  Fix issue with _pretty_size with large numbers of bytes
  
  See merge request BuildStream/buildstream!799

• b4eec489
  by Jim MacArthur at 2018-11-06T16:44:14Z

  _sandboxremote.py: Use the standard SandboxError.
  
  Replaces the custom one which was erroneously added during development.
  Fixes #746.
  

• 1f7acf74
  by Jim MacArthur at 2018-11-06T17:16:31Z

  Merge branch 'jmac/remote-sandbox-errors' into 'master'
  
  _sandboxremote.py: Use the standard SandboxError
  
  Closes #746
  
  See merge request BuildStream/buildstream!931

• 8e64ccef
  by James Ennis at 2018-11-07T11:06:11Z

  using_config.rst: Update our instructions for how we declare caches in user config
  

• 09ab676d
  by James Ennis at 2018-11-07T11:06:11Z

  format_project.rst: Update instructions regarding how we declare remote caches in project.conf
  

• f514124f
  by James Ennis at 2018-11-07T11:06:11Z

  using_configuring_artifact_server.rst: Remove examples and redirect to user config/project config
  

• 029ba17d
  by richardmaw-codethink at 2018-11-07T11:32:20Z

  Merge branch 'jennis/docs_patch' into 'master'
  
  Update our instructions/examples regarding how we declare remote caches
  
  See merge request BuildStream/buildstream!932

• 261e2cd3
  by Jim MacArthur at 2018-11-07T12:02:43Z

  element.py: Use _force_rmtree instead of custom code.
  
  There are arguments for and against using onerror - see issue #735
  for details.
  

• 8931e42c
  by Jim MacArthur at 2018-11-07T12:26:06Z

  Merge branch 'jmac/make_writable_combination' into 'master'
  
  Use _force_rmtree instead of custom code.
  
  Closes #735
  
  See merge request BuildStream/buildstream!927

• 6ccfab0b
  by Valentin David at 2018-11-07T16:28:47Z

  Fix cache corruption by scripts when layout and integration commands are used
  
  Root directory was marked as a non-artifact mount, so not using
  SafeHardLink.  However integration commands executed with write access
  to the root directory.
  
  Fixes #749
  

• f24e20e9
  by richardmaw-codethink at 2018-11-07T16:59:50Z

  Merge branch 'valentindavid/script-artifact-corruption' into 'master'
  
  Fix cache corruption by scripts when layout and integration commands are used
  
  Closes #749
  
  See merge request BuildStream/buildstream!934

• e28f668a
  by Valentin David at 2018-11-08T09:34:11Z

  Fix infinite recursion in default strip debug command
  
  Fixes #645.
  

• 83d15350
  by Valentin David at 2018-11-08T10:01:39Z

  Merge branch 'valentindavid/fix-recursive-strip-debug' into 'master'
  
  Fix infinite recursion in default strip debug command
  
  Closes #645
  
  See merge request BuildStream/buildstream!918

• 9f0e12f1
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Move bwrap checks in platform/linux.py
  
  Remove the bwraps checks from _site.py and put them in platform.linux
  which is the only place where they are run.
  
  This allows the removal of a double level of caching, making
  reasoning about tests easier
  

• cf2e0059
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  conftest.py: Ensure platform is not maintained between tests
  
  This removes the `_instance` on the platform object that we use
  for caching and not recreating the object everytime at the start
  of every test.
  
  This is to ensure our tests share the least amount of state.
  
  The performance penalty is from 5 to 10% accross the whole test suite.
  The readings were done 5 times for each before and after the change
  and on the same computer.
  

• c51ba01b
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Test that helpful messages are raised when missing dependencies
  
  This adds a `reason` to the SandboxEror thrown in sandboxdummy
  to be able to understand where the error comes from
  

• f116b9b7
  by Benjamin Schubert at 2018-11-08T11:07:12Z

  Merge branch 'bschubert/tests-no-chroot-linux' into 'master'
  
  test behavior when there is no sandbox on Linux
  
  Closes #696 and #736
  
  See merge request BuildStream/buildstream!921

• cc2e6ae5
  by Valentin David at 2018-11-08T12:41:36Z

  Fix bug with root mounted as non-artifact in script plugin.
  
  The issue was introduced by 6ccfab0b1b25990e406446d5cbe5aee83a5e158a.
  

• e578a89f
  by Valentin David at 2018-11-08T13:11:10Z

  Merge branch 'valentindavid/fix-script-install-root' into 'master'
  
  Fix bug with root mounted as non-artifact in script plugin.
  
  Closes #756 and #749
  
  See merge request BuildStream/buildstream!941

• c0a8bb66
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: WIP again to be kind to reviewers
  
  As someone coming from GitHub to GitLab, I was pleasantly surprised by
  the 'filter by WIP status' option.
  
  To make sure we get the most out of it, add a guideline to keep the
  filter clean, to reduce reviewer burden.
  

• f7231e90
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: non-WIP should always be landable
  
  I was recently surprised that we don't prefer addressing review comments
  in 'fixup!' commits.
  
  Coming from GitHub, I've found that fixup commits make it easier for
  reviewers to see what has changed since their last review. The idea is
  to use '--autosquash' before landing to clean up the history again.
  
  It's a pleasant surprise that it's easy to keep track of what changed in
  merge-requests between pushes, so we can always keep the history clean.
  
  Document this, so folks like me can see the light sooner.
  

• f7643440
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: emphasise the 'why' in commits
  
  Explain why it's useful to mention the decisions made for a change and
  provide links for background, e.g. issue numbers.
  
  My general experience of git histories is that folks don't do enough
  explaining, so it's great we have a contributing section for it.
  
  I've found it's easier for folks to stick to rules when they know the
  practical reasons for them, so provide some.
  

• dd5e7b04
  by Angelos Evripiotis at 2018-11-08T16:17:04Z

  Merge branch 'aevri/contributing_gitlab' into 'master'
  
  Add more to GitLab-relevant parts of contributing
  
  See merge request BuildStream/buildstream!935

• fe33e328
  by James Ennis at 2018-11-08T17:54:18Z

  using_config.rst: Add documentation to showing how to impose quotas on the local cache
  
  This patch partially resolves #700
  

• 09faf002
  by James Ennis at 2018-11-08T17:54:18Z

  artifactcache.py: Fix misleading error message when using % cache quota
  
  Due to the changed Exception message, this patch also changes
  the test_parse_size_over_1024T test in misc.py
  

• d153453c
  by Javier Jardón at 2018-11-08T18:22:54Z

  Merge branch 'jennis/quota_declaration_fix' into 'master'
  
  Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota
  
  Closes #700
  
  See merge request BuildStream/buildstream!939

• 7738f6df
  by Daniel Silverstone at 2018-11-09T11:25:44Z

  plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
  
  Previously, debug messages from `Plugin.__del__()` went to stdout
  which was a problem for the test suite in extremely rare and hard to
  predict circumstances.  This corrects that by sending the message to
  stderr where all the rest of the messages tend to go.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 62f59eaa
  by Valentin David at 2018-11-09T13:30:06Z

  Merge branch 'danielsilverstone-ct/plugin-destroy-debug-to-stderr' into 'master'
  
  plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
  
  See merge request BuildStream/buildstream!944

• 35ec6b6a
  by Jim MacArthur at 2018-11-09T14:58:22Z

  _casbaseddirectory.py: Enable direct CAS-to-CAS import.
  
  Fixes issue #574.
  

• b93100ec
  by Jim MacArthur at 2018-11-09T14:58:22Z

  _casbaseddirectory: Fix order of list_modified_paths
  
  This will now return paths in the same order as list_relative_paths.
  

• 78691fa0
  by Jim MacArthur at 2018-11-09T14:58:22Z

  virtual_directory_import.py: New test
  

• 7ce6581b
  by Jim MacArthur at 2018-11-09T15:25:46Z

  Merge branch 'jmac/cas_to_cas_v2' into 'master'
  
  Direct CAS-to-CAS import
  
  Closes #574
  
  See merge request BuildStream/buildstream!911

• 6f4351ec
  by Benjamin Schubert at 2018-11-09T16:53:42Z

  source.py: don't let StopIteration propagate to silence() contextmanager
  
  As per PEP 0479 (https://www.python.org/dev/peps/pep-0479/),
  StopIteration thrown in context managers are not valid starting from
  Python 3.7.
  

• e6e03451
  by richardmaw-codethink at 2018-11-12T11:05:15Z

  Merge branch 'bschubert/fix-silence-stopiteration-pep-0479' into 'master'
  
  source.py: don't let StopIteration propagate to silence() contextmanager
  
  See merge request BuildStream/buildstream!945

• ae5ccd76
  by Phillip Smyth at 2018-11-12T14:57:51Z

  _project.py: Validate nodes early in Project._load
  
  _project.py: Added validate_nodes() helper function to prevent duplicate lists
  
  element-path was being used before node validation resulting in uncaught errors
  

• 56a54161
  by Phillip Smyth at 2018-11-12T14:57:51Z

  tests/frontend: Add test for invalid element-path
  

• e55a9703
  by Jürg Billeter at 2018-11-12T15:44:45Z

  Merge branch 'element-path_not_validated' into 'master'
  
  Element path not validated before use
  
  See merge request BuildStream/buildstream!937

• c87d6e45
  by Valentin David at 2018-11-14T11:43:33Z

  Run tests on aarch64
  
  Fixes #755
  

• 16b01489
  by Valentin David at 2018-11-14T11:43:33Z

  Disable cachekey tests on other architectures than x86_64
  

• 85046b29
  by Valentin David at 2018-11-14T11:43:33Z

  Disable tests on example on other architectures than x86_64
  

• dd5a073b
  by Valentin David at 2018-11-14T11:43:33Z

  Add support for aarch64 in a test
  

• c5d72ae7
  by Valentin David at 2018-11-14T11:52:53Z

  Test aarch64 only overnight
  

• 264a57f6
  by Javier Jardón at 2018-11-14T12:22:03Z

  Merge branch 'valentindavid/integration-tests-aarch64' into 'master'
  
  Run tests on aarch64
  
  Closes #369 and #755
  
  See merge request BuildStream/buildstream!948

• da735e56
  by Richard Maw at 2018-11-14T13:30:34Z

  _platform/linux.py: Move get_bwrap_version into _site.py
  
  It's inconvenient to have to create a Linux platform to parse the bwrap version
  and we want to get the version in a consistent manner.
  

• e7633500
  by Richard Maw at 2018-11-14T13:30:34Z

  buildstream/sandbox/_sandboxbwrap.py: Distinguish sandbox failure from command failure
  
  If `bwrap` fails to set up the sandbox and start the payload command
  it won't write an exit-code in --json-status-fd,
  so we can report if it was a sandboxing failure if we don't get exit-code status
  and a payload command failure if we do and it's non-zero.
  
  Closes https://gitlab.com/BuildStream/buildstream/issues/286
  

• e9e08823
  by Richard Maw at 2018-11-14T13:30:34Z

  tests/testutils/site.py: Check for bwrap supporting --json-status-fd
  

• 90ca007e
  by Richard Maw at 2018-11-14T13:30:34Z

  tests/integration/sandbox-bwrap.py: Test distinguishing sandbox exit code from command
  

• 327b19dd
  by richardmaw-codethink at 2018-11-14T13:59:16Z

  Merge branch 'richardmaw/distinguish-sandboxing-build-fail' into 'master'
  
  Distinguish between bubblewrap sandboxing failure and command failure
  
  Closes #286
  
  See merge request BuildStream/buildstream!868

• 082b3811
  by Angelos Evripiotis at 2018-11-15T19:48:12Z

  contributing: add guidance on unit tests
  
  Decrease uncertainty around whether unit tests are welcome in the
  project or not.
  

• b5b79056
  by Jürg Billeter at 2018-11-15T20:24:58Z

  Merge branch 'aevri/unit_tests' into 'master'
  
  contributing: add guidance on unit tests
  
  See merge request BuildStream/buildstream!943

• 199bfff1
  by Tom Pollard at 2018-11-17T13:07:59Z

  Add cli main and user config option for 'pull-buildtrees' context.
  
  _context.py: Add pull_buildtrees global user context, the default
  of which is set to False via the addition of pull-buildtrees to
  userconfig.yaml cache group.
  
  _frontend/app.py & cli.py: Add --pull-buildtrees as a bst main
  option, which when passed will override the default or user defined
  context for pull_buildtrees.
  
  tests/completions/completions.py: Update for the added flag.
  

• 454dcaf1
  by Tom Pollard at 2018-11-17T13:07:59Z

  Don't pull artifact build trees by default.
  
  The addition of cached build trees being included in element
  artifacts has led to mostly redundant download overheads when
  pulling from a remote artifact server. As such the default behaviour
  of pull shouldn't fetch the build tree object if available.
  
  element.py: extend relevant pull logic for specified subdir
  consideration and ensure push logic does not lead to partial
  artifact pushes. Change assumption that the buildtree can be
  extracted if artifact is cached. __cached_buildtree() and
  __pull_directories() helpers added.
  
  _artifactcache/: artifactcache.py & cascache.py inclusion of
  helper functions for subdir artifact checking & fetching, fetch
  logic extended to only pull required artifact directories.
  extract & checkout updated to handle full/partial operation.
  
  tests/: addition of integration test pullbuildtrees.py,
  buildtrees.py adapted cli options, testutils/artifactshare.py
  has_artifact changed to return artifact digest on true condition.
  

• 0d309478
  by Jürg Billeter at 2018-11-17T13:49:58Z

  Merge branch 'tpollard/494' into 'master'
  
  Don't pull artifact buildtrees by default
  
  See merge request BuildStream/buildstream!786

• 27932739
  by Benjamin Schubert at 2018-11-19T09:46:39Z

  plugins/sources/pip.py: also look for python version named "python"
  
  In some virtual environments, we might not have the binary with the
  exact python version, which would lead the current pip plugin to
  fail to find a working pip version.
  
  This adds "python" at the start of the list of valid python versions
  

• ea2de561
  by Benjamin Schubert at 2018-11-19T10:19:24Z

  Merge branch 'bschubert/fix-pip-python' into 'master'
  
  plugins/sources/pip.py: also look for python version named "python"
  
  Closes #758
  
  See merge request BuildStream/buildstream!942

• f23b6031
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  tests/frontend/buildtrack.py: check for success after invocation
  
  Previously the result of the first call in test_build_track was not
  checked to be successful, which would lead to potential erros later on
  that were hard to debug.
  
  Let's check it everytime
  

• a6defc0b
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  Fix os.rename in git source element to correctly handle error codes
  
  According to the documentation
  (https://www.unix.com/man-page/POSIX/3posix/rename/), when the directory
  already is there, either EEXIST or ENOTEMPTY could be thrown.
  
  Previously only ENOTEMPTY was checked.
  
  Done:
    - Separated the move into its own function
    - Check for both errors
    - Create unit tests for it, covering most test cases
  

• 88089d2d
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  Extract atomic move function to utils.py
  
  Moving atomically a file/directory can be tricky since different
  errors might be raised for the same underlying problem.
  
  Having a utility function to reduce this discrepancies will help
  in ensuring we have correct behavior
  

• fd9e46be
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  cascache.py: use move_atomic instead of manual error checking
  

• d32e0b83
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  sources/pip.py: Use move_atomic instead of manual os.rename
  
  This uses move_atomic insteand of the manual os.rename and manual error
  checking and throws a SourceError for consistency with other modules.
  

• 6f837118
  by Benjamin Schubert at 2018-11-19T12:22:40Z

  Merge branch 'bschubert/fix-atomic-move-git-repo' into 'master'
  
  Fix os.rename in git source element to correctly handle error codes
  
  See merge request BuildStream/buildstream!938

• 30b72244
  by Chandan Singh at 2018-11-19T12:35:01Z

  buildstream/_pipeline.py: Simplify error message for assert_consistent()
  
  Currently, `Pipeline.assert_consistent()` prints an error message like
  the one below when certain elements have inconsistent sources:
  
  ```
  ...
        Element: hello.bst is inconsistent
          Source tar source at hello.bst [line 16 column 2] is missing ref
  ...
  ```
  
  Drop the word "Source" from the beginning of the message as
  "Source <source-kind> source ..." is kind of awkward to read.
  

• 76c5d2f8
  by Chandan Singh at 2018-11-19T12:35:01Z

  Add `bst source-checkout` command
  
  As discussed in
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
  add `bst source-checkout` command. This will allow users to checkout
  sources for a given target.
  
  * _frontend/cli.py: Add source-checkout command
  * _pipeline.py: Add assert_sources_cached() method
  * _stream.py: Add source_checkout method, abstract out
    __check_location_writable() method that used to part of checkout()
  

• 9f629638
  by Chandan Singh at 2018-11-19T13:03:36Z

  Merge branch 'chandan/source-checkout' into 'master'
  
  Add `bst source-checkout` command
  
  See merge request BuildStream/buildstream!820

• bc827cc8
  by Abderrahim Kitouni at 2018-11-19T14:55:15Z

  plugins/elements/cmake.yaml: always specify variable types
  
  cmake sometimes misinterprets relative paths as relative to the current directory
  if this is not specified. See freedesktop-sdk/freedesktop-sdk#431
  
  adjust tests/format/variables.py accordingly.
  

• 5fbc5f41
  by Valentin David at 2018-11-19T15:31:05Z

  Merge branch 'abderrahim/cmake-variable-types' into 'master'
  
  plugins/elements/cmake.yaml: always specify variable types
  
  Closes #761
  
  See merge request BuildStream/buildstream!947

• 06e28860
  by Benjamin Schubert at 2018-11-19T15:52:24Z

  Don't cache sandbox errors
  
  Sandbox errors (like missing host tools) are dependent on the host
  system and rarely on what is actually done.
  
  It is therefore better to not cache them as they are subject to
  change between two runs.
  
  Also add test to ensure sandbox failure are not cached
  

• dd36cfbc
  by Benjamin Schubert at 2018-11-19T15:52:24Z

  element.py: Only cache on element error or success
  
  Reverse the way we were handling caching by only caching
  when we know it's a problem with the build or the build is a
  success in order not to cache transient errors.
  

• eac7274d
  by Benjamin Schubert at 2018-11-19T16:29:07Z

  Merge branch 'bschubert/dont-cache-errors-from-host-tools' into 'master'
  
  Don't cache sandbox failures
  
  Closes #727
  
  See merge request BuildStream/buildstream!895

• 625dfe1f
  by Chandan Singh at 2018-11-19T16:41:17Z

  NEWS: Add entry for the new source-checkout command
  
  `bst source-checkout` command was recently added in
  https://gitlab.com/BuildStream/buildstream/merge_requests/820. Add a
  NEWS entry for the new command.
  

• d4f12184
  by Chandan Singh at 2018-11-19T17:15:10Z

  Merge branch 'chandan/source-checkout-news' into 'master'
  
  NEWS: Add entry for the new source-checkout command
  
  See merge request BuildStream/buildstream!960

• ba6c96e8
  by Richard Maw at 2018-11-19T19:54:30Z

  CasBasedDirectory: Change constructor to take a CASCache instead of a Context
  
  The Context was only used to obtain a reference to the CASCache
  and set the unused cas_directory field.
  

• 8a0dc3a3
  by Jürg Billeter at 2018-11-19T19:54:30Z

  tests/testutils/artifactshare.py: Do not create a fake context
  
  The fake context did not set the cache quota, triggering an error. With
  CASCache now separate from ArtifactCache, we can instantiate a CASCache
  without context.
  

• 8722aced
  by Jürg Billeter at 2018-11-19T19:54:30Z

  tests/plugins/pipeline.py: Load context default values
  

• fc56ffa4
  by Jürg Billeter at 2018-11-19T19:54:30Z

  _context.py: Drop duplicated default values for user configuration
  
  The default values are in userconfig.yaml, together with the
  documentation. The default values should not be duplicated in
  _context.py.
  

• 3f663d82
  by Jürg Billeter at 2018-11-19T20:36:57Z

  Merge branch 'juerg/context-default-values' into 'master'
  
  _context.py: Drop duplicated default values for user configuration
  
  See merge request BuildStream/buildstream!953

• b498cce7
  by Angelos Evripiotis at 2018-11-20T11:17:38Z

  _yaml: document node_get()'s default_value arg
  

• eb2d376f
  by Angelos Evripiotis at 2018-11-20T11:17:38Z

  _context: allow 'terminate' for scheduler.on-error
  
  Enable this option of 'terminate', which is mentioned in userconfig.yaml
  and handled in _frontend/app.py:_handle_failure(). It appears to have
  been left out of the valid_actions as an oversight.
  
  Originally introduced in
  https://gitlab.com/BuildStream/buildstream/commit/2622d5da9bd9fefd87436613d6e3e9770fdd0f28
  

• b81c4333
  by Angelos Evripiotis at 2018-11-20T11:17:38Z

  _context: refactor, extract _node_get_option_str
  
  Use a new helper function to simplify working with nodes that can only
  accept certain strings. This will be used when adding the prompt.*
  config options.
  
  In later work we can see if this function would be useful elsewhere, and
  could be added to '_yaml.py'.
  

• 27ca6593
  by Angelos Evripiotis at 2018-11-20T11:19:33Z

  Add prompt.auto-init buildstream.conf option
  
  Provide an option in buildstream.conf to disable the 'Would you like to
  ...' prompt when we cannot resolve a project.
  
  Some users prefer not to be interrupted by such prompts, so pave the way
  to creating options to disable all those that might get in the way.
  
  Follow the example of the advice.* options 'git-config', and create a
  namespace for these UI options grouped by behaviour, rather than an
  over-reaching 'ui.*' namespace. In later work perhaps we'll also add
  'advice.*' options.
  
  Add a NEWS item for this.
  

• 7ae3a3d2
  by Angelos Evripiotis at 2018-11-20T11:43:49Z

  Add prompt.workspace-... options
  
  Provide options in project.conf to disable the 'Are you sure ...'
  prompts when making destructive changes:
  
  - Add prompt.really-workspace-close-remove-dir
  - Add prompt.really-workspace-reset-hard
  
  Add a NEWS item for these.
  

• 8071c00c
  by Angelos Evripiotis at 2018-11-20T12:12:11Z

  Merge branch 'aevri/prompt-config' into 'master'
  
  Add 'prompt.*' config options to buildstream.conf
  
  See merge request BuildStream/buildstream!887

• fd41b0b5
  by Raoul Hidalgo Charman at 2018-11-20T16:32:10Z

  _sandboxremote.py: Add sigterm handler that sends CancelOperation
  

• c6306b88
  by Jürg Billeter at 2018-11-20T16:59:45Z

  Merge branch '725-job-cancellation-on-remote-builds' into 'master'
  
  _sandboxremote.py: Add sigterm handler that sends CancelOperation
  
  Closes #725
  
  See merge request BuildStream/buildstream!900

• 9fa8a881
  by James Ennis at 2018-11-20T17:00:42Z

  _yaml.py: Fix incorrect error message
  
  This patch ensures that we receive an appropriate error
  message if we specify an absolute path that leads within
  the project.
  

• fb8860db
  by Jürg Billeter at 2018-11-20T17:32:58Z

  Merge branch 'jennis/less_restrictive_pathways' into 'master'
  
  Fix warning when using an absolute path which exists inside the project
  
  See merge request BuildStream/buildstream!956

• 87751350
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-ci.yml: Use CI_PROJECT_DIR instead of $(pwd) for cache
  

• 8668df9f
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  Export variables globally for less repetition.
  
  This doesn't change the test behavior and enable for better
  composability of tests
  

• cc6a7f0d
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-cy.yml: make linux-tests-template more composable.
  
  This will allow for further de-duplication of work.
  

• eb37dd89
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-ci.yml: remove unused artifacts for unix tests
  
  The unix-logs is always empty, there is therefore no reasons of keeping
  it there
  

• e030b48e
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-ci.yml: use template for unix tests
  
  This makes all tests use the same template, which makes reasoning
  about them simpler
  

• 57e6358f
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  Ensure tests requiring bubblewrap are correctly marked
  

• e70dd1f8
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-ci.yml: Add linux without brwap/ostree to ensure tests behave
  
  This allows us to make sure that tests behave nicely while missing brwap
  and ostree.
  

• f7cb4386
  by Benjamin Schubert at 2018-11-20T18:10:48Z

  gitlab-ci.yml: Extract test command as a variable
  
  This removes the need of having it synchronized in multiple places
  

• 7f316a76
  by Benjamin Schubert at 2018-11-20T18:41:03Z

  Merge branch 'bschubert/add-fedora-nobwrap-tests' into 'master'
  
  Add tests for fedora with no bubblewrap available
  
  See merge request BuildStream/buildstream!949

• a50c598e
  by Benjamin Schubert at 2018-11-21T09:24:15Z

  utils.py: Ensure move_atomic booleans are keyword only arguments
  
  This is to be coherent with the rest of the codebase
  

• b2f604b8
  by Benjamin Schubert at 2018-11-21T09:24:15Z

  utils.py: improve documentation of move_atomic
  

• daca0c01
  by Benjamin Schubert at 2018-11-21T09:52:42Z

  Merge branch 'bschubert/mr938-comments' into 'master'
  
  Followup on MR 938, addressing additional comments
  
  See merge request BuildStream/buildstream!958

• 050cd1bd
  by Tom Pollard at 2018-11-21T11:37:48Z

  tests/plugin/pipeline.py: Avoid using host user conf
  
  create_pipeline was passing no config parameter to context.load,
  leading to _context.py loading in the host's buildstream user conf
  when running tests locally, potentially leading to unwanted errors.
  

• 58353aad
  by Jürg Billeter at 2018-11-21T14:29:49Z

  Merge branch 'tpollard/pipelinehostconfig' into 'master'
  
  tests/plugin/pipeline.py: Avoid using host user conf
  
  See merge request BuildStream/buildstream!961

• 2022c3e8
  by William Salmon at 2018-11-21T15:53:03Z

  tests/frontend/workspace.py: Refactor support function
  
  The Refactor now makes it possible to test for the forth coming multi
  element workspace open commands and to test for failure.
  

• 7bdd44b2
  by William Salmon at 2018-11-21T15:53:03Z

  Workspace CLI update
  
  This is to update the workspace CLI to as agreed on the mailing list
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00046.html
  
  This patch also introduces the default workspace directory.
  

• 6612e708
  by William Salmon at 2018-11-21T15:53:03Z

  Added a news item for workspace open changes
  

• 4d6e2cd3
  by Will Salmon at 2018-11-21T16:20:26Z

  Merge branch 'willsalmon/defaultWorkspaces' into 'master'
  
  Updated Workspace CLI
  
  See merge request BuildStream/buildstream!897

• dddf0eaa
  by Valentin David at 2018-11-22T11:12:53Z

  .gitlab-ci.yml: Bump version of fd.o SDK and bst_external in overnight tests
  

• abef70fe
  by Javier Jardón at 2018-11-22T11:41:31Z

  Merge branch 'valentindavid/overnight-tests-bump-fdo-sdk-version' into 'master'
  
  Bump version of fd.o SDK and bst_external in overnight tests
  
  See merge request BuildStream/buildstream!962

• 5c208698
  by Jim MacArthur at 2018-11-26T10:23:31Z

  Docs: Add remote execution architecture documentation.
  

• 5af455a4
  by Jim MacArthur at 2018-11-26T10:55:22Z

  Merge branch 'jmac/execution-architecture-docs' into 'master'
  
  Docs: Add remote execution architecture documentation.
  
  See merge request BuildStream/buildstream!923

• 4ca892e3
  by Jim MacArthur at 2018-11-27T09:30:13Z

  tests: Avoid hangs due to exceptions in subprocesses
  

• d09a18b8
  by Jim MacArthur at 2018-11-27T10:31:07Z

  Merge branch 'jmac/fix-test-hangs-2' into 'master'
  
  Avoid hanging artifact cache tests
  
  See merge request BuildStream/buildstream!964

• df60258a
  by Jürg Billeter at 2018-11-27T12:11:24Z

  source.py: Add optional _get_local_path() method
  

• 42809933
  by Jürg Billeter at 2018-11-27T12:11:24Z

  local.py: Implement _get_local_path()
  

• eb5b10f7
  by Jürg Billeter at 2018-11-27T12:11:24Z

  _loader.py: Use Source._get_local_path() for subprojects
  
  This eliminates unneeded copies of the project directory for junctions
  with a single local source.
  

• b2ea208c
  by Jürg Billeter at 2018-11-27T12:50:46Z

  Merge branch 'juerg/local-junctions' into 'master'
  
  Optimization for local junctions
  
  See merge request BuildStream/buildstream!290

• 223c0073
  by Jürg Billeter at 2018-11-27T13:41:09Z

  tests/integration/manual.py: Add test for command logging
  

• 38d16c62
  by Jürg Billeter at 2018-11-27T13:41:09Z

  sandbox: Deduplicate code to process cwd, env, and command arguments
  

• 85f5b574
  by Jürg Billeter at 2018-11-27T13:41:09Z

  sandbox/sandbox.py: Add SandboxFlags.NONE
  
  This improves readability.
  

• bdd6925f
  by Jürg Billeter at 2018-11-27T13:41:09Z

  element.py: Use SandboxFlags.NONE instead of 0
  

• 69005c76
  by Jürg Billeter at 2018-11-27T13:41:09Z

  scriptelement.py: Use SandboxFlags.NONE instead of 0
  

• f1767de2
  by Jürg Billeter at 2018-11-27T13:41:09Z

  sandbox/sandbox.py: Add command batching API
  
  This adds the batch() context manager.
  

• 87f7c99a
  by Jürg Billeter at 2018-11-27T13:41:09Z

  element.py: Pass element object to sandbox
  
  This allows the sandbox to output messages using the plugin id.
  

• 024d584d
  by Jürg Billeter at 2018-11-27T13:41:09Z

  element.py: Cache artifact also on SandboxCommandError
  

• bb7c3213
  by Jürg Billeter at 2018-11-27T13:41:09Z

  element.py: Defer workspace.prepared setting in case of pending commands
  

• 04f05b1a
  by Jürg Billeter at 2018-11-27T13:41:09Z

  element.py: Support command batching for integration commands
  

• 7827ac50
  by Jürg Billeter at 2018-11-27T13:44:49Z

  element.py: Add batch_prepare_assemble() method
  
  This allows batching of all sandbox commands aross prepare() and
  assemble().
  

• 63db90b2
  by Jürg Billeter at 2018-11-27T13:44:49Z

  buildelement.py: Support batching for integration and build commands
  

• 73dec86e
  by Jürg Billeter at 2018-11-27T13:44:49Z

  scriptelement.py: Support batching for integration and build commands
  

• 79442e0a
  by Jürg Billeter at 2018-11-27T13:44:49Z

  compose.py: Support batching for integration commands
  

• 6018b3bd
  by Jürg Billeter at 2018-11-27T13:44:49Z

  sandbox/_sandboxremote.py: Implement command batching
  
  Execute batched commands in a single shell script.
  

• ec3d4486
  by Jürg Billeter at 2018-11-27T13:44:49Z

  NEWS: Add entry for command batching
  

• 127d332f
  by Jürg Billeter at 2018-11-27T14:16:50Z

  Merge branch 'juerg/command-batching' into 'master'
  
  Command batching
  
  Closes #675
  
  See merge request BuildStream/buildstream!915

• d5b5538d
  by Jim MacArthur at 2018-11-27T15:32:19Z

  artifactcache: Move ArtifactCacheSpec code into CASRemoteSpec.
  
  There is nothing in ArtifactCacheSpec that's actually specific to
  artifacts, so I've made it a CAS class so we can use the same
  spec for remote execution.
  

• f224d4cb
  by Jim MacArthur at 2018-11-27T15:32:19Z

  artifactcache.py, _context.py: Move CASCache object into context
  
  Since the artifact cache and remote execution share the same
  local CAS store, they should share the same CASCache object. Moving
  this into context allows us to do this.
  

• 926c1446
  by Jim MacArthur at 2018-11-27T15:33:34Z

  _sandboxremote: Remove references to artifact cache.
  
  Uses the new config options introduced earlier to specify remote
  exec storage and execution service.
  

• d7cdb03e
  by Jim MacArthur at 2018-11-27T15:33:36Z

  Documentation: Update to show new config options
  

• 95d8244f
  by Jim MacArthur at 2018-11-27T15:33:36Z

  tests/sandboxes/remote-exec-config.py: New test.
  
  Tests that we get a sensible error message when we specify incomplete
  or wrong information in the configuration for remote execution.
  

• 7f2f62eb
  by Jim MacArthur at 2018-11-27T15:33:36Z

  artifactcache.py: Remove unused verify_digest_pushed
  

• c55ca3a5
  by Jim MacArthur at 2018-11-27T15:33:36Z

  Docs: Correct URLs in artifact cache example
  
  The port number must come after the hostname and before the path.
  The path is ignored by CASCache anyway, so including it is misleading.
  

• 2a8a3b19
  by Jim MacArthur at 2018-11-27T18:17:03Z

  Merge branch 'jmac/remote_execution_split' into 'master'
  
  Split remote execution from artifact cache
  
  Closes #750
  
  See merge request BuildStream/buildstream!946

• 499c70fd
  by Valentin David at 2018-11-28T09:11:21Z

  setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52
  

• 3513580c
  by Valentin David at 2018-11-28T14:02:32Z

  Merge branch 'valentindavid/ruamel_0_15_41' into 'master'
  
  setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52
  
  See merge request BuildStream/buildstream!975

• 26cdee08
  by Valentin David at 2018-11-28T14:29:52Z

  Use f_bavail to query available space. Not f_bfree.
  
  f_bfree space might not be usable. In practice we see failures in big
  disks because f_bfree is over 2GB and f_bavail is 0. We get ENOSPC if
  writing on disk then.
  

• 58ca298f
  by Valentin David at 2018-11-28T14:29:52Z

  Make cache clients not fail when a blob is not available.
  
  We plan to make cache incomplete. That is some blobs are missing.  For
  most of cases we will delete references when requested if they are
  incomplete. But there will be corner cases where objects are removed
  after the reference is requested.
  

• 227fa26d
  by Valentin David at 2018-11-28T14:29:52Z

  Fix type error in RPC messages
  

• 5ef19a0b
  by Valentin David at 2018-11-28T14:29:52Z

  Avoid copying temporary file when adding object to CAS in server.
  
  The file is already a temporary file and does not need copy.  ENOSPC
  is thrown during that copy in issue #609.
  
  Fixes #678.
  

• 8d2946ff
  by Valentin David at 2018-11-28T14:29:52Z

  "Fallocate" object disk space to avoid getting NOSPC error when writing
  
  This locks the temporary object file so that cleanup does not need to
  be done for every write.
  

• b587953a
  by Valentin David at 2018-11-28T14:29:52Z

  Update mtimes of objects for requested references.
  
  This also remove references when some objects are missing. This is in
  preparation for the move from reference to object garbage collection.
  

• a64f667d
  by Valentin David at 2018-11-28T14:29:52Z

  Move cas server from ref-based to object-based garbage collection.
  

• 353b90dd
  by Valentin David at 2018-11-28T14:29:52Z

  Cleanup cache in cas server more agressively
  
  When there is less than 2GB left, it cleans up have 10GB available.
  These values are configurable.
  

• ba9afa98
  by Valentin David at 2018-11-28T14:29:52Z

  Lock cache cleanup in cas server
  
  Cleaning up in parallel might slow down the cleaning process
  

• 9a458402
  by Valentin David at 2018-11-29T08:47:40Z

  Merge branch 'valentindavid/cache_server_fill_up' into 'master'
  
  Fix cleanup of cache in server when disk is full
  
  Closes #678
  
  See merge request BuildStream/buildstream!830

• 47b7a9ba
  by Jim MacArthur at 2018-11-29T09:58:48Z

  _sandboxremote.py: Remove unnecessary tests.
  
  push_message and push_directory will both raise assertion failures if
  they fail to send the digest to the server. Checking the digest
  returned by each only tests that the content was hashed locally, which
  cannot reasonably fail.
  

• 4a8d0565
  by Jim MacArthur at 2018-11-29T10:42:31Z

  Merge branch 'jmac/no-verify-digests' into 'master'
  
  _sandboxremote.py: Remove unnecessary tests.
  
  Closes #786
  
  See merge request BuildStream/buildstream!976

• 2e78e0d1
  by Valentin David at 2018-11-29T13:18:17Z

  Add support for .netrc in remote/tar/zip plugins
  
  Fixes #723.
  

• a6144100
  by Valentin David at 2018-11-29T13:18:17Z

  .gitlab-ci.yml: Update image versions where forgotten
  

• f6c184f5
  by Valentin David at 2018-11-29T16:15:56Z

  Merge branch 'valentindavid/netrc' into 'master'
  
  Add support for .netrc in remote/tar/zip plugins
  
  Closes #723
  
  See merge request BuildStream/buildstream!908

• 1f31aefc
  by Angelos Evripiotis at 2018-11-30T09:34:38Z

  man/: update with changes since Apr 2018
  
  It seems that the regenerated man pages haven't been committed for some
  time. Add them all in one lump now.
  
  Had to manually strip the version number from the generated files.
  
  Had to manually fix the NAME field of bst.1 to not be 'bst - None'.
  
  Oddly had to invoke click-man directly in order to get the changes to
  be applied, the usual setup.py route mentioned in contributing.rst
  didn't work.
  
  Eyeballed the result of invoking man on each of the commands, seems
  good.
  

• 033a5ad9
  by Jürg Billeter at 2018-11-30T10:09:11Z

  Merge branch 'aevri/update_man' into 'master'
  
  man/: update with changes since Apr 2018
  
  See merge request BuildStream/buildstream!981

• 1123b9a1
  by Tom Pollard at 2018-11-30T13:30:35Z

  _stream.py: Ability to pull missing buildtrees outside of pull/build
  
  Adds helper function _buildtree_pull_required() to determine if a
  pullqueue should be constructed, for commands outside of bst pull
  and build where it is determined that an element's buildtree
  artifact is to be required given the respective semantics and
  config. Utilised in push() to attempt to mitigate skipping the push
  of partial elements without the user having to have preceded it with
  an explicit pull.
  
  cli.py: Add new behaviour to push command description
  
  element.py: Move _cached_buildtree() to be non local private method,
  use _KeyStrength types to reduce duplication.
  
  tests/integration/pullbuildtrees.py also updated to cover this
  use-case.
  

• ad293ed3
  by Jürg Billeter at 2018-11-30T14:10:06Z

  Merge branch 'tpollard/774' into 'master'
  
  _stream.py: Ability to pull missing buildtrees outside of pull/build
  
  Closes #774
  
  See merge request BuildStream/buildstream!978

• 0d634c21
  by Phillip Smyth at 2018-11-30T14:16:36Z

  plugin.py: Add BAD_ELEMENT_SUFFIX warning
  

• d97ef822
  by Phillip Smyth at 2018-11-30T14:16:36Z

  _project.py: Check for fatal_warnings earlier
  

• 59d442e5
  by Phillip Smyth at 2018-11-30T15:41:57Z

  loader: Added file suffix check and helper function
  
  _loader/loader.py: Added Suffix check and warning for named elements and add helper function
  

• 2d6ec07d
  by Phillip Smyth at 2018-11-30T15:41:57Z

  cli.py: no autocomplete on files without .bst
  

• ac0bc4c0
  by Phillip Smyth at 2018-11-30T16:50:35Z

  Added tests for .bst suffix and completions
  
  buildcheckout.py: Add mandatory .bst suffix tests
  completions.py: Add test for fail on invalid suffix
  Added required files for testing
  integration/source-determinism.py: renamed test elements to end with .bst
  loader/__init__.py: initialised context properly with load and a message handler
  

• d4bc662a
  by Phillip Smyth at 2018-11-30T16:50:50Z

  NEWS: Added update about the Mandatory .bst change
  

• 7cea464a
  by Jürg Billeter at 2018-11-30T17:21:37Z

  Merge branch 'mandatory_suffix' into 'master'
  
  Mandatory .bst suffix
  
  See merge request BuildStream/buildstream!967

• cc5ef68c
  by Tristan Van Berkom at 2018-12-03T08:41:42Z

  MANIFEST.in: Include the sources for rebuilding the documentation session HTML
  
  This was previously missing, causing our CI to forget to rebuild the sessions
  when updating the documentation.
  
  This fixes issue #778
  

• 87ba19d7
  by Tristan Van Berkom at 2018-12-03T09:19:54Z

  Merge branch 'tristan/fix-doc-session-rebuilds' into 'master'
  
  MANIFEST.in: Include the sources for rebuilding the documentation session HTML
  
  Closes #778
  
  See merge request BuildStream/buildstream!984

• 1b24148f
  by Chandan Singh at 2018-12-03T09:46:13Z

  Ensure `--deps=none` option works for `bst checkout`
  
  Currently, `bst checkout --deps none` command always produces empty
  output. Fix this issue and add regression test for the same.
  
  - element_enums.py: Add Scope.NONE.
  
  - element.py: Ensure Scope.NONE works correctly in addition to
    Scope.RUN/Scope.ALL in Element.dependencies() and Element.search().
  
  - tests/frontend/buildcheckout.py: Fix tests for `--deps none`.
  
  Fixes #670.
  

• 8ac1e757
  by Chandan Singh at 2018-12-03T09:46:14Z

  Add `--deps build` option to `bst checkout`
  
  As discussed in
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
  add `--deps build` option to `bst checkout`. This will allow users to
  checkout the all build dependencies of a given element using a single
  command.
  
  - _frontend/cli.py: Add `--deps build` option for `bst checkout`.
  
  - element.py: Support `deps='build'` in Element._prepare_sandbox().
  
  - tests/frontend/buildcheckout.py: Ensure `--deps build` works as
    expected.
  

• ac60f47a
  by Benjamin Schubert at 2018-12-03T09:46:14Z

  Refactor and simplify _prepare_sandbox for elements
  
  Before we would have a intricate logics with multiple arguments
  that might get ignored.
  
  This simplifies the design and introduces a bool `shell` instead
  of having two different variables concerned about scope
  

30 changed files:

• .gitignore
• .gitlab-ci.yml
• CONTRIBUTING.rst
• MANIFEST.in
• NEWS
• README.rst
• buildstream/__init__.py
• buildstream/_artifactcache/artifactcache.py
• buildstream/_artifactcache/cascache.py
• buildstream/_artifactcache/casserver.py
• buildstream/_context.py
• buildstream/_elementfactory.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/status.py
• buildstream/_frontend/widget.py
• buildstream/_fuse/hardlinks.py
• buildstream/_fuse/mount.py
• buildstream/_includes.py
• buildstream/_loader/loadelement.py
• buildstream/_loader/loader.py
• buildstream/_message.py
• buildstream/_options/optionbool.py
• buildstream/_options/optionpool.py
• buildstream/_pipeline.py
• buildstream/_platform/darwin.py
• buildstream/_platform/linux.py
• buildstream/_project.py
• buildstream/_scheduler/jobs/job.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.