[Notes] [Git][BuildStream/buildstream][tpollard/386] 48 commits: Documentation typos and fixes

[Tom Pollard <gitlab mg gitlab com>]

Title: GitLab

Tom Pollard pushed to branch tpollard/386 at BuildStream / buildstream

Commits:

• 7ceacfea
  by William Salmon at 2018-07-26T14:32:49Z

  Documentation typos and fixes
  
  Daniel went through the Doc's etc and made a list of suggestions.
  These are the result of the most easily patched suggestions and
  one or two thing that I noticed as well.
  

• 8c9fd9e4
  by Javier Jardón at 2018-07-26T15:32:16Z

  Merge branch 'willsalmon/documentation_form_notes' into 'master'
  
  Documentation typos and fixes
  
  See merge request BuildStream/buildstream!569

• 9ebd7fe1
  by Josh Smith at 2018-07-27T04:41:26Z

  _exceptions.py: Modify BstError API to allow optional retry
  
  job.py: Changes to the logic surrounding retry attempts and child process return codes
  element.py, source.py: ElementError and SourceError also implement this change.
  
  These exceptions now have an optional parameter of temporary which defaults to false. This will potentially break
  backwards compatibility where exceptions were previously raised and a retry was intended.
  
  To trigger a retry, one must now raise their SourceError or ElementError with temporary=True.
  
  This aims to fix #397.
  

• ca224425
  by Josh Smith at 2018-07-27T04:41:26Z

  _downloadablefilesource.py: Update network errors to be temporary
  
  This follows the change in 67ecd97a05279a3b7570ad59f05bf0a5973ef04c.
  

• 0a7cfcdf
  by Josh Smith at 2018-07-27T04:41:26Z

  tests/sources: deb, tar and zip all test for retry behaviour
  
  This is following 5d6418a06a3a0262df5f6b9e6e1a36578515fd4a where
  DownloadableFileSource was modified to raise 'temporary'
  errors when relating to file downloading.
  

• 9b2bc78f
  by Josh Smith at 2018-07-27T04:41:26Z

  cascache.py: Update cache push/pull errors to be temporary
  
  Further work needs to be done for the current grpc exceptions which are reraised.
  

• 4986527f
  by Josh Smith at 2018-07-27T04:41:26Z

  plugin.py: Extend Plugin.call() API
  
  Plugin.call() now takes fail_temporarily as an optional parameter,
  when supplied it will cause subsequent failures to trigger temporary
  errors as opposed to permanent errors.
  
  This also extends Plugin.check_output() which makes use of Plugin.call()
  

• a9a1048e
  by Josh Smith at 2018-07-27T04:41:26Z

  git.py: Update git plugin to fail temporarily on clone failures
  
  The git plugin will now make use of the fail_temporarily parameter
  to Plugin.call(), allowing failures to trigger a retry.
  

• aa071cb6
  by Tristan Van Berkom at 2018-07-27T06:13:53Z

  Merge branch 'tristan/temporary-failures' into 'master'
  
  Temorary failures
  
  Closes #397
  
  See merge request BuildStream/buildstream!574

• 4d86e7c5
  by Tristan Van Berkom at 2018-07-27T06:43:48Z

  element.py, source.py, plugin.py: Adjusting API version since annotations.
  
  Since we backported the temporary/permanent failures patch for #397
  into the `bst-1.2` branch, we need to adjust the since versions in master
  down to 1.2.
  

• fb222ba1
  by Josh Smith at 2018-07-27T08:50:45Z

  cascache.py: Raise ArtifactError on grpc error
  
  This aims to fix issue #502.
  

• 48916b8a
  by Tristan Maat at 2018-07-27T10:22:41Z

  Merge branch 'Qinusty/502' into 'master'
  
  cascache.py: Raise ArtifactError on grpc error
  
  Closes #502
  
  See merge request BuildStream/buildstream!576

• 23e080b9
  by James Ennis at 2018-07-27T11:13:50Z

  README.md: Minor corrections
  

• 196cfffc
  by James Ennis at 2018-07-27T11:13:50Z

  README.md: Direct users towards the tutorial before existing projects:
  
    This patch also removes displaying the projects that live within
    the buildstream-examples repository, as we wish to deprecate this
    repo and migrate the two examples.
    - #438 and #439
  
    In addition to this, both examples do not 'build' in their current
    state.
  

• 2b93574e
  by James Ennis at 2018-07-27T11:13:50Z

  main_install.rst: Add note to refer Windows and macOS users to docker
  

• 63e2320e
  by Josh Smith at 2018-07-27T11:16:55Z

  .gitignore: Ignore .pytest_cache/
  

• 6ea97b17
  by Javier Jardón at 2018-07-27T11:54:20Z

  Merge branch 'jennis/doc_improvements' into 'master'
  
  Refer readers to our tutorial before referring them to existing bst projects
  
  See merge request BuildStream/buildstream!578

• 8b46e874
  by Jonathan Maw at 2018-07-27T12:24:56Z

  project: Parse and store mirrors
  

• a2e9c62a
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Move _ALIAS_SEPARATOR into utils
  
  The separator is useful in source files other than _project.py
  

• 7c993ac0
  by Jonathan Maw at 2018-07-27T12:24:56Z

  source: Store the url aliases or use an override
  
  This is part of a later plan to implement mirroring without forcing
  everyone to update their sources. We use the expected calls to
  Source.translate_url() when running Source.configure() to extract the
  aliases from the URL. Multiple aliases must be extracted because
  sources exist that may fetch from multiple aliases (for example, git
  submodules)
  
  Later, we want to substitute another URI where the alias normally reads
  from the project - We accomplish this by re-instantiating the Source
  with the alias overrides passed as an argument to the constructor.
  

• 2889003c
  by Jonathan Maw at 2018-07-27T12:24:56Z

  source: When fetching, try to fetch from mirrors first
  
  **KLUDGE WARNING**: This involves making the source store its "meta"
  object so that it's possible to create a copy of the source inside the
  fetch queue, instead of back when the pipeline was being loaded.
  
  This adds the SourceFetcher class, which is intended for sources that
  fetch from multiple URLs (e.g. the git source and its submodules)
  
  Fix when fetching
  

• f81e8e7b
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Set default mirror via command-line or user config
  
  In user config (buildstream.conf), it is set with the "default-mirror"
  field.
  
  On the command-line, it is set with "--default-mirror"
  

• 755ed898
  by Jonathan Maw at 2018-07-27T12:24:56Z

  Add mirrored source tracking
  

• 19c01a56
  by Jonathan Maw at 2018-07-27T12:24:56Z

  bzr.py: Improve mirror support
  
  This fixes:
  * Bzr repositories pulling from the branch they were created with.
  * Bzr's _ensure_mirror() not actually checking that it successfully
    mirrored the ref.
  

• 84872141
  by Jonathan Maw at 2018-07-27T12:24:56Z

  git.py: Improve mirror support
  
  When implementing fetching from mirrors, I encountered some problems
  with the git source:
  
  1. The mirror URL was using translate_url()'s output, so if a different
     alias was used, then fetching from the mirror would go to a different
     directory, and be inaccessible.
  2. After fixing that, fetching was unable to pull from a URL other than
     the one used at repository creation, meaning it wouldn't actually
     pull from the mirror.
  

• bd51a0b2
  by Jonathan Maw at 2018-07-27T12:24:56Z

  testutils: Add a helper to copy a testutils repo
  
  This is helpful if you want to test what happens when you have one repo
  that has diverged from another. By copying the repo you're sure they
  start with shared history.
  
  This is especially useful when mirroring.
  

• 909120ab
  by Jonathan Maw at 2018-07-27T12:24:56Z

  tests: Add mirrored fetching and tracking tests
  

• 1cbc2e17
  by Jonathan Maw at 2018-07-27T12:24:56Z

  doc: Add documentation for source mirroring
  

• 202d9d26
  by Jonathan Maw at 2018-07-27T12:24:56Z

  NEWS: Add item for source mirroring
  

• 2b23898d
  by Jonathan Maw at 2018-07-27T12:24:56Z

  doc: Add tutorials for setting up git and tar mirrors
  

• bd4d0355
  by Jonathan Maw at 2018-07-27T13:08:00Z

  Merge branch '328-support-for-downloading-sources-from-mirrors' into 'master'
  
  Resolve "Support for downloading sources from mirrors"
  
  Closes #328
  
  See merge request BuildStream/buildstream!404

• cb4693b2
  by Josh Smith at 2018-07-27T14:00:26Z

  Merge branch 'Qinusty/pytest_cache_gitignore'
  

• 8a96679a
  by Josh Smith at 2018-07-27T14:01:07Z

  Merge branch 'master' of gitlab.com:BuildStream/buildstream
  

• f5c8ff61
  by Josh Smith at 2018-07-27T14:10:45Z

  _context.py: Cache size is now restricted to available disk space
  
  This address issue #491.
  
  When attempting to run buildstream with a configuration specifying
  a cache quota larger than your available disk space, buildstream
  will alert the user and exit.
  
  Note:
  This takes into consideration your current cache usage and
  therefore restricts the overall size of your artifact cache folder.
  

• 32ddb544
  by Qinusty at 2018-07-27T14:57:30Z

  Merge branch 'Qinusty/491' into 'master'
  
  Cache quota is now restricted to available disk space
  
  See merge request BuildStream/buildstream!563

• cb8c99e5
  by Tristan Maat at 2018-07-29T07:52:06Z

  buildelement.py: Only display one activity message (#507)
  

• 08b7562b
  by Tristan Van Berkom at 2018-07-29T10:23:43Z

  Merge branch '507-some-log-lines-appear-to-be-duplicates' into 'master'
  
  Resolve "Some log lines appear to be duplicates"
  
  Closes #507
  
  See merge request BuildStream/buildstream!573

• 3b7158b3
  by Tristan Van Berkom at 2018-07-29T10:25:02Z

  source.py: Bring down since version for source mirroring APIs
  
  Since we have now backported this to `bst-1.2`, the APIs have
  been introduced in 1.2 and not 1.4
  

• e84f8b24
  by Tristan Van Berkom at 2018-07-29T10:25:02Z

  NEWS: Bring down since version of backported features to 1.1.5
  
  These features will be released in 1.1.5 since they have been
  backported, lets reflect this in the NEWS file in master.
  
  Also this adds the backported `remote` plugin to the NEWS
  

• df18d38c
  by Tristan Van Berkom at 2018-07-29T10:26:08Z

  git.py: Handle concurrent download completions properly
  
  Use os.rename() to rename the cloned temporary repository into
  place in the source cache, and issue a STATUS message when discarding
  a duplicate clone, in the case where the same repository is cloned
  twice in parallel.
  
  The problem with using shutil.move() is that it will create the source
  directory in a subdirectory of the destination when the destination
  exists, so it's behavior depends on whether the destination exists.
  
  This shutil.move() behavior has so far hidden the race condition
  where a duplicate repo is created in a subdirectory, as you need
  to have three concurrent downloads of the same repo in order to
  trigger the error.
  
  This fixes issue #503
  

• 92714b03
  by Tristan Van Berkom at 2018-07-29T11:18:52Z

  Merge branch 'tristan/fix-spurious-errors' into 'master'
  
  git.py: Handle concurrent download completions properly
  
  Closes #503
  
  See merge request BuildStream/buildstream!583

• 8568d272
  by Phil Dawson at 2018-07-30T07:58:07Z

  examples: add project demonstrating a junction element
  
  This is part of the work towards #437.
  

• efa24e7e
  by Phil Dawson at 2018-07-30T07:58:07Z

  junctions.run: Add terminal session output for junctions walkthrough.
  
  This is part of the work towards #437.
  

• bf175bf7
  by Phil Dawson at 2018-07-30T07:58:07Z

  junction-elements.rst: Add junctions walkthrough
  
  This is part of the work towards #437.
  

• e8931048
  by James Ennis at 2018-07-30T07:58:07Z

  using_advanced_features.rst: Added file
  

• b4f25190
  by Phil Dawson at 2018-07-30T07:58:07Z

  tests/examples/junctions.py: Add tests for junctions walkthrough
  
  This is part of the work towards issue #437.
  

• aff76ae9
  by Phil Dawson at 2018-07-30T09:15:49Z

  Merge branch 'phil/437-junction-tutorial' into 'master'
  
  Phil/437 junction tutorial
  
  See merge request BuildStream/buildstream!550

• c313ad17
  by Tom Pollard at 2018-07-30T10:57:49Z

  widget.py: Limit failure summary to currently failing elements
  
  Change widget.py print_summary() to only print the failure
  messages of elements in the Failure Summary that failed on the
  current try.
  

30 changed files:

• .gitignore
• HACKING.rst
• NEWS
• README.rst
• buildstream/__init__.py
• buildstream/_artifactcache/cascache.py
• buildstream/_context.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/widget.py
• buildstream/_loader/loader.py
• buildstream/_project.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_versions.py
• buildstream/buildelement.py
• buildstream/element.py
• buildstream/plugin.py
• buildstream/plugins/sources/_downloadablefilesource.py
• buildstream/plugins/sources/bzr.py
• buildstream/plugins/sources/git.py
• buildstream/source.py
• buildstream/utils.py
• doc/Makefile
• + doc/examples/junctions/elements/callHello.bst
• + doc/examples/junctions/elements/hello-junction.bst
• + doc/examples/junctions/files/callHello.sh
• + doc/examples/junctions/project.conf
• + doc/sessions/junctions.run
• + doc/source/advanced-features/junction-elements.rst

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.