-
b498cce7
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_yaml: document node_get()'s default_value arg
-
eb2d376f
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_context: allow 'terminate' for scheduler.on-error
Enable this option of 'terminate', which is mentioned in userconfig.yaml
and handled in _frontend/app.py:_handle_failure(). It appears to have
been left out of the valid_actions as an oversight.
Originally introduced in
https://gitlab.com/BuildStream/buildstream/commit/2622d5da9bd9fefd87436613d6e3e9770fdd0f28
-
b81c4333
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_context: refactor, extract _node_get_option_str
Use a new helper function to simplify working with nodes that can only
accept certain strings. This will be used when adding the prompt.*
config options.
In later work we can see if this function would be useful elsewhere, and
could be added to '_yaml.py'.
-
27ca6593
by Angelos Evripiotis
at 2018-11-20T11:19:33Z
Add prompt.auto-init buildstream.conf option
Provide an option in buildstream.conf to disable the 'Would you like to
...' prompt when we cannot resolve a project.
Some users prefer not to be interrupted by such prompts, so pave the way
to creating options to disable all those that might get in the way.
Follow the example of the advice.* options 'git-config', and create a
namespace for these UI options grouped by behaviour, rather than an
over-reaching 'ui.*' namespace. In later work perhaps we'll also add
'advice.*' options.
Add a NEWS item for this.
-
7ae3a3d2
by Angelos Evripiotis
at 2018-11-20T11:43:49Z
Add prompt.workspace-... options
Provide options in project.conf to disable the 'Are you sure ...'
prompts when making destructive changes:
- Add prompt.really-workspace-close-remove-dir
- Add prompt.really-workspace-reset-hard
Add a NEWS item for these.
-
8071c00c
by Angelos Evripiotis
at 2018-11-20T12:12:11Z
Merge branch 'aevri/prompt-config' into 'master'
Add 'prompt.*' config options to buildstream.conf
See merge request BuildStream/buildstream!887
-
fd41b0b5
by Raoul Hidalgo Charman
at 2018-11-20T16:32:10Z
_sandboxremote.py: Add sigterm handler that sends CancelOperation
-
c6306b88
by Jürg Billeter
at 2018-11-20T16:59:45Z
Merge branch '725-job-cancellation-on-remote-builds' into 'master'
_sandboxremote.py: Add sigterm handler that sends CancelOperation
Closes #725
See merge request BuildStream/buildstream!900
-
9fa8a881
by James Ennis
at 2018-11-20T17:00:42Z
_yaml.py: Fix incorrect error message
This patch ensures that we receive an appropriate error
message if we specify an absolute path that leads within
the project.
-
fb8860db
by Jürg Billeter
at 2018-11-20T17:32:58Z
Merge branch 'jennis/less_restrictive_pathways' into 'master'
Fix warning when using an absolute path which exists inside the project
See merge request BuildStream/buildstream!956
-
87751350
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Use CI_PROJECT_DIR instead of $(pwd) for cache
-
8668df9f
by Benjamin Schubert
at 2018-11-20T18:10:48Z
Export variables globally for less repetition.
This doesn't change the test behavior and enable for better
composability of tests
-
cc6a7f0d
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-cy.yml: make linux-tests-template more composable.
This will allow for further de-duplication of work.
-
eb37dd89
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: remove unused artifacts for unix tests
The unix-logs is always empty, there is therefore no reasons of keeping
it there
-
e030b48e
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: use template for unix tests
This makes all tests use the same template, which makes reasoning
about them simpler
-
57e6358f
by Benjamin Schubert
at 2018-11-20T18:10:48Z
Ensure tests requiring bubblewrap are correctly marked
-
e70dd1f8
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Add linux without brwap/ostree to ensure tests behave
This allows us to make sure that tests behave nicely while missing brwap
and ostree.
-
f7cb4386
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Extract test command as a variable
This removes the need of having it synchronized in multiple places
-
7f316a76
by Benjamin Schubert
at 2018-11-20T18:41:03Z
Merge branch 'bschubert/add-fedora-nobwrap-tests' into 'master'
Add tests for fedora with no bubblewrap available
See merge request BuildStream/buildstream!949
-
a50c598e
by Benjamin Schubert
at 2018-11-21T09:24:15Z
utils.py: Ensure move_atomic booleans are keyword only arguments
This is to be coherent with the rest of the codebase
-
b2f604b8
by Benjamin Schubert
at 2018-11-21T09:24:15Z
utils.py: improve documentation of move_atomic
-
daca0c01
by Benjamin Schubert
at 2018-11-21T09:52:42Z
Merge branch 'bschubert/mr938-comments' into 'master'
Followup on MR 938, addressing additional comments
See merge request BuildStream/buildstream!958
-
050cd1bd
by Tom Pollard
at 2018-11-21T11:37:48Z
tests/plugin/pipeline.py: Avoid using host user conf
create_pipeline was passing no config parameter to context.load,
leading to _context.py loading in the host's buildstream user conf
when running tests locally, potentially leading to unwanted errors.
-
58353aad
by Jürg Billeter
at 2018-11-21T14:29:49Z
Merge branch 'tpollard/pipelinehostconfig' into 'master'
tests/plugin/pipeline.py: Avoid using host user conf
See merge request BuildStream/buildstream!961
-
2022c3e8
by William Salmon
at 2018-11-21T15:53:03Z
tests/frontend/workspace.py: Refactor support function
The Refactor now makes it possible to test for the forth coming multi
element workspace open commands and to test for failure.
-
7bdd44b2
by William Salmon
at 2018-11-21T15:53:03Z
Workspace CLI update
This is to update the workspace CLI to as agreed on the mailing list
https://mail.gnome.org/archives/buildstream-list/2018-September/msg00046.html
This patch also introduces the default workspace directory.
-
6612e708
by William Salmon
at 2018-11-21T15:53:03Z
Added a news item for workspace open changes
-
4d6e2cd3
by Will Salmon
at 2018-11-21T16:20:26Z
Merge branch 'willsalmon/defaultWorkspaces' into 'master'
Updated Workspace CLI
See merge request BuildStream/buildstream!897
-
dddf0eaa
by Valentin David
at 2018-11-22T11:12:53Z
.gitlab-ci.yml: Bump version of fd.o SDK and bst_external in overnight tests
-
abef70fe
by Javier Jardón
at 2018-11-22T11:41:31Z
Merge branch 'valentindavid/overnight-tests-bump-fdo-sdk-version' into 'master'
Bump version of fd.o SDK and bst_external in overnight tests
See merge request BuildStream/buildstream!962
-
5c208698
by Jim MacArthur
at 2018-11-26T10:23:31Z
Docs: Add remote execution architecture documentation.
-
5af455a4
by Jim MacArthur
at 2018-11-26T10:55:22Z
Merge branch 'jmac/execution-architecture-docs' into 'master'
Docs: Add remote execution architecture documentation.
See merge request BuildStream/buildstream!923
-
4ca892e3
by Jim MacArthur
at 2018-11-27T09:30:13Z
tests: Avoid hangs due to exceptions in subprocesses
-
d09a18b8
by Jim MacArthur
at 2018-11-27T10:31:07Z
Merge branch 'jmac/fix-test-hangs-2' into 'master'
Avoid hanging artifact cache tests
See merge request BuildStream/buildstream!964
-
df60258a
by Jürg Billeter
at 2018-11-27T12:11:24Z
source.py: Add optional _get_local_path() method
-
42809933
by Jürg Billeter
at 2018-11-27T12:11:24Z
local.py: Implement _get_local_path()
-
eb5b10f7
by Jürg Billeter
at 2018-11-27T12:11:24Z
_loader.py: Use Source._get_local_path() for subprojects
This eliminates unneeded copies of the project directory for junctions
with a single local source.
-
b2ea208c
by Jürg Billeter
at 2018-11-27T12:50:46Z
Merge branch 'juerg/local-junctions' into 'master'
Optimization for local junctions
See merge request BuildStream/buildstream!290
-
223c0073
by Jürg Billeter
at 2018-11-27T13:41:09Z
tests/integration/manual.py: Add test for command logging
-
38d16c62
by Jürg Billeter
at 2018-11-27T13:41:09Z
sandbox: Deduplicate code to process cwd, env, and command arguments
-
85f5b574
by Jürg Billeter
at 2018-11-27T13:41:09Z
sandbox/sandbox.py: Add SandboxFlags.NONE
This improves readability.
-
bdd6925f
by Jürg Billeter
at 2018-11-27T13:41:09Z
element.py: Use SandboxFlags.NONE instead of 0
-
69005c76
by Jürg Billeter
at 2018-11-27T13:41:09Z
scriptelement.py: Use SandboxFlags.NONE instead of 0
-
f1767de2
by Jürg Billeter
at 2018-11-27T13:41:09Z
sandbox/sandbox.py: Add command batching API
This adds the batch() context manager.
-
87f7c99a
by Jürg Billeter
at 2018-11-27T13:41:09Z
element.py: Pass element object to sandbox
This allows the sandbox to output messages using the plugin id.
-
024d584d
by Jürg Billeter
at 2018-11-27T13:41:09Z
element.py: Cache artifact also on SandboxCommandError
-
bb7c3213
by Jürg Billeter
at 2018-11-27T13:41:09Z
element.py: Defer workspace.prepared setting in case of pending commands
-
04f05b1a
by Jürg Billeter
at 2018-11-27T13:41:09Z
element.py: Support command batching for integration commands
-
7827ac50
by Jürg Billeter
at 2018-11-27T13:44:49Z
element.py: Add batch_prepare_assemble() method
This allows batching of all sandbox commands aross prepare() and
assemble().
-
63db90b2
by Jürg Billeter
at 2018-11-27T13:44:49Z
buildelement.py: Support batching for integration and build commands
-
73dec86e
by Jürg Billeter
at 2018-11-27T13:44:49Z
scriptelement.py: Support batching for integration and build commands
-
79442e0a
by Jürg Billeter
at 2018-11-27T13:44:49Z
compose.py: Support batching for integration commands
-
6018b3bd
by Jürg Billeter
at 2018-11-27T13:44:49Z
sandbox/_sandboxremote.py: Implement command batching
Execute batched commands in a single shell script.
-
ec3d4486
by Jürg Billeter
at 2018-11-27T13:44:49Z
NEWS: Add entry for command batching
-
127d332f
by Jürg Billeter
at 2018-11-27T14:16:50Z
Merge branch 'juerg/command-batching' into 'master'
Command batching
Closes #675
See merge request BuildStream/buildstream!915
-
d5b5538d
by Jim MacArthur
at 2018-11-27T15:32:19Z
artifactcache: Move ArtifactCacheSpec code into CASRemoteSpec.
There is nothing in ArtifactCacheSpec that's actually specific to
artifacts, so I've made it a CAS class so we can use the same
spec for remote execution.
-
f224d4cb
by Jim MacArthur
at 2018-11-27T15:32:19Z
artifactcache.py, _context.py: Move CASCache object into context
Since the artifact cache and remote execution share the same
local CAS store, they should share the same CASCache object. Moving
this into context allows us to do this.
-
926c1446
by Jim MacArthur
at 2018-11-27T15:33:34Z
_sandboxremote: Remove references to artifact cache.
Uses the new config options introduced earlier to specify remote
exec storage and execution service.
-
d7cdb03e
by Jim MacArthur
at 2018-11-27T15:33:36Z
Documentation: Update to show new config options
-
95d8244f
by Jim MacArthur
at 2018-11-27T15:33:36Z
tests/sandboxes/remote-exec-config.py: New test.
Tests that we get a sensible error message when we specify incomplete
or wrong information in the configuration for remote execution.
-
7f2f62eb
by Jim MacArthur
at 2018-11-27T15:33:36Z
artifactcache.py: Remove unused verify_digest_pushed
-
c55ca3a5
by Jim MacArthur
at 2018-11-27T15:33:36Z
Docs: Correct URLs in artifact cache example
The port number must come after the hostname and before the path.
The path is ignored by CASCache anyway, so including it is misleading.
-
2a8a3b19
by Jim MacArthur
at 2018-11-27T18:17:03Z
Merge branch 'jmac/remote_execution_split' into 'master'
Split remote execution from artifact cache
Closes #750
See merge request BuildStream/buildstream!946
-
499c70fd
by Valentin David
at 2018-11-28T09:11:21Z
setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52
-
3513580c
by Valentin David
at 2018-11-28T14:02:32Z
Merge branch 'valentindavid/ruamel_0_15_41' into 'master'
setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52
See merge request BuildStream/buildstream!975
-
26cdee08
by Valentin David
at 2018-11-28T14:29:52Z
Use f_bavail to query available space. Not f_bfree.
f_bfree space might not be usable. In practice we see failures in big
disks because f_bfree is over 2GB and f_bavail is 0. We get ENOSPC if
writing on disk then.
-
58ca298f
by Valentin David
at 2018-11-28T14:29:52Z
Make cache clients not fail when a blob is not available.
We plan to make cache incomplete. That is some blobs are missing. For
most of cases we will delete references when requested if they are
incomplete. But there will be corner cases where objects are removed
after the reference is requested.
-
227fa26d
by Valentin David
at 2018-11-28T14:29:52Z
Fix type error in RPC messages
-
5ef19a0b
by Valentin David
at 2018-11-28T14:29:52Z
Avoid copying temporary file when adding object to CAS in server.
The file is already a temporary file and does not need copy. ENOSPC
is thrown during that copy in issue #609.
Fixes #678.
-
8d2946ff
by Valentin David
at 2018-11-28T14:29:52Z
"Fallocate" object disk space to avoid getting NOSPC error when writing
This locks the temporary object file so that cleanup does not need to
be done for every write.
-
b587953a
by Valentin David
at 2018-11-28T14:29:52Z
Update mtimes of objects for requested references.
This also remove references when some objects are missing. This is in
preparation for the move from reference to object garbage collection.
-
a64f667d
by Valentin David
at 2018-11-28T14:29:52Z
Move cas server from ref-based to object-based garbage collection.
-
353b90dd
by Valentin David
at 2018-11-28T14:29:52Z
Cleanup cache in cas server more agressively
When there is less than 2GB left, it cleans up have 10GB available.
These values are configurable.
-
ba9afa98
by Valentin David
at 2018-11-28T14:29:52Z
Lock cache cleanup in cas server
Cleaning up in parallel might slow down the cleaning process
-
9a458402
by Valentin David
at 2018-11-29T08:47:40Z
Merge branch 'valentindavid/cache_server_fill_up' into 'master'
Fix cleanup of cache in server when disk is full
Closes #678
See merge request BuildStream/buildstream!830
-
47b7a9ba
by Jim MacArthur
at 2018-11-29T09:58:48Z
_sandboxremote.py: Remove unnecessary tests.
push_message and push_directory will both raise assertion failures if
they fail to send the digest to the server. Checking the digest
returned by each only tests that the content was hashed locally, which
cannot reasonably fail.
-
4a8d0565
by Jim MacArthur
at 2018-11-29T10:42:31Z
Merge branch 'jmac/no-verify-digests' into 'master'
_sandboxremote.py: Remove unnecessary tests.
Closes #786
See merge request BuildStream/buildstream!976
-
2e78e0d1
by Valentin David
at 2018-11-29T13:18:17Z
Add support for .netrc in remote/tar/zip plugins
Fixes #723.
-
a6144100
by Valentin David
at 2018-11-29T13:18:17Z
.gitlab-ci.yml: Update image versions where forgotten
-
f6c184f5
by Valentin David
at 2018-11-29T16:15:56Z
Merge branch 'valentindavid/netrc' into 'master'
Add support for .netrc in remote/tar/zip plugins
Closes #723
See merge request BuildStream/buildstream!908
-
1f31aefc
by Angelos Evripiotis
at 2018-11-30T09:34:38Z
man/: update with changes since Apr 2018
It seems that the regenerated man pages haven't been committed for some
time. Add them all in one lump now.
Had to manually strip the version number from the generated files.
Had to manually fix the NAME field of bst.1 to not be 'bst - None'.
Oddly had to invoke click-man directly in order to get the changes to
be applied, the usual setup.py route mentioned in contributing.rst
didn't work.
Eyeballed the result of invoking man on each of the commands, seems
good.
-
033a5ad9
by Jürg Billeter
at 2018-11-30T10:09:11Z
Merge branch 'aevri/update_man' into 'master'
man/: update with changes since Apr 2018
See merge request BuildStream/buildstream!981
-
1123b9a1
by Tom Pollard
at 2018-11-30T13:30:35Z
_stream.py: Ability to pull missing buildtrees outside of pull/build
Adds helper function _buildtree_pull_required() to determine if a
pullqueue should be constructed, for commands outside of bst pull
and build where it is determined that an element's buildtree
artifact is to be required given the respective semantics and
config. Utilised in push() to attempt to mitigate skipping the push
of partial elements without the user having to have preceded it with
an explicit pull.
cli.py: Add new behaviour to push command description
element.py: Move _cached_buildtree() to be non local private method,
use _KeyStrength types to reduce duplication.
tests/integration/pullbuildtrees.py also updated to cover this
use-case.
-
ad293ed3
by Jürg Billeter
at 2018-11-30T14:10:06Z
Merge branch 'tpollard/774' into 'master'
_stream.py: Ability to pull missing buildtrees outside of pull/build
Closes #774
See merge request BuildStream/buildstream!978
-
0d634c21
by Phillip Smyth
at 2018-11-30T14:16:36Z
plugin.py: Add BAD_ELEMENT_SUFFIX warning
-
d97ef822
by Phillip Smyth
at 2018-11-30T14:16:36Z
_project.py: Check for fatal_warnings earlier
-
59d442e5
by Phillip Smyth
at 2018-11-30T15:41:57Z
loader: Added file suffix check and helper function
_loader/loader.py: Added Suffix check and warning for named elements and add helper function
-
2d6ec07d
by Phillip Smyth
at 2018-11-30T15:41:57Z
cli.py: no autocomplete on files without .bst
-
ac0bc4c0
by Phillip Smyth
at 2018-11-30T16:50:35Z
Added tests for .bst suffix and completions
buildcheckout.py: Add mandatory .bst suffix tests
completions.py: Add test for fail on invalid suffix
Added required files for testing
integration/source-determinism.py: renamed test elements to end with .bst
loader/__init__.py: initialised context properly with load and a message handler
-
d4bc662a
by Phillip Smyth
at 2018-11-30T16:50:50Z
NEWS: Added update about the Mandatory .bst change
-
7cea464a
by Jürg Billeter
at 2018-11-30T17:21:37Z
Merge branch 'mandatory_suffix' into 'master'
Mandatory .bst suffix
See merge request BuildStream/buildstream!967
-
cc5ef68c
by Tristan Van Berkom
at 2018-12-03T08:41:42Z
MANIFEST.in: Include the sources for rebuilding the documentation session HTML
This was previously missing, causing our CI to forget to rebuild the sessions
when updating the documentation.
This fixes issue #778
-
87ba19d7
by Tristan Van Berkom
at 2018-12-03T09:19:54Z
Merge branch 'tristan/fix-doc-session-rebuilds' into 'master'
MANIFEST.in: Include the sources for rebuilding the documentation session HTML
Closes #778
See merge request BuildStream/buildstream!984
-
1b24148f
by Chandan Singh
at 2018-12-03T09:46:13Z
Ensure `--deps=none` option works for `bst checkout`
Currently, `bst checkout --deps none` command always produces empty
output. Fix this issue and add regression test for the same.
- element_enums.py: Add Scope.NONE.
- element.py: Ensure Scope.NONE works correctly in addition to
Scope.RUN/Scope.ALL in Element.dependencies() and Element.search().
- tests/frontend/buildcheckout.py: Fix tests for `--deps none`.
Fixes #670.
-
8ac1e757
by Chandan Singh
at 2018-12-03T09:46:14Z
Add `--deps build` option to `bst checkout`
As discussed in
https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
add `--deps build` option to `bst checkout`. This will allow users to
checkout the all build dependencies of a given element using a single
command.
- _frontend/cli.py: Add `--deps build` option for `bst checkout`.
- element.py: Support `deps='build'` in Element._prepare_sandbox().
- tests/frontend/buildcheckout.py: Ensure `--deps build` works as
expected.
-
ac60f47a
by Benjamin Schubert
at 2018-12-03T09:46:14Z
Refactor and simplify _prepare_sandbox for elements
Before we would have a intricate logics with multiple arguments
that might get ignored.
This simplifies the design and introduces a bool `shell` instead
of having two different variables concerned about scope
-
b1430fb5
by Tristan Van Berkom
at 2018-12-03T11:01:54Z
Merge branch 'chandan/bst-checkout-build' into 'master'
Add `--deps build` option to `bst checkout`
Closes #670
See merge request BuildStream/buildstream!819
-
f1f58f48
by Benjamin Schubert
at 2018-12-03T16:03:57Z
Mock storage space checks for tests.
Fix #530
- Extract free space computation in a function for easier
mocking
- Mock space computation during cache quota tests
- Mock cache size during cachque quota tests
- Add two more tests when the configuration would require
to much storage space