[Notes] [Git][BuildStream/buildstream][bschubert/tests-no-chroot-linux]

Benjamin Schubert pushed to branch bschubert/tests-no-chroot-linux at BuildStream / buildstream

Commits:

8e64ccef

by James Ennis at 2018-11-07T11:06:11Z

using_config.rst: Update our instructions for how we declare caches in user config

09ab676d

by James Ennis at 2018-11-07T11:06:11Z

format_project.rst: Update instructions regarding how we declare remote caches in project.conf

f514124f

by James Ennis at 2018-11-07T11:06:11Z

using_configuring_artifact_server.rst: Remove examples and redirect to user config/project config

029ba17d

by richardmaw-codethink at 2018-11-07T11:32:20Z

Merge branch 'jennis/docs_patch' into 'master'

Update our instructions/examples regarding how we declare remote caches

See merge request BuildStream/buildstream!932

302923c8

by Benjamin Schubert at 2018-11-07T11:32:49Z

Ensure platform is not maintained between tests

This removes the `_instance` on the platform object that we use
for caching and not recreating the object everytime at the start
of every test.

This is to ensure our tests share the least amount of state.

The performance penalty is from 2 to 5% accross the whole test suite.
The readings were done 5 times for each before and after the change
and on the same computer.

c8d98b57

by Benjamin Schubert at 2018-11-07T11:32:49Z

Ensure helpful messages are raised when missing dependencies

We need to supress the brwap version caching for it to work.

This caching leaks for every test but removing it every time seems
too prohibitive, given that, normally, the bwrap version would not
change.

On a fedora 29 laptop, the performance degradation was around 5-10%
over the whole test suite if used globally.

6 changed files:

buildstream/sandbox/_sandboxdummy.py
conftest.py
doc/source/format_project.rst
doc/source/using_config.rst
doc/source/using_configuring_artifact_server.rst
+ tests/integration/missing_dependencies.py

Changes:

buildstream/sandbox/_sandboxdummy.py

@@ -42,4 +42,5 @@ class SandboxDummy(Sandbox):
                                 "'{}'".format(command[0]),
                                 reason='missing-command')
 -        raise SandboxError("This platform does not support local builds: {}".format(self._reason))
 +        raise SandboxError("This platform does not support local builds: {}".format(self._reason),
 +                           reason="unavailable-local-sandbox")

conftest.py

@@ -23,6 +23,8 @@ import shutil
  import pytest
 +from buildstream._platform.platform import Platform
++
  def pytest_addoption(parser):
      parser.addoption('--integration', action='store_true', default=False,
@@ -52,3 +54,8 @@ def integration_cache(request):
          shutil.rmtree(os.path.join(cache_dir, 'artifacts'))
      except FileNotFoundError:
          pass
++
++
 +@pytest.fixture(autouse=True)
 +def clean_platform_cache():
 +    Platform._instance = None

doc/source/format_project.rst

@@ -190,19 +190,34 @@ for more detail.
  Artifact server
  ~~~~~~~~~~~~~~~
  If you have setup an :ref:`artifact server <artifacts>` for your
 -project then it is convenient to configure this in your ``project.conf``
 +project then it is convenient to configure the following in your ``project.conf``
  so that users need not have any additional configuration to communicate
  with an artifact share.
  .. code:: yaml
 +  #
 +  # Artifacts
 +  #
    artifacts:
 +    # A remote cache from which to download prebuilt artifacts
 +    - url: https://foo.com/artifacts:11001
 +      server.cert: server.crt
 +    # A remote cache from which to upload/download built/prebuilt artifacts
 +    - url: https://foo.com/artifacts:11002
 +      server-cert: server.crt
 +      client-cert: client.crt
 +      client-key: client.key
 -    # A url from which to download prebuilt artifacts
 -    url: https://foo.com/artifacts
 +.. note::
++
 +    You can also specify a list of different caches here; earlier entries in the
 +    list will have higher priority than later ones.
++
 +The use of ports are required to distinguish between pull only access and
 +push/pull access. For information regarding the server/client certificates
 +and keys, please see: :ref:`Key pair for the server <server_authentication>`.
 -You can also specify a list of caches here; earlier entries in the list
 -will have higher priority than later ones.
  Remote execution
  ~~~~~~~~~~~~~~~~

doc/source/using_config.rst

@@ -32,38 +32,75 @@ the supported configurations on a project wide basis are listed here.
  Artifact server
  ~~~~~~~~~~~~~~~
 -The project you build will often specify a :ref:`remote artifact cache
 -<artifacts>` already, but you may want to specify extra caches. There are two
 -ways to do this.  You can add one or more global caches:
 +Although project's often specify a :ref:`remote artifact cache <artifacts>` in
 +their ``project.conf``, you may also want to specify extra caches.
 -**Example**
 +Assuming that your host/server is reachable on the internet as ``artifacts.com``
 +(for example), there are two ways to declare remote caches in your user
 +configuration:
++
 +1. Adding global caches:
  .. code:: yaml
 +   #
 +   # Artifacts
 +   #
     artifacts:
 -     url: https://artifacts.com/artifacts
 +     # Add a cache to pull from
 +     - url: https://artifacts.com/artifacts:11001
 +       server-cert: server.crt
 +     # Add a cache to push/pull to/from
 +     - url: https://artifacts.com/artifacts:11002
 +       server-cert: server.crt
 +       client-cert: client.crt
 +       client-key: client.key
 +       push: true
 +     # Add another cache to pull from
 +     - url: https://anothercache.com/artifacts:8080
 +       server-cert: another_server.crt
++
 +.. note::
 -Caches listed there will be considered lower priority than those specified
 -by the project configuration.
 +    Caches declared here will be used by **all** BuildStream project's on the user's
 +    machine and are considered a lower priority than those specified in the project
 +    configuration.
 -You can also add project-specific caches:
 -**Example**
 +2. Specifying caches for a specific project within the user configuration:
  .. code:: yaml
     projects:
       project-name:
         artifacts:
 -         - url: https://artifacts.com/artifacts1
 -         - url: ssh://user artifacts com/artifacts2
 +         # Add a cache to pull from
 +         - url: https://artifacts.com/artifacts:11001
 +           server-cert: server.crt
 +         # Add a cache to push/pull to/from
 +         - url: https://artifacts.com/artifacts:11002
 +           server-cert: server.crt
 +           client-cert: client.crt
 +           client-key: client.key
             push: true
 +         # Add another cache to pull from
 +         - url: https://ourprojectcache.com/artifacts:8080
 +           server-cert: project_server.crt
++
++
 +.. note::
++
 +    Caches listed here will be considered a higher priority than those specified
 +    by the project. Furthermore, for a given list of URLs, earlier entries will
 +    have higher priority.
++
++
 +Notice that the use of different ports for the same server distinguishes between
 +pull only access and push/pull access. For information regarding this and the
 +server/client certificates and keys, please see:
 +:ref:`Key pair for the server <server_authentication>`.
 -Caches listed here will be considered higher priority than those specified
 -by the project.
 -If you give a list of URLs, earlier entries in the list will have higher
 -priority than later ones.
  Strict build plan
  ~~~~~~~~~~~~~~~~~

doc/source/using_configuring_artifact_server.rst

@@ -98,6 +98,8 @@ Command reference
     :prog: bst-artifact-server
 +.. _server_authentication:
++
  Key pair for the server
  ~~~~~~~~~~~~~~~~~~~~~~~
@@ -237,52 +239,12 @@ We can then check if the services are successfully running with:
  For more information on systemd services see:
  `Creating Systemd Service Files <https://www.devdungeon.com/content/creating-systemd-service-files>`_.
 -User configuration
 -~~~~~~~~~~~~~~~~~~
 -The user configuration for artifacts is documented with the rest
 -of the :ref:`user configuration documentation <user_config>`.
+-
 -Note that for self-signed certificates, the public key fields are mandatory.
+-
 -Assuming you have the same setup used in this document, and that your
 -host is reachable on the internet as ``artifacts.com`` (for example),
 -then a user can use the following user configuration:
+-
 -Pull-only:
+-
 -.. code:: yaml
+-
 -   #
 -   #    Artifacts
 -   #
 -   artifacts:
+-
 -     url: https://artifacts.com:11001
+-
 -     # Optional server certificate if not trusted by system root certificates
 -     server-cert: server.crt
 +Declaring remote artifact caches
 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 +Remote artifact caches can be declared within either:
 -Pull and push:
+-
 -.. code:: yaml
+-
 -   #
 -   #    Artifacts
 -   #
 -   artifacts:
+-
 -     url: https://artifacts.com:11002
+-
 -     # Optional server certificate if not trusted by system root certificates
 -     server-cert: server.crt
+-
 -     # Optional client key pair for authentication
 -     client-key: client.key
 -     client-cert: client.crt
+-
 -     push: true
+-
 -.. note::
 +1. The :ref:`project configuration <project_essentials_artifacts>`, or
 +2. The :ref:`user configuration <config_artifacts>`.
 -    Equivalent statements can be delcared in a project's configuration file
 -    (the ``project.conf``).
 +Please follow the above links to see examples showing how we declare remote
 +caches in both the project configuration and the user configuration, respectively.

tests/integration/missing_dependencies.py

 +import os
 +import pytest
 +from tests.testutils import cli
 +from tests.testutils.site import IS_LINUX
++
 +from buildstream import _site, _yaml
 +from buildstream._exceptions import ErrorDomain
++
++
 +pytestmark = pytest.mark.integration
++
++
 +# Project directory
 +DATA_DIR = os.path.join(
 +    os.path.dirname(os.path.realpath(__file__)),
 +    'project',
 +)
++
++
 +@pytest.fixture
 +def clean_bwrap_cache():
 +    def wipe():
 +        _site._bwrap_major = None
 +        _site._bwrap_minor = None
 +        _site._bwrap_patch = None
++
 +    wipe()
 +    yield
 +    wipe()
++
++
 +@pytest.mark.skipif(not IS_LINUX, reason='Only available on Linux')
 +@pytest.mark.datafiles(DATA_DIR)
 +@pytest.mark.usefixtures("clean_bwrap_cache")
 +def test_missing_brwap_has_nice_error_message(cli, datafiles):
 +    project = os.path.join(datafiles.dirname, datafiles.basename)
 +    element_path = os.path.join(project, 'elements', 'element.bst')
++
 +    # Write out our test target
 +    element = {
 +        'kind': 'script',
 +        'depends': [
 +            {
 +                'filename': 'base.bst',
 +                'type': 'build',
 +            },
 +        ],
 +        'config': {
 +            'commands': [
 +                'false',
 +            ],
 +        },
 +    }
 +    _yaml.dump(element, element_path)
++
 +    # Build without access to host tools, this should fail with a nice error
 +    result = cli.run(
 +        project=project, args=['build', 'element.bst'], env={'PATH': ''})
 +    result.assert_task_error(ErrorDomain.SANDBOX, 'unavailable-local-sandbox')
 +    assert "not found" in result.stderr
++
++
 +@pytest.mark.skipif(not IS_LINUX, reason='Only available on Linux')
 +@pytest.mark.datafiles(DATA_DIR)
 +@pytest.mark.usefixtures("clean_bwrap_cache")
 +def test_old_brwap_has_nice_error_message(cli, datafiles, tmp_path):
 +    import buildstream._platform
 +    buildstream._platform.Platform._instance = None
++
 +    bwrap = tmp_path.joinpath('bin/bwrap')
 +    bwrap.parent.mkdir()
 +    with bwrap.open('w') as fp:
 +        fp.write('''
 +            #!/bin/bash
 +            echo bubblewrap 0.0.1
 +        '''.strip())
++
 +    bwrap.chmod(0o755)
++
 +    project = os.path.join(datafiles.dirname, datafiles.basename)
 +    element_path = os.path.join(project, 'elements', 'element3.bst')
++
 +    # Write out our test target
 +    element = {
 +        'kind': 'script',
 +        'depends': [
 +            {
 +                'filename': 'base.bst',
 +                'type': 'build',
 +            },
 +        ],
 +        'config': {
 +            'commands': [
 +                'true && true && false',
 +            ],
 +        },
 +    }
 +    _yaml.dump(element, element_path)
++
 +    # Build without access to host tools, this should fail with a nice error
 +    result = cli.run(
 +        project=project,
 +        args=['--debug', '--verbose', 'build', 'element3.bst'],
 +        env={'PATH': str(tmp_path.joinpath('bin'))})
 +    result.assert_task_error(ErrorDomain.SANDBOX, 'unavailable-local-sandbox')
 +    assert "too old" in result.stderr

[Notes] [Git][BuildStream/buildstream][bschubert/tests-no-chroot-linux] 6 commits: using_config.rst: Update our instructions for how we declare caches in user config

Benjamin Schubert pushed to branch bschubert/tests-no-chroot-linux at BuildStream / buildstream

Commits:

6 changed files:

Changes: