GNOME.org
 

[Notes] [Git][BuildStream/buildstream][laurence/update-readme] 18 commits: plugin.py: Add API to allow plugins to raise deprecation warnings

Title: GitLab

Laurence Urhegyi pushed to branch laurence/update-readme at BuildStream / buildstream

Commits:

15 changed files:

Changes:

  • .gitlab-ci.yml
    ... ... @@ -12,7 +12,7 @@ stages:
    12 12 
     variables:
    13 13 
       PYTEST_ADDOPTS: "--color=yes"
    14 14 
       INTEGRATION_CACHE: "${CI_PROJECT_DIR}/cache/integration-cache"
    15 
    -  TEST_COMMAND: "tox -- --color=yes --integration"
    15 
    +  TEST_COMMAND: "tox -- --color=yes --integration -n 2"
    16 16 
       COVERAGE_PREFIX: "${CI_JOB_NAME}."
    17 17
    18 18

  • .gitlab/merge_request_templates/stale_MR_message
    1 
    + The below is simply some boilerplate to text for use when commenting on
    2 
    + MRs that have not seen enough recent activity.
    3 
    +
    4 
    + Hi, thanks for your contribution! Unfortunately this MR is now over
    5 
    + a month without an update from the author, as per our policy I'm WIP'ing
    6 
    + this to keep our queue tidy. Please do clear the WIP status if you come
    7 
    + back to work on it. Cheers!
    8 
    +
    9 
    + Hi! Since another month has gone by on this MR without an update from the
    10 
    + author, as per our policy I'm going to close this to keep our queue tidy.
    11 
    + Please do re-open the MR if you come back to work on it. Cheers!
    \ No newline at end of file

  • CONTRIBUTING.rst
    ... ... @@ -105,6 +105,15 @@ Consider marking a merge request as WIP again if you are taking a while to
    105 105 
     address a review point. This signals that the next action is on you, and it
    106 106 
     won't appear in a reviewer's search for non-WIP merge requests to review.
    107 107
    108 
    +As a general rule of thumb, after a month of no activity from the submitter of
    109 
    +a non-WIP MR, we'll put it back into WIP with a polite note. Then after another
    110 
    +month with no activity we'll close the MR off entirely with another note.
    111 
    +In this way we are trying to ensure all of the MRs in our backlog are relevant
    112 
    +and up to date. We have some `boilerplate text
    113 
    +<https://gitlab.com/BuildStream/buildstream/blob/master/.gitlab/merge_request_templates/stale_MR_message.md>`_,
    114 
    +to help us when writing these notes.
    115 
    +
    116 
    +
    108 117
    109 118 
     Organized commits
    110 119 
     ~~~~~~~~~~~~~~~~~
    ... ... @@ -1589,6 +1598,15 @@ can run ``tox`` with ``-r`` or ``--recreate`` option.
    1589 1598
    1590 1599 
          ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'
    1591 1600
    1601 
    +.. tip::
    1602 
    +
    1603 
    +   We also have an environment called 'venv' which takes any arguments
    1604 
    +   you give it and runs them inside the same virtualenv we use for our
    1605 
    +   tests::
    1606 
    +
    1607 
    +     tox -e venv -- <your command(s) here>
    1608 
    +
    1609 
    +   Any commands after ``--`` will be run a virtualenv managed by tox.
    1592 1610
    1593 1611 
     Observing coverage
    1594 1612 
     ~~~~~~~~~~~~~~~~~~

  • buildstream/_versions.py
    ... ... @@ -23,7 +23,7 @@
    23 23 
     # This version is bumped whenever enhancements are made
    24 24 
     # to the `project.conf` format or the core element format.
    25 25 
     #
    26 
    -BST_FORMAT_VERSION = 21
    26 
    +BST_FORMAT_VERSION = 22
    27 27
    28 28
    29 29 
     # The base BuildStream artifact version

  • buildstream/plugin.py
    ... ... @@ -164,6 +164,25 @@ class Plugin():
    164 164 
            core format version :ref:`core format version <project_format_version>`.
    165 165 
         """
    166 166
    167 
    +    BST_PLUGIN_DEPRECATED = False
    168 
    +    """True if this element plugin has been deprecated.
    169 
    +
    170 
    +    If this is set to true, BuildStream will emmit a deprecation
    171 
    +    warning when this plugin is loaded. This deprecation warning may
    172 
    +    be suppressed on a plugin by plugin basis by setting
    173 
    +    ``suppress-deprecation-warnings: true`` in the relevent section of
    174 
    +    the project's :ref:`plugin configuration overrides <project_overrides>`.
    175 
    +
    176 
    +    """
    177 
    +
    178 
    +    BST_PLUGIN_DEPRECATION_MESSAGE = ""
    179 
    +    """ The message printed when this element shows a deprecation warning.
    180 
    +
    181 
    +    This should be set if BST_PLUGIN_DEPRECATED is True and should direct the user
    182 
    +    to the deprecated plug-in's replacement.
    183 
    +
    184 
    +    """
    185 
    +
    167 186 
         def __init__(self, name, context, project, provenance, type_tag):
    168 187
    169 188 
             self.name = name
    ... ... @@ -188,6 +207,12 @@ class Plugin():
    188 207 
             self.__kind = modulename.split('.')[-1]
    189 208 
             self.debug("Created: {}".format(self))
    190 209
    210 
    +        # If this plugin has been deprecated, emit a warning.
    211 
    +        if self.BST_PLUGIN_DEPRECATED and not self.__deprecation_warning_silenced():
    212 
    +            detail = "Using deprecated plugin {}: {}".format(self.__kind,
    213 
    +                                                             self.BST_PLUGIN_DEPRECATION_MESSAGE)
    214 
    +            self.__message(MessageType.WARN, detail)
    215 
    +
    191 216 
         def __del__(self):
    192 217 
             # Dont send anything through the Message() pipeline at destruction time,
    193 218 
             # any subsequent lookup of plugin by unique id would raise KeyError.
    ... ... @@ -767,6 +792,20 @@ class Plugin():
    767 792 
             else:
    768 793 
                 return self.name
    769 794
    795 
    +    def __deprecation_warning_silenced(self):
    796 
    +        if not self.BST_PLUGIN_DEPRECATED:
    797 
    +            return False
    798 
    +        else:
    799 
    +            silenced_warnings = set()
    800 
    +            project = self.__project
    801 
    +            plugin_overrides = {**project.element_overrides, **project.source_overrides}
    802 
    +
    803 
    +            for key, value in self.node_items(plugin_overrides):
    804 
    +                if value.get('suppress-deprecation-warnings', False):
    805 
    +                    silenced_warnings.add(key)
    806 
    +
    807 
    +            return self.get_kind() in silenced_warnings
    808 
    +
    770 809
    771 810 
     # Hold on to a lookup table by counter of all instantiated plugins.
    772 811 
     # We use this to send the id back from child processes so we can lookup

  • doc/source/arch_cachekeys.rst
    ... ... @@ -10,14 +10,14 @@ for the purpose of reusing artifacts in a well-defined, predictable way.
    10 10
    11 11 
     Structure
    12 12 
     ---------
    13 
    -Cache keys are SHA256 hash values generated from a pickled Python dict that
    13 
    +Cache keys are SHA256 hash values generated from a UTF-8 JSON document that
    14 14 
     includes:
    15 15
    16 
    -* Environment (e.g., project configuration and variables)
    17 
    -* Element configuration (details depend on element kind, ``Element.get_unique_key()``)
    18 
    -* Sources (``Source.get_unique_key()``)
    19 
    -* Dependencies (depending on cache key type, see below)
    20 
    -* Public data
    16 
    +* Environment (e.g., project configuration and variables).
    17 
    +* Element configuration (details depend on element kind, ``Element.get_unique_key()``).
    18 
    +* Sources (``Source.get_unique_key()``).
    19 
    +* Dependencies (depending on cache key type, see below).
    20 
    +* Public data.
    21 21
    22 22 
     Cache key types
    23 23 
     ---------------
    ... ... @@ -42,6 +42,9 @@ or the environment changes but it will not change when a dependency is updated.
    42 42 
     For elements without build dependencies the ``strong`` cache key is identical
    43 43 
     to the ``weak`` cache key.
    44 44
    45 
    +Note that dependencies which are not required at build time do not affect
    46 
    +either kind of key.
    47 
    +
    45 48 
     Strict build plan
    46 49 
     -----------------
    47 50 
     This is the default build plan that exclusively uses ``strong`` cache keys

  • doc/source/arch_dependency_model.rst
    ... ... @@ -42,7 +42,8 @@ Scope
    42 42 
     * **Scope.RUN**
    43 43
    44 44 
       In the :func:`Scope.RUN <buildstream.types.Scope.RUN>` scope, only elements
    45 
    -  which are required to run are considered, including the element itself.
    45 
    +  which are required to run are considered, including the element itself. Note
    46 
    +  that these are transitive - the service also requires the base runtime.
    46 47
    47 48 
       This is used when for example, launching a ``bst shell`` environment
    48 49 
       for the purpose of running, or in any case we need to consider which
    ... ... @@ -60,7 +61,7 @@ Scope
    60 61 
       .. image:: images/arch-dependency-model-build.svg
    61 62 
          :align: center
    62 63
    63 
    -  Note that build type dependencies are not transient, which is why the
    64 
    +  Note that build type dependencies are not transitive, which is why the
    64 65 
       *Bootstrap* element is not selected when pulling in the *Compiler* to
    65 66 
       build the *Application*.
    66 67

  • doc/source/arch_sandboxing.rst
    ... ... @@ -51,7 +51,7 @@ well.
    51 51 
     Filesystem access
    52 52 
     ~~~~~~~~~~~~~~~~~
    53 53
    54 
    -The filesystem inside sandboxes should be read only during element assembly,
    54 
    +The filesystem inside sandboxes should be read-only during element assembly,
    55 55 
     except for certain directories which element plugins can mark as being
    56 56 
     read/write. Most elements plugins derive from :mod:`BuildElement
    57 57 
     <buildstream.buildelement>`, which marks ``%{build-root}`` and
    ... ... @@ -158,17 +158,17 @@ and will refuse to push any artifacts built on such a system to a remote cache.
    158 158 
     For more information, see `issue #92
    159 159 
     <https://gitlab.com/BuildStream/buildstream/issues/92>`_.
    160 160
    161 
    -The Linux platform can operate as a standard user provided user namespace
    161 
    +The Linux platform can operate as a standard user, if user namespace
    162 162 
     support is available. If user namespace support is not available you have the
    163 163 
     option of installing bubblewrap as a setuid binary to avoid needing to run the
    164 164 
     entire ``bst`` process as the ``root`` user.
    165 165
    166 
    -The artifact cache on Linux systems is implemented using `OSTree
    167 
    -<https://github.com/ostreedev/ostree>`_, which can allow us to stage artifacts
    168 
    -using hardlinks instead of copying them. To avoid cache corruption it is
    169 
    -vital that hardlinked files cannot be overwritten. In cases where the root
    170 
    -filesystem inside the sandbox needs to be writable, a custom FUSE filesystem
    171 
    -named SafeHardlinks is used which provides a copy-on-write layer.
    166 
    +The artifact cache on Linux systems is implemented using a content-addressable
    167 
    +hardlink farm, which can allow us to stage artifacts using hardlinks instead of
    168 
    +copying them. To avoid cache corruption it is vital that hardlinked files
    169 
    +cannot be overwritten. In cases where the root filesystem inside the sandbox
    170 
    +needs to be writable, a custom FUSE filesystem named SafeHardlinks is used
    171 
    +which provides a copy-on-write layer.
    172 172
    173 173 
     Some of the operations on filesystem metadata listed above are not prohibited
    174 174 
     by the sandbox, but will instead be silently dropped when an artifact is

  • doc/source/arch_scheduler.rst
    ... ... @@ -19,9 +19,9 @@ The Job class has the following responsibilities:
    19 19 
     * Offering an abstract method for subclasses to handle the outcome of
    20 20 
       a job when it completes.
    21 21
    22 
    -* Forcefully terminating it's subprocess.
    22 
    +* Forcefully terminating its subprocess.
    23 23
    24 
    -* Suspending and resuming it's subprocess.
    24 
    +* Suspending and resuming its subprocess.
    25 25
    26 26 
     * Declaring the types of resources it will require, and which resources
    27 27 
       it will require exclusively.
    ... ... @@ -73,7 +73,7 @@ The Queue implementations are:
    73 73
    74 74 
       The pull queue tries to obtain a built artifact from a remote artifact server,
    75 75 
       it should be placed in advance of the fetch queue if used, since we can safely
    76 
    -  avoid fetching if fetching is not imerative, and we already have a cached
    76 
    +  avoid fetching if fetching is not imperative, and we already have a cached
    77 77 
       artifact.
    78 78
    79 79 
     * **Fetch**
    ... ... @@ -84,7 +84,7 @@ The Queue implementations are:
    84 84
    85 85 
     * **Build**
    86 86
    87 
    -  The build queue attempts to build the element if it's artifact is not locally
    87 
    +  The build queue attempts to build the element if its artifact is not locally
    88 88 
       present.
    89 89
    90 90 
     * **Push**

  • tests/plugins/deprecationwarnings/deprecationwarnings.py
    1 
    +import pytest
    2 
    +import tempfile
    3 
    +import os
    4 
    +from buildstream.plugintestutils import cli
    5 
    +from buildstream import _yaml
    6 
    +import buildstream.plugins.elements.manual
    7 
    +
    8 
    +
    9 
    +DATA_DIR = os.path.join(
    10 
    +    os.path.dirname(os.path.realpath(__file__)),
    11 
    +    "project"
    12 
    +)
    13 
    +
    14 
    +_DEPRECATION_MESSAGE = "Here is some detail."
    15 
    +_DEPRECATION_WARNING = "Using deprecated plugin deprecated_plugin: {}".format(_DEPRECATION_MESSAGE)
    16 
    +
    17 
    +
    18 
    +@pytest.mark.datafiles(DATA_DIR)
    19 
    +def test_deprecation_warning_present(cli, datafiles):
    20 
    +    project = os.path.join(datafiles.dirname, datafiles.basename)
    21 
    +    result = cli.run(project=project, args=['show', 'deprecated.bst'])
    22 
    +    result.assert_success()
    23 
    +    assert _DEPRECATION_WARNING in result.stderr
    24 
    +
    25 
    +
    26 
    +@pytest.mark.datafiles(DATA_DIR)
    27 
    +def test_suppress_deprecation_warning(cli, datafiles):
    28 
    +    project = os.path.join(datafiles.dirname, datafiles.basename)
    29 
    +    result = cli.run(project=project, args=['show', 'manual.bst'])
    30 
    +
    31 
    +    element_overrides = "elements:\n" \
    32 
    +                        "  deprecated_plugin:\n" \
    33 
    +                        "    suppress-deprecation-warnings : True\n"
    34 
    +
    35 
    +    project_conf = os.path.join(project, 'project.conf')
    36 
    +    with open(project_conf, 'a') as f:
    37 
    +        f.write(element_overrides)
    38 
    +
    39 
    +    result = cli.run(project=project, args=['show', 'deprecated.bst'])
    40 
    +    result.assert_success()
    41 
    +    assert _DEPRECATION_WARNING not in result.stderr

  • tests/plugins/deprecationwarnings/project/elements/deprecated.bst
    1 
    +kind: deprecated_plugin
    \ No newline at end of file

  • tests/plugins/deprecationwarnings/project/plugins/elements/deprecated_plugin.py
    1 
    +from buildstream import BuildElement, SandboxFlags
    2 
    +
    3 
    +
    4 
    +class DeprecatedPlugin(BuildElement):
    5 
    +    BST_PLUGIN_DEPRECATED = True
    6 
    +    BST_PLUGIN_DEPRECATION_MESSAGE = "Here is some detail."
    7 
    +
    8 
    +
    9 
    +# Plugin entry point
    10 
    +def setup():
    11 
    +    return DeprecatedPlugin

  • tests/plugins/deprecationwarnings/project/plugins/elements/deprecated_plugin.yaml
    1 
    +# Deprecated-plugin build element does not provide any default
    2 
    +# build commands
    3 
    +config:
    4 
    +
    5 
    +  # Commands for configuring the software
    6 
    +  #
    7 
    +  configure-commands: []
    8 
    +
    9 
    +  # Commands for building the software
    10 
    +  #
    11 
    +  build-commands: []
    12 
    +
    13 
    +  # Commands for installing the software into a
    14 
    +  # destination folder
    15 
    +  #
    16 
    +  install-commands: []
    17 
    +
    18 
    +  # Commands for stripping installed binaries
    19 
    +  #
    20 
    +  strip-commands:
    21 
    +  - |
    22 
    +    %{strip-binaries}
    \ No newline at end of file

  • tests/plugins/deprecationwarnings/project/project.conf
    1 
    +# Unique project name
    2 
    +name: deprecation-warnings
    3 
    +
    4 
    +# Required BuildStream format version
    5 
    +format-version: 20
    6 
    +
    7 
    +# Subdirectory where elements are stored
    8 
    +element-path: elements
    9 
    +
    10 
    +plugins:
    11 
    +
    12 
    +- origin: local
    13 
    +  path: plugins/elements
    14 
    +  elements:
    15 
    +    deprecated_plugin: 0

  • tox.ini
    ... ... @@ -91,3 +91,14 @@ commands =
    91 91 
     deps =
    92 92 
         click-man >= 0.3.0
    93 93 
         -rrequirements/requirements.txt
    94 
    +
    95 
    +#
    96 
    +# Usefull for running arbitrary scripts in a BuildStream virtual env
    97 
    +#
    98 
    +[testenv:venv]
    99 
    +commands = {posargs}
    100 
    +deps =
    101 
    +    -rrequirements/requirements.txt
    102 
    +    -rrequirements/dev-requirements.txt
    103 
    +    -rrequirements/plugin-requirements.txt
    104 
    +whitelist_externals = *

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.

    [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]