[gnome-build-meta.wiki] Update OpenQA for GNOME developers
- From: Sam Thursfield <sthursfield src gnome org>
- To: commits-list gnome org
- Cc:
- Subject: [gnome-build-meta.wiki] Update OpenQA for GNOME developers
- Date: Fri, 20 Aug 2021 16:06:05 +0000 (UTC)
commit 076a13bdf5026703372c31c554f7e28ad4cf1fb2
Author: Sam Thursfield <sam afuera me uk>
Date: Fri Aug 20 16:06:04 2021 +0000
Update OpenQA for GNOME developers
openqa/OpenQA-for-GNOME-developers.md | 26 +++++++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)
---
diff --git a/openqa/OpenQA-for-GNOME-developers.md b/openqa/OpenQA-for-GNOME-developers.md
index 05094db..38c9a7e 100644
--- a/openqa/OpenQA-for-GNOME-developers.md
+++ b/openqa/OpenQA-for-GNOME-developers.md
@@ -34,17 +34,33 @@ Tests can fail for several reasons. You will see the failure in the Web UI at ht
First check the "Logs & Assets" tab. Most useful is `autoinst-log.txt`, containing the log of installing the
ISO and running the testcases - this should log what went wrong. The `video.ogv` file can also be useful to
see what went wrong.
-If you see "no candidate needle matched" or "`assert_screen` failure", this indicates a graphical change
meaning the screenshot no longer matches. If the UI change is intentional then we will need to update the
needle. See below.
+If you see "no candidate needle matched" or "`assert_screen` failure", this indicates a graphical change
meaning the screenshot no longer matches. If the UI change is intentional then we will need to update the
needle.
-If in doubt about a test failure, [open a gnome-build-meta
issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
+The OpenQA Web UI lets you compare the actual screenshot with the expected ones. Select a needle in
'Candidate needles and tags', then use the orange slider to see differences. The 'landscape' and 'eye' icons
in the list switch between area-only or full diff. In the screenshot below, we have the expected image
(installation completed) on the left, and the actual screenshot (installation still in progress) on the right.
+![image](uploads/e1b3415068ed01db4ce68aada5972b0f/image.png)
+
+If you think the test failure represents a bug, open an issue against the relevant component. Otherwise, or
if in doubt, [open a gnome-build-meta issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
## How to update a test
-OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) allows you to
-update the needle from the web UI.
+Test scripts live in [gnome-build-meta](https://gitlab.gnome.org/GNOME/gnome-build-meta/) in the `openqa/`
folder. If you need to modify the test itself, create a branch of gnome-build-meta as normal with your
change. The test pipeline will run with your version of the tests, and once it passes we can merge this to
master.
+
+Needles live in [openqa-needles](https://gitlab.gnome.org/GNOME/openqa-needles) folder, and the `master`
branch is used by all tests. Multiple needles can have the same name, and OpenQA will pass a test if any one
needle matches. If you need to modify a needle, the workflow is to create a new one, and the preferred way is
to use OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) as follows.
+
+ 1. Ensure you have 'operator' permissions on OpenQA - ask an admin or open an issue if you don't have this.
+ 2. Open the failed test in the Web UI.
+ 3. Click the 'Create new needle' button to open the developer mode page
+ 4. Use the 'Screenshot and Areas' section to create a new needle
+ * Usually you 'Take image from' the screenshot and 'Copy areas from' the most recent needle.
+ * Click and drag on the screenshot to create, move and resize match areas
+ * Click an area once to select it, enabling the "Selected area" controls
+ * Click on area again to change its [type](http://open.qa/docs/#_areas) - 'exclude' matches can be very
useful when updating needles for UI changes.
+ 5. Now use 'Basics of Needle' to commit your change to
[openqa-needles.git](https://gitlab.gnome.org/GNOME/openqa-needles):
+ * Write a short commit message describing why you made the change
+ * Click 'Save' to commit and push from the OpenQA webUI.
-**FIXME: do you need to be an admin to use 'developer mode'? Write more here.**
+Now rerun the test which hopefully passes.
## Adding more tests
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]