[BuildStream] bst CLI design and consistency (UI)

[Chandan Singh <chandan chandansingh net>]

Hi all,

As we have been making some breaking changes to the bst CLI in the 1.4 cycle,
various questions have come up. We have been trying to answer them in isolation
so far, it has been difficult to do so in the absence of high-level design
goals. With this thread, I am hoping we can take a more holistic approach to
designing the CLI in a way that is consistent and intuitive.

Design principles
=================

- Consistency: Following the same patterns across various subcommands, and
  within the same command when used in various ways.
- Intuitive: Hard to define, but I would say that the name of the command and
  the options should be enough to provide a basic understanding of what the
  command does.
- Minimal top-level: With this point I mean that we should avoid having too
  many top-level commands. Related commands should be grouped together in a
  subcommand, where applicable.
- Avoiding surprises: Commands should not do unexpected things.
- Brevity: Users should not need to type excessive amounts of text to do the
  necessary operations.

Current issues
==============

These are some areas where I think we can improve:

1. `source-checkout` will soon be the only command with a dash in the name
   (once it replaces `source-bundle`). Also see this ML thread [0].
2. `checkout` and `source-checkout` should support multiple elements. See
   GitLab issue #771. [1]
3. The syntax for handling elements in `bst shell`, as proposed in !909 [2]
   will introduce a new syntax that may not be obvious to new users.

Proposed changes
================

`bst artifact` subcommand
-------------------------

As was already proposed before in a previous ML thread [3], the `checkout`,
`pull` and `push` should be folded into the `bst artifact` subcommand.

`bst source` subcommand
-----------------------

Similar to how we have the `artifact` subcommand to deal with artifacts
(and `workspace` subcommand to deal with workspaces), maybe we should also have
a `source` subcommand to deal with sources. I would propose to move `fetch`,
`source-checkout` (as `source checkout`) and `track` under the `source`
subcommand.

If we do both the changes mentioned above, our top-level CLI will look
something like this:

    $ bst
    ...
    Commands:
      artifact      Manipulate built artifacts
      build         Build elements in a pipeline
      help          Print usage information
      init          Initialize a new BuildStream project
      shell         Shell into an element's sandbox environment
      show          Show elements in the pipeline
      source        Manipulate sources for elements
      workspace     Manipulate developer workspaces

---

Unified way of dealing with multiple elements
---------------------------------------------

At present, all BuildStream commands support multiple elements with the
exception of `checkout`, `source-checkout` and `shell`. For all these commands
we already have issues to support multiple elements. Since all commands are
planned to support multiple elements, it would be really nice if they all were
consistent in the way that they do it.

Due to limitations of `Click`, and to a certain degree to avoid ambiguity, it
seems like the way forward is to have only one kind of positional
argument, i.e. ELEMENTS. I would propose that everything else be supplied via
optional arguments (like --option).

If the above premise makes sense, then we would need to make the following
changes:

checkout and source-checkout
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Similar to how `workspace open` used to behave, `checkout` and
`source-checkout` currently accept ELEMENT and DIRECTORY as postional
arguments. To support multiple elements, I would propose the following changes
similar to how we handle the same in `workspace open`:

1. Add a configuration option to set default location for checking out
   elements, making the directory attribute optional. This will be a BREAKING
   change.
2. Convert the DIRECTORY attribute to a `--directory` optional argument.
3. Add support for multiple elements.

shell
~~~~~

Rather than introducing a new `-e` syntax for handling multiple elements, I am
wondering if we can do something similar by removing all positional arguments
except ELEMENTS, i.e. removing the CMD positional argument. Something like:

1. Convert COMMAND positional argument to a `-c`/`---command` optional
   argument. This will be a BREAKING change.
2. Add support for multiple elements.

---

Thanks for reading it this far, I'll look forward to hear your thoughts and
suggestions.


[0]: https://mail.gnome.org/archives/buildstream-list/2018-November/msg00076.html
[1]: https://gitlab.com/BuildStream/buildstream/issues/771
[2]: https://gitlab.com/BuildStream/buildstream/merge_requests/909
[3]: https://mail.gnome.org/archives/buildstream-list/2018-July/msg00051.html

Cheers,
Chandan